2024-03-16T08:39:05.659 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T08:39:05.659 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305 --owner scheduled_pdonnell@teuthology --name pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305/orig.config.yaml 2024-03-16T08:39:05.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:39:05.809 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305 branch: wip-batrick-testing-20240314.183243 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7602305' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTbtdlTgoD62hC8MA7sGYj7zhxCZE928MyRAME/9TidtgTgSQZSKV4FTznZXn2dbL4FRWoyT2Ti/kY27swpgxE= smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPjtRrt2ZraSIWALpw5k1RtZcMzXMjx8UNRAfILXNtwlAFBpbw7dKfex/N5L/pe0C8pvGn7YZi/qiT9ohiYJ4+k= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_13:00:41 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-16T08:39:05.809 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa; will attempt to use it 2024-03-16T08:39:05.810 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks 2024-03-16T08:39:05.810 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-16T08:39:05.811 INFO:teuthology.task.internal:Checking packages... 2024-03-16T08:39:05.834 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-16T08:39:05.834 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T08:39:05.834 INFO:teuthology.packaging:ref: None 2024-03-16T08:39:05.834 INFO:teuthology.packaging:tag: None 2024-03-16T08:39:05.834 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-16T08:39:05.834 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:39:05.834 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-16T08:39:05.972 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162.g3d78f5ba 2024-03-16T08:39:05.974 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-16T08:39:05.980 INFO:teuthology.task.internal:no buildpackages task found 2024-03-16T08:39:05.980 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-16T08:39:06.027 INFO:teuthology.task.internal:Saving configuration 2024-03-16T08:39:06.047 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-16T08:39:06.058 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T08:39:06.084 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:34:22.360652', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTbtdlTgoD62hC8MA7sGYj7zhxCZE928MyRAME/9TidtgTgSQZSKV4FTznZXn2dbL4FRWoyT2Ti/kY27swpgxE='} 2024-03-16T08:39:06.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:34:22.362401', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPjtRrt2ZraSIWALpw5k1RtZcMzXMjx8UNRAfILXNtwlAFBpbw7dKfex/N5L/pe0C8pvGn7YZi/qiT9ohiYJ4+k='} 2024-03-16T08:39:06.104 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-16T08:39:06.110 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-16T08:39:06.110 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-16T08:39:06.110 INFO:teuthology.run_tasks:Running task console_log... 2024-03-16T08:39:06.199 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0f4fea2160>, signals=[15]) 2024-03-16T08:39:06.200 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-16T08:39:06.208 INFO:teuthology.task.internal:Opening connections... 2024-03-16T08:39:06.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:39:06.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:39:06.318 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-03-16T08:39:06.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:39:06.424 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-16T08:39:06.430 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-03-16T08:39:06.447 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-03-16T08:39:06.447 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-03-16T08:39:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-03-16T08:39:06.505 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T08:39:06.505 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-03-16T08:39:06.505 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T08:39:06.505 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T08:39:06.505 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T08:39:06.505 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-03-16T08:39:06.528 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-03-16T08:39:06.546 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-03-16T08:39:06.546 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-03-16T08:39:06.606 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-03-16T08:39:06.606 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T08:39:06.607 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T08:39:06.608 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-03-16T08:39:06.628 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-16T08:39:06.637 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-16T08:39:06.656 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-16T08:39:06.656 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-03-16T08:39:06.658 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-03-16T08:39:06.673 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-16T08:39:06.682 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-16T08:39:06.682 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-03-16T08:39:06.716 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-03-16T08:39:06.743 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-16T08:39:06.799 INFO:teuthology.run_tasks:Running task kernel... 2024-03-16T08:39:06.813 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-16T08:39:06.814 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-16T08:39:06.814 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T08:39:06.814 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T08:39:06.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:39:06.830 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-03-16T08:39:06.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:39:06.832 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-03-16T08:39:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-361.el9.x86_64 2024-03-16T08:39:06.889 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-361.el9.x86_64 2024-03-16T08:39:06.889 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-03-16T08:39:06.890 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-361.el9.x86_64 2024-03-16T08:39:06.891 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-361.el9.x86_64 2024-03-16T08:39:06.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-03-16T08:39:09.334 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - AppStream 8.9 MB/s | 6.7 MB 00:00 2024-03-16T08:39:09.457 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - AppStream 7.9 MB/s | 6.7 MB 00:00 2024-03-16T08:39:10.809 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-16T08:39:10.966 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2024-03-16T08:39:12.158 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:01 2024-03-16T08:39:12.598 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-03-16T08:39:14.578 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-03-16T08:39:18.643 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-16T08:39:20.152 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 15 kB 00:00 2024-03-16T08:39:34.863 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 919 kB/s | 19 MB 00:20 2024-03-16T08:39:39.181 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.1 MB 00:00 2024-03-16T08:39:40.790 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2024-03-16T08:39:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-16T08:39:47.558 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-03-16T08:39:49.790 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-16T08:39:49.790 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-16T08:39:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:39:49.892 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:39:49.892 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-03-16T08:39:49.892 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:39:49.892 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:39:49.892 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-16T08:39:49.892 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 100 M 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 123 M 2024-03-16T08:39:49.893 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:39:51.377 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 680 kB/s | 21 MB 00:31 2024-03-16T08:39:56.541 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-03-16T08:39:58.906 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-16T08:39:58.907 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-16T08:39:59.010 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:39:59.018 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-03-16T08:39:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:39:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 100 M 2024-03-16T08:39:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 123 M 2024-03-16T08:39:59.019 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:39:59.409 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 18 MB/s | 6.0 MB 00:00 2024-03-16T08:40:07.133 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 359 kB/s | 6.0 MB 00:17 2024-03-16T08:40:12.033 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 1.6 MB/s | 21 MB 00:12 2024-03-16T08:40:20.598 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 1.6 MB/s | 33 MB 00:21 2024-03-16T08:40:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 589 kB/s | 21 MB 00:36 2024-03-16T08:40:28.456 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 1.3 MB/s | 39 MB 00:29 2024-03-16T08:40:28.456 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:40:28.456 INFO:teuthology.orchestra.run.smithi007.stdout:Total 3.4 MB/s | 100 MB 00:29 2024-03-16T08:40:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:40:29.813 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:40:29.813 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:40:31.247 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:40:31.247 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:40:34.120 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:40:34.799 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-16T08:40:34.934 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:40:37.870 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:40:38.054 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:40:43.165 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:40:43.291 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:40:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:40:50.262 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 668 kB/s | 39 MB 01:00 2024-03-16T08:40:54.970 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 715 kB/s | 33 MB 00:47 2024-03-16T08:40:54.970 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:40:54.970 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.5 MB/s | 100 MB 01:05 2024-03-16T08:40:55.891 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:40:56.348 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:40:56.348 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:40:57.749 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:40:57.749 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:41:00.607 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:41:01.286 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-16T08:41:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:41:04.313 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:41:04.497 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:41:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:41:09.726 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:19.489 INFO:teuthology.orchestra.run.smithi007.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T08:41:19.489 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:41:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-16T08:41:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:41:20.071 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:41:20.524 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:41:21.068 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T08:41:21.220 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T08:41:21.221 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T08:41:21.221 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-16T08:41:21.221 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T08:41:21.221 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T08:41:21.221 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-16T08:41:21.221 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T08:41:21.221 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T08:41:21.347 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T08:41:21.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T08:41:21.395 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:41:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T08:41:21.397 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T08:41:21.397 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-427.el9.x86_64 2024-03-16T08:41:21.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T08:41:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-03-16T08:41:23.390 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T08:41:23.421 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-03-16T08:41:23.424 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-03-16T08:41:23.439 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.TjhJgLrcjc 2024-03-16T08:41:23.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TjhJgLrcjc 2024-03-16T08:41:23.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.TjhJgLrcjc 2024-03-16T08:41:23.653 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.TjhJgLrcjc is 6KB 2024-03-16T08:41:23.702 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.TjhJgLrcjc 2024-03-16T08:41:23.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T08:41:23.784 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T08:41:23.784 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T08:41:23.784 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T08:41:23.784 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-16T08:41:23.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-16T08:41:23.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-03-16T08:41:45.640 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:45.641 INFO:teuthology.orchestra.run.smithi019.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T08:41:45.641 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:41:45.726 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:46.237 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:46.237 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-16T08:41:46.238 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:41:46.238 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:41:46.702 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:41:46.702 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:41:46.702 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:41:46.702 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-16T08:41:46.702 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-16T08:41:46.703 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-16T08:41:46.703 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-16T08:41:46.703 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:41:46.703 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:41:47.215 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T08:41:47.366 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T08:41:47.366 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T08:41:47.367 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-16T08:41:47.367 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T08:41:47.367 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T08:41:47.367 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-16T08:41:47.367 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T08:41:47.367 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T08:41:47.493 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T08:41:47.493 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T08:41:47.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-03-16T08:41:47.537 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-03-16T08:41:47.537 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-03-16T08:41:47.537 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-03-16T08:41:47.537 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2024-03-16T08:41:47.537 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-03-16T08:41:47.537 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T08:41:47.537 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:41:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T08:41:47.539 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T08:41:47.539 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T08:41:47.539 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-03-16T08:41:47.539 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:41:47.539 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T08:41:47.540 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T08:41:47.540 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 5.14.0-427.el9.x86_64 2024-03-16T08:41:47.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T08:41:48.166 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-03-16T08:41:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T08:41:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-03-16T08:41:49.689 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-03-16T08:41:49.704 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.JokarJUJJq 2024-03-16T08:41:49.704 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JokarJUJJq 2024-03-16T08:41:49.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.JokarJUJJq 2024-03-16T08:41:49.922 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.JokarJUJJq is 6KB 2024-03-16T08:41:49.971 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.JokarJUJJq 2024-03-16T08:41:49.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T08:41:50.054 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T08:41:50.054 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T08:41:50.054 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T08:41:50.054 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-16T08:41:50.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-16T08:41:50.253 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-03-16T08:41:53.942 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-16T08:41:53.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-16T08:41:53.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:42:12.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-03-16T08:42:20.260 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-16T08:42:20.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-16T08:42:20.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:42:21.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-16T08:42:21.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:42:38.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-16T08:42:47.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-16T08:42:47.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:43:21.419 DEBUG:teuthology.orchestra.remote:timed out 2024-03-16T08:43:33.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-16T08:43:33.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:43:33.702 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-03-16T08:43:34.088 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-16T08:43:34.089 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-427.el9.x86_64"... 2024-03-16T08:43:34.089 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-03-16T08:43:34.105 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-427.el9.x86_64 2024-03-16T08:43:34.105 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-16T08:43:34.105 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T08:43:34.106 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T08:43:35.107 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-16T08:43:35.107 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-16T08:43:35.184 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-03-16T08:43:35.216 DEBUG:teuthology.parallel:result is None 2024-03-16T08:43:47.764 DEBUG:teuthology.orchestra.remote:timed out 2024-03-16T08:43:59.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-16T08:43:59.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:44:00.124 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-03-16T08:44:00.549 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-16T08:44:00.550 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-427.el9.x86_64"... 2024-03-16T08:44:00.550 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-03-16T08:44:00.566 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-427.el9.x86_64 2024-03-16T08:44:00.567 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-16T08:44:00.567 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T08:44:00.567 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T08:44:01.568 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-16T08:44:01.568 DEBUG:teuthology.orchestra.run.smithi019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-16T08:44:01.623 INFO:teuthology.orchestra.run.smithi019.stdout:ttyS1 2024-03-16T08:44:01.659 DEBUG:teuthology.parallel:result is None 2024-03-16T08:44:01.659 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-16T08:44:01.667 INFO:teuthology.task.internal:Creating test directory... 2024-03-16T08:44:01.668 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T08:44:01.670 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T08:44:01.687 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-16T08:44:01.703 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-16T08:44:01.736 INFO:teuthology.task.internal:Creating archive directory... 2024-03-16T08:44:01.737 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T08:44:01.739 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T08:44:01.763 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-16T08:44:01.769 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-16T08:44:01.769 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T08:44:01.800 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T08:44:01.835 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:44:01.836 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:44:01.849 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:44:01.850 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:44:01.851 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-16T08:44:01.859 INFO:teuthology.task.internal:Configuring sudo... 2024-03-16T08:44:01.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T08:44:01.893 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T08:44:01.923 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-16T08:44:01.932 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-16T08:44:01.932 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T08:44:01.963 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T08:44:01.980 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:44:02.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:44:02.107 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:44:02.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:44:02.233 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:44:02.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T08:44:02.301 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:44:02.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:44:02.399 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:44:02.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:44:02.531 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:44:02.532 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T08:44:02.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-03-16T08:44:02.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-03-16T08:44:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T08:44:02.702 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T08:44:03.030 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-16T08:44:03.037 INFO:teuthology.task.internal:Starting timer... 2024-03-16T08:44:03.037 INFO:teuthology.run_tasks:Running task pcp... 2024-03-16T08:44:03.102 INFO:teuthology.run_tasks:Running task selinux... 2024-03-16T08:44:03.131 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-16T08:44:03.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-03-16T08:44:03.195 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-03-16T08:44:03.197 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-03-16T08:44:03.263 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-03-16T08:44:03.264 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-16T08:44:03.265 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-03-16T08:44:03.290 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-03-16T08:44:03.291 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-03-16T08:44:03.311 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-03-16T08:44:03.311 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi019.front.sepia.ceph.com': 'permissive'} 2024-03-16T08:44:03.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T08:44:03.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:44:03.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T08:44:03.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:44:03.388 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-16T08:44:03.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-03-16T08:44:03.433 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-03-16T08:44:03.471 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-16T08:44:03.481 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-16T08:44:03.487 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-16T08:44:03.613 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-16T08:44:03.630 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-16T08:44:03.631 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi019.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-16T08:48:20.156 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi019.front.sepia.ceph.com')] 2024-03-16T08:48:20.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-16T08:48:20.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:48:20.252 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-03-16T08:48:20.334 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-16T08:48:20.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-16T08:48:20.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:48:20.423 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-03-16T08:48:20.509 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-16T08:48:20.509 INFO:teuthology.run_tasks:Running task clock... 2024-03-16T08:48:20.520 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-16T08:48:20.520 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T08:48:20.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T08:48:20.523 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T08:48:20.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T08:48:20.556 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T08:48:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T08:48:20.580 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T08:48:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T08:48:20.604 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-03-16T08:48:20.618 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-03-16T08:48:20.627 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-03-16T08:48:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T08:48:20.641 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-03-16T08:48:20.651 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T08:48:20.657 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T08:48:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T08:48:20.701 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-03-16T08:48:20.705 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T08:48:20.706 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-03-16T08:48:20.706 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:48:20.706 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:48:20.706 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:48:20.706 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:48:20.732 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-03-16T08:48:20.736 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T08:48:20.737 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-03-16T08:48:20.737 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:48:20.737 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:48:20.737 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:48:20.737 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T08:48:20.738 INFO:teuthology.run_tasks:Running task install... 2024-03-16T08:48:20.746 DEBUG:teuthology.task.install:project ceph 2024-03-16T08:48:20.747 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-16T08:48:20.747 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-16T08:48:20.747 INFO:teuthology.task.install:Using flavor: default 2024-03-16T08:48:20.753 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-16T08:48:20.753 INFO:teuthology.task.install:extra packages: [] 2024-03-16T08:48:20.754 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T08:48:20.754 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T08:48:20.754 INFO:teuthology.packaging:ref: None 2024-03-16T08:48:20.754 INFO:teuthology.packaging:tag: None 2024-03-16T08:48:20.754 INFO:teuthology.packaging:branch: quincy 2024-03-16T08:48:20.755 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:48:20.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-16T08:48:20.757 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T08:48:20.757 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T08:48:20.757 INFO:teuthology.packaging:ref: None 2024-03-16T08:48:20.757 INFO:teuthology.packaging:tag: None 2024-03-16T08:48:20.757 INFO:teuthology.packaging:branch: quincy 2024-03-16T08:48:20.757 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:48:20.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-16T08:48:20.984 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/1df27247a676373423b9b5769ede23dff6dccf8f/centos/9/flavors/default/ 2024-03-16T08:48:20.984 INFO:teuthology.task.install.rpm:Package version is 17.2.7-648.g1df27247 2024-03-16T08:48:21.000 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/1df27247a676373423b9b5769ede23dff6dccf8f/centos/9/flavors/default/ 2024-03-16T08:48:21.000 INFO:teuthology.task.install.rpm:Package version is 17.2.7-648.g1df27247 2024-03-16T08:48:21.096 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/1df27247a676373423b9b5769ede23dff6dccf8f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/1df27247a676373423b9b5769ede23dff6dccf8f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/1df27247a676373423b9b5769ede23dff6dccf8f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T08:48:21.096 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:48:21.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T08:48:21.124 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-16T08:48:21.124 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T08:48:21.124 INFO:teuthology.packaging:ref: None 2024-03-16T08:48:21.124 INFO:teuthology.packaging:tag: None 2024-03-16T08:48:21.124 INFO:teuthology.packaging:branch: quincy 2024-03-16T08:48:21.124 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:48:21.124 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T08:48:21.169 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/1df27247a676373423b9b5769ede23dff6dccf8f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/1df27247a676373423b9b5769ede23dff6dccf8f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/1df27247a676373423b9b5769ede23dff6dccf8f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T08:48:21.169 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:48:21.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T08:48:21.190 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T08:48:21.197 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-16T08:48:21.197 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T08:48:21.197 INFO:teuthology.packaging:ref: None 2024-03-16T08:48:21.197 INFO:teuthology.packaging:tag: None 2024-03-16T08:48:21.197 INFO:teuthology.packaging:branch: quincy 2024-03-16T08:48:21.197 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:48:21.198 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T08:48:21.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T08:48:21.270 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T08:48:21.335 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-03-16T08:48:21.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-03-16T08:48:21.344 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T08:48:21.411 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-03-16T08:48:21.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-03-16T08:48:21.598 INFO:teuthology.orchestra.run.smithi019.stdout:74 files removed 2024-03-16T08:48:21.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw 2024-03-16T08:48:21.669 INFO:teuthology.orchestra.run.smithi007.stdout:74 files removed 2024-03-16T08:48:21.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw 2024-03-16T08:48:22.340 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 241 kB/s | 77 kB 00:00 2024-03-16T08:48:22.356 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 189 kB/s | 77 kB 00:00 2024-03-16T08:48:22.716 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 28 kB/s | 9.1 kB 00:00 2024-03-16T08:48:22.781 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 24 kB/s | 9.1 kB 00:00 2024-03-16T08:48:23.091 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 5.3 kB/s | 1.8 kB 00:00 2024-03-16T08:48:23.182 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 4.9 kB/s | 1.8 kB 00:00 2024-03-16T08:48:23.510 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-16T08:48:23.818 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-16T08:48:24.986 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-16T08:48:25.268 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-03-16T08:48:26.051 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-16T08:48:26.067 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-16T08:48:28.055 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-16T08:48:32.322 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-16T08:48:34.120 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 15 kB 00:00 2024-03-16T08:48:35.524 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-16T08:48:40.647 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-16T08:48:43.322 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:17.2.7-648.g1df27247.el9 ceph 14 M 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:17.2.7-648.g1df27247.el9 ceph 3.1 M 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:17.2.7-648.g1df27247.el9 ceph 2.9 M 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:17.2.7-648.g1df27247.el9 ceph 5.0 M 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:17.2.7-648.g1df27247.el9 ceph 23 M 2024-03-16T08:48:43.325 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:17.2.7-648.g1df27247.el9 ceph 25 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:17.2.7-648.g1df27247.el9 ceph 636 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:17.2.7-648.g1df27247.el9 ceph 458 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:17.2.7-648.g1df27247.el9 ceph 4.9 M 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:17.2.7-648.g1df27247.el9 ceph 45 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:17.2.7-648.g1df27247.el9 ceph 116 k 2024-03-16T08:48:43.326 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:17.2.7-648.g1df27247.el9 ceph 156 k 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:17.2.7-648.g1df27247.el9 ceph 320 k 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:17.2.7-648.g1df27247.el9 ceph 293 k 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:17.2.7-648.g1df27247.el9 ceph 99 k 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout:Install 22 Packages 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 2 Packages 2024-03-16T08:48:43.327 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:48:43.328 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 58 M 2024-03-16T08:48:43.328 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:48:43.931 INFO:teuthology.orchestra.run.smithi007.stdout:(1/24): ceph-base-17.2.7-648.g1df27247.el9.x86_ 9.1 MB/s | 5.0 MB 00:00 2024-03-16T08:48:43.981 INFO:teuthology.orchestra.run.smithi007.stdout:(2/24): ceph-selinux-17.2.7-648.g1df27247.el9.x 504 kB/s | 25 kB 00:00 2024-03-16T08:48:44.182 INFO:teuthology.orchestra.run.smithi007.stdout:(3/24): libcephfs2-17.2.7-648.g1df27247.el9.x86 3.1 MB/s | 636 kB 00:00 2024-03-16T08:48:44.240 INFO:teuthology.orchestra.run.smithi007.stdout:(4/24): libradosstriper1-17.2.7-648.g1df27247.e 7.7 MB/s | 458 kB 00:00 2024-03-16T08:48:44.607 INFO:teuthology.orchestra.run.smithi007.stdout:(5/24): librgw2-17.2.7-648.g1df27247.el9.x86_64 13 MB/s | 4.9 MB 00:00 2024-03-16T08:48:44.649 INFO:teuthology.orchestra.run.smithi007.stdout:(6/24): python3-ceph-argparse-17.2.7-648.g1df27 1.1 MB/s | 45 kB 00:00 2024-03-16T08:48:44.716 INFO:teuthology.orchestra.run.smithi007.stdout:(7/24): python3-ceph-common-17.2.7-648.g1df2724 1.7 MB/s | 116 kB 00:00 2024-03-16T08:48:44.775 INFO:teuthology.orchestra.run.smithi007.stdout:(8/24): python3-cephfs-17.2.7-648.g1df27247.el9 2.6 MB/s | 156 kB 00:00 2024-03-16T08:48:44.975 INFO:teuthology.orchestra.run.smithi007.stdout:(9/24): python3-rados-17.2.7-648.g1df27247.el9. 1.6 MB/s | 320 kB 00:00 2024-03-16T08:48:45.058 INFO:teuthology.orchestra.run.smithi007.stdout:(10/24): python3-rbd-17.2.7-648.g1df27247.el9.x 3.4 MB/s | 293 kB 00:00 2024-03-16T08:48:45.109 INFO:teuthology.orchestra.run.smithi007.stdout:(11/24): python3-rgw-17.2.7-648.g1df27247.el9.x 1.9 MB/s | 99 kB 00:00 2024-03-16T08:48:45.217 INFO:teuthology.orchestra.run.smithi007.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-03-16T08:48:45.251 INFO:teuthology.orchestra.run.smithi007.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-16T08:48:45.318 INFO:teuthology.orchestra.run.smithi007.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-16T08:48:45.360 INFO:teuthology.orchestra.run.smithi007.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-16T08:48:45.510 INFO:teuthology.orchestra.run.smithi007.stdout:(16/24): ceph-radosgw-17.2.7-648.g1df27247.el9. 6.6 MB/s | 14 MB 00:02 2024-03-16T08:48:45.535 INFO:teuthology.orchestra.run.smithi007.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 235 kB/s | 41 kB 00:00 2024-03-16T08:48:45.602 INFO:teuthology.orchestra.run.smithi007.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 727 kB/s | 49 kB 00:00 2024-03-16T08:48:45.636 INFO:teuthology.orchestra.run.smithi007.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.4 MB/s | 309 kB 00:00 2024-03-16T08:48:45.661 INFO:teuthology.orchestra.run.smithi007.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-16T08:48:45.728 INFO:teuthology.orchestra.run.smithi007.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-16T08:48:46.195 INFO:teuthology.orchestra.run.smithi007.stdout:(22/24): ceph-common-17.2.7-648.g1df27247.el9.x 8.0 MB/s | 23 MB 00:02 2024-03-16T08:48:46.245 INFO:teuthology.orchestra.run.smithi007.stdout:(23/24): librados2-17.2.7-648.g1df27247.el9.x86 5.3 MB/s | 3.1 MB 00:00 2024-03-16T08:48:46.296 INFO:teuthology.orchestra.run.smithi007.stdout:(24/24): librbd1-17.2.7-648.g1df27247.el9.x86_6 5.2 MB/s | 2.9 MB 00:00 2024-03-16T08:48:46.299 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:48:46.299 INFO:teuthology.orchestra.run.smithi007.stdout:Total 19 MB/s | 58 MB 00:02 2024-03-16T08:48:46.299 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:48:46.334 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:48:46.334 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:48:46.811 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:48:46.811 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:48:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:48:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-16T08:48:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:17.2.7-648.g1df27247.el9.x86_64 2/26 2024-03-16T08:48:48.624 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:17.2.7-648.g1df27247.el9.x86_64 2/26 2024-03-16T08:48:48.712 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:17.2.7-648.g1df27247.el9.x86_64 3/26 2024-03-16T08:48:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 4/26 2024-03-16T08:48:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 4/26 2024-03-16T08:48:48.850 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-16T08:48:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-16T08:48:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-16T08:48:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:17.2.7-648.g1df27247.el9.x86_64 8/26 2024-03-16T08:48:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:17.2.7-648.g1df27247.el9.x86_64 8/26 2024-03-16T08:48:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-16T08:48:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:17.2.7-648.g1df27247.el9.x 10/26 2024-03-16T08:48:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:17.2.7-648.g1df27247.el9.x86_64 11/26 2024-03-16T08:48:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:17.2.7-648.g1df27247.el9.x86_64 12/26 2024-03-16T08:48:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:17.2.7-648.g1df27247.el9.x86_64 13/26 2024-03-16T08:48:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:17.2.7-648.g1df27247.el9.x86_64 13/26 2024-03-16T08:48:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-16T08:48:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-16T08:48:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-16T08:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-16T08:48:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:17.2.7-648.g1df27247.el9.x86_64 18/26 2024-03-16T08:48:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:17.2.7-648.g1df27247.el9.x86_64 18/26 2024-03-16T08:48:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:17.2.7-648.g1df27247.el9.x86_64 19/26 2024-03-16T08:48:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:17.2.7-648.g1df27247.el9.x86 20/26 2024-03-16T08:48:52.816 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 753 kB/s | 19 MB 00:25 2024-03-16T08:48:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 21/26 2024-03-16T08:48:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 21/26 2024-03-16T08:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 21/26 2024-03-16T08:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 22/26 2024-03-16T08:48:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 22/26 2024-03-16T08:48:53.632 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T08:48:53.632 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T08:48:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:48:53.678 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:17.2.7-648.g1df27247.el9.x86_64 23/26 2024-03-16T08:48:57.967 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.1 MB 00:01 2024-03-16T08:48:59.501 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 15 kB 00:00 2024-03-16T08:49:00.436 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-03-16T08:49:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:17.2.7-648.g1df27247.el9.x86_64 23/26 2024-03-16T08:49:03.809 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-03-16T08:49:03.809 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-03-16T08:49:03.809 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-03-16T08:49:03.809 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-03-16T08:49:03.809 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-03-16T08:49:03.809 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-03-16T08:49:03.810 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-03-16T08:49:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:05.122 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:17.2.7-648.g1df27247.el9.x86_64 24/26 2024-03-16T08:49:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-648.g1df27247.el9.x86_64 24/26 2024-03-16T08:49:05.183 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:05.183 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T08:49:05.183 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:49:05.183 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:49:05.183 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:05.185 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-16T08:49:05.267 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-16T08:49:05.267 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-16T08:49:05.502 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 1/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 2/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:17.2.7-648.g1df27247.el9.x86_64 3/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:17.2.7-648.g1df27247.el9.x86_64 4/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 5/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:17.2.7-648.g1df27247.el9.x86_64 6/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:17.2.7-648.g1df27247.el9.x86_64 7/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:17.2.7-648.g1df27247.el9.x 8/26 2024-03-16T08:49:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:17.2.7-648.g1df27247.el9.x86 9/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:17.2.7-648.g1df27247.el9.x86_64 10/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:17.2.7-648.g1df27247.el9.x86_64 11/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:17.2.7-648.g1df27247.el9.x86_64 12/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:17.2.7-648.g1df27247.el9.x86_64 13/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-16T08:49:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:17.2.7-648.g1df27247.el9.x86_64 23/26 2024-03-16T08:49:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-16T08:49:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:17.2.7-648.g1df27247.el9.x86_64 25/26 2024-03-16T08:49:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-16T08:49:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:07.332 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-03-16T08:49:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T08:49:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T08:49:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:07.335 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:49:07.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-test 2024-03-16T08:49:08.125 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:49:08.135 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:49:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:49:08.135 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:49:08.135 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:49:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:17.2.7-648.g1df27247.el9 ceph 14 M 2024-03-16T08:49:08.135 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-03-16T08:49:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:17.2.7-648.g1df27247.el9 ceph 3.1 M 2024-03-16T08:49:08.135 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:17.2.7-648.g1df27247.el9 ceph 2.9 M 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:17.2.7-648.g1df27247.el9 ceph 5.0 M 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:17.2.7-648.g1df27247.el9 ceph 23 M 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:17.2.7-648.g1df27247.el9 ceph 25 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:17.2.7-648.g1df27247.el9 ceph 636 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:17.2.7-648.g1df27247.el9 ceph 458 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:17.2.7-648.g1df27247.el9 ceph 4.9 M 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T08:49:08.136 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:17.2.7-648.g1df27247.el9 ceph 45 k 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:17.2.7-648.g1df27247.el9 ceph 116 k 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:17.2.7-648.g1df27247.el9 ceph 156 k 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:17.2.7-648.g1df27247.el9 ceph 320 k 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:17.2.7-648.g1df27247.el9 ceph 293 k 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:17.2.7-648.g1df27247.el9 ceph 99 k 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout:Install 22 Packages 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 2 Packages 2024-03-16T08:49:08.137 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:08.138 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 58 M 2024-03-16T08:49:08.138 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:49:08.368 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:28 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:49:08.527 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:49:08.528 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:49:08.528 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:08.528 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:49:08.528 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:17.2.7-648.g1df27247.el9 ceph 48 M 2024-03-16T08:49:08.528 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:17.2.7-648.g1df27247.el9 ceph 163 k 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout:Install 6 Packages 2024-03-16T08:49:08.529 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:08.530 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 49 M 2024-03-16T08:49:08.530 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 196 M 2024-03-16T08:49:08.530 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:49:08.981 INFO:teuthology.orchestra.run.smithi007.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-03-16T08:49:09.204 INFO:teuthology.orchestra.run.smithi019.stdout:(1/24): ceph-base-17.2.7-648.g1df27247.el9.x86_ 5.7 MB/s | 5.0 MB 00:00 2024-03-16T08:49:09.220 INFO:teuthology.orchestra.run.smithi007.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-03-16T08:49:09.256 INFO:teuthology.orchestra.run.smithi007.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.7 MB/s | 63 kB 00:00 2024-03-16T08:49:09.287 INFO:teuthology.orchestra.run.smithi019.stdout:(2/24): ceph-selinux-17.2.7-648.g1df27247.el9.x 302 kB/s | 25 kB 00:00 2024-03-16T08:49:09.382 INFO:teuthology.orchestra.run.smithi007.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-16T08:49:09.404 INFO:teuthology.orchestra.run.smithi019.stdout:(3/24): libcephfs2-17.2.7-648.g1df27247.el9.x86 5.3 MB/s | 636 kB 00:00 2024-03-16T08:49:09.407 INFO:teuthology.orchestra.run.smithi007.stdout:(5/6): libcephsqlite-17.2.7-648.g1df27247.el9.x 269 kB/s | 163 kB 00:00 2024-03-16T08:49:09.513 INFO:teuthology.orchestra.run.smithi019.stdout:(4/24): libradosstriper1-17.2.7-648.g1df27247.e 4.1 MB/s | 458 kB 00:00 2024-03-16T08:49:10.296 INFO:teuthology.orchestra.run.smithi019.stdout:(5/24): ceph-radosgw-17.2.7-648.g1df27247.el9.x 7.1 MB/s | 14 MB 00:01 2024-03-16T08:49:10.363 INFO:teuthology.orchestra.run.smithi019.stdout:(6/24): librgw2-17.2.7-648.g1df27247.el9.x86_64 5.8 MB/s | 4.9 MB 00:00 2024-03-16T08:49:10.380 INFO:teuthology.orchestra.run.smithi019.stdout:(7/24): python3-ceph-argparse-17.2.7-648.g1df27 540 kB/s | 45 kB 00:00 2024-03-16T08:49:10.422 INFO:teuthology.orchestra.run.smithi019.stdout:(8/24): python3-ceph-common-17.2.7-648.g1df2724 1.9 MB/s | 116 kB 00:00 2024-03-16T08:49:10.447 INFO:teuthology.orchestra.run.smithi019.stdout:(9/24): python3-cephfs-17.2.7-648.g1df27247.el9 2.3 MB/s | 156 kB 00:00 2024-03-16T08:49:10.498 INFO:teuthology.orchestra.run.smithi019.stdout:(10/24): python3-rados-17.2.7-648.g1df27247.el9 4.2 MB/s | 320 kB 00:00 2024-03-16T08:49:10.707 INFO:teuthology.orchestra.run.smithi019.stdout:(11/24): ceph-common-17.2.7-648.g1df27247.el9.x 9.5 MB/s | 23 MB 00:02 2024-03-16T08:49:10.757 INFO:teuthology.orchestra.run.smithi019.stdout:(12/24): python3-rgw-17.2.7-648.g1df27247.el9.x 384 kB/s | 99 kB 00:00 2024-03-16T08:49:10.807 INFO:teuthology.orchestra.run.smithi019.stdout:(13/24): python3-rbd-17.2.7-648.g1df27247.el9.x 815 kB/s | 293 kB 00:00 2024-03-16T08:49:10.882 INFO:teuthology.orchestra.run.smithi019.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 909 kB/s | 159 kB 00:00 2024-03-16T08:49:10.907 INFO:teuthology.orchestra.run.smithi019.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 353 kB/s | 44 kB 00:00 2024-03-16T08:49:10.966 INFO:teuthology.orchestra.run.smithi019.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 4.9 MB/s | 661 kB 00:00 2024-03-16T08:49:10.991 INFO:teuthology.orchestra.run.smithi019.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 282 kB 00:00 2024-03-16T08:49:11.017 INFO:teuthology.orchestra.run.smithi019.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 379 kB/s | 41 kB 00:00 2024-03-16T08:49:11.384 INFO:teuthology.orchestra.run.smithi019.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 132 kB/s | 49 kB 00:00 2024-03-16T08:49:11.400 INFO:teuthology.orchestra.run.smithi019.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 176 kB/s | 67 kB 00:00 2024-03-16T08:49:11.717 INFO:teuthology.orchestra.run.smithi019.stdout:(21/24): librados2-17.2.7-648.g1df27247.el9.x86 9.7 MB/s | 3.1 MB 00:00 2024-03-16T08:49:11.742 INFO:teuthology.orchestra.run.smithi019.stdout:(22/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 398 kB/s | 309 kB 00:00 2024-03-16T08:49:11.943 INFO:teuthology.orchestra.run.smithi019.stdout:(23/24): librbd1-17.2.7-648.g1df27247.el9.x86_6 13 MB/s | 2.9 MB 00:00 2024-03-16T08:49:12.209 INFO:teuthology.orchestra.run.smithi019.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-03-16T08:49:12.215 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:12.215 INFO:teuthology.orchestra.run.smithi019.stdout:Total 14 MB/s | 58 MB 00:04 2024-03-16T08:49:12.215 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:49:12.253 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:49:12.253 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:49:12.719 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:49:12.719 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:49:14.084 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:49:14.422 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-16T08:49:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:17.2.7-648.g1df27247.el9.x86_64 2/26 2024-03-16T08:49:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:17.2.7-648.g1df27247.el9.x86_64 2/26 2024-03-16T08:49:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:17.2.7-648.g1df27247.el9.x86_64 3/26 2024-03-16T08:49:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 4/26 2024-03-16T08:49:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 4/26 2024-03-16T08:49:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-16T08:49:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-16T08:49:15.346 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-16T08:49:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:17.2.7-648.g1df27247.el9.x86_64 8/26 2024-03-16T08:49:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:17.2.7-648.g1df27247.el9.x86_64 8/26 2024-03-16T08:49:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-16T08:49:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:17.2.7-648.g1df27247.el9.x 10/26 2024-03-16T08:49:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:17.2.7-648.g1df27247.el9.x86_64 11/26 2024-03-16T08:49:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:17.2.7-648.g1df27247.el9.x86_64 12/26 2024-03-16T08:49:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:17.2.7-648.g1df27247.el9.x86_64 13/26 2024-03-16T08:49:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:17.2.7-648.g1df27247.el9.x86_64 13/26 2024-03-16T08:49:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-16T08:49:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-16T08:49:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-16T08:49:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-16T08:49:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:17.2.7-648.g1df27247.el9.x86_64 18/26 2024-03-16T08:49:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:17.2.7-648.g1df27247.el9.x86_64 18/26 2024-03-16T08:49:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:17.2.7-648.g1df27247.el9.x86_64 19/26 2024-03-16T08:49:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:17.2.7-648.g1df27247.el9.x86 20/26 2024-03-16T08:49:18.822 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 21/26 2024-03-16T08:49:18.864 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 21/26 2024-03-16T08:49:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 21/26 2024-03-16T08:49:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 22/26 2024-03-16T08:49:19.572 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 22/26 2024-03-16T08:49:19.572 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T08:49:19.572 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T08:49:19.573 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:19.610 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:17.2.7-648.g1df27247.el9.x86_64 23/26 2024-03-16T08:49:20.756 INFO:teuthology.orchestra.run.smithi007.stdout:(6/6): ceph-test-17.2.7-648.g1df27247.el9.x86_6 4.0 MB/s | 48 MB 00:11 2024-03-16T08:49:20.758 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:20.758 INFO:teuthology.orchestra.run.smithi007.stdout:Total 4.0 MB/s | 49 MB 00:12 2024-03-16T08:49:20.771 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:49:20.790 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:49:20.790 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:49:21.195 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:49:21.195 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:49:21.793 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:49:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-16T08:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-16T08:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-16T08:49:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-16T08:49:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:17.2.7-648.g1df27247.el9.x86_64 5/6 2024-03-16T08:49:26.677 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:17.2.7-648.g1df27247.el9.x86_64 5/6 2024-03-16T08:49:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:17.2.7-648.g1df27247.el9.x86_64 6/6 2024-03-16T08:49:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-test-2:17.2.7-648.g1df27247.el9.x86_64 6/6 2024-03-16T08:49:29.140 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:17.2.7-648.g1df27247.el9.x86_64 1/6 2024-03-16T08:49:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:17.2.7-648.g1df27247.el9.x86_64 2/6 2024-03-16T08:49:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-16T08:49:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-16T08:49:29.141 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-16T08:49:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-16T08:49:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:29.554 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:49:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:29.554 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-16.el9.x86_64 2024-03-16T08:49:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T08:49:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T08:49:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T08:49:29.555 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:49:29.569 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:17.2.7-648.g1df27247.el9.x86_64 23/26 2024-03-16T08:49:29.569 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-03-16T08:49:29.570 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-03-16T08:49:29.570 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-03-16T08:49:29.570 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-03-16T08:49:29.570 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-03-16T08:49:29.570 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-03-16T08:49:29.570 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-03-16T08:49:29.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:29.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph 2024-03-16T08:49:30.379 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:50 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:49:30.544 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================== 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================== 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:17.2.7-648.g1df27247.el9 ceph 6.5 k 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:17.2.7-648.g1df27247.el9 ceph 2.1 M 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:17.2.7-648.g1df27247.el9 ceph 1.4 M 2024-03-16T08:49:30.547 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 239 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:17.2.7-648.g1df27247.el9 ceph 3.7 M 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:17.2.7-648.g1df27247.el9 ceph 15 M 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T08:49:30.548 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T08:49:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T08:49:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T08:49:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T08:49:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T08:49:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:30.550 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:49:30.550 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================== 2024-03-16T08:49:30.550 INFO:teuthology.orchestra.run.smithi007.stdout:Install 33 Packages 2024-03-16T08:49:30.550 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:30.551 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 27 M 2024-03-16T08:49:30.551 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 100 M 2024-03-16T08:49:30.551 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:49:30.887 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:17.2.7-648.g1df27247.el9.x86_64 24/26 2024-03-16T08:49:30.948 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-648.g1df27247.el9.x86_64 24/26 2024-03-16T08:49:30.948 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:30.948 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T08:49:30.948 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:49:30.948 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T08:49:30.948 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:30.950 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-16T08:49:31.040 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-16T08:49:31.040 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-16T08:49:31.259 INFO:teuthology.orchestra.run.smithi007.stdout:(1/33): ceph-17.2.7-648.g1df27247.el9.x86_64.rp 32 kB/s | 6.5 kB 00:00 2024-03-16T08:49:31.451 INFO:teuthology.orchestra.run.smithi007.stdout:(2/33): ceph-mgr-17.2.7-648.g1df27247.el9.x86_6 3.5 MB/s | 1.4 MB 00:00 2024-03-16T08:49:31.835 INFO:teuthology.orchestra.run.smithi007.stdout:(3/33): ceph-mds-17.2.7-648.g1df27247.el9.x86_6 2.6 MB/s | 2.1 MB 00:00 2024-03-16T08:49:31.943 INFO:teuthology.orchestra.run.smithi007.stdout:(4/33): ceph-mgr-modules-core-17.2.7-648.g1df27 2.2 MB/s | 239 kB 00:00 2024-03-16T08:49:32.052 INFO:teuthology.orchestra.run.smithi007.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-16T08:49:32.077 INFO:teuthology.orchestra.run.smithi007.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-03-16T08:49:32.102 INFO:teuthology.orchestra.run.smithi007.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.8 MB/s | 47 kB 00:00 2024-03-16T08:49:32.136 INFO:teuthology.orchestra.run.smithi007.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-16T08:49:32.211 INFO:teuthology.orchestra.run.smithi007.stdout:(9/33): ceph-mon-17.2.7-648.g1df27247.el9.x86_6 3.9 MB/s | 3.7 MB 00:00 2024-03-16T08:49:32.236 INFO:teuthology.orchestra.run.smithi007.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 241 kB 00:00 2024-03-16T08:49:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-16T08:49:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 1/26 2024-03-16T08:49:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 2/26 2024-03-16T08:49:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:17.2.7-648.g1df27247.el9.x86_64 3/26 2024-03-16T08:49:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:17.2.7-648.g1df27247.el9.x86_64 4/26 2024-03-16T08:49:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 5/26 2024-03-16T08:49:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:17.2.7-648.g1df27247.el9.x86_64 6/26 2024-03-16T08:49:32.249 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:17.2.7-648.g1df27247.el9.x86_64 7/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:17.2.7-648.g1df27247.el9.x 8/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:17.2.7-648.g1df27247.el9.x86 9/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:17.2.7-648.g1df27247.el9.x86_64 10/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:17.2.7-648.g1df27247.el9.x86_64 11/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:17.2.7-648.g1df27247.el9.x86_64 12/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:17.2.7-648.g1df27247.el9.x86_64 13/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-16T08:49:32.250 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-16T08:49:32.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-16T08:49:32.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:17.2.7-648.g1df27247.el9.x86_64 23/26 2024-03-16T08:49:32.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-16T08:49:32.251 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:17.2.7-648.g1df27247.el9.x86_64 25/26 2024-03-16T08:49:32.295 INFO:teuthology.orchestra.run.smithi007.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 14 MB/s | 1.1 MB 00:00 2024-03-16T08:49:32.312 INFO:teuthology.orchestra.run.smithi007.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-16T08:49:32.337 INFO:teuthology.orchestra.run.smithi007.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2024-03-16T08:49:32.363 INFO:teuthology.orchestra.run.smithi007.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-16T08:49:32.538 INFO:teuthology.orchestra.run.smithi007.stdout:(15/33): ceph-osd-17.2.7-648.g1df27247.el9.x86_ 14 MB/s | 15 MB 00:01 2024-03-16T08:49:32.647 INFO:teuthology.orchestra.run.smithi007.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 404 kB/s | 125 kB 00:00 2024-03-16T08:49:32.822 INFO:teuthology.orchestra.run.smithi007.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 95 kB/s | 43 kB 00:00 2024-03-16T08:49:32.989 INFO:teuthology.orchestra.run.smithi007.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 381 kB/s | 172 kB 00:00 2024-03-16T08:49:33.022 INFO:teuthology.orchestra.run.smithi007.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 53 kB/s | 11 kB 00:00 2024-03-16T08:49:33.198 INFO:teuthology.orchestra.run.smithi007.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 651 kB/s | 358 kB 00:00 2024-03-16T08:49:33.273 INFO:teuthology.orchestra.run.smithi007.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 62 kB/s | 18 kB 00:00 2024-03-16T08:49:33.315 INFO:teuthology.orchestra.run.smithi007.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 79 kB/s | 23 kB 00:00 2024-03-16T08:49:33.361 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-16T08:49:33.361 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:33.361 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-03-16T08:49:33.361 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.361 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.362 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:33.363 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:49:33.516 INFO:teuthology.orchestra.run.smithi007.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 61 kB/s | 19 kB 00:00 2024-03-16T08:49:33.557 INFO:teuthology.orchestra.run.smithi007.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 69 kB/s | 20 kB 00:00 2024-03-16T08:49:33.668 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-test 2024-03-16T08:49:33.683 INFO:teuthology.orchestra.run.smithi007.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 126 kB/s | 46 kB 00:00 2024-03-16T08:49:33.891 INFO:teuthology.orchestra.run.smithi007.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 210 kB/s | 79 kB 00:00 2024-03-16T08:49:33.958 INFO:teuthology.orchestra.run.smithi007.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 60 kB/s | 16 kB 00:00 2024-03-16T08:49:34.108 INFO:teuthology.orchestra.run.smithi007.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 494 kB/s | 272 kB 00:00 2024-03-16T08:49:34.234 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:29 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:49:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 126 kB/s | 36 kB 00:00 2024-03-16T08:49:34.259 INFO:teuthology.orchestra.run.smithi007.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 245 kB/s | 90 kB 00:00 2024-03-16T08:49:34.388 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:49:34.388 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:17.2.7-648.g1df27247.el9 ceph 48 M 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:17.2.7-648.g1df27247.el9 ceph 163 k 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:49:34.389 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:49:34.390 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2024-03-16T08:49:34.390 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:34.390 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 49 M 2024-03-16T08:49:34.390 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 196 M 2024-03-16T08:49:34.390 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:49:34.534 INFO:teuthology.orchestra.run.smithi007.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 73 kB/s | 20 kB 00:00 2024-03-16T08:49:34.622 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-03-16T08:49:34.651 INFO:teuthology.orchestra.run.smithi007.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 423 kB/s | 230 kB 00:00 2024-03-16T08:49:34.664 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-16T08:49:34.697 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-16T08:49:34.823 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-16T08:49:34.848 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): libcephsqlite-17.2.7-648.g1df27247.el9.x 399 kB/s | 163 kB 00:00 2024-03-16T08:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 683 kB/s | 427 kB 00:00 2024-03-16T08:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout:Total 6.2 MB/s | 27 MB 00:04 2024-03-16T08:49:34.886 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:49:34.930 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:49:34.930 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:49:36.670 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:49:36.721 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-16T08:49:36.754 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-16T08:49:36.786 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-16T08:49:36.884 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-16T08:49:36.968 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-16T08:49:37.014 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-16T08:49:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-16T08:49:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-16T08:49:37.187 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-16T08:49:37.246 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-16T08:49:37.310 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-16T08:49:37.506 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-16T08:49:37.558 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-16T08:49:37.597 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-16T08:49:37.631 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-16T08:49:37.661 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-16T08:49:37.740 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-16T08:49:37.780 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-16T08:49:37.813 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-16T08:49:37.854 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-16T08:49:37.890 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-16T08:49:37.923 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-16T08:49:38.018 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-16T08:49:38.074 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): ceph-test-17.2.7-648.g1df27247.el9.x86_6 13 MB/s | 48 MB 00:03 2024-03-16T08:49:38.076 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:38.076 INFO:teuthology.orchestra.run.smithi019.stdout:Total 13 MB/s | 49 MB 00:03 2024-03-16T08:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-16T08:49:38.090 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:49:38.112 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:49:38.113 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:49:38.156 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-16T08:49:38.297 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-16T08:49:38.430 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-16T08:49:38.519 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:49:38.520 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-648.g1df27247.el9.n 28/33 2024-03-16T08:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 29/33 2024-03-16T08:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 29/33 2024-03-16T08:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T08:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:39.114 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:49:39.191 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-16T08:49:39.224 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-16T08:49:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-16T08:49:39.299 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-16T08:49:39.326 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:17.2.7-648.g1df27247.el9.x86_64 5/6 2024-03-16T08:49:40.052 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:17.2.7-648.g1df27247.el9.x86_64 30/33 2024-03-16T08:49:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:17.2.7-648.g1df27247.el9.x86_64 30/33 2024-03-16T08:49:40.077 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:40.077 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T08:49:40.077 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:49:40.077 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:49:40.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:17.2.7-648.g1df27247.el9.x86_64 31/33 2024-03-16T08:49:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:17.2.7-648.g1df27247.el9.x86_64 31/33 2024-03-16T08:49:40.500 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:40.500 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T08:49:40.500 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:49:40.500 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:49:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:40.711 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:17.2.7-648.g1df27247.el9.x86_64 32/33 2024-03-16T08:49:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:17.2.7-648.g1df27247.el9.x86_64 32/33 2024-03-16T08:49:40.734 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:40.734 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T08:49:40.734 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:49:40.734 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:49:40.734 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:40.803 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:17.2.7-648.g1df27247.el9.x86_64 33/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-2:17.2.7-648.g1df27247.el9.x86_64 33/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:17.2.7-648.g1df27247.el9.x86_64 1/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:17.2.7-648.g1df27247.el9.x86_64 2/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 3/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:17.2.7-648.g1df27247.el9.x86_64 4/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:17.2.7-648.g1df27247.el9.x86_64 5/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-648.g1df27247.el9.n 6/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-16T08:49:42.378 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-16T08:49:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-16T08:49:42.380 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-16T08:49:42.380 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-16T08:49:42.381 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-16T08:49:42.381 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-16T08:49:42.381 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-16T08:49:42.381 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-16T08:49:42.381 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-16T08:49:42.381 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-16T08:49:42.381 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T08:49:43.283 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T08:49:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:43.285 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:49:43.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-base 2024-03-16T08:49:43.995 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:17.2.7-648.g1df27247.el9.x86_64 5/6 2024-03-16T08:49:44.086 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:17.2.7-648.g1df27247.el9.x86_64 6/6 2024-03-16T08:49:44.251 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:04 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:49:44.352 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:49:44.407 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:49:44.408 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:49:44.408 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:49:44.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install cephadm 2024-03-16T08:49:45.040 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:05 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:49:45.195 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:49:45.196 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:49:45.196 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:45.196 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:49:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 82 k 2024-03-16T08:49:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:45.196 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:49:45.197 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:45.197 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-03-16T08:49:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:45.197 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 82 k 2024-03-16T08:49:45.197 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 370 k 2024-03-16T08:49:45.197 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:49:45.402 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm-17.2.7-648.g1df27247.el9.noarch.rpm 399 kB/s | 82 kB 00:00 2024-03-16T08:49:45.403 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:45.403 INFO:teuthology.orchestra.run.smithi007.stdout:Total 396 kB/s | 82 kB 00:00 2024-03-16T08:49:45.403 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:49:45.408 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:49:45.408 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:49:45.410 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:49:45.411 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:49:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:49:45.949 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/1 2024-03-16T08:49:46.042 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/1 2024-03-16T08:49:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/1 2024-03-16T08:49:46.536 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-test-2:17.2.7-648.g1df27247.el9.x86_64 6/6 2024-03-16T08:49:46.536 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:17.2.7-648.g1df27247.el9.x86_64 1/6 2024-03-16T08:49:46.536 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:17.2.7-648.g1df27247.el9.x86_64 2/6 2024-03-16T08:49:46.536 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-16T08:49:46.536 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-16T08:49:46.536 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-16T08:49:46.758 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/1 2024-03-16T08:49:46.758 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:46.759 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:49:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:49:46.759 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:46.759 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:49:46.904 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-16T08:49:46.904 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:46.904 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:49:46.904 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:46.904 INFO:teuthology.orchestra.run.smithi019.stdout: jq-1.6-16.el9.x86_64 2024-03-16T08:49:46.904 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:46.905 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T08:49:46.905 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T08:49:46.905 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T08:49:46.905 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:46.905 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:49:46.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-immutable-object-cache 2024-03-16T08:49:47.174 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph 2024-03-16T08:49:47.542 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:07 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:49:47.695 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:49:47.695 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-648.g1df27247.el9 ceph 137 k 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-03-16T08:49:47.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:47.697 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 137 k 2024-03-16T08:49:47.697 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 379 k 2024-03-16T08:49:47.697 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:49:47.752 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:42 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:49:47.921 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-immutable-object-cache-17.2.7-648.g1df2724 613 kB/s | 137 kB 00:00 2024-03-16T08:49:47.922 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:47.922 INFO:teuthology.orchestra.run.smithi007.stdout:Total 609 kB/s | 137 kB 00:00 2024-03-16T08:49:47.923 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:49:47.924 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:49:47.927 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2024-03-16T08:49:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:49:47.927 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2024-03-16T08:49:47.927 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:49:47.927 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:17.2.7-648.g1df27247.el9 ceph 6.5 k 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:17.2.7-648.g1df27247.el9 ceph 2.1 M 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:17.2.7-648.g1df27247.el9 ceph 1.4 M 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 239 k 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:17.2.7-648.g1df27247.el9 ceph 3.7 M 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:17.2.7-648.g1df27247.el9 ceph 15 M 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T08:49:47.928 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T08:49:47.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T08:49:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T08:49:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T08:49:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T08:49:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T08:49:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T08:49:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T08:49:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T08:49:47.930 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2024-03-16T08:49:47.931 INFO:teuthology.orchestra.run.smithi019.stdout:Install 33 Packages 2024-03-16T08:49:47.932 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:47.932 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:49:47.932 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:49:47.933 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 27 M 2024-03-16T08:49:47.933 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 100 M 2024-03-16T08:49:47.933 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:49:47.983 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:49:47.983 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:49:48.130 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-648.g1df27247.e 1/1 2024-03-16T08:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-648.g1df27247.e 1/1 2024-03-16T08:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T08:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:48.783 INFO:teuthology.orchestra.run.smithi019.stdout:(1/33): ceph-17.2.7-648.g1df27247.el9.x86_64.rp 32 kB/s | 6.5 kB 00:00 2024-03-16T08:49:48.983 INFO:teuthology.orchestra.run.smithi019.stdout:(2/33): ceph-mds-17.2.7-648.g1df27247.el9.x86_6 5.1 MB/s | 2.1 MB 00:00 2024-03-16T08:49:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-648.g1df27247.e 1/1 2024-03-16T08:49:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:49.347 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:49:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:49.348 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:49:49.367 INFO:teuthology.orchestra.run.smithi019.stdout:(3/33): ceph-mgr-17.2.7-648.g1df27247.el9.x86_6 1.8 MB/s | 1.4 MB 00:00 2024-03-16T08:49:49.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr 2024-03-16T08:49:49.801 INFO:teuthology.orchestra.run.smithi019.stdout:(4/33): ceph-osd-17.2.7-648.g1df27247.el9.x86_6 19 MB/s | 15 MB 00:00 2024-03-16T08:49:49.867 INFO:teuthology.orchestra.run.smithi019.stdout:(5/33): ceph-mon-17.2.7-648.g1df27247.el9.x86_6 3.4 MB/s | 3.7 MB 00:01 2024-03-16T08:49:49.926 INFO:teuthology.orchestra.run.smithi019.stdout:(6/33): ceph-mgr-modules-core-17.2.7-648.g1df27 427 kB/s | 239 kB 00:00 2024-03-16T08:49:50.009 INFO:teuthology.orchestra.run.smithi019.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 692 kB/s | 144 kB 00:00 2024-03-16T08:49:50.035 INFO:teuthology.orchestra.run.smithi019.stdout:(8/33): python3-markupsafe-1.1.1-12.el9.x86_64. 225 kB/s | 32 kB 00:00 2024-03-16T08:49:50.060 INFO:teuthology.orchestra.run.smithi019.stdout:(9/33): python3-pytz-2021.1-5.el9.noarch.rpm 444 kB/s | 47 kB 00:00 2024-03-16T08:49:50.085 INFO:teuthology.orchestra.run.smithi019.stdout:(10/33): python3-toml-0.10.2-6.el9.noarch.rpm 513 kB/s | 39 kB 00:00 2024-03-16T08:49:50.110 INFO:teuthology.orchestra.run.smithi019.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-16T08:49:50.144 INFO:teuthology.orchestra.run.smithi019.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 14 MB/s | 1.1 MB 00:00 2024-03-16T08:49:50.169 INFO:teuthology.orchestra.run.smithi019.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-16T08:49:50.178 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:10 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:49:50.194 INFO:teuthology.orchestra.run.smithi019.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 1.5 MB/s | 124 kB 00:00 2024-03-16T08:49:50.220 INFO:teuthology.orchestra.run.smithi019.stdout:(15/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-16T08:49:50.280 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:49:50.312 INFO:teuthology.orchestra.run.smithi019.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 882 kB/s | 125 kB 00:00 2024-03-16T08:49:50.336 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:49:50.337 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:49:50.337 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:49:50.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-dashboard 2024-03-16T08:49:50.579 INFO:teuthology.orchestra.run.smithi019.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 113 kB/s | 43 kB 00:00 2024-03-16T08:49:50.662 INFO:teuthology.orchestra.run.smithi019.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-03-16T08:49:50.712 INFO:teuthology.orchestra.run.smithi019.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 349 kB/s | 172 kB 00:00 2024-03-16T08:49:50.804 INFO:teuthology.orchestra.run.smithi019.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 728 kB/s | 358 kB 00:00 2024-03-16T08:49:50.830 INFO:teuthology.orchestra.run.smithi019.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 106 kB/s | 18 kB 00:00 2024-03-16T08:49:50.855 INFO:teuthology.orchestra.run.smithi019.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 163 kB/s | 23 kB 00:00 2024-03-16T08:49:50.888 INFO:teuthology.orchestra.run.smithi019.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 232 kB/s | 19 kB 00:00 2024-03-16T08:49:50.914 INFO:teuthology.orchestra.run.smithi019.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 235 kB/s | 20 kB 00:00 2024-03-16T08:49:50.956 INFO:teuthology.orchestra.run.smithi019.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 461 kB/s | 46 kB 00:00 2024-03-16T08:49:50.991 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:10 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:49:51.006 INFO:teuthology.orchestra.run.smithi019.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 672 kB/s | 79 kB 00:00 2024-03-16T08:49:51.081 INFO:teuthology.orchestra.run.smithi019.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 131 kB/s | 16 kB 00:00 2024-03-16T08:49:51.107 INFO:teuthology.orchestra.run.smithi019.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-03-16T08:49:51.131 INFO:teuthology.orchestra.run.smithi019.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 719 kB/s | 90 kB 00:00 2024-03-16T08:49:51.148 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:49:51.149 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:51.149 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 1.7 M 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 23 k 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 15 k 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T08:49:51.150 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T08:49:51.151 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-03-16T08:49:51.151 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-16T08:49:51.151 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:51.151 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:49:51.151 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:49:51.151 INFO:teuthology.orchestra.run.smithi007.stdout:Install 7 Packages 2024-03-16T08:49:51.151 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:51.152 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 2.0 M 2024-03-16T08:49:51.152 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 8.1 M 2024-03-16T08:49:51.152 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:49:51.157 INFO:teuthology.orchestra.run.smithi019.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 477 kB/s | 36 kB 00:00 2024-03-16T08:49:51.249 INFO:teuthology.orchestra.run.smithi019.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-03-16T08:49:51.307 INFO:teuthology.orchestra.run.smithi019.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.4 MB/s | 427 kB 00:00 2024-03-16T08:49:51.324 INFO:teuthology.orchestra.run.smithi019.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 119 kB/s | 20 kB 00:00 2024-03-16T08:49:51.332 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:51.332 INFO:teuthology.orchestra.run.smithi019.stdout:Total 7.9 MB/s | 27 MB 00:03 2024-03-16T08:49:51.346 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:49:51.389 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:49:51.389 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:49:51.543 INFO:teuthology.orchestra.run.smithi007.stdout:(1/7): ceph-grafana-dashboards-17.2.7-648.g1df2 119 kB/s | 23 kB 00:00 2024-03-16T08:49:51.568 INFO:teuthology.orchestra.run.smithi007.stdout:(2/7): ceph-prometheus-alerts-17.2.7-648.g1df27 66 kB/s | 15 kB 00:00 2024-03-16T08:49:51.730 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:49:51.730 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:49:51.769 INFO:teuthology.orchestra.run.smithi007.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 40 kB/s | 9.0 kB 00:00 2024-03-16T08:49:51.860 INFO:teuthology.orchestra.run.smithi007.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 140 kB/s | 41 kB 00:00 2024-03-16T08:49:51.969 INFO:teuthology.orchestra.run.smithi007.stdout:(5/7): ceph-mgr-dashboard-17.2.7-648.g1df27247. 2.7 MB/s | 1.7 MB 00:00 2024-03-16T08:49:52.053 INFO:teuthology.orchestra.run.smithi007.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 108 kB/s | 31 kB 00:00 2024-03-16T08:49:52.344 INFO:teuthology.orchestra.run.smithi007.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 390 kB/s | 188 kB 00:00 2024-03-16T08:49:52.347 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:52.347 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.7 MB/s | 2.0 MB 00:01 2024-03-16T08:49:52.347 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:49:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:49:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:49:52.422 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:49:52.422 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-16T08:49:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-16T08:49:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-16T08:49:52.899 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-16T08:49:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-648.g1df27247.el9.no 5/7 2024-03-16T08:49:53.090 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:49:53.135 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-16T08:49:53.160 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-16T08:49:53.167 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-648.g1df27247.el9.n 6/7 2024-03-16T08:49:53.185 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-16T08:49:53.198 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-648.g1df27247.el9.noarch 7/7 2024-03-16T08:49:53.280 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-16T08:49:53.364 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-16T08:49:53.412 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-16T08:49:53.494 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-16T08:49:53.547 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-16T08:49:53.584 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-16T08:49:53.634 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-16T08:49:53.689 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-16T08:49:53.741 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-648.g1df27247.el9.noarch 7/7 2024-03-16T08:49:53.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-648.g1df27247.el9.n 1/7 2024-03-16T08:49:53.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-648.g1df27247.el9.noarch 2/7 2024-03-16T08:49:53.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-648.g1df27247.el9.no 3/7 2024-03-16T08:49:53.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-16T08:49:53.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-16T08:49:53.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-16T08:49:53.880 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-16T08:49:53.931 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-16T08:49:53.961 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-16T08:49:53.995 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-16T08:49:54.025 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-16T08:49:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-16T08:49:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:54.050 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:49:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:49:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:49:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:49:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-16T08:49:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-16T08:49:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T08:49:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T08:49:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:54.051 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:49:54.104 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-16T08:49:54.135 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-16T08:49:54.168 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-16T08:49:54.210 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-16T08:49:54.253 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-16T08:49:54.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-16T08:49:54.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-16T08:49:54.379 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-16T08:49:54.418 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-16T08:49:54.494 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-16T08:49:54.634 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-16T08:49:54.766 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-16T08:49:54.839 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:14 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:49:54.923 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-648.g1df27247.el9.n 28/33 2024-03-16T08:49:54.947 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 29/33 2024-03-16T08:49:54.972 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 29/33 2024-03-16T08:49:54.972 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:54.972 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T08:49:54.972 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:49:54.973 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T08:49:54.973 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:54.997 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:49:54.998 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================== 2024-03-16T08:49:54.998 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:49:54.998 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================== 2024-03-16T08:49:54.998 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:49:54.998 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 7.4 M 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:54.999 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:49:55.000 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================== 2024-03-16T08:49:55.000 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-03-16T08:49:55.000 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:49:55.000 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 37 M 2024-03-16T08:49:55.000 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 209 M 2024-03-16T08:49:55.000 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:49:55.393 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 32 kB 00:00 2024-03-16T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 536 kB/s | 17 kB 00:00 2024-03-16T08:49:55.486 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-03-16T08:49:55.502 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 34 kB 00:00 2024-03-16T08:49:55.678 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2024-03-16T08:49:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-03-16T08:49:55.787 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2024-03-16T08:49:56.087 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-03-16T08:49:56.287 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): ceph-mgr-diskprediction-local-17.2.7-64 7.2 MB/s | 7.4 MB 00:01 2024-03-16T08:49:56.382 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:17.2.7-648.g1df27247.el9.x86_64 30/33 2024-03-16T08:49:56.407 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:17.2.7-648.g1df27247.el9.x86_64 30/33 2024-03-16T08:49:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T08:49:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:49:56.408 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T08:49:56.408 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-03-16T08:49:56.596 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 384 kB/s | 195 kB 00:00 2024-03-16T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 956 kB/s | 803 kB 00:00 2024-03-16T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stdout:Total 23 MB/s | 37 MB 00:01 2024-03-16T08:49:56.674 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:49:56.700 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:49:56.700 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:49:56.807 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:17.2.7-648.g1df27247.el9.x86_64 31/33 2024-03-16T08:49:56.830 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:17.2.7-648.g1df27247.el9.x86_64 31/33 2024-03-16T08:49:56.830 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:56.831 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T08:49:56.831 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:49:56.831 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T08:49:56.831 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:57.033 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:17.2.7-648.g1df27247.el9.x86_64 32/33 2024-03-16T08:49:57.056 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:17.2.7-648.g1df27247.el9.x86_64 32/33 2024-03-16T08:49:57.056 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:49:57.056 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T08:49:57.056 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:49:57.056 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T08:49:57.056 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:57.117 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:17.2.7-648.g1df27247.el9.x86_64 33/33 2024-03-16T08:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:49:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:49:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-16T08:49:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-16T08:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-16T08:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-16T08:49:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-16T08:49:58.370 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-16T08:49:58.448 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-2:17.2.7-648.g1df27247.el9.x86_64 33/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:17.2.7-648.g1df27247.el9.x86_64 1/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:17.2.7-648.g1df27247.el9.x86_64 2/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 3/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:17.2.7-648.g1df27247.el9.x86_64 4/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:17.2.7-648.g1df27247.el9.x86_64 5/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-648.g1df27247.el9.n 6/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-16T08:49:58.852 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-16T08:49:58.853 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-16T08:49:58.854 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-16T08:49:58.854 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-16T08:49:58.854 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-16T08:49:58.854 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-16T08:49:58.854 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-16T08:49:58.854 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-16T08:49:58.854 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-16T08:49:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-16T08:49:59.254 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-16T08:49:59.806 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-16T08:49:59.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:59.806 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:49:59.806 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:59.806 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T08:49:59.808 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T08:49:59.809 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T08:49:59.809 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T08:49:59.809 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T08:49:59.809 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T08:49:59.809 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:49:59.809 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:00.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-base 2024-03-16T08:50:00.554 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-16T08:50:00.604 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:55 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:00.704 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-base-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:00.760 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:00.761 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:00.761 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:00.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install cephadm 2024-03-16T08:50:01.400 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:56 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:01.557 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 82 k 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 82 k 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 370 k 2024-03-16T08:50:01.558 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-16T08:50:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-648.g1df272 12/12 2024-03-16T08:50:01.772 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm-17.2.7-648.g1df27247.el9.noarch.rpm 383 kB/s | 82 kB 00:00 2024-03-16T08:50:01.772 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:01.773 INFO:teuthology.orchestra.run.smithi019.stdout:Total 380 kB/s | 82 kB 00:00 2024-03-16T08:50:01.773 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:01.778 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:01.778 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:01.780 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:01.780 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/1 2024-03-16T08:50:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/1 2024-03-16T08:50:02.925 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/1 2024-03-16T08:50:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/1 2024-03-16T08:50:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:03.243 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:03.243 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:03.427 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-immutable-object-cache 2024-03-16T08:50:04.001 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:58 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:04.152 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-648.g1df27247.el9 ceph 137 k 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-03-16T08:50:04.153 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 137 k 2024-03-16T08:50:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 379 k 2024-03-16T08:50:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:04.228 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-648.g1df272 12/12 2024-03-16T08:50:04.228 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-648.g1df272 1/12 2024-03-16T08:50:04.228 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-16T08:50:04.228 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-16T08:50:04.228 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-16T08:50:04.228 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-16T08:50:04.228 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-16T08:50:04.229 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-16T08:50:04.229 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-16T08:50:04.229 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-16T08:50:04.229 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-16T08:50:04.229 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-16T08:50:04.378 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-immutable-object-cache-17.2.7-648.g1df2724 614 kB/s | 137 kB 00:00 2024-03-16T08:50:04.379 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:04.379 INFO:teuthology.orchestra.run.smithi019.stdout:Total 610 kB/s | 137 kB 00:00 2024-03-16T08:50:04.379 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:04.387 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:04.387 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:04.441 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:04.442 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:04.705 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:04.770 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-648.g1df27247.e 1/1 2024-03-16T08:50:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-648.g1df27247.e 1/1 2024-03-16T08:50:04.794 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:50:04.794 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T08:50:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T08:50:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T08:50:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T08:50:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T08:50:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T08:50:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T08:50:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T08:50:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T08:50:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:04.802 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:05.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-rook 2024-03-16T08:50:05.735 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:25 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-648.g1df27247.e 1/1 2024-03-16T08:50:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:05.839 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:05.839 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:05.890 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 47 k 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T08:50:05.892 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout:Install 13 Packages 2024-03-16T08:50:05.893 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:05.894 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 2.0 M 2024-03-16T08:50:05.894 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 26 M 2024-03-16T08:50:05.894 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:50:06.071 INFO:teuthology.orchestra.run.smithi007.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 149 kB/s | 18 kB 00:00 2024-03-16T08:50:06.073 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr 2024-03-16T08:50:06.088 INFO:teuthology.orchestra.run.smithi007.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 176 kB/s | 24 kB 00:00 2024-03-16T08:50:06.146 INFO:teuthology.orchestra.run.smithi007.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-16T08:50:06.163 INFO:teuthology.orchestra.run.smithi007.stdout:(4/13): ceph-mgr-rook-17.2.7-648.g1df27247.el9. 222 kB/s | 47 kB 00:00 2024-03-16T08:50:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2024-03-16T08:50:06.214 INFO:teuthology.orchestra.run.smithi007.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.1 MB/s | 211 kB 00:00 2024-03-16T08:50:06.231 INFO:teuthology.orchestra.run.smithi007.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 43 kB 00:00 2024-03-16T08:50:06.456 INFO:teuthology.orchestra.run.smithi007.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 121 kB/s | 32 kB 00:00 2024-03-16T08:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2024-03-16T08:50:06.598 INFO:teuthology.orchestra.run.smithi007.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-03-16T08:50:06.624 INFO:teuthology.orchestra.run.smithi007.stdout:(11/13): python3-google-auth-2.28.0-1.el9.noarc 516 kB/s | 203 kB 00:00 2024-03-16T08:50:06.644 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:01 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 824 kB/s | 90 kB 00:00 2024-03-16T08:50:06.745 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-mgr-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:06.749 INFO:teuthology.orchestra.run.smithi007.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.5 MB/s | 1.0 MB 00:00 2024-03-16T08:50:06.752 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.4 MB/s | 2.0 MB 00:00 2024-03-16T08:50:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:50:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:50:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:50:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:06.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-dashboard 2024-03-16T08:50:06.868 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:50:06.868 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:50:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:50:07.429 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:02 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-16T08:50:07.586 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:07.586 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:07.586 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:50:07.586 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:07.586 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:07.586 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 1.7 M 2024-03-16T08:50:07.586 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 23 k 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 15 k 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout:Install 7 Packages 2024-03-16T08:50:07.587 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:07.588 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.0 M 2024-03-16T08:50:07.588 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 8.1 M 2024-03-16T08:50:07.588 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:07.635 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-16T08:50:07.698 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-16T08:50:07.735 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-16T08:50:07.777 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-16T08:50:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-16T08:50:07.990 INFO:teuthology.orchestra.run.smithi019.stdout:(1/7): ceph-grafana-dashboards-17.2.7-648.g1df2 111 kB/s | 23 kB 00:00 2024-03-16T08:50:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-16T08:50:08.015 INFO:teuthology.orchestra.run.smithi019.stdout:(2/7): ceph-prometheus-alerts-17.2.7-648.g1df27 62 kB/s | 15 kB 00:00 2024-03-16T08:50:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-16T08:50:08.132 INFO:teuthology.orchestra.run.smithi019.stdout:(3/7): ceph-mgr-dashboard-17.2.7-648.g1df27247. 4.8 MB/s | 1.7 MB 00:00 2024-03-16T08:50:08.190 INFO:teuthology.orchestra.run.smithi019.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 45 kB/s | 9.0 kB 00:00 2024-03-16T08:50:08.332 INFO:teuthology.orchestra.run.smithi019.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 129 kB/s | 41 kB 00:00 2024-03-16T08:50:08.391 INFO:teuthology.orchestra.run.smithi019.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 119 kB/s | 31 kB 00:00 2024-03-16T08:50:08.566 INFO:teuthology.orchestra.run.smithi019.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 502 kB/s | 188 kB 00:00 2024-03-16T08:50:08.568 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:08.568 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.0 MB/s | 2.0 MB 00:00 2024-03-16T08:50:08.569 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:08.577 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:08.578 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:08.653 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:08.653 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-16T08:50:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-16T08:50:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-16T08:50:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-16T08:50:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:17.2.7-648.g1df27247.el9.noarch 13/13 2024-03-16T08:50:08.964 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:09.034 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-16T08:50:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-16T08:50:09.124 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-16T08:50:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-16T08:50:09.184 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-648.g1df27247.el9.no 5/7 2024-03-16T08:50:09.426 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-648.g1df27247.el9.n 6/7 2024-03-16T08:50:09.453 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-648.g1df27247.el9.noarch 7/7 2024-03-16T08:50:09.741 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-648.g1df27247.el9.noarch 13/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:17.2.7-648.g1df27247.el9.noarch 1/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-16T08:50:09.742 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-16T08:50:09.743 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-16T08:50:09.743 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-16T08:50:10.003 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-648.g1df27247.el9.noarch 7/7 2024-03-16T08:50:10.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-648.g1df27247.el9.n 1/7 2024-03-16T08:50:10.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-648.g1df27247.el9.noarch 2/7 2024-03-16T08:50:10.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-648.g1df27247.el9.no 3/7 2024-03-16T08:50:10.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-16T08:50:10.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-16T08:50:10.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-16T08:50:10.100 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T08:50:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T08:50:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T08:50:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T08:50:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T08:50:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T08:50:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T08:50:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T08:50:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:10.102 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-16T08:50:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:10.337 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-16T08:50:10.337 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-16T08:50:10.338 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T08:50:10.338 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T08:50:10.338 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:10.338 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:10.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-cephadm 2024-03-16T08:50:10.545 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-16T08:50:10.912 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:30 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:11.067 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:11.068 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-03-16T08:50:11.068 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:50:11.068 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-03-16T08:50:11.068 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:50:11.068 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 122 k 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout:Install 6 Packages 2024-03-16T08:50:11.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:11.070 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 6.8 M 2024-03-16T08:50:11.070 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 33 M 2024-03-16T08:50:11.070 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:50:11.113 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:06 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:11.271 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:11.272 INFO:teuthology.orchestra.run.smithi019.stdout:=========================================================================================== 2024-03-16T08:50:11.272 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:50:11.272 INFO:teuthology.orchestra.run.smithi019.stdout:=========================================================================================== 2024-03-16T08:50:11.272 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:11.272 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 7.4 M 2024-03-16T08:50:11.272 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout:=========================================================================================== 2024-03-16T08:50:11.273 INFO:teuthology.orchestra.run.smithi019.stdout:Install 12 Packages 2024-03-16T08:50:11.274 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:11.274 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 37 M 2024-03-16T08:50:11.274 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 209 M 2024-03-16T08:50:11.274 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:11.523 INFO:teuthology.orchestra.run.smithi019.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 217 kB/s | 32 kB 00:00 2024-03-16T08:50:11.556 INFO:teuthology.orchestra.run.smithi019.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 516 kB/s | 17 kB 00:00 2024-03-16T08:50:11.568 INFO:teuthology.orchestra.run.smithi007.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-03-16T08:50:11.590 INFO:teuthology.orchestra.run.smithi019.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-16T08:50:11.593 INFO:teuthology.orchestra.run.smithi007.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 886 kB/s | 248 kB 00:00 2024-03-16T08:50:11.665 INFO:teuthology.orchestra.run.smithi019.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-03-16T08:50:11.694 INFO:teuthology.orchestra.run.smithi007.stdout:(3/6): ceph-mgr-cephadm-17.2.7-648.g1df27247.el 320 kB/s | 122 kB 00:00 2024-03-16T08:50:11.782 INFO:teuthology.orchestra.run.smithi019.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2024-03-16T08:50:11.793 INFO:teuthology.orchestra.run.smithi007.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 288 kB/s | 58 kB 00:00 2024-03-16T08:50:11.852 INFO:teuthology.orchestra.run.smithi007.stdout:(5/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 1.9 MB/s | 548 kB 00:00 2024-03-16T08:50:11.866 INFO:teuthology.orchestra.run.smithi019.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2024-03-16T08:50:11.877 INFO:teuthology.orchestra.run.smithi007.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 279 kB/s | 51 kB 00:00 2024-03-16T08:50:11.880 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:11.881 INFO:teuthology.orchestra.run.smithi007.stdout:Total 8.4 MB/s | 6.8 MB 00:00 2024-03-16T08:50:11.891 INFO:teuthology.orchestra.run.smithi019.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2024-03-16T08:50:11.893 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:50:11.903 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:50:11.903 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:50:11.998 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:50:11.999 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:50:12.183 INFO:teuthology.orchestra.run.smithi019.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 50 MB/s | 16 MB 00:00 2024-03-16T08:50:12.275 INFO:teuthology.orchestra.run.smithi019.stdout:(9/12): ceph-mgr-diskprediction-local-17.2.7-64 8.3 MB/s | 7.4 MB 00:00 2024-03-16T08:50:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:50:12.350 INFO:teuthology.orchestra.run.smithi019.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-03-16T08:50:12.401 INFO:teuthology.orchestra.run.smithi019.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 899 kB/s | 195 kB 00:00 2024-03-16T08:50:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-16T08:50:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-16T08:50:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 453 kB/s | 245 kB 00:00 2024-03-16T08:50:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:Total 24 MB/s | 37 MB 00:01 2024-03-16T08:50:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:12.890 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:12.890 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-16T08:50:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-16T08:50:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-16T08:50:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-648.g1df27247.el9.noarch 6/6 2024-03-16T08:50:13.356 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:13.356 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:14.097 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-648.g1df27247.el9.noarch 6/6 2024-03-16T08:50:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/6 2024-03-16T08:50:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-16T08:50:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-16T08:50:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-16T08:50:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-16T08:50:14.182 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-16T08:50:14.214 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-16T08:50:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-16T08:50:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-16T08:50:14.429 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:14.430 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-16T08:50:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-16T08:50:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-16T08:50:14.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-fuse 2024-03-16T08:50:15.246 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:35 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-16T08:50:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-16T08:50:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:50:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:17.2.7-648.g1df27247.el9 ceph 812 k 2024-03-16T08:50:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-16T08:50:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T08:50:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:50:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:Install 2 Packages 2024-03-16T08:50:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:15.402 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 891 k 2024-03-16T08:50:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 2.6 M 2024-03-16T08:50:15.403 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:50:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-16T08:50:15.530 INFO:teuthology.orchestra.run.smithi007.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 619 kB/s | 79 kB 00:00 2024-03-16T08:50:15.713 INFO:teuthology.orchestra.run.smithi007.stdout:(2/2): ceph-fuse-17.2.7-648.g1df27247.el9.x86_6 2.6 MB/s | 812 kB 00:00 2024-03-16T08:50:15.714 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:15.715 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.8 MB/s | 891 kB 00:00 2024-03-16T08:50:15.715 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:50:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:50:15.737 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:50:15.813 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:50:15.813 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:50:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:50:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-16T08:50:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:17.2.7-648.g1df27247.el9.x86_64 2/2 2024-03-16T08:50:16.771 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-16T08:50:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-fuse-2:17.2.7-648.g1df27247.el9.x86_64 2/2 2024-03-16T08:50:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:17.2.7-648.g1df27247.el9.x86_64 1/2 2024-03-16T08:50:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-16T08:50:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:17.206 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:17.2.7-648.g1df27247.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-16T08:50:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:17.207 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:17.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados-devel 2024-03-16T08:50:17.847 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-16T08:50:17.881 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-648.g1df272 12/12 2024-03-16T08:50:18.154 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:38 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:18.308 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:18.309 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:18.309 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-03-16T08:50:18.309 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:18.309 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:50:18.309 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:17.2.7-648.g1df27247.el9 ceph 126 k 2024-03-16T08:50:18.309 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:18.309 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:50:18.310 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:18.310 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-03-16T08:50:18.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:18.310 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 126 k 2024-03-16T08:50:18.310 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 448 k 2024-03-16T08:50:18.310 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:50:18.651 INFO:teuthology.orchestra.run.smithi007.stdout:librados-devel-17.2.7-648.g1df27247.el9.x86_64. 370 kB/s | 126 kB 00:00 2024-03-16T08:50:18.651 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:18.652 INFO:teuthology.orchestra.run.smithi007.stdout:Total 369 kB/s | 126 kB 00:00 2024-03-16T08:50:18.652 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:50:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:50:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:50:18.715 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:50:18.715 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:50:18.874 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:50:18.972 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:19.424 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:19.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:19.714 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:19.715 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:19.715 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:19.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs2 2024-03-16T08:50:20.438 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-648.g1df272 12/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-648.g1df272 1/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-16T08:50:20.439 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-16T08:50:20.562 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:40 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:20.664 INFO:teuthology.orchestra.run.smithi007.stdout:Package libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:20.721 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:20.722 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:50:20.722 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:20.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs-devel 2024-03-16T08:50:20.910 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-16T08:50:20.910 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:20.910 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:20.910 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:20.911 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:21.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-rook 2024-03-16T08:50:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:41 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:21.531 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:21.532 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:21.532 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-03-16T08:50:21.532 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:21.532 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:50:21.532 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:17.2.7-648.g1df27247.el9 ceph 25 k 2024-03-16T08:50:21.532 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:21.532 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:50:21.532 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:21.533 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-03-16T08:50:21.533 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:21.533 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 25 k 2024-03-16T08:50:21.533 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 110 k 2024-03-16T08:50:21.533 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:50:21.774 INFO:teuthology.orchestra.run.smithi007.stdout:libcephfs-devel-17.2.7-648.g1df27247.el9.x86_64 104 kB/s | 25 kB 00:00 2024-03-16T08:50:21.775 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:21.775 INFO:teuthology.orchestra.run.smithi007.stdout:Total 103 kB/s | 25 kB 00:00 2024-03-16T08:50:21.775 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:50:21.779 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:50:21.779 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:50:21.780 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:50:21.781 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:50:21.824 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:16 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:50:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:21.980 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:21.982 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-03-16T08:50:21.982 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:50:21.982 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-03-16T08:50:21.982 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:21.982 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 47 k 2024-03-16T08:50:21.982 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:50:21.982 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T08:50:21.982 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:21.983 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-03-16T08:50:21.984 INFO:teuthology.orchestra.run.smithi019.stdout:Install 13 Packages 2024-03-16T08:50:21.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:21.984 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.0 M 2024-03-16T08:50:21.984 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 26 M 2024-03-16T08:50:21.984 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:22.166 INFO:teuthology.orchestra.run.smithi019.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 187 kB/s | 24 kB 00:00 2024-03-16T08:50:22.191 INFO:teuthology.orchestra.run.smithi019.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 116 kB/s | 18 kB 00:00 2024-03-16T08:50:22.241 INFO:teuthology.orchestra.run.smithi019.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-16T08:50:22.266 INFO:teuthology.orchestra.run.smithi019.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-16T08:50:22.292 INFO:teuthology.orchestra.run.smithi019.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-16T08:50:22.317 INFO:teuthology.orchestra.run.smithi019.stdout:(6/13): ceph-mgr-rook-17.2.7-648.g1df27247.el9. 168 kB/s | 47 kB 00:00 2024-03-16T08:50:22.344 INFO:teuthology.orchestra.run.smithi019.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 567 kB/s | 43 kB 00:00 2024-03-16T08:50:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:22.584 INFO:teuthology.orchestra.run.smithi019.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-03-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi019.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 99 kB/s | 32 kB 00:00 2024-03-16T08:50:22.629 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:22.940 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:22.941 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:22.941 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:22.941 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:22.941 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:22.942 INFO:teuthology.orchestra.run.smithi019.stdout:(10/13): python3-google-auth-2.28.0-1.el9.noarc 426 kB/s | 203 kB 00:00 2024-03-16T08:50:22.942 INFO:teuthology.orchestra.run.smithi019.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 263 kB/s | 59 kB 00:00 2024-03-16T08:50:22.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados2 2024-03-16T08:50:22.977 INFO:teuthology.orchestra.run.smithi019.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 566 kB/s | 90 kB 00:00 2024-03-16T08:50:23.119 INFO:teuthology.orchestra.run.smithi019.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-03-16T08:50:23.121 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:23.121 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.8 MB/s | 2.0 MB 00:01 2024-03-16T08:50:23.122 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:23.138 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:23.138 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:23.248 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:23.249 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:23.529 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:43 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:23.628 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:23.683 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:23.684 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:50:23.685 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:23.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librbd1 2024-03-16T08:50:23.810 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:23.982 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-16T08:50:24.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-16T08:50:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-16T08:50:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-16T08:50:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-16T08:50:24.268 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-16T08:50:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:44 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:24.386 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-16T08:50:24.425 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:24.448 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-16T08:50:24.481 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:24.482 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:50:24.482 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:24.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rados 2024-03-16T08:50:25.132 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:45 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:25.204 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-16T08:50:25.231 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rados-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:25.261 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-16T08:50:25.286 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:25.288 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:50:25.288 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:25.294 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-16T08:50:25.341 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-16T08:50:25.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rgw 2024-03-16T08:50:25.364 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:17.2.7-648.g1df27247.el9.noarch 13/13 2024-03-16T08:50:25.934 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:45 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:26.034 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rgw-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:26.089 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:50:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-648.g1df27247.el9.noarch 13/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:17.2.7-648.g1df27247.el9.noarch 1/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-16T08:50:26.152 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-16T08:50:26.153 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-16T08:50:26.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-cephfs 2024-03-16T08:50:26.526 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-16T08:50:26.526 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:26.526 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:26.526 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:26.526 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T08:50:26.526 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T08:50:26.526 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T08:50:26.526 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:26.733 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:46 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:26.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-cephadm 2024-03-16T08:50:26.833 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-cephfs-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:26.888 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:26.889 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:50:26.889 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:26.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rbd 2024-03-16T08:50:27.308 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:22 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:27.463 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:17.2.7-648.g1df27247.el9 ceph-noarch 122 k 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T08:50:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T08:50:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T08:50:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:27.465 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:27.465 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-03-16T08:50:27.465 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2024-03-16T08:50:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:27.465 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 6.8 M 2024-03-16T08:50:27.466 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 33 M 2024-03-16T08:50:27.466 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:27.542 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:47 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:27.642 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rbd-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:27.698 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:27.699 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-03-16T08:50:27.699 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:27.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-fuse 2024-03-16T08:50:28.103 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): ceph-mgr-cephadm-17.2.7-648.g1df27247.el 290 kB/s | 122 kB 00:00 2024-03-16T08:50:28.306 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 9.3 MB/s | 5.8 MB 00:00 2024-03-16T08:50:28.339 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:48 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:28.429 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 462 kB/s | 58 kB 00:00 2024-03-16T08:50:28.471 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.2 MB/s | 51 kB 00:00 2024-03-16T08:50:28.494 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:28.495 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:17.2.7-648.g1df27247.el9 ceph 82 k 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 82 k 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 222 k 2024-03-16T08:50:28.496 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:50:28.529 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 1.3 MB/s | 548 kB 00:00 2024-03-16T08:50:28.694 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-fuse-17.2.7-648.g1df27247.el9.x86_64.rpm 413 kB/s | 82 kB 00:00 2024-03-16T08:50:28.695 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:28.695 INFO:teuthology.orchestra.run.smithi007.stdout:Total 411 kB/s | 82 kB 00:00 2024-03-16T08:50:28.695 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:50:28.702 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:50:28.703 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:50:28.759 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:50:28.759 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:50:28.787 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 225 kB/s | 248 kB 00:01 2024-03-16T08:50:28.790 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:28.790 INFO:teuthology.orchestra.run.smithi019.stdout:Total 5.1 MB/s | 6.8 MB 00:01 2024-03-16T08:50:28.804 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:28.815 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:28.815 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:28.901 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:50:28.912 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:28.913 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:29.224 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:29.394 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-16T08:50:29.413 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-fuse-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:29.449 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-16T08:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:29.737 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:29.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-mirror 2024-03-16T08:50:30.004 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-16T08:50:30.089 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-16T08:50:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-16T08:50:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-648.g1df27247.el9.noarch 6/6 2024-03-16T08:50:30.535 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:50 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:30.690 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:30.691 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-03-16T08:50:30.691 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:30.691 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:50:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:17.2.7-648.g1df27247.el9 ceph 2.9 M 2024-03-16T08:50:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:30.692 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:50:30.692 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:30.692 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-03-16T08:50:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:30.692 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 2.9 M 2024-03-16T08:50:30.692 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 12 M 2024-03-16T08:50:30.692 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:50:30.979 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-648.g1df27247.el9.noarch 6/6 2024-03-16T08:50:30.979 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-648.g1df27247.el9.noarch 1/6 2024-03-16T08:50:30.979 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-16T08:50:30.980 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-16T08:50:30.980 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-16T08:50:30.980 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-16T08:50:31.256 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-mirror-17.2.7-648.g1df27247.el9.x86_64.rpm 5.2 MB/s | 2.9 MB 00:00 2024-03-16T08:50:31.256 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:31.257 INFO:teuthology.orchestra.run.smithi007.stdout:Total 5.2 MB/s | 2.9 MB 00:00 2024-03-16T08:50:31.257 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:50:31.267 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:50:31.267 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:17.2.7-648.g1df27247.el9.noarch 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T08:50:31.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:31.301 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:31.340 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:50:31.341 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:50:31.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-fuse 2024-03-16T08:50:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:50:31.849 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:31.873 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:31.873 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:50:31.873 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T08:50:31.873 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:50:31.873 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:50:31.873 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:32.108 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:27 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:32.263 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:32.264 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:32.264 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-16T08:50:32.264 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:32.264 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:32.264 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:17.2.7-648.g1df27247.el9 ceph 812 k 2024-03-16T08:50:32.264 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-16T08:50:32.264 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T08:50:32.265 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:32.265 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:32.265 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:32.265 INFO:teuthology.orchestra.run.smithi019.stdout:Install 2 Packages 2024-03-16T08:50:32.265 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:32.265 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 891 k 2024-03-16T08:50:32.265 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 2.6 M 2024-03-16T08:50:32.265 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:32.391 INFO:teuthology.orchestra.run.smithi019.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 626 kB/s | 79 kB 00:00 2024-03-16T08:50:32.575 INFO:teuthology.orchestra.run.smithi019.stdout:(2/2): ceph-fuse-17.2.7-648.g1df27247.el9.x86_6 2.6 MB/s | 812 kB 00:00 2024-03-16T08:50:32.576 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:32.576 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.8 MB/s | 891 kB 00:00 2024-03-16T08:50:32.576 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:32.598 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:32.598 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:32.676 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:32.676 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:32.861 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:32.982 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:32.982 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:32.982 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:32.982 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:32.982 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:32.982 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:32.997 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-16T08:50:33.068 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:17.2.7-648.g1df27247.el9.x86_64 2/2 2024-03-16T08:50:33.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-nbd 2024-03-16T08:50:33.766 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-fuse-2:17.2.7-648.g1df27247.el9.x86_64 2/2 2024-03-16T08:50:33.766 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:17.2.7-648.g1df27247.el9.x86_64 1/2 2024-03-16T08:50:33.779 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:53 ago on Sat 16 Mar 2024 08:48:40 AM UTC. 2024-03-16T08:50:33.935 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:17.2.7-648.g1df27247.el9 ceph 165 k 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 165 k 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 477 k 2024-03-16T08:50:33.936 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-16T08:50:34.072 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-16T08:50:34.072 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:34.072 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:34.072 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:17.2.7-648.g1df27247.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-16T08:50:34.072 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:34.073 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-nbd-17.2.7-648.g1df27247.el9.x86_64.rpm 742 kB/s | 165 kB 00:00 2024-03-16T08:50:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:Total 737 kB/s | 165 kB 00:00 2024-03-16T08:50:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-16T08:50:34.168 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-16T08:50:34.168 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-16T08:50:34.227 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-16T08:50:34.227 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-16T08:50:34.321 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados-devel 2024-03-16T08:50:34.393 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-16T08:50:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:34.897 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:29 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:34.949 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-nbd-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:35.052 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:35.052 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:35.052 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-03-16T08:50:35.052 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:35.052 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:35.053 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:17.2.7-648.g1df27247.el9 ceph 126 k 2024-03-16T08:50:35.053 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:35.053 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:35.053 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:35.053 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-03-16T08:50:35.053 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:35.053 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 126 k 2024-03-16T08:50:35.054 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 448 k 2024-03-16T08:50:35.054 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:35.252 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-16T08:50:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:50:35.252 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-16T08:50:35.363 INFO:teuthology.orchestra.run.smithi019.stdout:librados-devel-17.2.7-648.g1df27247.el9.x86_64. 409 kB/s | 126 kB 00:00 2024-03-16T08:50:35.363 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:35.363 INFO:teuthology.orchestra.run.smithi019.stdout:Total 407 kB/s | 126 kB 00:00 2024-03-16T08:50:35.363 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:35.369 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:35.369 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:35.426 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:35.427 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:35.488 DEBUG:teuthology.parallel:result is None 2024-03-16T08:50:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:35.680 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:36.099 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:36.447 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:36.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:36.447 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:36.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs2 2024-03-16T08:50:37.268 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:32 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:37.371 INFO:teuthology.orchestra.run.smithi019.stdout:Package libcephfs2-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:37.427 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:37.429 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:37.429 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:37.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs-devel 2024-03-16T08:50:38.065 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:33 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:38.221 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:38.221 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:38.221 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:17.2.7-648.g1df27247.el9 ceph 25 k 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 25 k 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 110 k 2024-03-16T08:50:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:38.391 INFO:teuthology.orchestra.run.smithi019.stdout:libcephfs-devel-17.2.7-648.g1df27247.el9.x86_64 148 kB/s | 25 kB 00:00 2024-03-16T08:50:38.392 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:38.392 INFO:teuthology.orchestra.run.smithi019.stdout:Total 147 kB/s | 25 kB 00:00 2024-03-16T08:50:38.392 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:38.396 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:38.396 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:38.397 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:38.398 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:38.483 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:38.583 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:38.980 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:39.262 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:39.263 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:39.263 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:39.263 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:39.423 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados2 2024-03-16T08:50:39.991 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:34 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:40.091 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:40.147 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:40.148 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:40.148 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:40.207 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librbd1 2024-03-16T08:50:40.779 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:35 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:40.879 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:40.934 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:40.935 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:40.935 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:41.002 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rados 2024-03-16T08:50:41.578 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:36 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:41.678 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rados-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:41.734 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:41.735 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:41.735 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:41.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rgw 2024-03-16T08:50:42.376 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:37 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:42.475 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rgw-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:42.531 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:42.532 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:42.532 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:42.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-cephfs 2024-03-16T08:50:43.166 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:38 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:43.269 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-cephfs-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:43.324 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:43.325 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:43.325 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:43.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rbd 2024-03-16T08:50:43.959 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:38 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:44.059 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rbd-2:17.2.7-648.g1df27247.el9.x86_64 is already installed. 2024-03-16T08:50:44.116 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:44.117 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-03-16T08:50:44.117 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:44.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-fuse 2024-03-16T08:50:44.755 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:39 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:44.911 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:44.911 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:44.911 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:17.2.7-648.g1df27247.el9 ceph 82 k 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 82 k 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 222 k 2024-03-16T08:50:44.912 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:45.228 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-fuse-17.2.7-648.g1df27247.el9.x86_64.rpm 259 kB/s | 82 kB 00:00 2024-03-16T08:50:45.228 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:45.228 INFO:teuthology.orchestra.run.smithi019.stdout:Total 258 kB/s | 82 kB 00:00 2024-03-16T08:50:45.229 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:45.236 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:45.236 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:45.292 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:45.293 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:45.441 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:45.544 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-fuse-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:46.312 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:46.312 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:46.312 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:46.312 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:46.312 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:46.312 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:46.562 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-mirror 2024-03-16T08:50:47.143 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:42 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:47.299 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:17.2.7-648.g1df27247.el9 ceph 2.9 M 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.9 M 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 12 M 2024-03-16T08:50:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:47.681 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-mirror-17.2.7-648.g1df27247.el9.x86_64.rpm 7.7 MB/s | 2.9 MB 00:00 2024-03-16T08:50:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:Total 7.7 MB/s | 2.9 MB 00:00 2024-03-16T08:50:47.682 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:47.691 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:47.692 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:47.766 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:47.767 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:48.193 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:48.240 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:48.264 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T08:50:48.264 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T08:50:48.264 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:50:48.264 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T08:50:48.264 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:49.428 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:49.428 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:49.428 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:49.428 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:49.428 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:49.428 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:49.662 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-nbd 2024-03-16T08:50:50.236 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:45 ago on Sat 16 Mar 2024 08:49:05 AM UTC. 2024-03-16T08:50:50.393 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-16T08:50:50.393 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:17.2.7-648.g1df27247.el9 ceph 165 k 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 165 k 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 477 k 2024-03-16T08:50:50.394 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-16T08:50:50.618 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-nbd-17.2.7-648.g1df27247.el9.x86_64.rpm 739 kB/s | 165 kB 00:00 2024-03-16T08:50:50.619 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:50:50.619 INFO:teuthology.orchestra.run.smithi019.stdout:Total 734 kB/s | 165 kB 00:00 2024-03-16T08:50:50.619 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-16T08:50:50.627 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-16T08:50:50.627 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-16T08:50:50.686 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-16T08:50:50.686 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-16T08:50:50.835 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-16T08:50:50.952 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:51.397 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-nbd-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:51.735 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:17.2.7-648.g1df27247.el9.x86_64 1/1 2024-03-16T08:50:51.736 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:51.736 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-16T08:50:51.736 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:17.2.7-648.g1df27247.el9.x86_64 2024-03-16T08:50:51.736 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:50:51.736 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-16T08:50:51.980 DEBUG:teuthology.parallel:result is None 2024-03-16T08:50:51.981 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T08:50:51.981 INFO:teuthology.packaging:ref: None 2024-03-16T08:50:51.981 INFO:teuthology.packaging:tag: None 2024-03-16T08:50:51.981 INFO:teuthology.packaging:branch: quincy 2024-03-16T08:50:51.981 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:50:51.981 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-16T08:50:52.329 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T08:50:52.353 INFO:teuthology.orchestra.run.smithi007.stdout:17.2.7-648.g1df27247.el9 2024-03-16T08:50:52.354 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-648.g1df27247.el9 2024-03-16T08:50:52.354 INFO:teuthology.task.install:The correct ceph version 17.2.7-648.g1df27247 is installed. 2024-03-16T08:50:52.355 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T08:50:52.356 INFO:teuthology.packaging:ref: None 2024-03-16T08:50:52.356 INFO:teuthology.packaging:tag: None 2024-03-16T08:50:52.356 INFO:teuthology.packaging:branch: quincy 2024-03-16T08:50:52.356 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:50:52.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-16T08:50:52.754 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T08:50:52.777 INFO:teuthology.orchestra.run.smithi019.stdout:17.2.7-648.g1df27247.el9 2024-03-16T08:50:52.777 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-648.g1df27247.el9 2024-03-16T08:50:52.778 INFO:teuthology.task.install:The correct ceph version 17.2.7-648.g1df27247 is installed. 2024-03-16T08:50:52.779 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-16T08:50:52.780 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:50:52.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T08:50:52.812 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:50:52.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T08:50:52.848 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-16T08:50:52.848 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:50:52.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T08:50:52.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T08:50:52.952 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:50:52.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T08:50:52.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T08:50:53.050 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-16T08:50:53.050 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:50:53.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T08:50:53.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T08:50:53.148 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:50:53.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T08:50:53.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T08:50:53.244 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-16T08:50:53.244 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:50:53.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T08:50:53.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T08:50:53.337 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:50:53.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T08:50:53.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T08:50:53.843 INFO:teuthology.run_tasks:Running task print... 2024-03-16T08:50:53.921 INFO:teuthology.task.print:**** done install task... 2024-03-16T08:50:53.921 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-16T08:50:54.053 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-16T08:50:54.053 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-16T08:50:54.054 INFO:tasks.cephadm:Cluster fsid is 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:50:54.054 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-16T08:50:54.054 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-16T08:50:54.054 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi007': '172.21.15.7', 'mon.smithi019': '172.21.15.19'} 2024-03-16T08:50:54.054 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-16T08:50:54.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-03-16T08:50:54.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-03-16T08:50:54.113 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-16T08:50:54.113 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T08:50:54.331 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 16 08:50 /home/ubuntu/cephtest/cephadm 2024-03-16T08:50:54.331 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T08:50:54.543 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 16 08:50 /home/ubuntu/cephtest/cephadm 2024-03-16T08:50:54.543 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T08:50:54.559 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T08:50:54.587 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-16T08:50:54.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-16T08:50:54.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-16T08:50:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-16T08:50:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-16T08:51:24.233 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-16T08:51:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)", 2024-03-16T08:51:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "0bdcfccf14ced818364a26df730f679c4000432cea6541bcb4a6dbd30237ef40", 2024-03-16T08:51:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-03-16T08:51:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc" 2024-03-16T08:51:24.233 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-16T08:51:24.234 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-16T08:51:27.893 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T08:51:27.893 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)", 2024-03-16T08:51:27.893 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "0bdcfccf14ced818364a26df730f679c4000432cea6541bcb4a6dbd30237ef40", 2024-03-16T08:51:27.893 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-03-16T08:51:27.894 INFO:teuthology.orchestra.run.smithi007.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc" 2024-03-16T08:51:27.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-16T08:51:27.894 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T08:51:27.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-03-16T08:51:27.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-03-16T08:51:28.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-03-16T08:51:28.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-03-16T08:51:28.079 INFO:tasks.cephadm:Writing seed config... 2024-03-16T08:51:28.080 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-16T08:51:28.080 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-16T08:51:28.080 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-16T08:51:28.080 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-16T08:51:28.080 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-16T08:51:28.081 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-16T08:51:28.082 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-16T08:51:28.083 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-16T08:51:28.083 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-16T08:51:28.083 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-16T08:51:28.083 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-16T08:51:28.083 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-16T08:51:28.083 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-16T08:51:28.083 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-16T08:51:28.084 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:51:28.084 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-16T08:51:28.098 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4c46b77e-e372-11ee-95c9-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-16T08:51:28.098 DEBUG:teuthology.orchestra.run.smithi007:mon.smithi007> sudo journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007.service 2024-03-16T08:51:28.140 INFO:tasks.cephadm:Bootstrapping... 2024-03-16T08:51:28.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:51:28.349 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:51:28.349 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '4c46b77e-e372-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-03-16T08:51:28.374 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T08:51:28.374 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-16T08:51:28.374 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-03-16T08:51:28.397 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T08:51:28.398 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-03-16T08:51:28.398 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-03-16T08:51:28.405 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T08:51:28.405 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T08:51:28.411 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T08:51:28.411 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-16T08:51:28.418 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-03-16T08:51:28.424 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-03-16T08:51:28.424 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-03-16T08:51:28.424 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-03-16T08:51:28.447 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T08:51:28.447 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-16T08:51:28.447 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-03-16T08:51:28.447 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-03-16T08:51:28.454 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T08:51:28.454 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T08:51:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T08:51:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-16T08:51:28.467 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-03-16T08:51:28.473 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-03-16T08:51:28.473 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-03-16T08:51:28.473 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-03-16T08:51:28.473 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:51:28.473 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 139692422008992 on /run/cephadm/4c46b77e-e372-11ee-95c9-87774f69a715.lock 2024-03-16T08:51:28.473 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 139692422008992 acquired on /run/cephadm/4c46b77e-e372-11ee-95c9-87774f69a715.lock 2024-03-16T08:51:28.474 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-03-16T08:51:28.474 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-03-16T08:51:28.474 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-03-16T08:51:28.478 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-03-16T08:51:28.478 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-03-16T08:51:28.480 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-16T08:51:28.480 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-16T08:51:28.481 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-16T08:51:28.483 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-16T08:51:28.483 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-16T08:51:28.483 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T08:51:28.483 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-16T08:51:28.484 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-16T08:51:28.484 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T08:51:28.484 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-03-16T08:51:28.484 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-03-16T08:51:28.484 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-16T08:51:28.484 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-16T08:51:28.485 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-16T08:51:28.977 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 0bdcfccf14ced818364a26df730f679c4000432cea6541bcb4a6dbd30237ef40 2024-03-16T08:51:28.977 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-16T08:51:28.977 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-03-16T08:51:28.977 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-16T08:51:28.977 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:39d52701371a4fdea6d6b829412141823a19927f69c03256d8b0e0cfb8dec82e 2024-03-16T08:51:28.977 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:0bdcfccf14ced818364a26df730f679c4000432cea6541bcb4a6dbd30237ef40 2024-03-16T08:51:28.978 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-16T08:51:29.874 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable) 2024-03-16T08:51:29.874 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable) 2024-03-16T08:51:29.874 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-03-16T08:51:30.704 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-03-16T08:51:30.705 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-03-16T08:51:31.623 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCTXfVltVufARAAmQU8l7522utSFYCwkF95Ww== 2024-03-16T08:51:32.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCTXfVld7wdOBAAkaczEv6eGMuMMFulRqbV5w== 2024-03-16T08:51:33.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQCUXfVl7KOjMRAAZApnuAXqh1cA8tSk4JZlHw== 2024-03-16T08:51:33.518 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-03-16T08:51:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T08:51:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-16T08:51:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:51:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T08:51:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for smithi007 [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T08:51:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = octopus 2024-03-16T08:51:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:51:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T08:51:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:51:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-03-16T08:51:35.270 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.smithi007 on 2024-03-16T08:51:35.466 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-16T08:51:35.606 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-16T08:51:35.756 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4c46b77e-e372-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-4c46b77e-e372-11ee-95c9-87774f69a715.target. 2024-03-16T08:51:35.756 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4c46b77e-e372-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-4c46b77e-e372-11ee-95c9-87774f69a715.target. 2024-03-16T08:51:35.921 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007 2024-03-16T08:51:35.921 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007.service: Unit ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007.service not loaded. 2024-03-16T08:51:36.092 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4c46b77e-e372-11ee-95c9-87774f69a715.target.wants/ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007.service → /etc/systemd/system/ceph-4c46b77e-e372-11ee-95c9-87774f69a715@.service. 2024-03-16T08:51:36.762 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T08:51:36.762 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-16T08:51:36.770 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T08:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-16T08:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-16T08:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T08:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-03-16T08:51:36.771 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-03-16T08:51:36.900 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:36 smithi007 systemd[1]: Started Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T08:51:37.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-03-16T08:51:37.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:51:37.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-16T08:51:37.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:37.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi007 (age 0.601391s) 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.219+0000 7fc79b6de700 1 Processor -- start 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.219+0000 7fc79b6de700 1 -- start start 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.219+0000 7fc79b6de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794102710 0x7fc794102b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:37.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.219+0000 7fc79b6de700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794103060 con 0x7fc794102710 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.220+0000 7fc79947a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794102710 0x7fc794102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.220+0000 7fc79947a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794102710 0x7fc794102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42834/0 (socket says 172.21.15.7:42834) 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.220+0000 7fc79947a700 1 -- 172.21.15.7:0/1697003098 learned_addr learned my addr 172.21.15.7:0/1697003098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.220+0000 7fc79947a700 1 -- 172.21.15.7:0/1697003098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7941031a0 con 0x7fc794102710 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.220+0000 7fc79947a700 1 --2- 172.21.15.7:0/1697003098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794102710 0x7fc794102b20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc7900098e0 tx=0x7fc790009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57761003b2540d78 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.220+0000 7fc78bfff700 1 -- 172.21.15.7:0/1697003098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc79000b860 con 0x7fc794102710 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.220+0000 7fc78bfff700 1 -- 172.21.15.7:0/1697003098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79000b9c0 con 0x7fc794102710 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.220+0000 7fc78bfff700 1 -- 172.21.15.7:0/1697003098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc79000bc90 con 0x7fc794102710 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.221+0000 7fc79b6de700 1 -- 172.21.15.7:0/1697003098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794102710 msgr2=0x7fc794102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.221+0000 7fc79b6de700 1 --2- 172.21.15.7:0/1697003098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794102710 0x7fc794102b20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc7900098e0 tx=0x7fc790009250 comp rx=0 tx=0).stop 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.221+0000 7fc79b6de700 1 -- 172.21.15.7:0/1697003098 shutdown_connections 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.221+0000 7fc79b6de700 1 --2- 172.21.15.7:0/1697003098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794102710 0x7fc794102b20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.221+0000 7fc79b6de700 1 -- 172.21.15.7:0/1697003098 >> 172.21.15.7:0/1697003098 conn(0x7fc7940fde50 msgr2=0x7fc7941002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.221+0000 7fc79b6de700 1 -- 172.21.15.7:0/1697003098 shutdown_connections 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.221+0000 7fc79b6de700 1 -- 172.21.15.7:0/1697003098 wait complete. 2024-03-16T08:51:37.855 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.221+0000 7fc79b6de700 1 Processor -- start 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79b6de700 1 -- start start 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79b6de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794110ed0 0x7fc7941112e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79b6de700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794111820 con 0x7fc794110ed0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79947a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794110ed0 0x7fc7941112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79947a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794110ed0 0x7fc7941112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42838/0 (socket says 172.21.15.7:42838) 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79947a700 1 -- 172.21.15.7:0/382115281 learned_addr learned my addr 172.21.15.7:0/382115281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79947a700 1 -- 172.21.15.7:0/382115281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900094f0 con 0x7fc794110ed0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79947a700 1 --2- 172.21.15.7:0/382115281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794110ed0 0x7fc7941112e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc790009290 tx=0x7fc790016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc789ffb700 1 -- 172.21.15.7:0/382115281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc79000bc90 con 0x7fc794110ed0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc789ffb700 1 -- 172.21.15.7:0/382115281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc790020070 con 0x7fc794110ed0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc794111a20 con 0x7fc794110ed0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.222+0000 7fc789ffb700 1 -- 172.21.15.7:0/382115281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc79001b5f0 con 0x7fc794110ed0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.223+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7941123b0 con 0x7fc794110ed0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.223+0000 7fc789ffb700 1 -- 172.21.15.7:0/382115281 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc790022030 con 0x7fc794110ed0 2024-03-16T08:51:37.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.223+0000 7fc789ffb700 1 -- 172.21.15.7:0/382115281 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc79001c7a0 con 0x7fc794110ed0 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.224+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79405f5f0 con 0x7fc794110ed0 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.227+0000 7fc789ffb700 1 -- 172.21.15.7:0/382115281 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc79001bb60 con 0x7fc794110ed0 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.299+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc794112e80 con 0x7fc794110ed0 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.299+0000 7fc789ffb700 1 -- 172.21.15.7:0/382115281 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fc79001bdb0 con 0x7fc794110ed0 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.300+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794110ed0 msgr2=0x7fc7941112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.300+0000 7fc79b6de700 1 --2- 172.21.15.7:0/382115281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794110ed0 0x7fc7941112e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc790009290 tx=0x7fc790016000 comp rx=0 tx=0).stop 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.300+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 shutdown_connections 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.300+0000 7fc79b6de700 1 --2- 172.21.15.7:0/382115281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc794110ed0 0x7fc7941112e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.300+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 >> 172.21.15.7:0/382115281 conn(0x7fc7940fde50 msgr2=0x7fc794107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.300+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 shutdown_connections 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:37.300+0000 7fc79b6de700 1 -- 172.21.15.7:0/382115281 wait complete. 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-03-16T08:51:37.857 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-03-16T08:51:37.974 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:37 smithi007 ceph-mon[29097]: mon.smithi007 is new leader, mons smithi007 in quorum (ranks 0) 2024-03-16T08:51:37.974 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:37 smithi007 ceph-mon[29097]: monmap e1: 1 mons at {smithi007=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-16T08:51:37.975 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:37 smithi007 ceph-mon[29097]: fsmap 2024-03-16T08:51:37.975 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:37 smithi007 ceph-mon[29097]: osdmap e1: 0 total, 0 up, 0 in 2024-03-16T08:51:37.975 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:37 smithi007 ceph-mon[29097]: mgrmap e1: no daemons active 2024-03-16T08:51:37.975 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:37 smithi007 ceph-mon[29097]: from='client.? 172.21.15.7:0/382115281' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-16T08:51:38.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:38.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T08:51:38.859 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T08:51:38.860 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.277+0000 7f961599d700 1 Processor -- start 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.277+0000 7f961599d700 1 -- start start 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.277+0000 7f961599d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f9610102e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.277+0000 7f961599d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9610103340 con 0x7f96101029f0 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.278+0000 7f961499b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f9610102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.278+0000 7f961499b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f9610102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42842/0 (socket says 172.21.15.7:42842) 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.278+0000 7f961499b700 1 -- 172.21.15.7:0/1651560949 learned_addr learned my addr 172.21.15.7:0/1651560949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.278+0000 7f961499b700 1 -- 172.21.15.7:0/1651560949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610103480 con 0x7f96101029f0 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.278+0000 7f961499b700 1 --2- 172.21.15.7:0/1651560949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f9610102e00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f960400bb80 tx=0x7f960400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd297f28ea1492ab server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.278+0000 7f960f7fe700 1 -- 172.21.15.7:0/1651560949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f960400dd00 con 0x7f96101029f0 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.278+0000 7f960f7fe700 1 -- 172.21.15.7:0/1651560949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960400de60 con 0x7f96101029f0 2024-03-16T08:51:38.861 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.278+0000 7f960f7fe700 1 -- 172.21.15.7:0/1651560949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96040105f0 con 0x7f96101029f0 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 -- 172.21.15.7:0/1651560949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 msgr2=0x7f9610102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 --2- 172.21.15.7:0/1651560949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f9610102e00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f960400bb80 tx=0x7f960400b4f0 comp rx=0 tx=0).stop 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 -- 172.21.15.7:0/1651560949 shutdown_connections 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 --2- 172.21.15.7:0/1651560949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f9610102e00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 -- 172.21.15.7:0/1651560949 >> 172.21.15.7:0/1651560949 conn(0x7f96100fc320 msgr2=0x7f96100fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 -- 172.21.15.7:0/1651560949 shutdown_connections 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 -- 172.21.15.7:0/1651560949 wait complete. 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 Processor -- start 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.279+0000 7f961599d700 1 -- start start 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f961599d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f961010e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f961599d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961010eee0 con 0x7f96101029f0 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f961499b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f961010e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f961499b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f961010e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42852/0 (socket says 172.21.15.7:42852) 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f961499b700 1 -- 172.21.15.7:0/3564393383 learned_addr learned my addr 172.21.15.7:0/3564393383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f961499b700 1 -- 172.21.15.7:0/3564393383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960400b790 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f961499b700 1 --2- 172.21.15.7:0/3564393383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f961010e9a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9604000c00 tx=0x7f960401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f960d7fa700 1 -- 172.21.15.7:0/3564393383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96040189b0 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f960d7fa700 1 -- 172.21.15.7:0/3564393383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960401d070 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.280+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961010f0e0 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.281+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961010fc50 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.281+0000 7f960d7fa700 1 -- 172.21.15.7:0/3564393383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9604018cd0 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.281+0000 7f960d7fa700 1 -- 172.21.15.7:0/3564393383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f960402e460 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.281+0000 7f960d7fa700 1 -- 172.21.15.7:0/3564393383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9604014070 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.281+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f40052e0 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.284+0000 7f960d7fa700 1 -- 172.21.15.7:0/3564393383 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f960401f050 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.353+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f95f4005860 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.356+0000 7f960d7fa700 1 -- 172.21.15.7:0/3564393383 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9604018e30 con 0x7f96101029f0 2024-03-16T08:51:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.356+0000 7f960d7fa700 1 -- 172.21.15.7:0/3564393383 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f9604038bf0 con 0x7f96101029f0 2024-03-16T08:51:38.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.357+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 msgr2=0x7f961010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:38.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.357+0000 7f961599d700 1 --2- 172.21.15.7:0/3564393383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f961010e9a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9604000c00 tx=0x7f960401a000 comp rx=0 tx=0).stop 2024-03-16T08:51:38.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.357+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 shutdown_connections 2024-03-16T08:51:38.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.357+0000 7f961599d700 1 --2- 172.21.15.7:0/3564393383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96101029f0 0x7f961010e9a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:38.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.357+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 >> 172.21.15.7:0/3564393383 conn(0x7f96100fc320 msgr2=0x7f96100fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:38.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.357+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 shutdown_connections 2024-03-16T08:51:38.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:38.357+0000 7f961599d700 1 -- 172.21.15.7:0/3564393383 wait complete. 2024-03-16T08:51:38.864 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-03-16T08:51:38.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:38 smithi007 ceph-mon[29097]: from='client.? 172.21.15.7:0/3564393383' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T08:51:38.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:38 smithi007 ceph-mon[29097]: from='client.? 172.21.15.7:0/3564393383' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-16T08:51:39.932 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:39 smithi007 ceph-mon[29097]: from='client.? 172.21.15.7:0/259834579' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:51:39.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.272+0000 7f36d4906700 1 Processor -- start 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.272+0000 7f36d4906700 1 -- start start 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.273+0000 7f36d4906700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0105410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.273+0000 7f36d4906700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0076ee0 con 0x7f36d0076ad0 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.273+0000 7f36ce59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.273+0000 7f36ce59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0105410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42868/0 (socket says 172.21.15.7:42868) 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.273+0000 7f36ce59c700 1 -- 172.21.15.7:0/2943233781 learned_addr learned my addr 172.21.15.7:0/2943233781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.273+0000 7f36ce59c700 1 -- 172.21.15.7:0/2943233781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d0105950 con 0x7f36d0076ad0 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.273+0000 7f36ce59c700 1 --2- 172.21.15.7:0/2943233781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0105410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f36b80098e0 tx=0x7f36b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9f7b62f63fc1de server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.273+0000 7f36cd59a700 1 -- 172.21.15.7:0/2943233781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36b800b860 con 0x7f36d0076ad0 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36cd59a700 1 -- 172.21.15.7:0/2943233781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f36b800be50 con 0x7f36d0076ad0 2024-03-16T08:51:39.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36cd59a700 1 -- 172.21.15.7:0/2943233781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36b8010380 con 0x7f36d0076ad0 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36d4906700 1 -- 172.21.15.7:0/2943233781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 msgr2=0x7f36d0105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36d4906700 1 --2- 172.21.15.7:0/2943233781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0105410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f36b80098e0 tx=0x7f36b8009250 comp rx=0 tx=0).stop 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36d4906700 1 -- 172.21.15.7:0/2943233781 shutdown_connections 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36d4906700 1 --2- 172.21.15.7:0/2943233781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0105410 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36d4906700 1 -- 172.21.15.7:0/2943233781 >> 172.21.15.7:0/2943233781 conn(0x7f36d00fcde0 msgr2=0x7f36d00ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36d4906700 1 -- 172.21.15.7:0/2943233781 shutdown_connections 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.274+0000 7f36d4906700 1 -- 172.21.15.7:0/2943233781 wait complete. 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36d4906700 1 Processor -- start 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36d4906700 1 -- start start 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36d4906700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36d4906700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d0111ac0 con 0x7f36d0076ad0 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36ce59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36ce59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42878/0 (socket says 172.21.15.7:42878) 2024-03-16T08:51:39.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36ce59c700 1 -- 172.21.15.7:0/259834579 learned_addr learned my addr 172.21.15.7:0/259834579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36ce59c700 1 -- 172.21.15.7:0/259834579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b80094f0 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.275+0000 7f36ce59c700 1 --2- 172.21.15.7:0/259834579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0111580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f36b800b440 tx=0x7f36b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.276+0000 7f36c6ffd700 1 -- 172.21.15.7:0/259834579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36b8010760 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.276+0000 7f36c6ffd700 1 -- 172.21.15.7:0/259834579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f36b8010d50 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.276+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d0111cc0 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.276+0000 7f36c6ffd700 1 -- 172.21.15.7:0/259834579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36b801adc0 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.276+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d0112830 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.276+0000 7f36c6ffd700 1 -- 172.21.15.7:0/259834579 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f36b8019690 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.276+0000 7f36c6ffd700 1 -- 172.21.15.7:0/259834579 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f36b8024900 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.277+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d005f5f0 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.279+0000 7f36c6ffd700 1 -- 172.21.15.7:0/259834579 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f36b8029060 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.347+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f36d0106790 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.348+0000 7f36c6ffd700 1 -- 172.21.15.7:0/259834579 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f36b801a5f0 con 0x7f36d0076ad0 2024-03-16T08:51:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.348+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 msgr2=0x7f36d0111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:39.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.348+0000 7f36d4906700 1 --2- 172.21.15.7:0/259834579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0111580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f36b800b440 tx=0x7f36b8009250 comp rx=0 tx=0).stop 2024-03-16T08:51:39.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.349+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 shutdown_connections 2024-03-16T08:51:39.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.349+0000 7f36d4906700 1 --2- 172.21.15.7:0/259834579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36d0076ad0 0x7f36d0111580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:39.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.349+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 >> 172.21.15.7:0/259834579 conn(0x7f36d00fcde0 msgr2=0x7f36d00fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:39.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.349+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 shutdown_connections 2024-03-16T08:51:39.942 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:39.349+0000 7f36d4906700 1 -- 172.21.15.7:0/259834579 wait complete. 2024-03-16T08:51:39.942 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-03-16T08:51:40.211 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:39 smithi007 systemd[1]: Stopping Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T08:51:40.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:40 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[29093]: 2024-03-16T08:51:40.208+0000 7fab176f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi007 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T08:51:40.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:40 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[29093]: 2024-03-16T08:51:40.208+0000 7fab176f5700 -1 mon.smithi007@0(leader) e1 *** Got Signal Terminated *** 2024-03-16T08:51:40.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:40 smithi007 podman[29343]: 2024-03-16 08:51:40.362744064 +0000 UTC m=+0.266064483 container died 6e05b8685c206fe65eb79d9c3fedd00a240cb26745089fca85f288e7c6d24302 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-1df2724, io.buildah.version=1.33.5) 2024-03-16T08:51:40.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:40 smithi007 podman[29343]: 2024-03-16 08:51:40.708492204 +0000 UTC m=+0.611812622 container cleanup 6e05b8685c206fe65eb79d9c3fedd00a240cb26745089fca85f288e7c6d24302 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-1df2724, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T08:51:40.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:40 smithi007 bash[29343]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007 2024-03-16T08:51:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:40 smithi007 podman[29357]: 2024-03-16 08:51:40.959390322 +0000 UTC m=+0.594800997 container remove 6e05b8685c206fe65eb79d9c3fedd00a240cb26745089fca85f288e7c6d24302 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=quincy-1df2724, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-16T08:51:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007.service: Deactivated successfully. 2024-03-16T08:51:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 systemd[1]: Stopped Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T08:51:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 systemd[1]: Starting Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T08:51:41.526 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-16T08:51:41.571 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 podman[29451]: 2024-03-16 08:51:41.27684334 +0000 UTC m=+0.100035615 container create aaa292e58c256d214655e6adedfb221c3fc3223fc8c21bbd025f68a28346176c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-1df2724, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 podman[29451]: 2024-03-16 08:51:41.194070565 +0000 UTC m=+0.017262844 image pull 0bdcfccf14ced818364a26df730f679c4000432cea6541bcb4a6dbd30237ef40 quay.ceph.io/ceph-ci/ceph:quincy 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 podman[29451]: 2024-03-16 08:51:41.471058438 +0000 UTC m=+0.294250733 container init aaa292e58c256d214655e6adedfb221c3fc3223fc8c21bbd025f68a28346176c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, RELEASE=quincy-1df2724, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 podman[29451]: 2024-03-16 08:51:41.474087947 +0000 UTC m=+0.297280225 container start aaa292e58c256d214655e6adedfb221c3fc3223fc8c21bbd025f68a28346176c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=quincy-1df2724, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable), process ceph-mon, pid 2 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: pidfile_write: ignore empty --pid-file 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: load: jerasure load: lrc 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: RocksDB version: 6.15.5 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Compile date Mar 15 2024 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: DB SUMMARY 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: DB Session ID: 45AX9G65J1Q4F8PVJ3V4 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: CURRENT file: CURRENT 2024-03-16T08:51:41.572 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T08:51:41.573 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-16T08:51:41.573 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi007/store.db dir, Total Num: 1, files: 000008.sst 2024-03-16T08:51:41.573 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi007/store.db: 000010.log size: 97246 ; 2024-03-16T08:51:41.573 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.error_if_exists: 0 2024-03-16T08:51:41.573 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.create_if_missing: 0 2024-03-16T08:51:41.573 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.paranoid_checks: 1 2024-03-16T08:51:41.574 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T08:51:41.574 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.env: 0x55abba184a80 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.fs: Posix File System 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.info_log: 0x55abbb543c00 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.statistics: (nil) 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.use_fsync: 0 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_log_file_size: 0 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.allow_fallocate: 1 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.use_direct_reads: 0 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T08:51:41.575 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.db_log_dir: 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi007/store.db 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.write_buffer_manager: 0x55abbc2b8ab0 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T08:51:41.576 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.unordered_write: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.row_cache: None 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.wal_filter: None 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.preserve_deletes: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.two_write_queues: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T08:51:41.577 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.atomic_flush: 0 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.log_readahead_size: 0 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_background_jobs: 2 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_background_compactions: -1 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_subcompactions: 1 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T08:51:41.578 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_open_files: -1 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_background_flushes: -1 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Compression algorithms supported: 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: kZSTD supported: 0 2024-03-16T08:51:41.579 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: kXpressCompression supported: 0 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: kZlibCompression supported: 1 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: kSnappyCompression supported: 1 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: kLZ4Compression supported: 1 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: kBZip2Compression supported: 0 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi007/store.db/MANIFEST-000009 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.merge_operator: 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_filter: None 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T08:51:41.580 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55abbb598f58) 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_type: 0 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_block_index_type: 0 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_shortening: 1 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: hash_index_allow_collision: 1 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: checksum: 1 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: no_block_cache: 0 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache: 0x55abbb5cb090 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_options: 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: capacity : 536870912 2024-03-16T08:51:41.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: num_shard_bits : 4 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: strict_capacity_limit : 0 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_compressed: (nil) 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: persistent_cache: (nil) 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_size: 4096 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_size_deviation: 10 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_restart_interval: 16 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_block_restart_interval: 1 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: metadata_block_size: 4096 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: partition_filters: 0 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: use_delta_encoding: 1 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: whole_key_filtering: 1 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: verify_compression: 0 2024-03-16T08:51:41.582 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: format_version: 4 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: enable_index_compression: 1 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_align: 0 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compression: NoCompression 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.num_levels: 7 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T08:51:41.583 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T08:51:41.584 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T08:51:41.585 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.arena_block_size: 4194304 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T08:51:41.586 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.table_properties_collectors: 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.inplace_update_support: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.bloom_locality: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.max_successive_merges: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.ttl: 2592000 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.enable_blob_files: false 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.min_blob_size: 0 2024-03-16T08:51:41.587 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi007/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579101499791, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579101500613, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92518, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90613, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83328, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1710579101, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "a672d714-f643-4b7b-b813-2bc1dbcde421", "db_session_id": "45AX9G65J1Q4F8PVJ3V4"}} 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579101500947, "job": 1, "event": "recovery_finished"} 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi007/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55abbb55f180 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: DB pointer 0x55abbb634000 2024-03-16T08:51:41.588 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** DB Stats ** 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: L0 2/0 92.15 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.0 0.00 0.00 1 0.001 0 0 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Sum 2/0 92.15 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.0 0.00 0.00 1 0.001 0 0 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.0 0.00 0.00 1 0.001 0 0 2024-03-16T08:51:41.589 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 154.0 0.00 0.00 1 0.001 0 0 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative compaction: 0.00 GB write, 29.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval compaction: 0.00 GB write, 29.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T08:51:41.590 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: L0 2/0 92.15 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.0 0.00 0.00 1 0.001 0 0 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Sum 2/0 92.15 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.0 0.00 0.00 1 0.001 0 0 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 154.0 0.00 0.00 1 0.001 0 0 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:51:41.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative compaction: 0.00 GB write, 29.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: starting mon.smithi007 rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi007 fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007@-1(???) e1 preinit fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007@-1(???).mds e1 new map 2024-03-16T08:51:41.592 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007@-1(???).mds e1 print_map 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: e1 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: legacy client fscid: -1 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: No filesystems configured 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 bash[29451]: aaa292e58c256d214655e6adedfb221c3fc3223fc8c21bbd025f68a28346176c 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 systemd[1]: Started Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mon.smithi007 is new leader, mons smithi007 in quorum (ranks 0) 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: monmap e1: 1 mons at {smithi007=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-16T08:51:41.593 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: fsmap 2024-03-16T08:51:41.594 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: osdmap e1: 0 total, 0 up, 0 in 2024-03-16T08:51:41.594 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:41 smithi007 ceph-mon[29470]: mgrmap e1: no daemons active 2024-03-16T08:51:42.631 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.007+0000 7fd90b79d700 1 Processor -- start 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.007+0000 7fd90b79d700 1 -- start start 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.007+0000 7fd90b79d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.007+0000 7fd90b79d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9041030c0 con 0x7fd904102770 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.008+0000 7fd909539700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.008+0000 7fd909539700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42894/0 (socket says 172.21.15.7:42894) 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.008+0000 7fd909539700 1 -- 172.21.15.7:0/1889468892 learned_addr learned my addr 172.21.15.7:0/1889468892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.008+0000 7fd909539700 1 -- 172.21.15.7:0/1889468892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904103200 con 0x7fd904102770 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.008+0000 7fd909539700 1 --2- 172.21.15.7:0/1889468892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0098e0 tx=0x7fd8fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed6def17e32b9a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.008+0000 7fd903fff700 1 -- 172.21.15.7:0/1889468892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8fc00b860 con 0x7fd904102770 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.008+0000 7fd903fff700 1 -- 172.21.15.7:0/1889468892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd8fc00be50 con 0x7fd904102770 2024-03-16T08:51:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.008+0000 7fd903fff700 1 -- 172.21.15.7:0/1889468892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8fc010380 con 0x7fd904102770 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.009+0000 7fd90b79d700 1 -- 172.21.15.7:0/1889468892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 msgr2=0x7fd904102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.009+0000 7fd90b79d700 1 --2- 172.21.15.7:0/1889468892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0098e0 tx=0x7fd8fc009250 comp rx=0 tx=0).stop 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.009+0000 7fd90b79d700 1 -- 172.21.15.7:0/1889468892 shutdown_connections 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.009+0000 7fd90b79d700 1 --2- 172.21.15.7:0/1889468892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904102b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.009+0000 7fd90b79d700 1 -- 172.21.15.7:0/1889468892 >> 172.21.15.7:0/1889468892 conn(0x7fd9040fde90 msgr2=0x7fd904100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.009+0000 7fd90b79d700 1 -- 172.21.15.7:0/1889468892 shutdown_connections 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.009+0000 7fd90b79d700 1 -- 172.21.15.7:0/1889468892 wait complete. 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd90b79d700 1 Processor -- start 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd90b79d700 1 -- start start 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd90b79d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd90b79d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904111ac0 con 0x7fd904102770 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd909539700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd909539700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42902/0 (socket says 172.21.15.7:42902) 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd909539700 1 -- 172.21.15.7:0/1572422932 learned_addr learned my addr 172.21.15.7:0/1572422932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd909539700 1 -- 172.21.15.7:0/1572422932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0094f0 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd909539700 1 --2- 172.21.15.7:0/1572422932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904111580 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00b440 tx=0x7fd8fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.010+0000 7fd901ffb700 1 -- 172.21.15.7:0/1572422932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8fc010760 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.011+0000 7fd901ffb700 1 -- 172.21.15.7:0/1572422932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd8fc010d50 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.011+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd904111cc0 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.011+0000 7fd901ffb700 1 -- 172.21.15.7:0/1572422932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8fc01ae20 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.011+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd904112830 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.011+0000 7fd901ffb700 1 -- 172.21.15.7:0/1572422932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd8fc02e6c0 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.011+0000 7fd901ffb700 1 -- 172.21.15.7:0/1572422932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd8fc0248c0 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.012+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90405f5f0 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.014+0000 7fd901ffb700 1 -- 172.21.15.7:0/1572422932 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd8fc029070 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.082+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fd904104540 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.085+0000 7fd901ffb700 1 -- 172.21.15.7:0/1572422932 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd8fc01a6a0 con 0x7fd904102770 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.085+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 msgr2=0x7fd904111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.085+0000 7fd90b79d700 1 --2- 172.21.15.7:0/1572422932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904111580 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00b440 tx=0x7fd8fc009250 comp rx=0 tx=0).stop 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.085+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 shutdown_connections 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.085+0000 7fd90b79d700 1 --2- 172.21.15.7:0/1572422932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd904102770 0x7fd904111580 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.085+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 >> 172.21.15.7:0/1572422932 conn(0x7fd9040fde90 msgr2=0x7fd9040feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.085+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 shutdown_connections 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:42.085+0000 7fd90b79d700 1 -- 172.21.15.7:0/1572422932 wait complete. 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-03-16T08:51:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 9283 ... 2024-03-16T08:51:42.806 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mgr.smithi007.uamwtp 2024-03-16T08:51:42.807 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mgr.smithi007.uamwtp.service: Unit ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mgr.smithi007.uamwtp.service not loaded. 2024-03-16T08:51:42.955 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4c46b77e-e372-11ee-95c9-87774f69a715.target.wants/ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mgr.smithi007.uamwtp.service → /etc/systemd/system/ceph-4c46b77e-e372-11ee-95c9-87774f69a715@.service. 2024-03-16T08:51:43.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:43 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1572422932' entity='client.admin' 2024-03-16T08:51:43.530 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T08:51:43.530 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-16T08:51:43.538 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T08:51:43.538 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-16T08:51:43.538 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-16T08:51:43.539 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T08:51:43.547 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T08:51:43.548 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-16T08:51:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T08:51:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-16T08:51:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-16T08:51:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-16T08:51:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-03-16T08:51:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-03-16T08:51:44.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:44 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3243964018' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "4c46b77e-e372-11ee-95c9-87774f69a715", 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T08:51:44.577 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "smithi007" 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:44.578 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T08:51:44.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T08:51:44.579 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T08:51:44.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T08:51:44.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T08:51:44.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:44.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T08:51:44.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T08:51:36.700509+0000", 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.981+0000 7f2a76083700 1 Processor -- start 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.981+0000 7f2a76083700 1 -- start start 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.981+0000 7f2a76083700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70105390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.981+0000 7f2a76083700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a701058d0 con 0x7f2a70076b40 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.981+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.981+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70105390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57114/0 (socket says 172.21.15.7:57114) 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.981+0000 7f2a6f7fe700 1 -- 172.21.15.7:0/967377218 learned_addr learned my addr 172.21.15.7:0/967377218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.982+0000 7f2a6f7fe700 1 -- 172.21.15.7:0/967377218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70105a10 con 0x7f2a70076b40 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.982+0000 7f2a6f7fe700 1 --2- 172.21.15.7:0/967377218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70105390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a600098e0 tx=0x7f2a60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c4ad6dc39b2ee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.982+0000 7f2a6e7fc700 1 -- 172.21.15.7:0/967377218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a6000b860 con 0x7f2a70076b40 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.982+0000 7f2a6e7fc700 1 -- 172.21.15.7:0/967377218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2a6000be50 con 0x7f2a70076b40 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.982+0000 7f2a6e7fc700 1 -- 172.21.15.7:0/967377218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a60010380 con 0x7f2a70076b40 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 -- 172.21.15.7:0/967377218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 msgr2=0x7f2a70105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 --2- 172.21.15.7:0/967377218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70105390 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2a600098e0 tx=0x7f2a60009250 comp rx=0 tx=0).stop 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 -- 172.21.15.7:0/967377218 shutdown_connections 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 --2- 172.21.15.7:0/967377218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70105390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 -- 172.21.15.7:0/967377218 >> 172.21.15.7:0/967377218 conn(0x7f2a700fcd80 msgr2=0x7f2a700ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 -- 172.21.15.7:0/967377218 shutdown_connections 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 -- 172.21.15.7:0/967377218 wait complete. 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 Processor -- start 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.983+0000 7f2a76083700 1 -- start start 2024-03-16T08:51:44.591 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a76083700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a76083700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a70103220 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57126/0 (socket says 172.21.15.7:57126) 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a6f7fe700 1 -- 172.21.15.7:0/3243964018 learned_addr learned my addr 172.21.15.7:0/3243964018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a6f7fe700 1 -- 172.21.15.7:0/3243964018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600094f0 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a6f7fe700 1 --2- 172.21.15.7:0/3243964018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70102ce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a60006b00 tx=0x7f2a60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a57fff700 1 -- 172.21.15.7:0/3243964018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a600107c0 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a57fff700 1 -- 172.21.15.7:0/3243964018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2a60010db0 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a70101290 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a57fff700 1 -- 172.21.15.7:0/3243964018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2a6001acf0 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.984+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a70101780 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.985+0000 7f2a57fff700 1 -- 172.21.15.7:0/3243964018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2a60024440 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.985+0000 7f2a57fff700 1 -- 172.21.15.7:0/3243964018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2a60019710 con 0x7f2a70076b40 2024-03-16T08:51:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.985+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7004bed0 con 0x7f2a70076b40 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:43.988+0000 7f2a57fff700 1 -- 172.21.15.7:0/3243964018 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2a6001a530 con 0x7f2a70076b40 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.060+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2a70101a20 con 0x7f2a70076b40 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.060+0000 7f2a57fff700 1 -- 172.21.15.7:0/3243964018 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f2a60010920 con 0x7f2a70076b40 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.061+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 msgr2=0x7f2a70102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.061+0000 7f2a76083700 1 --2- 172.21.15.7:0/3243964018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70102ce0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2a60006b00 tx=0x7f2a60009250 comp rx=0 tx=0).stop 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.061+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 shutdown_connections 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.061+0000 7f2a76083700 1 --2- 172.21.15.7:0/3243964018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a70076b40 0x7f2a70102ce0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.061+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 >> 172.21.15.7:0/3243964018 conn(0x7f2a700fcd80 msgr2=0x7f2a700fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.062+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 shutdown_connections 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:44.062+0000 7f2a76083700 1 -- 172.21.15.7:0/3243964018 wait complete. 2024-03-16T08:51:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-03-16T08:51:47.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:47 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3813493635' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T08:51:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: Activating manager daemon smithi007.uamwtp 2024-03-16T08:51:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: mgrmap e2: smithi007.uamwtp(active, starting, since 0.00285043s) 2024-03-16T08:51:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:51:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:51:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:51:50.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:51:50.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T08:51:50.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: Manager daemon smithi007.uamwtp is now available 2024-03-16T08:51:50.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:51:50.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T08:51:50.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' 2024-03-16T08:51:50.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' 2024-03-16T08:51:50.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:50 smithi007 ceph-mon[29470]: from='mgr.14100 172.21.15.7:0/3156124424' entity='mgr.smithi007.uamwtp' 2024-03-16T08:51:52.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:52 smithi007 ceph-mon[29470]: mgrmap e3: smithi007.uamwtp(active, since 1.00666s) 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "4c46b77e-e372-11ee-95c9-87774f69a715", 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T08:51:52.592 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "smithi007" 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T08:51:52.593 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T08:51:52.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T08:51:52.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T08:51:52.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:52.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:51:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T08:51:36.700509+0000", 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.027+0000 7f2f3024b700 1 Processor -- start 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.027+0000 7f2f3024b700 1 -- start start 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.027+0000 7f2f3024b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f28101af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.027+0000 7f2f3024b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f28102030 con 0x7f2f281016e0 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.027+0000 7f2f2dfe7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f28101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.027+0000 7f2f2dfe7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f28101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57132/0 (socket says 172.21.15.7:57132) 2024-03-16T08:51:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.027+0000 7f2f2dfe7700 1 -- 172.21.15.7:0/1809169523 learned_addr learned my addr 172.21.15.7:0/1809169523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.028+0000 7f2f2dfe7700 1 -- 172.21.15.7:0/1809169523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f28102170 con 0x7f2f281016e0 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.028+0000 7f2f2dfe7700 1 --2- 172.21.15.7:0/1809169523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f28101af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400bb80 tx=0x7f2f2400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b48193d07ef64e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.028+0000 7f2f2cfe5700 1 -- 172.21.15.7:0/1809169523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f2400dd00 con 0x7f2f281016e0 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.028+0000 7f2f2cfe5700 1 -- 172.21.15.7:0/1809169523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2f2400de60 con 0x7f2f281016e0 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.028+0000 7f2f2cfe5700 1 -- 172.21.15.7:0/1809169523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f240188a0 con 0x7f2f281016e0 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 -- 172.21.15.7:0/1809169523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 msgr2=0x7f2f28101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 --2- 172.21.15.7:0/1809169523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f28101af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400bb80 tx=0x7f2f2400b4f0 comp rx=0 tx=0).stop 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 -- 172.21.15.7:0/1809169523 shutdown_connections 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 --2- 172.21.15.7:0/1809169523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f28101af0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 -- 172.21.15.7:0/1809169523 >> 172.21.15.7:0/1809169523 conn(0x7f2f280fce40 msgr2=0x7f2f280ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 -- 172.21.15.7:0/1809169523 shutdown_connections 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 -- 172.21.15.7:0/1809169523 wait complete. 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 Processor -- start 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.029+0000 7f2f3024b700 1 -- start start 2024-03-16T08:51:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f3024b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f2810f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f3024b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2810f850 con 0x7f2f281016e0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f2dfe7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f2810f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f2dfe7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f2810f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57136/0 (socket says 172.21.15.7:57136) 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f2dfe7700 1 -- 172.21.15.7:0/3813493635 learned_addr learned my addr 172.21.15.7:0/3813493635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f2dfe7700 1 -- 172.21.15.7:0/3813493635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2400b790 con 0x7f2f281016e0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f2dfe7700 1 --2- 172.21.15.7:0/3813493635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f2810f310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f24018600 tx=0x7f2f2400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f1e7fc700 1 -- 172.21.15.7:0/3813493635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f24010b70 con 0x7f2f281016e0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2810fa50 con 0x7f2f281016e0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f1e7fc700 1 -- 172.21.15.7:0/3813493635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2f2401d050 con 0x7f2f281016e0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.030+0000 7f2f1e7fc700 1 -- 172.21.15.7:0/3813493635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f240225b0 con 0x7f2f281016e0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.031+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f281105c0 con 0x7f2f281016e0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.031+0000 7f2f1e7fc700 1 -- 172.21.15.7:0/3813493635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2f240227b0 con 0x7f2f281016e0 2024-03-16T08:51:52.606 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.031+0000 7f2f1e7fc700 1 -- 172.21.15.7:0/3813493635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2f24014070 con 0x7f2f281016e0 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.031+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2805f5f0 con 0x7f2f281016e0 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.034+0000 7f2f1e7fc700 1 -- 172.21.15.7:0/3813493635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2f24010cd0 con 0x7f2f281016e0 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.107+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2f281034b0 con 0x7f2f281016e0 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.107+0000 7f2f1e7fc700 1 -- 172.21.15.7:0/3813493635 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f2f24023a00 con 0x7f2f281016e0 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.108+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 msgr2=0x7f2f2810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.108+0000 7f2f3024b700 1 --2- 172.21.15.7:0/3813493635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f2810f310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f24018600 tx=0x7f2f2400b4f0 comp rx=0 tx=0).stop 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.108+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 shutdown_connections 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.108+0000 7f2f3024b700 1 --2- 172.21.15.7:0/3813493635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281016e0 0x7f2f2810f310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.108+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 >> 172.21.15.7:0/3813493635 conn(0x7f2f280fce40 msgr2=0x7f2f28105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.108+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 shutdown_connections 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:47.108+0000 7f2f3024b700 1 -- 172.21.15.7:0/3813493635 wait complete. 2024-03-16T08:51:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-03-16T08:51:54.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:54 smithi007 ceph-mon[29470]: mgrmap e4: smithi007.uamwtp(active, since 3s) 2024-03-16T08:51:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:55 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/24365738' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "4c46b77e-e372-11ee-95c9-87774f69a715", 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-16T08:51:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "smithi007" 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T08:51:56.192 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T08:51:56.193 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:56.194 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T08:51:36.700509+0000", 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.398+0000 7fc0042d6700 1 Processor -- start 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.399+0000 7fc0042d6700 1 -- start start 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.399+0000 7fc0042d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.399+0000 7fc0042d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc103070 con 0x7fbffc102720 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.399+0000 7fc002072700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.399+0000 7fc002072700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53228/0 (socket says 172.21.15.7:53228) 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.399+0000 7fc002072700 1 -- 172.21.15.7:0/128471842 learned_addr learned my addr 172.21.15.7:0/128471842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:56.195 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.399+0000 7fc002072700 1 -- 172.21.15.7:0/128471842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc1031b0 con 0x7fbffc102720 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc002072700 1 --2- 172.21.15.7:0/128471842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc102b30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0098e0 tx=0x7fbfec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18df47a7f23609ba server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc001070700 1 -- 172.21.15.7:0/128471842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfec00b860 con 0x7fbffc102720 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc001070700 1 -- 172.21.15.7:0/128471842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbfec00be50 con 0x7fbffc102720 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc001070700 1 -- 172.21.15.7:0/128471842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfec010380 con 0x7fbffc102720 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc0042d6700 1 -- 172.21.15.7:0/128471842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 msgr2=0x7fbffc102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc0042d6700 1 --2- 172.21.15.7:0/128471842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc102b30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0098e0 tx=0x7fbfec009250 comp rx=0 tx=0).stop 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc0042d6700 1 -- 172.21.15.7:0/128471842 shutdown_connections 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc0042d6700 1 --2- 172.21.15.7:0/128471842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc102b30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc0042d6700 1 -- 172.21.15.7:0/128471842 >> 172.21.15.7:0/128471842 conn(0x7fbffc0fdee0 msgr2=0x7fbffc1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc0042d6700 1 -- 172.21.15.7:0/128471842 shutdown_connections 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.400+0000 7fc0042d6700 1 -- 172.21.15.7:0/128471842 wait complete. 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.401+0000 7fc0042d6700 1 Processor -- start 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.401+0000 7fc0042d6700 1 -- start start 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.401+0000 7fc0042d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:56.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.401+0000 7fc0042d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc079110 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.401+0000 7fc002072700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.401+0000 7fc002072700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53234/0 (socket says 172.21.15.7:53234) 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.401+0000 7fc002072700 1 -- 172.21.15.7:0/24365738 learned_addr learned my addr 172.21.15.7:0/24365738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.401+0000 7fc002072700 1 -- 172.21.15.7:0/24365738 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec0094f0 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.402+0000 7fc002072700 1 --2- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc078bd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.402+0000 7fbff2ffd700 1 -- 172.21.15.7:0/24365738 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfec0107c0 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.402+0000 7fbff2ffd700 1 -- 172.21.15.7:0/24365738 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbfec010db0 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.402+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc077180 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.402+0000 7fbff2ffd700 1 -- 172.21.15.7:0/24365738 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfec01ad00 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.402+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc077670 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.403+0000 7fbff2ffd700 1 -- 172.21.15.7:0/24365738 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fbfec0193c0 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.403+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc04bed0 con 0x7fbffc102720 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.403+0000 7fbff2ffd700 1 --2- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7fbfe8036740 0x7fbfe8038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:56.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.403+0000 7fbff2ffd700 1 -- 172.21.15.7:0/24365738 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbfec04a840 con 0x7fbffc102720 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.403+0000 7fc001871700 1 --2- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7fbfe8036740 0x7fbfe8038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.403+0000 7fc001871700 1 --2- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7fbfe8036740 0x7fbfe8038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009990 tx=0x7fbff8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.408+0000 7fbff2ffd700 1 -- 172.21.15.7:0/24365738 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fbfec02c3a0 con 0x7fbffc102720 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.677+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbffc077dc0 con 0x7fbffc102720 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.677+0000 7fbff2ffd700 1 -- 172.21.15.7:0/24365738 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fbfec022340 con 0x7fbffc102720 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.679+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7fbfe8036740 msgr2=0x7fbfe8038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.679+0000 7fc0042d6700 1 --2- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7fbfe8036740 0x7fbfe8038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009990 tx=0x7fbff8006e40 comp rx=0 tx=0).stop 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.679+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 msgr2=0x7fbffc078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.679+0000 7fc0042d6700 1 --2- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc078bd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec009250 comp rx=0 tx=0).stop 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.680+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 shutdown_connections 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.680+0000 7fc0042d6700 1 --2- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7fbfe8036740 0x7fbfe8038bf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.680+0000 7fc0042d6700 1 --2- 172.21.15.7:0/24365738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc102720 0x7fbffc078bd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.680+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 >> 172.21.15.7:0/24365738 conn(0x7fbffc0fdee0 msgr2=0x7fbffc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:56.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.680+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 shutdown_connections 2024-03-16T08:51:56.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:55.680+0000 7fc0042d6700 1 -- 172.21.15.7:0/24365738 wait complete. 2024-03-16T08:51:56.199 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-03-16T08:51:57.341 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:57.341 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T08:51:57.341 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:51:57.341 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T08:51:57.341 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T08:51:57.341 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T08:51:57.341 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T08:51:57.341 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.592+0000 7ff5e8e84700 1 Processor -- start 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.592+0000 7ff5e8e84700 1 -- start start 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.592+0000 7ff5e8e84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e4101af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.592+0000 7ff5e8e84700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4102030 con 0x7ff5e41016e0 2024-03-16T08:51:57.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.593+0000 7ff5e259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e4101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.593+0000 7ff5e259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e4101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53240/0 (socket says 172.21.15.7:53240) 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.593+0000 7ff5e259c700 1 -- 172.21.15.7:0/147318259 learned_addr learned my addr 172.21.15.7:0/147318259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.593+0000 7ff5e259c700 1 -- 172.21.15.7:0/147318259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4102170 con 0x7ff5e41016e0 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.593+0000 7ff5e259c700 1 --2- 172.21.15.7:0/147318259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e4101af0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40098e0 tx=0x7ff5d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534f04e5845a36e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.593+0000 7ff5e1d9b700 1 -- 172.21.15.7:0/147318259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5d400b860 con 0x7ff5e41016e0 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.593+0000 7ff5e1d9b700 1 -- 172.21.15.7:0/147318259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff5d400be50 con 0x7ff5e41016e0 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.593+0000 7ff5e1d9b700 1 -- 172.21.15.7:0/147318259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5d4010380 con 0x7ff5e41016e0 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 -- 172.21.15.7:0/147318259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 msgr2=0x7ff5e4101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 --2- 172.21.15.7:0/147318259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e4101af0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40098e0 tx=0x7ff5d4009250 comp rx=0 tx=0).stop 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 -- 172.21.15.7:0/147318259 shutdown_connections 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 --2- 172.21.15.7:0/147318259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e4101af0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 -- 172.21.15.7:0/147318259 >> 172.21.15.7:0/147318259 conn(0x7ff5e40fce40 msgr2=0x7ff5e40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 -- 172.21.15.7:0/147318259 shutdown_connections 2024-03-16T08:51:57.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 -- 172.21.15.7:0/147318259 wait complete. 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 Processor -- start 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.594+0000 7ff5e8e84700 1 -- start start 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e8e84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e41115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e8e84700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4111af0 con 0x7ff5e41016e0 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e41115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e41115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53250/0 (socket says 172.21.15.7:53250) 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e259c700 1 -- 172.21.15.7:0/98130793 learned_addr learned my addr 172.21.15.7:0/98130793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e259c700 1 -- 172.21.15.7:0/98130793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d40094f0 con 0x7ff5e41016e0 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e259c700 1 --2- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e41115b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400ba90 tx=0x7ff5d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5daffd700 1 -- 172.21.15.7:0/98130793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5d40107c0 con 0x7ff5e41016e0 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5daffd700 1 -- 172.21.15.7:0/98130793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff5d4010db0 con 0x7ff5e41016e0 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5daffd700 1 -- 172.21.15.7:0/98130793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5d401add0 con 0x7ff5e41016e0 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e4111cf0 con 0x7ff5e41016e0 2024-03-16T08:51:57.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.595+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e4112860 con 0x7ff5e41016e0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.596+0000 7ff5daffd700 1 -- 172.21.15.7:0/98130793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7ff5d4010920 con 0x7ff5e41016e0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.596+0000 7ff5daffd700 1 --2- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7ff5d0036790 0x7ff5d0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.596+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e405f5f0 con 0x7ff5e41016e0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.596+0000 7ff5daffd700 1 -- 172.21.15.7:0/98130793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff5d404a5a0 con 0x7ff5e41016e0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.596+0000 7ff5dbfff700 1 --2- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7ff5d0036790 0x7ff5d0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.597+0000 7ff5dbfff700 1 --2- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7ff5d0036790 0x7ff5d0038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009990 tx=0x7ff5cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.601+0000 7ff5daffd700 1 -- 172.21.15.7:0/98130793 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7ff5d401a430 con 0x7ff5e41016e0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.792+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff5e404bed0 con 0x7ff5e41016e0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.793+0000 7ff5daffd700 1 -- 172.21.15.7:0/98130793 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff5d4018350 con 0x7ff5e41016e0 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7ff5d0036790 msgr2=0x7ff5d0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 --2- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7ff5d0036790 0x7ff5d0038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009990 tx=0x7ff5cc006e40 comp rx=0 tx=0).stop 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 msgr2=0x7ff5e41115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 --2- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e41115b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400ba90 tx=0x7ff5d4009250 comp rx=0 tx=0).stop 2024-03-16T08:51:57.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 shutdown_connections 2024-03-16T08:51:57.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 --2- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7ff5d0036790 0x7ff5d0038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:57.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 --2- 172.21.15.7:0/98130793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5e41016e0 0x7ff5e41115b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:57.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 >> 172.21.15.7:0/98130793 conn(0x7ff5e40fce40 msgr2=0x7ff5e40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:57.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 shutdown_connections 2024-03-16T08:51:57.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:56.795+0000 7ff5e8e84700 1 -- 172.21.15.7:0/98130793 wait complete. 2024-03-16T08:51:57.346 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-03-16T08:51:57.709 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:57 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/98130793' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T08:51:58.761 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:58 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3957176692' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-16T08:51:59.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.707+0000 7f846892e700 1 Processor -- start 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.707+0000 7f846892e700 1 -- start start 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.707+0000 7f846892e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f8464101a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.707+0000 7f846892e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464101f70 con 0x7f8464101620 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.707+0000 7f846259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f8464101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.707+0000 7f846259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f8464101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53266/0 (socket says 172.21.15.7:53266) 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.707+0000 7f846259c700 1 -- 172.21.15.7:0/2520946667 learned_addr learned my addr 172.21.15.7:0/2520946667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.708+0000 7f846259c700 1 -- 172.21.15.7:0/2520946667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84641020b0 con 0x7f8464101620 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.708+0000 7f846259c700 1 --2- 172.21.15.7:0/2520946667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f8464101a30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f845400bb80 tx=0x7f845400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36f64345e7da084 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:59.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.708+0000 7f846159a700 1 -- 172.21.15.7:0/2520946667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f845400dd00 con 0x7f8464101620 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.708+0000 7f846159a700 1 -- 172.21.15.7:0/2520946667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f845400de60 con 0x7f8464101620 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.708+0000 7f846159a700 1 -- 172.21.15.7:0/2520946667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84540188a0 con 0x7f8464101620 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.709+0000 7f846892e700 1 -- 172.21.15.7:0/2520946667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 msgr2=0x7f8464101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.709+0000 7f846892e700 1 --2- 172.21.15.7:0/2520946667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f8464101a30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f845400bb80 tx=0x7f845400b4f0 comp rx=0 tx=0).stop 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.709+0000 7f846892e700 1 -- 172.21.15.7:0/2520946667 shutdown_connections 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.709+0000 7f846892e700 1 --2- 172.21.15.7:0/2520946667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f8464101a30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.709+0000 7f846892e700 1 -- 172.21.15.7:0/2520946667 >> 172.21.15.7:0/2520946667 conn(0x7f84640fcdc0 msgr2=0x7f84640ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.709+0000 7f846892e700 1 -- 172.21.15.7:0/2520946667 shutdown_connections 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.709+0000 7f846892e700 1 -- 172.21.15.7:0/2520946667 wait complete. 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846892e700 1 Processor -- start 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846892e700 1 -- start start 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846892e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f84641136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846892e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464113c10 con 0x7f8464101620 2024-03-16T08:51:59.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f84641136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f84641136d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53282/0 (socket says 172.21.15.7:53282) 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846259c700 1 -- 172.21.15.7:0/3957176692 learned_addr learned my addr 172.21.15.7:0/3957176692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846259c700 1 -- 172.21.15.7:0/3957176692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845400b790 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.710+0000 7f846259c700 1 --2- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f84641136d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8454010fa0 tx=0x7f845400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.711+0000 7f8452ffd700 1 -- 172.21.15.7:0/3957176692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84540188a0 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.711+0000 7f8452ffd700 1 -- 172.21.15.7:0/3957176692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8454010c90 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.711+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8464113e10 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.711+0000 7f8452ffd700 1 -- 172.21.15.7:0/3957176692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8454021510 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.711+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8464114980 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.712+0000 7f8452ffd700 1 -- 172.21.15.7:0/3957176692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f84540223f0 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.712+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846405f5f0 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.712+0000 7f8452ffd700 1 --2- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f844c036790 0x7f844c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.712+0000 7f8452ffd700 1 -- 172.21.15.7:0/3957176692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8454014070 con 0x7f8464101620 2024-03-16T08:51:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.712+0000 7f8461d9b700 1 --2- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f844c036790 0x7f844c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.712+0000 7f8461d9b700 1 --2- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f844c036790 0x7f844c038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8458009990 tx=0x7f8458006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.717+0000 7f8452ffd700 1 -- 172.21.15.7:0/3957176692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f84540226a0 con 0x7f8464101620 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:57.965+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f846404bed0 con 0x7f8464101620 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.511+0000 7f8452ffd700 1 -- 172.21.15.7:0/3957176692 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f845402b400 con 0x7f8464101620 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.512+0000 7f8452ffd700 1 -- 172.21.15.7:0/3957176692 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f845400a820 con 0x7f8464101620 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f844c036790 msgr2=0x7f844c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 --2- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f844c036790 0x7f844c038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8458009990 tx=0x7f8458006e40 comp rx=0 tx=0).stop 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 msgr2=0x7f84641136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 --2- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f84641136d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8454010fa0 tx=0x7f845400b4f0 comp rx=0 tx=0).stop 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 shutdown_connections 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 --2- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f844c036790 0x7f844c038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 --2- 172.21.15.7:0/3957176692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8464101620 0x7f84641136d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 >> 172.21.15.7:0/3957176692 conn(0x7f84640fcdc0 msgr2=0x7f8464105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 shutdown_connections 2024-03-16T08:51:59.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:58.515+0000 7f846892e700 1 -- 172.21.15.7:0/3957176692 wait complete. 2024-03-16T08:51:59.729 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:59 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3957176692' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-16T08:51:59.729 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:51:59 smithi007 ceph-mon[29470]: mgrmap e5: smithi007.uamwtp(active, since 8s) 2024-03-16T08:52:00.658 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:00 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3523289362' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T08:52:02.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:52:02.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-16T08:52:02.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "smithi007.uamwtp", 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.521+0000 7f000ae5c700 1 Processor -- start 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.521+0000 7f000ae5c700 1 -- start start 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.521+0000 7f000ae5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004102b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.521+0000 7f000ae5c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004103060 con 0x7f0004102710 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.521+0000 7f0008bf8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.521+0000 7f0008bf8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53318/0 (socket says 172.21.15.7:53318) 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.521+0000 7f0008bf8700 1 -- 172.21.15.7:0/803895435 learned_addr learned my addr 172.21.15.7:0/803895435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.522+0000 7f0008bf8700 1 -- 172.21.15.7:0/803895435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00041031a0 con 0x7f0004102710 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.522+0000 7f0008bf8700 1 --2- 172.21.15.7:0/803895435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004102b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efff80098e0 tx=0x7efff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec218f2d6e24fb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.522+0000 7f00037fe700 1 -- 172.21.15.7:0/803895435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efff800b860 con 0x7f0004102710 2024-03-16T08:52:02.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.522+0000 7f00037fe700 1 -- 172.21.15.7:0/803895435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efff800be50 con 0x7f0004102710 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.522+0000 7f00037fe700 1 -- 172.21.15.7:0/803895435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efff8010380 con 0x7f0004102710 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 -- 172.21.15.7:0/803895435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 msgr2=0x7f0004102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 --2- 172.21.15.7:0/803895435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004102b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efff80098e0 tx=0x7efff8009250 comp rx=0 tx=0).stop 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 -- 172.21.15.7:0/803895435 shutdown_connections 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 --2- 172.21.15.7:0/803895435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004102b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 -- 172.21.15.7:0/803895435 >> 172.21.15.7:0/803895435 conn(0x7f00040fdef0 msgr2=0x7f0004100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 -- 172.21.15.7:0/803895435 shutdown_connections 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 -- 172.21.15.7:0/803895435 wait complete. 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 Processor -- start 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.523+0000 7f000ae5c700 1 -- start start 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f000ae5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004111560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f000ae5c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004111aa0 con 0x7f0004102710 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f0008bf8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004111560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:02.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f0008bf8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004111560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53332/0 (socket says 172.21.15.7:53332) 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f0008bf8700 1 -- 172.21.15.7:0/3523289362 learned_addr learned my addr 172.21.15.7:0/3523289362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f0008bf8700 1 -- 172.21.15.7:0/3523289362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff80094f0 con 0x7f0004102710 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f0008bf8700 1 --2- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004111560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efff800b440 tx=0x7efff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f00017fa700 1 -- 172.21.15.7:0/3523289362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efff8010760 con 0x7f0004102710 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.524+0000 7f00017fa700 1 -- 172.21.15.7:0/3523289362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efff8010d50 con 0x7f0004102710 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.525+0000 7f00017fa700 1 -- 172.21.15.7:0/3523289362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efff801ae20 con 0x7f0004102710 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.525+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0004111ca0 con 0x7f0004102710 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.525+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0004112810 con 0x7f0004102710 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.525+0000 7f00017fa700 1 -- 172.21.15.7:0/3523289362 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7efff802e440 con 0x7f0004102710 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.526+0000 7f00017fa700 1 --2- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7efff40367e0 0x7efff4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.526+0000 7f00017fa700 1 -- 172.21.15.7:0/3523289362 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7efff804a830 con 0x7f0004102710 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.526+0000 7f0003fff700 1 -- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7efff40367e0 msgr2=0x7efff4038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/490303564 2024-03-16T08:52:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.526+0000 7f0003fff700 1 --2- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7efff40367e0 0x7efff4038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.526+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000405f5f0 con 0x7f0004102710 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.531+0000 7f00017fa700 1 -- 172.21.15.7:0/3523289362 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7efff8018940 con 0x7f0004102710 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.727+0000 7f0003fff700 1 -- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7efff40367e0 msgr2=0x7efff4038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/490303564 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.727+0000 7f0003fff700 1 --2- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7efff40367e0 0x7efff4038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.779+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0004112a50 con 0x7f0004102710 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.780+0000 7f00017fa700 1 -- 172.21.15.7:0/3523289362 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7efff8018350 con 0x7f0004102710 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7efff40367e0 msgr2=0x7efff4038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 --2- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7efff40367e0 0x7efff4038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 msgr2=0x7f0004111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 --2- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004111560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efff800b440 tx=0x7efff8009250 comp rx=0 tx=0).stop 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 shutdown_connections 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 --2- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7efff40367e0 0x7efff4038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 --2- 172.21.15.7:0/3523289362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0004102710 0x7f0004111560 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 >> 172.21.15.7:0/3523289362 conn(0x7f00040fdef0 msgr2=0x7f00040fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 shutdown_connections 2024-03-16T08:52:02.128 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:51:59.782+0000 7f000ae5c700 1 -- 172.21.15.7:0/3523289362 wait complete. 2024-03-16T08:52:02.128 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-03-16T08:52:02.128 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: Active manager daemon smithi007.uamwtp restarted 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: Activating manager daemon smithi007.uamwtp 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: osdmap e2: 0 total, 0 up, 0 in 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: mgrmap e6: smithi007.uamwtp(active, starting, since 0.00316092s) 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:52:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: Manager daemon smithi007.uamwtp is now available 2024-03-16T08:52:05.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:05.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:05.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:52:05.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:52:05.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:05 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:52:06.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:06 smithi007 ceph-mon[29470]: Found migration_current of "None". Setting to last migration. 2024-03-16T08:52:06.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:06 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:52:06.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:06 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T08:52:06.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:06 smithi007 ceph-mon[29470]: mgrmap e7: smithi007.uamwtp(active, since 1.00498s) 2024-03-16T08:52:06.795 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.543+0000 7f5781f8a700 1 Processor -- start 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.543+0000 7f5781f8a700 1 -- start start 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f5781f8a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f5781f8a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c103070 con 0x7f577c102720 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38238/0 (socket says 172.21.15.7:38238) 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f577b7fe700 1 -- 172.21.15.7:0/2201499142 learned_addr learned my addr 172.21.15.7:0/2201499142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f577b7fe700 1 -- 172.21.15.7:0/2201499142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c1031b0 con 0x7f577c102720 2024-03-16T08:52:06.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f577b7fe700 1 --2- 172.21.15.7:0/2201499142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c102b30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57640098e0 tx=0x7f5764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587eb307ea3e39af server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f577a7fc700 1 -- 172.21.15.7:0/2201499142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f576400b860 con 0x7f577c102720 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f577a7fc700 1 -- 172.21.15.7:0/2201499142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f576400be50 con 0x7f577c102720 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.544+0000 7f577a7fc700 1 -- 172.21.15.7:0/2201499142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5764010380 con 0x7f577c102720 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.545+0000 7f5781f8a700 1 -- 172.21.15.7:0/2201499142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 msgr2=0x7f577c102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.545+0000 7f5781f8a700 1 --2- 172.21.15.7:0/2201499142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c102b30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57640098e0 tx=0x7f5764009250 comp rx=0 tx=0).stop 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.545+0000 7f5781f8a700 1 -- 172.21.15.7:0/2201499142 shutdown_connections 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.545+0000 7f5781f8a700 1 --2- 172.21.15.7:0/2201499142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c102b30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.545+0000 7f5781f8a700 1 -- 172.21.15.7:0/2201499142 >> 172.21.15.7:0/2201499142 conn(0x7f577c0fde40 msgr2=0x7f577c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.545+0000 7f5781f8a700 1 -- 172.21.15.7:0/2201499142 shutdown_connections 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.545+0000 7f5781f8a700 1 -- 172.21.15.7:0/2201499142 wait complete. 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f5781f8a700 1 Processor -- start 2024-03-16T08:52:06.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f5781f8a700 1 -- start start 2024-03-16T08:52:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f5781f8a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f5781f8a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c079110 con 0x7f577c102720 2024-03-16T08:52:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38246/0 (socket says 172.21.15.7:38246) 2024-03-16T08:52:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f577b7fe700 1 -- 172.21.15.7:0/2033551031 learned_addr learned my addr 172.21.15.7:0/2033551031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f577b7fe700 1 -- 172.21.15.7:0/2033551031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640094f0 con 0x7f577c102720 2024-03-16T08:52:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.546+0000 7f577b7fe700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c078bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f5764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.547+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57640107c0 con 0x7f577c102720 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.547+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5764010db0 con 0x7f577c102720 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.547+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c077180 con 0x7f577c102720 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.547+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f576401ad00 con 0x7f577c102720 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.547+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c077670 con 0x7f577c102720 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.548+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f57640193c0 con 0x7f577c102720 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.548+0000 7f5773fff700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 0x7f5768038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.548+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 --> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5768039350 con 0x7f5768036790 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.548+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f576404a890 con 0x7f577c102720 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.548+0000 7f577affd700 1 -- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 msgr2=0x7f5768038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/490303564 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.548+0000 7f577affd700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 0x7f5768038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.749+0000 7f577affd700 1 -- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 msgr2=0x7f5768038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/490303564 2024-03-16T08:52:06.799 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:02.749+0000 7f577affd700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 0x7f5768038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:03.150+0000 7f577affd700 1 -- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 msgr2=0x7f5768038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/490303564 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:03.150+0000 7f577affd700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 0x7f5768038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:03.951+0000 7f577affd700 1 -- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 msgr2=0x7f5768038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/490303564 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:03.951+0000 7f577affd700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 0x7f5768038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:05.203+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7f57640198a0 con 0x7f577c102720 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:05.203+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 msgr2=0x7f5768038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:05.203+0000 7f5773fff700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 0x7f5768038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.204+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f5764016d50 con 0x7f577c102720 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.204+0000 7f5773fff700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f5768039fc0 0x7f576803c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.204+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5768039350 con 0x7f5768039fc0 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.204+0000 7f577affd700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f5768039fc0 0x7f576803c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.205+0000 7f577affd700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f5768039fc0 0x7f576803c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f576c0039d0 tx=0x7f576c009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.206+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f5768039350 con 0x7f5768039fc0 2024-03-16T08:52:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f577c04bed0 con 0x7f5768039fc0 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5773fff700 1 -- 172.21.15.7:0/2033551031 <== mgr.14118 v2:172.21.15.7:6800/3435258711 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f577c04bed0 con 0x7f5768039fc0 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f5768039fc0 msgr2=0x7f576803c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f5768039fc0 0x7f576803c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f576c0039d0 tx=0x7f576c009350 comp rx=0 tx=0).stop 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 msgr2=0x7f577c078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c078bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f5764009250 comp rx=0 tx=0).stop 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 shutdown_connections 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f5768039fc0 0x7f576803c3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c102720 0x7f577c078bd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 --2- 172.21.15.7:0/2033551031 >> [v2:172.21.15.7:6800/490303564,v1:172.21.15.7:6801/490303564] conn(0x7f5768036790 0x7f5768038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 >> 172.21.15.7:0/2033551031 conn(0x7f577c0fde40 msgr2=0x7f577c0fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 shutdown_connections 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:06.212+0000 7f5781f8a700 1 -- 172.21.15.7:0/2033551031 wait complete. 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-03-16T08:52:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-03-16T08:52:07.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:07 smithi007 ceph-mon[29470]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T08:52:07.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:07 smithi007 ceph-mon[29470]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T08:52:07.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:07 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:07.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:07 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:52:07.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:07 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:07.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:07 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:52:07.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.193+0000 7f096913b700 1 Processor -- start 2024-03-16T08:52:07.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.193+0000 7f096913b700 1 -- start start 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.193+0000 7f096913b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964105310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.193+0000 7f096913b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0964105850 con 0x7f0964076be0 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.193+0000 7f0962d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964105310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.193+0000 7f0962d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964105310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38310/0 (socket says 172.21.15.7:38310) 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.193+0000 7f0962d9d700 1 -- 172.21.15.7:0/3346104390 learned_addr learned my addr 172.21.15.7:0/3346104390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f0962d9d700 1 -- 172.21.15.7:0/3346104390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0964105990 con 0x7f0964076be0 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f0962d9d700 1 --2- 172.21.15.7:0/3346104390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964105310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09580098e0 tx=0x7f0958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bd8b2d1d04f82e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f0961d9b700 1 -- 172.21.15.7:0/3346104390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f095800b860 con 0x7f0964076be0 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f0961d9b700 1 -- 172.21.15.7:0/3346104390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f095800be50 con 0x7f0964076be0 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f0961d9b700 1 -- 172.21.15.7:0/3346104390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0958010380 con 0x7f0964076be0 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f096913b700 1 -- 172.21.15.7:0/3346104390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 msgr2=0x7f0964105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f096913b700 1 --2- 172.21.15.7:0/3346104390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964105310 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f09580098e0 tx=0x7f0958009250 comp rx=0 tx=0).stop 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f096913b700 1 -- 172.21.15.7:0/3346104390 shutdown_connections 2024-03-16T08:52:07.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f096913b700 1 --2- 172.21.15.7:0/3346104390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964105310 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.194+0000 7f096913b700 1 -- 172.21.15.7:0/3346104390 >> 172.21.15.7:0/3346104390 conn(0x7f09640fcdc0 msgr2=0x7f09640ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.195+0000 7f096913b700 1 -- 172.21.15.7:0/3346104390 shutdown_connections 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.195+0000 7f096913b700 1 -- 172.21.15.7:0/3346104390 wait complete. 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.195+0000 7f096913b700 1 Processor -- start 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.195+0000 7f096913b700 1 -- start start 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.195+0000 7f096913b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964111570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.195+0000 7f096913b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0964111ab0 con 0x7f0964076be0 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f0962d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964111570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f0962d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964111570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38314/0 (socket says 172.21.15.7:38314) 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f0962d9d700 1 -- 172.21.15.7:0/1464033726 learned_addr learned my addr 172.21.15.7:0/1464033726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f0962d9d700 1 -- 172.21.15.7:0/1464033726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09580094f0 con 0x7f0964076be0 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f0962d9d700 1 --2- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964111570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0958003050 tx=0x7f0958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f094b7fe700 1 -- 172.21.15.7:0/1464033726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0958010760 con 0x7f0964076be0 2024-03-16T08:52:07.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f094b7fe700 1 -- 172.21.15.7:0/1464033726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0958010d50 con 0x7f0964076be0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0964111cb0 con 0x7f0964076be0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f094b7fe700 1 -- 172.21.15.7:0/1464033726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f095801ae20 con 0x7f0964076be0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.196+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0964112820 con 0x7f0964076be0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.197+0000 7f094b7fe700 1 -- 172.21.15.7:0/1464033726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f095802e440 con 0x7f0964076be0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.197+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096405f5f0 con 0x7f0964076be0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.197+0000 7f094b7fe700 1 --2- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f094c036660 0x7f094c038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.197+0000 7f094b7fe700 1 -- 172.21.15.7:0/1464033726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f095804a460 con 0x7f0964076be0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.197+0000 7f096259c700 1 --2- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f094c036660 0x7f094c038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.198+0000 7f096259c700 1 --2- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f094c036660 0x7f094c038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0954009990 tx=0x7f0954006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.202+0000 7f094b7fe700 1 -- 172.21.15.7:0/1464033726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f0958023b00 con 0x7f0964076be0 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.410+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f09640fe0b0 con 0x7f094c036660 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.419+0000 7f094b7fe700 1 -- 172.21.15.7:0/1464033726 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f09640fe0b0 con 0x7f094c036660 2024-03-16T08:52:07.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f094c036660 msgr2=0x7f094c038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 --2- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f094c036660 0x7f094c038b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0954009990 tx=0x7f0954006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 msgr2=0x7f0964111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 --2- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964111570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0958003050 tx=0x7f0958009250 comp rx=0 tx=0).stop 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 shutdown_connections 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 --2- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f094c036660 0x7f094c038b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 --2- 172.21.15.7:0/1464033726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0964076be0 0x7f0964111570 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 >> 172.21.15.7:0/1464033726 conn(0x7f09640fcdc0 msgr2=0x7f09640fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 shutdown_connections 2024-03-16T08:52:07.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:07.421+0000 7f096913b700 1 -- 172.21.15.7:0/1464033726 wait complete. 2024-03-16T08:52:08.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:08 smithi007 ceph-mon[29470]: [16/Mar/2024:08:52:07] ENGINE Bus STARTING 2024-03-16T08:52:08.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:08 smithi007 ceph-mon[29470]: [16/Mar/2024:08:52:07] ENGINE Serving on https://172.21.15.7:7150 2024-03-16T08:52:08.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:08 smithi007 ceph-mon[29470]: [16/Mar/2024:08:52:07] ENGINE Bus STARTED 2024-03-16T08:52:08.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:08 smithi007 ceph-mon[29470]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:08.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:08 smithi007 ceph-mon[29470]: mgrmap e8: smithi007.uamwtp(active, since 2s) 2024-03-16T08:52:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-16T08:52:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.441+0000 7f1fd339d700 1 Processor -- start 2024-03-16T08:52:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.441+0000 7f1fd339d700 1 -- start start 2024-03-16T08:52:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.441+0000 7f1fd339d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc101840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.441+0000 7f1fd339d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc101d80 con 0x7f1fcc101430 2024-03-16T08:52:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.441+0000 7f1fd239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.442+0000 7f1fd239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38318/0 (socket says 172.21.15.7:38318) 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.442+0000 7f1fd239b700 1 -- 172.21.15.7:0/290963153 learned_addr learned my addr 172.21.15.7:0/290963153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.442+0000 7f1fd239b700 1 -- 172.21.15.7:0/290963153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc101ec0 con 0x7f1fcc101430 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.442+0000 7f1fd239b700 1 --2- 172.21.15.7:0/290963153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc101840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009e50 tx=0x7f1fbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fc3773f94bca63 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.442+0000 7f1fd1399700 1 -- 172.21.15.7:0/290963153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fbc0092e0 con 0x7f1fcc101430 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.442+0000 7f1fd1399700 1 -- 172.21.15.7:0/290963153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1fbc012430 con 0x7f1fcc101430 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.442+0000 7f1fd1399700 1 -- 172.21.15.7:0/290963153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fbc0075c0 con 0x7f1fcc101430 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 -- 172.21.15.7:0/290963153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 msgr2=0x7f1fcc101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 --2- 172.21.15.7:0/290963153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc101840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009e50 tx=0x7f1fbc009840 comp rx=0 tx=0).stop 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 -- 172.21.15.7:0/290963153 shutdown_connections 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 --2- 172.21.15.7:0/290963153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc101840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 -- 172.21.15.7:0/290963153 >> 172.21.15.7:0/290963153 conn(0x7f1fcc0fcb90 msgr2=0x7f1fcc0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:09.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 -- 172.21.15.7:0/290963153 shutdown_connections 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 -- 172.21.15.7:0/290963153 wait complete. 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 Processor -- start 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 -- start start 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc1135f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.443+0000 7f1fd339d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc113b30 con 0x7f1fcc101430 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fd239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc1135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fd239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc1135f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38330/0 (socket says 172.21.15.7:38330) 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fd239b700 1 -- 172.21.15.7:0/2558557421 learned_addr learned my addr 172.21.15.7:0/2558557421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fd239b700 1 -- 172.21.15.7:0/2558557421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009ae0 con 0x7f1fcc101430 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fd239b700 1 --2- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc1135f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009ab0 tx=0x7f1fbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fcaffd700 1 -- 172.21.15.7:0/2558557421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fbc0092e0 con 0x7f1fcc101430 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fcaffd700 1 -- 172.21.15.7:0/2558557421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1fbc007d10 con 0x7f1fcc101430 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc113d30 con 0x7f1fcc101430 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fcaffd700 1 -- 172.21.15.7:0/2558557421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fbc01c990 con 0x7f1fcc101430 2024-03-16T08:52:09.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.444+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc1148a0 con 0x7f1fcc101430 2024-03-16T08:52:09.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.445+0000 7f1fcaffd700 1 -- 172.21.15.7:0/2558557421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1fbc019070 con 0x7f1fcc101430 2024-03-16T08:52:09.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.445+0000 7f1fcaffd700 1 --2- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f1fb4036790 0x7f1fb4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:09.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.445+0000 7f1fcaffd700 1 -- 172.21.15.7:0/2558557421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1fbc049ed0 con 0x7f1fcc101430 2024-03-16T08:52:09.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.445+0000 7f1fd1b9a700 1 --2- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f1fb4036790 0x7f1fb4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:09.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.446+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc05f5f0 con 0x7f1fcc101430 2024-03-16T08:52:09.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.446+0000 7f1fd1b9a700 1 --2- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f1fb4036790 0x7f1fb4038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009990 tx=0x7f1fc0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.450+0000 7f1fcaffd700 1 -- 172.21.15.7:0/2558557421 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f1fbc017080 con 0x7f1fcc101430 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.646+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fcc1153c0 con 0x7f1fb4036790 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.646+0000 7f1fcaffd700 1 -- 172.21.15.7:0/2558557421 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f1fcc1153c0 con 0x7f1fb4036790 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f1fb4036790 msgr2=0x7f1fb4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 --2- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f1fb4036790 0x7f1fb4038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009990 tx=0x7f1fc0006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 msgr2=0x7f1fcc1135f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 --2- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc1135f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009ab0 tx=0x7f1fbc009840 comp rx=0 tx=0).stop 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 shutdown_connections 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 --2- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f1fb4036790 0x7f1fb4038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 --2- 172.21.15.7:0/2558557421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fcc101430 0x7f1fcc1135f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 >> 172.21.15.7:0/2558557421 conn(0x7f1fcc0fcb90 msgr2=0x7f1fcc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 shutdown_connections 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:08.648+0000 7f1fd339d700 1 -- 172.21.15.7:0/2558557421 wait complete. 2024-03-16T08:52:09.286 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-03-16T08:52:09.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:09 smithi007 ceph-mon[29470]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:11.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.491+0000 7face8e90700 1 Processor -- start 2024-03-16T08:52:11.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.491+0000 7face8e90700 1 -- start start 2024-03-16T08:52:11.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.491+0000 7face8e90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face4102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:11.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.491+0000 7face8e90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face4103080 con 0x7face4102730 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.491+0000 7face259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face4102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.491+0000 7face259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face4102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38346/0 (socket says 172.21.15.7:38346) 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.491+0000 7face259c700 1 -- 172.21.15.7:0/131755517 learned_addr learned my addr 172.21.15.7:0/131755517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.492+0000 7face259c700 1 -- 172.21.15.7:0/131755517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face41031c0 con 0x7face4102730 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.492+0000 7face259c700 1 --2- 172.21.15.7:0/131755517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face4102b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7facd40098e0 tx=0x7facd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9acbb8b8ffa515 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.492+0000 7face159a700 1 -- 172.21.15.7:0/131755517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd400b860 con 0x7face4102730 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.492+0000 7face159a700 1 -- 172.21.15.7:0/131755517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7facd400be50 con 0x7face4102730 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.492+0000 7face159a700 1 -- 172.21.15.7:0/131755517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd4010380 con 0x7face4102730 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 -- 172.21.15.7:0/131755517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 msgr2=0x7face4102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 --2- 172.21.15.7:0/131755517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face4102b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7facd40098e0 tx=0x7facd4009250 comp rx=0 tx=0).stop 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 -- 172.21.15.7:0/131755517 shutdown_connections 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 --2- 172.21.15.7:0/131755517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face4102b40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 -- 172.21.15.7:0/131755517 >> 172.21.15.7:0/131755517 conn(0x7face40fdf10 msgr2=0x7face4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 -- 172.21.15.7:0/131755517 shutdown_connections 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 -- 172.21.15.7:0/131755517 wait complete. 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 Processor -- start 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.493+0000 7face8e90700 1 -- start start 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face8e90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face41115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face8e90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face4111b30 con 0x7face4102730 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face41115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face41115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38350/0 (socket says 172.21.15.7:38350) 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face259c700 1 -- 172.21.15.7:0/778564908 learned_addr learned my addr 172.21.15.7:0/778564908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face259c700 1 -- 172.21.15.7:0/778564908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd40094f0 con 0x7face4102730 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face259c700 1 --2- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face41115f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7facd4006fd0 tx=0x7facd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7facd2ffd700 1 -- 172.21.15.7:0/778564908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd40107c0 con 0x7face4102730 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7facd2ffd700 1 -- 172.21.15.7:0/778564908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7facd4010db0 con 0x7face4102730 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face4111d30 con 0x7face4102730 2024-03-16T08:52:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7facd2ffd700 1 -- 172.21.15.7:0/778564908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd401ada0 con 0x7face4102730 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.494+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face41128a0 con 0x7face4102730 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.495+0000 7facd2ffd700 1 -- 172.21.15.7:0/778564908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7facd4010920 con 0x7face4102730 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.495+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face41121c0 con 0x7face4102730 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.495+0000 7facd2ffd700 1 --2- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7faccc0367e0 0x7faccc038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.495+0000 7facd2ffd700 1 -- 172.21.15.7:0/778564908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7facd4049d20 con 0x7face4102730 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.496+0000 7face1d9b700 1 --2- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7faccc0367e0 0x7faccc038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.496+0000 7face1d9b700 1 --2- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7faccc0367e0 0x7faccc038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7facd8009990 tx=0x7facd8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.500+0000 7facd2ffd700 1 -- 172.21.15.7:0/778564908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7facd4023360 con 0x7face4102730 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.693+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7face40ff2d0 con 0x7faccc0367e0 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.815+0000 7facd2ffd700 1 -- 172.21.15.7:0/778564908 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7face40ff2d0 con 0x7faccc0367e0 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7faccc0367e0 msgr2=0x7faccc038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 --2- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7faccc0367e0 0x7faccc038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7facd8009990 tx=0x7facd8006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 msgr2=0x7face41115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:11.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 --2- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face41115f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7facd4006fd0 tx=0x7facd4009250 comp rx=0 tx=0).stop 2024-03-16T08:52:11.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 shutdown_connections 2024-03-16T08:52:11.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 --2- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7faccc0367e0 0x7faccc038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:11.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 --2- 172.21.15.7:0/778564908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7face4102730 0x7face41115f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:11.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 >> 172.21.15.7:0/778564908 conn(0x7face40fdf10 msgr2=0x7face40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:11.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 shutdown_connections 2024-03-16T08:52:11.442 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:10.817+0000 7face8e90700 1 -- 172.21.15.7:0/778564908 wait complete. 2024-03-16T08:52:12.061 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:11 smithi007 ceph-mon[29470]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:12.061 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:11 smithi007 ceph-mon[29470]: Generating ssh key... 2024-03-16T08:52:12.061 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:11 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:12.061 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:11 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClsX2MYJ7lzPDENMN74inbHJmx0CYlNzRuWIjj54BPIwhrB9NBSmOa20N7EZeC/Fd6Cbq5IwuR6FmbvGTk2fYwJlp8GwnonVogxh8ihDPdiZ4je94jM2us+drwzms8RGoaUD7KNOLpYW9X2JC0+7nq807dbtOC7wyfS/mTuqho6r67Ckx0cCh6o8eHenpxYbluYKbQds0CrmZjzIDoA/fLlqrTIeLm+f0diIAjnPJfrpCFpJPUeMefQB1OIh9xQmRRH5qOZe2+jNqBhmsnEHnW4oe9ECQkj0RphdDQwvrvQAlZMnfQGyM6ptJjR+zxm9wSmd6TmIxFb/kt0DU55iaSGaCbtNWDkaTU83VpwnORXyLxyY0ceTx3qNnYAk+d4vXxSmVuwVpo1WURXOgK7v9clcnfAipkpkpvJp3u4k3GtMwGYFNWhxjFHS667DeiGJ+h0kgapt3vVLZ5xaFJPOmITkEY4VIaUST8N4mKnSFrwHp6MEcIOsvDH5fUwK5oKkU= ceph-4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.855+0000 7f60503bc700 1 Processor -- start 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.855+0000 7f60503bc700 1 -- start start 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.855+0000 7f60503bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.855+0000 7f60503bc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6048103070 con 0x7f6048102720 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.855+0000 7f604e158700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.855+0000 7f604e158700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38366/0 (socket says 172.21.15.7:38366) 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.855+0000 7f604e158700 1 -- 172.21.15.7:0/2384119638 learned_addr learned my addr 172.21.15.7:0/2384119638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:12.612 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.856+0000 7f604e158700 1 -- 172.21.15.7:0/2384119638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60481031b0 con 0x7f6048102720 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.856+0000 7f604e158700 1 --2- 172.21.15.7:0/2384119638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048102b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60440098e0 tx=0x7f6044009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c512aae004d95f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.856+0000 7f604d156700 1 -- 172.21.15.7:0/2384119638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f604400b860 con 0x7f6048102720 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.856+0000 7f604d156700 1 -- 172.21.15.7:0/2384119638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f604400be50 con 0x7f6048102720 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.856+0000 7f604d156700 1 -- 172.21.15.7:0/2384119638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6044010380 con 0x7f6048102720 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.857+0000 7f60503bc700 1 -- 172.21.15.7:0/2384119638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 msgr2=0x7f6048102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.857+0000 7f60503bc700 1 --2- 172.21.15.7:0/2384119638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048102b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60440098e0 tx=0x7f6044009250 comp rx=0 tx=0).stop 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.857+0000 7f60503bc700 1 -- 172.21.15.7:0/2384119638 shutdown_connections 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.857+0000 7f60503bc700 1 --2- 172.21.15.7:0/2384119638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048102b30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.857+0000 7f60503bc700 1 -- 172.21.15.7:0/2384119638 >> 172.21.15.7:0/2384119638 conn(0x7f60480fdee0 msgr2=0x7f60481002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.857+0000 7f60503bc700 1 -- 172.21.15.7:0/2384119638 shutdown_connections 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.857+0000 7f60503bc700 1 -- 172.21.15.7:0/2384119638 wait complete. 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f60503bc700 1 Processor -- start 2024-03-16T08:52:12.613 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f60503bc700 1 -- start start 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f60503bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048111560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f60503bc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6048111aa0 con 0x7f6048102720 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f604e158700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048111560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f604e158700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048111560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38382/0 (socket says 172.21.15.7:38382) 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f604e158700 1 -- 172.21.15.7:0/4151641518 learned_addr learned my addr 172.21.15.7:0/4151641518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f604e158700 1 -- 172.21.15.7:0/4151641518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440094f0 con 0x7f6048102720 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f604e158700 1 --2- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048111560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f6044009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.858+0000 7f603effd700 1 -- 172.21.15.7:0/4151641518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60440107c0 con 0x7f6048102720 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.859+0000 7f603effd700 1 -- 172.21.15.7:0/4151641518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6044010db0 con 0x7f6048102720 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.859+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6048111ca0 con 0x7f6048102720 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.859+0000 7f603effd700 1 -- 172.21.15.7:0/4151641518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f604401ad00 con 0x7f6048102720 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.859+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6048112810 con 0x7f6048102720 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.859+0000 7f603effd700 1 -- 172.21.15.7:0/4151641518 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f60440193c0 con 0x7f6048102720 2024-03-16T08:52:12.614 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.860+0000 7f603effd700 1 --2- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f6034036790 0x7f6034038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.860+0000 7f603effd700 1 -- 172.21.15.7:0/4151641518 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f604404a650 con 0x7f6048102720 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.860+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604805f5f0 con 0x7f6048102720 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.860+0000 7f604d957700 1 --2- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f6034036790 0x7f6034038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.860+0000 7f604d957700 1 --2- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f6034036790 0x7f6034038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6038009990 tx=0x7f6038006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:11.865+0000 7f603effd700 1 -- 172.21.15.7:0/4151641518 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f60440196c0 con 0x7f6048102720 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.059+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f60480ff170 con 0x7f6034036790 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.059+0000 7f603effd700 1 -- 172.21.15.7:0/4151641518 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f60480ff170 con 0x7f6034036790 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f6034036790 msgr2=0x7f6034038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 --2- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f6034036790 0x7f6034038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6038009990 tx=0x7f6038006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 msgr2=0x7f6048111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 --2- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048111560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f6044009250 comp rx=0 tx=0).stop 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 shutdown_connections 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 --2- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f6034036790 0x7f6034038c40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 --2- 172.21.15.7:0/4151641518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6048102720 0x7f6048111560 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:12.615 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 >> 172.21.15.7:0/4151641518 conn(0x7f60480fdee0 msgr2=0x7f60480fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:12.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 shutdown_connections 2024-03-16T08:52:12.616 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:12.061+0000 7f60503bc700 1 -- 172.21.15.7:0/4151641518 wait complete. 2024-03-16T08:52:12.616 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-16T08:52:12.616 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-03-16T08:52:12.616 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-03-16T08:52:12.914 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:12 smithi007 ceph-mon[29470]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:14.096 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:13 smithi007 ceph-mon[29470]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:14.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:14 smithi007 ceph-mon[29470]: Deploying cephadm binary to smithi007 2024-03-16T08:52:15.885 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-03-16T08:52:15.885 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.016+0000 7f97567c5700 1 Processor -- start 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.016+0000 7f97567c5700 1 -- start start 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.016+0000 7f97567c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.016+0000 7f97567c5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97501030c0 con 0x7f9750102770 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.016+0000 7f974ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.016+0000 7f974ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51954/0 (socket says 172.21.15.7:51954) 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.016+0000 7f974ffff700 1 -- 172.21.15.7:0/2035935122 learned_addr learned my addr 172.21.15.7:0/2035935122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.016+0000 7f974ffff700 1 -- 172.21.15.7:0/2035935122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750103200 con 0x7f9750102770 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f974ffff700 1 --2- 172.21.15.7:0/2035935122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750102b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f97400098e0 tx=0x7f9740009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2320516bb1353b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f974effd700 1 -- 172.21.15.7:0/2035935122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f974000b860 con 0x7f9750102770 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f974effd700 1 -- 172.21.15.7:0/2035935122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f974000be50 con 0x7f9750102770 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f974effd700 1 -- 172.21.15.7:0/2035935122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9740010380 con 0x7f9750102770 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f97567c5700 1 -- 172.21.15.7:0/2035935122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 msgr2=0x7f9750102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:15.886 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f97567c5700 1 --2- 172.21.15.7:0/2035935122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750102b80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f97400098e0 tx=0x7f9740009250 comp rx=0 tx=0).stop 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f97567c5700 1 -- 172.21.15.7:0/2035935122 shutdown_connections 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f97567c5700 1 --2- 172.21.15.7:0/2035935122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750102b80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f97567c5700 1 -- 172.21.15.7:0/2035935122 >> 172.21.15.7:0/2035935122 conn(0x7f97500fde90 msgr2=0x7f9750100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f97567c5700 1 -- 172.21.15.7:0/2035935122 shutdown_connections 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.017+0000 7f97567c5700 1 -- 172.21.15.7:0/2035935122 wait complete. 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.018+0000 7f97567c5700 1 Processor -- start 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.018+0000 7f97567c5700 1 -- start start 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.018+0000 7f97567c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.018+0000 7f97567c5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9750111ac0 con 0x7f9750102770 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.018+0000 7f974ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.018+0000 7f974ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51960/0 (socket says 172.21.15.7:51960) 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.018+0000 7f974ffff700 1 -- 172.21.15.7:0/929459347 learned_addr learned my addr 172.21.15.7:0/929459347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.019+0000 7f974ffff700 1 -- 172.21.15.7:0/929459347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400094f0 con 0x7f9750102770 2024-03-16T08:52:15.887 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.019+0000 7f974ffff700 1 --2- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750111580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f974000b440 tx=0x7f9740009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.019+0000 7f974cff9700 1 -- 172.21.15.7:0/929459347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9740010760 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.019+0000 7f974cff9700 1 -- 172.21.15.7:0/929459347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9740010d50 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.019+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9750111cc0 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.019+0000 7f974cff9700 1 -- 172.21.15.7:0/929459347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f974001ae20 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.019+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9750112830 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.020+0000 7f974cff9700 1 -- 172.21.15.7:0/929459347 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f974002e440 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.020+0000 7f974cff9700 1 --2- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9738036770 0x7f9738038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.020+0000 7f974cff9700 1 -- 172.21.15.7:0/929459347 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f974004a7a0 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.020+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975005f5f0 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.020+0000 7f974f7fe700 1 --2- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9738036770 0x7f9738038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.021+0000 7f974f7fe700 1 --2- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9738036770 0x7f9738038c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9744009990 tx=0x7f9744006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.025+0000 7f974cff9700 1 -- 172.21.15.7:0/929459347 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f9740018350 con 0x7f9750102770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:13.220+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) v1 -- 0x7f97500ff1d0 con 0x7f9738036770 2024-03-16T08:52:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.260+0000 7f974cff9700 1 -- 172.21.15.7:0/929459347 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f97500ff1d0 con 0x7f9738036770 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9738036770 msgr2=0x7f9738038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 --2- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9738036770 0x7f9738038c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9744009990 tx=0x7f9744006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 msgr2=0x7f9750111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 --2- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750111580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f974000b440 tx=0x7f9740009250 comp rx=0 tx=0).stop 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 shutdown_connections 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 --2- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9738036770 0x7f9738038c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 --2- 172.21.15.7:0/929459347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9750102770 0x7f9750111580 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 >> 172.21.15.7:0/929459347 conn(0x7f97500fde90 msgr2=0x7f97500feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 shutdown_connections 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:15.263+0000 7f97567c5700 1 -- 172.21.15.7:0/929459347 wait complete. 2024-03-16T08:52:15.889 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying mon service with default placement... 2024-03-16T08:52:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:16 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:16 smithi007 ceph-mon[29470]: Added host smithi007 2024-03-16T08:52:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:16 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:52:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-16T08:52:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.303+0000 7f71067e8700 1 Processor -- start 2024-03-16T08:52:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.303+0000 7f71067e8700 1 -- start start 2024-03-16T08:52:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.303+0000 7f71067e8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100101af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f71067e8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7100102030 con 0x7f71001016e0 2024-03-16T08:52:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f70fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f70fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51964/0 (socket says 172.21.15.7:51964) 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f70fffff700 1 -- 172.21.15.7:0/1542885993 learned_addr learned my addr 172.21.15.7:0/1542885993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f70fffff700 1 -- 172.21.15.7:0/1542885993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100102170 con 0x7f71001016e0 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f70fffff700 1 --2- 172.21.15.7:0/1542885993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100101af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f70f000bb80 tx=0x7f70f000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1f8c969a12b8dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f70ff7fe700 1 -- 172.21.15.7:0/1542885993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70f000dd00 con 0x7f71001016e0 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f70ff7fe700 1 -- 172.21.15.7:0/1542885993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f70f000de60 con 0x7f71001016e0 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.304+0000 7f70ff7fe700 1 -- 172.21.15.7:0/1542885993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70f00188a0 con 0x7f71001016e0 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.305+0000 7f71067e8700 1 -- 172.21.15.7:0/1542885993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 msgr2=0x7f7100101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.305+0000 7f71067e8700 1 --2- 172.21.15.7:0/1542885993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100101af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f70f000bb80 tx=0x7f70f000b4f0 comp rx=0 tx=0).stop 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.305+0000 7f71067e8700 1 -- 172.21.15.7:0/1542885993 shutdown_connections 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.305+0000 7f71067e8700 1 --2- 172.21.15.7:0/1542885993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100101af0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.305+0000 7f71067e8700 1 -- 172.21.15.7:0/1542885993 >> 172.21.15.7:0/1542885993 conn(0x7f71000fce40 msgr2=0x7f71000ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.305+0000 7f71067e8700 1 -- 172.21.15.7:0/1542885993 shutdown_connections 2024-03-16T08:52:17.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.305+0000 7f71067e8700 1 -- 172.21.15.7:0/1542885993 wait complete. 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f71067e8700 1 Processor -- start 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f71067e8700 1 -- start start 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f71067e8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f71067e8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7100113c60 con 0x7f71001016e0 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f70fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f70fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100113720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51966/0 (socket says 172.21.15.7:51966) 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f70fffff700 1 -- 172.21.15.7:0/585176997 learned_addr learned my addr 172.21.15.7:0/585176997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f70fffff700 1 -- 172.21.15.7:0/585176997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f000b790 con 0x7f71001016e0 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f70fffff700 1 --2- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100113720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70f00184f0 tx=0x7f70f000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.306+0000 7f70fd7fa700 1 -- 172.21.15.7:0/585176997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70f000dc70 con 0x7f71001016e0 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.307+0000 7f70fd7fa700 1 -- 172.21.15.7:0/585176997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f70f0010bd0 con 0x7f71001016e0 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.307+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7100113e60 con 0x7f71001016e0 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.307+0000 7f70fd7fa700 1 -- 172.21.15.7:0/585176997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70f0021550 con 0x7f71001016e0 2024-03-16T08:52:17.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.307+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71001149d0 con 0x7f71001016e0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.307+0000 7f70fd7fa700 1 -- 172.21.15.7:0/585176997 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f70f0021700 con 0x7f71001016e0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.307+0000 7f70fd7fa700 1 --2- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f70ec0367e0 0x7f70ec038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.308+0000 7f70fd7fa700 1 -- 172.21.15.7:0/585176997 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f70f0014070 con 0x7f71001016e0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.308+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710005f5f0 con 0x7f71001016e0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.308+0000 7f70f7fff700 1 --2- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f70ec0367e0 0x7f70ec038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.308+0000 7f70f7fff700 1 --2- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f70ec0367e0 0x7f70ec038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009990 tx=0x7f70e8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.312+0000 7f70fd7fa700 1 -- 172.21.15.7:0/585176997 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f70f0026030 con 0x7f71001016e0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.543+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f7100114cd0 con 0x7f70ec0367e0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.547+0000 7f70fd7fa700 1 -- 172.21.15.7:0/585176997 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7100114cd0 con 0x7f70ec0367e0 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f70ec0367e0 msgr2=0x7f70ec038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 --2- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f70ec0367e0 0x7f70ec038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009990 tx=0x7f70e8006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:17.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 msgr2=0x7f7100113720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 --2- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100113720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70f00184f0 tx=0x7f70f000b4f0 comp rx=0 tx=0).stop 2024-03-16T08:52:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 shutdown_connections 2024-03-16T08:52:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 --2- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f70ec0367e0 0x7f70ec038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 --2- 172.21.15.7:0/585176997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71001016e0 0x7f7100113720 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 >> 172.21.15.7:0/585176997 conn(0x7f71000fce40 msgr2=0x7f7100105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 shutdown_connections 2024-03-16T08:52:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:16.549+0000 7f71067e8700 1 -- 172.21.15.7:0/585176997 wait complete. 2024-03-16T08:52:17.240 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying mgr service with default placement... 2024-03-16T08:52:17.799 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:17 smithi007 ceph-mon[29470]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:17.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:17 smithi007 ceph-mon[29470]: Saving service mon spec with placement count:5 2024-03-16T08:52:17.800 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:17 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-16T08:52:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.664+0000 7f27607a5700 1 Processor -- start 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.664+0000 7f27607a5700 1 -- start start 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.664+0000 7f27607a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.664+0000 7f27607a5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758101410 con 0x7f2758100ac0 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.665+0000 7f275e541700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.665+0000 7f275e541700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51980/0 (socket says 172.21.15.7:51980) 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.665+0000 7f275e541700 1 -- 172.21.15.7:0/3990136045 learned_addr learned my addr 172.21.15.7:0/3990136045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.665+0000 7f275e541700 1 -- 172.21.15.7:0/3990136045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2758101550 con 0x7f2758100ac0 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.665+0000 7f275e541700 1 --2- 172.21.15.7:0/3990136045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758100ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2748009e50 tx=0x7f2748009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464b6cd9ea8253b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.665+0000 7f275d53f700 1 -- 172.21.15.7:0/3990136045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27480092e0 con 0x7f2758100ac0 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.665+0000 7f275d53f700 1 -- 172.21.15.7:0/3990136045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2748012430 con 0x7f2758100ac0 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.665+0000 7f275d53f700 1 -- 172.21.15.7:0/3990136045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27480075c0 con 0x7f2758100ac0 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 -- 172.21.15.7:0/3990136045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 msgr2=0x7f2758100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 --2- 172.21.15.7:0/3990136045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758100ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2748009e50 tx=0x7f2748009840 comp rx=0 tx=0).stop 2024-03-16T08:52:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 -- 172.21.15.7:0/3990136045 shutdown_connections 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 --2- 172.21.15.7:0/3990136045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758100ed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 -- 172.21.15.7:0/3990136045 >> 172.21.15.7:0/3990136045 conn(0x7f27580fc500 msgr2=0x7f27580fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 -- 172.21.15.7:0/3990136045 shutdown_connections 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 -- 172.21.15.7:0/3990136045 wait complete. 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 Processor -- start 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.666+0000 7f27607a5700 1 -- start start 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f27607a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758111790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f27607a5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758111cd0 con 0x7f2758100ac0 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f275e541700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758111790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f275e541700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758111790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51996/0 (socket says 172.21.15.7:51996) 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f275e541700 1 -- 172.21.15.7:0/3328944912 learned_addr learned my addr 172.21.15.7:0/3328944912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f275e541700 1 -- 172.21.15.7:0/3328944912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748009ae0 con 0x7f2758100ac0 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f275e541700 1 --2- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758111790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2748009ab0 tx=0x7f2748009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f2756ffd700 1 -- 172.21.15.7:0/3328944912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27480092e0 con 0x7f2758100ac0 2024-03-16T08:52:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f2756ffd700 1 -- 172.21.15.7:0/3328944912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2748007d10 con 0x7f2758100ac0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2758111ed0 con 0x7f2758100ac0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f2756ffd700 1 -- 172.21.15.7:0/3328944912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f274801c990 con 0x7f2758100ac0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.667+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2758112a40 con 0x7f2758100ac0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.668+0000 7f2756ffd700 1 -- 172.21.15.7:0/3328944912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2748019070 con 0x7f2758100ac0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.668+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275805f5f0 con 0x7f2758100ac0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.668+0000 7f2756ffd700 1 --2- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f27400367e0 0x7f2740038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.668+0000 7f2756ffd700 1 -- 172.21.15.7:0/3328944912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2748049e60 con 0x7f2758100ac0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.669+0000 7f275dd40700 1 --2- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f27400367e0 0x7f2740038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.669+0000 7f275dd40700 1 --2- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f27400367e0 0x7f2740038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f274c009990 tx=0x7f274c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.673+0000 7f2756ffd700 1 -- 172.21.15.7:0/3328944912 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f2748011980 con 0x7f2758100ac0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.879+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f27580fd7b0 con 0x7f27400367e0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.884+0000 7f2756ffd700 1 -- 172.21.15.7:0/3328944912 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f27580fd7b0 con 0x7f27400367e0 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.885+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f27400367e0 msgr2=0x7f2740038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.885+0000 7f27607a5700 1 --2- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f27400367e0 0x7f2740038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f274c009990 tx=0x7f274c006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.886+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 msgr2=0x7f2758111790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.886+0000 7f27607a5700 1 --2- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758111790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2748009ab0 tx=0x7f2748009840 comp rx=0 tx=0).stop 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.886+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 shutdown_connections 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.886+0000 7f27607a5700 1 --2- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f27400367e0 0x7f2740038c90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.886+0000 7f27607a5700 1 --2- 172.21.15.7:0/3328944912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2758100ac0 0x7f2758111790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.886+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 >> 172.21.15.7:0/3328944912 conn(0x7f27580fc500 msgr2=0x7f27580fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.886+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 shutdown_connections 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:17.886+0000 7f27607a5700 1 -- 172.21.15.7:0/3328944912 wait complete. 2024-03-16T08:52:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying crash service with default placement... 2024-03-16T08:52:19.183 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:18 smithi007 ceph-mon[29470]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:19.183 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:18 smithi007 ceph-mon[29470]: Saving service mgr spec with placement count:2 2024-03-16T08:52:19.183 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:18 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:20.423 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:20 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:20.423 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:20 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:20.424 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:20 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:20.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-16T08:52:20.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.614+0000 7f71bf091700 1 Processor -- start 2024-03-16T08:52:20.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.614+0000 7f71bf091700 1 -- start start 2024-03-16T08:52:20.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.614+0000 7f71bf091700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:20.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.614+0000 7f71bf091700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8076f00 con 0x7f71b8076af0 2024-03-16T08:52:20.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.614+0000 7f71bce2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:20.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.614+0000 7f71bce2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52012/0 (socket says 172.21.15.7:52012) 2024-03-16T08:52:20.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.614+0000 7f71bce2d700 1 -- 172.21.15.7:0/674898997 learned_addr learned my addr 172.21.15.7:0/674898997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.615+0000 7f71bce2d700 1 -- 172.21.15.7:0/674898997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b81059b0 con 0x7f71b8076af0 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.615+0000 7f71bce2d700 1 --2- 172.21.15.7:0/674898997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8105470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0098e0 tx=0x7f71ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb2f34d943da9cd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.615+0000 7f71b77fe700 1 -- 172.21.15.7:0/674898997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71ac00b860 con 0x7f71b8076af0 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.615+0000 7f71b77fe700 1 -- 172.21.15.7:0/674898997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71ac00be90 con 0x7f71b8076af0 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.615+0000 7f71b77fe700 1 -- 172.21.15.7:0/674898997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71ac0103c0 con 0x7f71b8076af0 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 -- 172.21.15.7:0/674898997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 msgr2=0x7f71b8105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 --2- 172.21.15.7:0/674898997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8105470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0098e0 tx=0x7f71ac009250 comp rx=0 tx=0).stop 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 -- 172.21.15.7:0/674898997 shutdown_connections 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 --2- 172.21.15.7:0/674898997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8105470 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 -- 172.21.15.7:0/674898997 >> 172.21.15.7:0/674898997 conn(0x7f71b80fce40 msgr2=0x7f71b80ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 -- 172.21.15.7:0/674898997 shutdown_connections 2024-03-16T08:52:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 -- 172.21.15.7:0/674898997 wait complete. 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 Processor -- start 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 -- start start 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.616+0000 7f71bf091700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8111b50 con 0x7f71b8076af0 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71bce2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71bce2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52024/0 (socket says 172.21.15.7:52024) 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71bce2d700 1 -- 172.21.15.7:0/3791664182 learned_addr learned my addr 172.21.15.7:0/3791664182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71bce2d700 1 -- 172.21.15.7:0/3791664182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0094f0 con 0x7f71b8076af0 2024-03-16T08:52:20.435 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71bce2d700 1 --2- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8111610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0094c0 tx=0x7f71ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71b57fa700 1 -- 172.21.15.7:0/3791664182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71ac0118f0 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71b57fa700 1 -- 172.21.15.7:0/3791664182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71ac010d60 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b8111d50 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71b57fa700 1 -- 172.21.15.7:0/3791664182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71ac01a430 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.617+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b81128c0 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.618+0000 7f71b57fa700 1 -- 172.21.15.7:0/3791664182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f71ac01a590 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.618+0000 7f71b57fa700 1 --2- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f71a8036790 0x7f71a8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.618+0000 7f71b57fa700 1 -- 172.21.15.7:0/3791664182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f71ac04a480 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.618+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b805f5f0 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.618+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f71a8036790 0x7f71a8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.619+0000 7f71b7fff700 1 --2- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f71a8036790 0x7f71a8038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009990 tx=0x7f71a4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.625+0000 7f71b57fa700 1 -- 172.21.15.7:0/3791664182 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f71ac02d3f0 con 0x7f71b8076af0 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.833+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f71b80fe1e0 con 0x7f71a8036790 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.839+0000 7f71b57fa700 1 -- 172.21.15.7:0/3791664182 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f71b80fe1e0 con 0x7f71a8036790 2024-03-16T08:52:20.436 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.840+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f71a8036790 msgr2=0x7f71a8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.840+0000 7f71bf091700 1 --2- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f71a8036790 0x7f71a8038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009990 tx=0x7f71a4006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.840+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 msgr2=0x7f71b8111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.840+0000 7f71bf091700 1 --2- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8111610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0094c0 tx=0x7f71ac009250 comp rx=0 tx=0).stop 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.841+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 shutdown_connections 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.841+0000 7f71bf091700 1 --2- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f71a8036790 0x7f71a8038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.841+0000 7f71bf091700 1 --2- 172.21.15.7:0/3791664182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71b8076af0 0x7f71b8111610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.841+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 >> 172.21.15.7:0/3791664182 conn(0x7f71b80fce40 msgr2=0x7f71b80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.841+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 shutdown_connections 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:19.841+0000 7f71bf091700 1 -- 172.21.15.7:0/3791664182 wait complete. 2024-03-16T08:52:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying prometheus service with default placement... 2024-03-16T08:52:21.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:21 smithi007 ceph-mon[29470]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:21.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:21 smithi007 ceph-mon[29470]: Saving service crash spec with placement * 2024-03-16T08:52:21.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:21 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.838+0000 7f41b36d1700 1 Processor -- start 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.838+0000 7f41b36d1700 1 -- start start 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.838+0000 7f41b36d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac0771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.838+0000 7f41b36d1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac079050 con 0x7f41ac078c40 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.838+0000 7f41b146d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac0771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.838+0000 7f41b146d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac0771a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52026/0 (socket says 172.21.15.7:52026) 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.838+0000 7f41b146d700 1 -- 172.21.15.7:0/1369999140 learned_addr learned my addr 172.21.15.7:0/1369999140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.839+0000 7f41b146d700 1 -- 172.21.15.7:0/1369999140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0776e0 con 0x7f41ac078c40 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.839+0000 7f41b146d700 1 --2- 172.21.15.7:0/1369999140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac0771a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41a80098e0 tx=0x7f41a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75926347e55a81e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:21.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.839+0000 7f41a3fff700 1 -- 172.21.15.7:0/1369999140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41a800b860 con 0x7f41ac078c40 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.839+0000 7f41a3fff700 1 -- 172.21.15.7:0/1369999140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41a800be90 con 0x7f41ac078c40 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.839+0000 7f41a3fff700 1 -- 172.21.15.7:0/1369999140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41a80103c0 con 0x7f41ac078c40 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.839+0000 7f41b36d1700 1 -- 172.21.15.7:0/1369999140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 msgr2=0x7f41ac0771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.839+0000 7f41b36d1700 1 --2- 172.21.15.7:0/1369999140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac0771a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41a80098e0 tx=0x7f41a8009250 comp rx=0 tx=0).stop 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 -- 172.21.15.7:0/1369999140 shutdown_connections 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 --2- 172.21.15.7:0/1369999140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac0771a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 -- 172.21.15.7:0/1369999140 >> 172.21.15.7:0/1369999140 conn(0x7f41ac0fde40 msgr2=0x7f41ac1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 -- 172.21.15.7:0/1369999140 shutdown_connections 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 -- 172.21.15.7:0/1369999140 wait complete. 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 Processor -- start 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 -- start start 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac1137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b36d1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac113d10 con 0x7f41ac078c40 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b146d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac1137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:21.925 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b146d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac1137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52032/0 (socket says 172.21.15.7:52032) 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.840+0000 7f41b146d700 1 -- 172.21.15.7:0/882640421 learned_addr learned my addr 172.21.15.7:0/882640421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.841+0000 7f41b146d700 1 -- 172.21.15.7:0/882640421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a80094f0 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.841+0000 7f41b146d700 1 --2- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac1137d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f41a80094c0 tx=0x7f41a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.841+0000 7f41a1ffb700 1 -- 172.21.15.7:0/882640421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41a8010890 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.841+0000 7f41a1ffb700 1 -- 172.21.15.7:0/882640421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41a8010ec0 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.841+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ac113f10 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.841+0000 7f41a1ffb700 1 -- 172.21.15.7:0/882640421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41a80243f0 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.841+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ac114a80 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.842+0000 7f41a1ffb700 1 -- 172.21.15.7:0/882640421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f41a802d3f0 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.842+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ac05f5f0 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.842+0000 7f41a1ffb700 1 --2- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f41980367e0 0x7f4198038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.842+0000 7f41a1ffb700 1 -- 172.21.15.7:0/882640421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f41a804a400 con 0x7f41ac078c40 2024-03-16T08:52:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.843+0000 7f41b0c6c700 1 --2- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f41980367e0 0x7f4198038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.843+0000 7f41b0c6c700 1 --2- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f41980367e0 0x7f4198038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f419c009990 tx=0x7f419c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:20.847+0000 7f41a1ffb700 1 -- 172.21.15.7:0/882640421 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f41a802b6b0 con 0x7f41ac078c40 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.066+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f41ac0ff180 con 0x7f41980367e0 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.070+0000 7f41a1ffb700 1 -- 172.21.15.7:0/882640421 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f41ac0ff180 con 0x7f41980367e0 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f41980367e0 msgr2=0x7f4198038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 --2- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f41980367e0 0x7f4198038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f419c009990 tx=0x7f419c006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 msgr2=0x7f41ac1137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 --2- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac1137d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f41a80094c0 tx=0x7f41a8009250 comp rx=0 tx=0).stop 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 shutdown_connections 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 --2- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f41980367e0 0x7f4198038c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 --2- 172.21.15.7:0/882640421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ac078c40 0x7f41ac1137d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 >> 172.21.15.7:0/882640421 conn(0x7f41ac0fde40 msgr2=0x7f41ac0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 shutdown_connections 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:21.072+0000 7f41b36d1700 1 -- 172.21.15.7:0/882640421 wait complete. 2024-03-16T08:52:21.927 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying grafana service with default placement... 2024-03-16T08:52:22.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:22 smithi007 ceph-mon[29470]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:22.938 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:22 smithi007 ceph-mon[29470]: Saving service prometheus spec with placement count:1 2024-03-16T08:52:23.180 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.338+0000 7fc599cba700 1 Processor -- start 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.338+0000 7fc599cba700 1 -- start start 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.338+0000 7fc599cba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.338+0000 7fc599cba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc594103070 con 0x7fc594102720 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.339+0000 7fc5937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.339+0000 7fc5937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52044/0 (socket says 172.21.15.7:52044) 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.339+0000 7fc5937fe700 1 -- 172.21.15.7:0/1525053829 learned_addr learned my addr 172.21.15.7:0/1525053829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.339+0000 7fc5937fe700 1 -- 172.21.15.7:0/1525053829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5941031b0 con 0x7fc594102720 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.339+0000 7fc5937fe700 1 --2- 172.21.15.7:0/1525053829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594102b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0098e0 tx=0x7fc57c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7563797fd7a72b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.339+0000 7fc5927fc700 1 -- 172.21.15.7:0/1525053829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc57c00b860 con 0x7fc594102720 2024-03-16T08:52:23.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.339+0000 7fc5927fc700 1 -- 172.21.15.7:0/1525053829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc57c00be90 con 0x7fc594102720 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.339+0000 7fc5927fc700 1 -- 172.21.15.7:0/1525053829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc57c0103c0 con 0x7fc594102720 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.340+0000 7fc599cba700 1 -- 172.21.15.7:0/1525053829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 msgr2=0x7fc594102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.340+0000 7fc599cba700 1 --2- 172.21.15.7:0/1525053829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594102b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0098e0 tx=0x7fc57c009250 comp rx=0 tx=0).stop 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.340+0000 7fc599cba700 1 -- 172.21.15.7:0/1525053829 shutdown_connections 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.340+0000 7fc599cba700 1 --2- 172.21.15.7:0/1525053829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594102b30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.340+0000 7fc599cba700 1 -- 172.21.15.7:0/1525053829 >> 172.21.15.7:0/1525053829 conn(0x7fc5940fde40 msgr2=0x7fc5941002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.340+0000 7fc599cba700 1 -- 172.21.15.7:0/1525053829 shutdown_connections 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.340+0000 7fc599cba700 1 -- 172.21.15.7:0/1525053829 wait complete. 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.340+0000 7fc599cba700 1 Processor -- start 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc599cba700 1 -- start start 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc599cba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc599cba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc594079110 con 0x7fc594102720 2024-03-16T08:52:23.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc5937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc5937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52058/0 (socket says 172.21.15.7:52058) 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc5937fe700 1 -- 172.21.15.7:0/3204548719 learned_addr learned my addr 172.21.15.7:0/3204548719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc5937fe700 1 -- 172.21.15.7:0/3204548719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c0094f0 con 0x7fc594102720 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc5937fe700 1 --2- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594078bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc57c011680 tx=0x7fc57c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.341+0000 7fc58bfff700 1 -- 172.21.15.7:0/3204548719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc57c010890 con 0x7fc594102720 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.342+0000 7fc58bfff700 1 -- 172.21.15.7:0/3204548719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc57c010ec0 con 0x7fc594102720 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.342+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc594077180 con 0x7fc594102720 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.342+0000 7fc58bfff700 1 -- 172.21.15.7:0/3204548719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc57c0243f0 con 0x7fc594102720 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.342+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc594077670 con 0x7fc594102720 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.342+0000 7fc58bfff700 1 -- 172.21.15.7:0/3204548719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc57c02d3f0 con 0x7fc594102720 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.343+0000 7fc58bfff700 1 --2- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fc5800367e0 0x7fc580038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:23.183 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.343+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59404bed0 con 0x7fc594102720 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.343+0000 7fc58bfff700 1 -- 172.21.15.7:0/3204548719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc57c04a3a0 con 0x7fc594102720 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.343+0000 7fc592ffd700 1 --2- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fc5800367e0 0x7fc580038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.343+0000 7fc592ffd700 1 --2- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fc5800367e0 0x7fc580038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc584009990 tx=0x7fc584006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.348+0000 7fc58bfff700 1 -- 172.21.15.7:0/3204548719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fc57c018ce0 con 0x7fc594102720 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.549+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fc594077d80 con 0x7fc5800367e0 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.553+0000 7fc58bfff700 1 -- 172.21.15.7:0/3204548719 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc594077d80 con 0x7fc5800367e0 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fc5800367e0 msgr2=0x7fc580038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 --2- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fc5800367e0 0x7fc580038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc584009990 tx=0x7fc584006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 msgr2=0x7fc594078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 --2- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594078bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc57c011680 tx=0x7fc57c009250 comp rx=0 tx=0).stop 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 shutdown_connections 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 --2- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fc5800367e0 0x7fc580038c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 --2- 172.21.15.7:0/3204548719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc594102720 0x7fc594078bd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 >> 172.21.15.7:0/3204548719 conn(0x7fc5940fde40 msgr2=0x7fc5940fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:23.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 shutdown_connections 2024-03-16T08:52:23.185 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:22.555+0000 7fc599cba700 1 -- 172.21.15.7:0/3204548719 wait complete. 2024-03-16T08:52:23.185 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying node-exporter service with default placement... 2024-03-16T08:52:23.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:23 smithi007 ceph-mon[29470]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:23.810 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:23 smithi007 ceph-mon[29470]: Saving service grafana spec with placement count:1 2024-03-16T08:52:23.811 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:23 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:24.429 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-16T08:52:24.429 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.593+0000 7f81ef9b2700 1 Processor -- start 2024-03-16T08:52:24.429 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.593+0000 7f81ef9b2700 1 -- start start 2024-03-16T08:52:24.429 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.593+0000 7f81ef9b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8107810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:24.429 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.593+0000 7f81ef9b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8103200 con 0x7f81e8076b50 2024-03-16T08:52:24.429 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.594+0000 7f81ed74e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8107810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.594+0000 7f81ed74e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8107810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57664/0 (socket says 172.21.15.7:57664) 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.594+0000 7f81ed74e700 1 -- 172.21.15.7:0/41343613 learned_addr learned my addr 172.21.15.7:0/41343613 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.594+0000 7f81ed74e700 1 -- 172.21.15.7:0/41343613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8107d50 con 0x7f81e8076b50 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.594+0000 7f81ed74e700 1 --2- 172.21.15.7:0/41343613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8107810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81e00098e0 tx=0x7f81e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7918d4fcb7930c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.594+0000 7f81e7fff700 1 -- 172.21.15.7:0/41343613 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81e000b860 con 0x7f81e8076b50 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.594+0000 7f81e7fff700 1 -- 172.21.15.7:0/41343613 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81e000be90 con 0x7f81e8076b50 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.594+0000 7f81e7fff700 1 -- 172.21.15.7:0/41343613 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81e00103c0 con 0x7f81e8076b50 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 -- 172.21.15.7:0/41343613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 msgr2=0x7f81e8107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 --2- 172.21.15.7:0/41343613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8107810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81e00098e0 tx=0x7f81e0009250 comp rx=0 tx=0).stop 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 -- 172.21.15.7:0/41343613 shutdown_connections 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 --2- 172.21.15.7:0/41343613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8107810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 -- 172.21.15.7:0/41343613 >> 172.21.15.7:0/41343613 conn(0x7f81e80fce40 msgr2=0x7f81e80ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 -- 172.21.15.7:0/41343613 shutdown_connections 2024-03-16T08:52:24.430 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 -- 172.21.15.7:0/41343613 wait complete. 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 Processor -- start 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.595+0000 7f81ef9b2700 1 -- start start 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81ef9b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81ef9b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8113d50 con 0x7f81e8076b50 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81ed74e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81ed74e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57678/0 (socket says 172.21.15.7:57678) 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81ed74e700 1 -- 172.21.15.7:0/999693458 learned_addr learned my addr 172.21.15.7:0/999693458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81ed74e700 1 -- 172.21.15.7:0/999693458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e00094f0 con 0x7f81e8076b50 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81ed74e700 1 --2- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8113810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f81e00105b0 tx=0x7f81e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81e5ffb700 1 -- 172.21.15.7:0/999693458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81e0010840 con 0x7f81e8076b50 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81e5ffb700 1 -- 172.21.15.7:0/999693458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81e0010e70 con 0x7f81e8076b50 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e8113f50 con 0x7f81e8076b50 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.596+0000 7f81e5ffb700 1 -- 172.21.15.7:0/999693458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81e001ae30 con 0x7f81e8076b50 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.597+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e8114ac0 con 0x7f81e8076b50 2024-03-16T08:52:24.431 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.597+0000 7f81e5ffb700 1 -- 172.21.15.7:0/999693458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f81e001a430 con 0x7f81e8076b50 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.597+0000 7f81e5ffb700 1 --2- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f81d00367e0 0x7f81d0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.597+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e805f5f0 con 0x7f81e8076b50 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.598+0000 7f81e5ffb700 1 -- 172.21.15.7:0/999693458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f81e004a170 con 0x7f81e8076b50 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.598+0000 7f81ecf4d700 1 --2- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f81d00367e0 0x7f81d0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.598+0000 7f81ecf4d700 1 --2- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f81d00367e0 0x7f81d0038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009990 tx=0x7f81dc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.602+0000 7f81e5ffb700 1 -- 172.21.15.7:0/999693458 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f81e002b8c0 con 0x7f81e8076b50 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.808+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f81e80fe230 con 0x7f81d00367e0 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.813+0000 7f81e5ffb700 1 -- 172.21.15.7:0/999693458 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f81e80fe230 con 0x7f81d00367e0 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f81d00367e0 msgr2=0x7f81d0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 --2- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f81d00367e0 0x7f81d0038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009990 tx=0x7f81dc006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 msgr2=0x7f81e8113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 --2- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8113810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f81e00105b0 tx=0x7f81e0009250 comp rx=0 tx=0).stop 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 shutdown_connections 2024-03-16T08:52:24.432 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 --2- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f81d00367e0 0x7f81d0038c90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:24.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 --2- 172.21.15.7:0/999693458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81e8076b50 0x7f81e8113810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:24.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 >> 172.21.15.7:0/999693458 conn(0x7f81e80fce40 msgr2=0x7f81e80fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:24.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 shutdown_connections 2024-03-16T08:52:24.433 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:23.815+0000 7f81ef9b2700 1 -- 172.21.15.7:0/999693458 wait complete. 2024-03-16T08:52:24.433 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying alertmanager service with default placement... 2024-03-16T08:52:24.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:24 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:24.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:24 smithi007 ceph-mon[29470]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:24.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:24 smithi007 ceph-mon[29470]: Saving service node-exporter spec with placement * 2024-03-16T08:52:24.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:24 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:24.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:24 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:25.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-16T08:52:25.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.896+0000 7ff5fd27c700 1 Processor -- start 2024-03-16T08:52:25.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.896+0000 7ff5fd27c700 1 -- start start 2024-03-16T08:52:25.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.896+0000 7ff5fd27c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f8101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:25.696 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.896+0000 7ff5fd27c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f8102500 con 0x7ff5f8101bb0 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.897+0000 7ff5f6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f8101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.897+0000 7ff5f6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f8101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57690/0 (socket says 172.21.15.7:57690) 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.897+0000 7ff5f6ffd700 1 -- 172.21.15.7:0/2645301229 learned_addr learned my addr 172.21.15.7:0/2645301229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.897+0000 7ff5f6ffd700 1 -- 172.21.15.7:0/2645301229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f8102640 con 0x7ff5f8101bb0 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.897+0000 7ff5f6ffd700 1 --2- 172.21.15.7:0/2645301229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f8101fc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098e0 tx=0x7ff5e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424e92d95fdcc856 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5f5ffb700 1 -- 172.21.15.7:0/2645301229 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5e800b860 con 0x7ff5f8101bb0 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5f5ffb700 1 -- 172.21.15.7:0/2645301229 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5e800be90 con 0x7ff5f8101bb0 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5f5ffb700 1 -- 172.21.15.7:0/2645301229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5e80103c0 con 0x7ff5f8101bb0 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5fd27c700 1 -- 172.21.15.7:0/2645301229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 msgr2=0x7ff5f8101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5fd27c700 1 --2- 172.21.15.7:0/2645301229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f8101fc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098e0 tx=0x7ff5e8009250 comp rx=0 tx=0).stop 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5fd27c700 1 -- 172.21.15.7:0/2645301229 shutdown_connections 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5fd27c700 1 --2- 172.21.15.7:0/2645301229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f8101fc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5fd27c700 1 -- 172.21.15.7:0/2645301229 >> 172.21.15.7:0/2645301229 conn(0x7ff5f80fc500 msgr2=0x7ff5f80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:25.697 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.898+0000 7ff5fd27c700 1 -- 172.21.15.7:0/2645301229 shutdown_connections 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.899+0000 7ff5fd27c700 1 -- 172.21.15.7:0/2645301229 wait complete. 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.899+0000 7ff5fd27c700 1 Processor -- start 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.899+0000 7ff5fd27c700 1 -- start start 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5fd27c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f81013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5fd27c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f81142f0 con 0x7ff5f8101bb0 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5f6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f81013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5f6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f81013b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57694/0 (socket says 172.21.15.7:57694) 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5f6ffd700 1 -- 172.21.15.7:0/4097034676 learned_addr learned my addr 172.21.15.7:0/4097034676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5f6ffd700 1 -- 172.21.15.7:0/4097034676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80094f0 con 0x7ff5f8101bb0 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5f6ffd700 1 --2- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f81013b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8006d40 tx=0x7ff5e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5df7fe700 1 -- 172.21.15.7:0/4097034676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5e8010840 con 0x7ff5f8101bb0 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.900+0000 7ff5df7fe700 1 -- 172.21.15.7:0/4097034676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5e8010e70 con 0x7ff5f8101bb0 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.901+0000 7ff5df7fe700 1 -- 172.21.15.7:0/4097034676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5e8024ea0 con 0x7ff5f8101bb0 2024-03-16T08:52:25.698 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.901+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f8114430 con 0x7ff5f8101bb0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.901+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f8114800 con 0x7ff5f8101bb0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.901+0000 7ff5df7fe700 1 -- 172.21.15.7:0/4097034676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ff5e801a4c0 con 0x7ff5f8101bb0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.901+0000 7ff5df7fe700 1 --2- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7ff5e00367e0 0x7ff5e0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.902+0000 7ff5df7fe700 1 -- 172.21.15.7:0/4097034676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff5e804b800 con 0x7ff5f8101bb0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.902+0000 7ff5f67fc700 1 --2- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7ff5e00367e0 0x7ff5e0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.902+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f804bed0 con 0x7ff5f8101bb0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.902+0000 7ff5f67fc700 1 --2- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7ff5e00367e0 0x7ff5e0038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009990 tx=0x7ff5ec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:24.907+0000 7ff5df7fe700 1 -- 172.21.15.7:0/4097034676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7ff5e8018940 con 0x7ff5f8101bb0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.115+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5f80fd7f0 con 0x7ff5e00367e0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.119+0000 7ff5df7fe700 1 -- 172.21.15.7:0/4097034676 <== mgr.14118 v2:172.21.15.7:6800/3435258711 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7ff5f80fd7f0 con 0x7ff5e00367e0 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7ff5e00367e0 msgr2=0x7ff5e0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 --2- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7ff5e00367e0 0x7ff5e0038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009990 tx=0x7ff5ec006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 msgr2=0x7ff5f81013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:25.699 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 --2- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f81013b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8006d40 tx=0x7ff5e8009250 comp rx=0 tx=0).stop 2024-03-16T08:52:25.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 shutdown_connections 2024-03-16T08:52:25.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 --2- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7ff5e00367e0 0x7ff5e0038c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:25.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 --2- 172.21.15.7:0/4097034676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5f8101bb0 0x7ff5f81013b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:25.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 >> 172.21.15.7:0/4097034676 conn(0x7ff5f80fc500 msgr2=0x7ff5f80fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:25.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.121+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 shutdown_connections 2024-03-16T08:52:25.700 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:25.122+0000 7ff5fd27c700 1 -- 172.21.15.7:0/4097034676 wait complete. 2024-03-16T08:52:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:26 smithi007 ceph-mon[29470]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:26 smithi007 ceph-mon[29470]: Saving service alertmanager spec with placement count:1 2024-03-16T08:52:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:26 smithi007 ceph-mon[29470]: from='mgr.14118 172.21.15.7:0/2002449092' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:26 smithi007 ceph-mon[29470]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:52:27.474 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:27 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3649901926' entity='client.admin' 2024-03-16T08:52:27.481 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d093a1700 1 Processor -- start 2024-03-16T08:52:27.481 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d093a1700 1 -- start start 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d093a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d093a1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d04102ee0 con 0x7f2d04102590 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57706/0 (socket says 172.21.15.7:57706) 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d03fff700 1 -- 172.21.15.7:0/2713620377 learned_addr learned my addr 172.21.15.7:0/2713620377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d03fff700 1 -- 172.21.15.7:0/2713620377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d04103020 con 0x7f2d04102590 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.091+0000 7f2d03fff700 1 --2- 172.21.15.7:0/2713620377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041029a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40098e0 tx=0x7f2cf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f3dd5158c11264 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d02ffd700 1 -- 172.21.15.7:0/2713620377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cf400b860 con 0x7f2d04102590 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d02ffd700 1 -- 172.21.15.7:0/2713620377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cf400be90 con 0x7f2d04102590 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d02ffd700 1 -- 172.21.15.7:0/2713620377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cf40103c0 con 0x7f2d04102590 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d093a1700 1 -- 172.21.15.7:0/2713620377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 msgr2=0x7f2d041029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d093a1700 1 --2- 172.21.15.7:0/2713620377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041029a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40098e0 tx=0x7f2cf4009250 comp rx=0 tx=0).stop 2024-03-16T08:52:27.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d093a1700 1 -- 172.21.15.7:0/2713620377 shutdown_connections 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d093a1700 1 --2- 172.21.15.7:0/2713620377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041029a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d093a1700 1 -- 172.21.15.7:0/2713620377 >> 172.21.15.7:0/2713620377 conn(0x7f2d040fdd50 msgr2=0x7f2d04100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d093a1700 1 -- 172.21.15.7:0/2713620377 shutdown_connections 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.092+0000 7f2d093a1700 1 -- 172.21.15.7:0/2713620377 wait complete. 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.093+0000 7f2d093a1700 1 Processor -- start 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.093+0000 7f2d093a1700 1 -- start start 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.093+0000 7f2d093a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041113a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.093+0000 7f2d093a1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d041118e0 con 0x7f2d04102590 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.093+0000 7f2d03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.093+0000 7f2d03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041113a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57710/0 (socket says 172.21.15.7:57710) 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.093+0000 7f2d03fff700 1 -- 172.21.15.7:0/3649901926 learned_addr learned my addr 172.21.15.7:0/3649901926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.094+0000 7f2d03fff700 1 -- 172.21.15.7:0/3649901926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40094f0 con 0x7f2d04102590 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.094+0000 7f2d03fff700 1 --2- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041113a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40090b0 tx=0x7f2cf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.094+0000 7f2d00ff9700 1 -- 172.21.15.7:0/3649901926 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cf4010840 con 0x7f2d04102590 2024-03-16T08:52:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.094+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d04111ae0 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.094+0000 7f2d00ff9700 1 -- 172.21.15.7:0/3649901926 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cf4010e70 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.094+0000 7f2d00ff9700 1 -- 172.21.15.7:0/3649901926 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cf4024e40 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.094+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d04112650 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.095+0000 7f2d00ff9700 1 -- 172.21.15.7:0/3649901926 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2cf401a530 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.095+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0405f5f0 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.095+0000 7f2d00ff9700 1 --2- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f2cec036740 0x7f2cec038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.095+0000 7f2d00ff9700 1 -- 172.21.15.7:0/3649901926 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f2cf404a7c0 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.095+0000 7f2d037fe700 1 --2- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f2cec036740 0x7f2cec038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.095+0000 7f2d037fe700 1 --2- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f2cec036740 0x7f2cec038bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009990 tx=0x7f2cf8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.100+0000 7f2d00ff9700 1 -- 172.21.15.7:0/3649901926 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f2cf4018940 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.300+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f2d0404bed0 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.303+0000 7f2d00ff9700 1 -- 172.21.15.7:0/3649901926 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f2cf4018350 con 0x7f2d04102590 2024-03-16T08:52:27.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f2cec036740 msgr2=0x7f2cec038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 --2- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f2cec036740 0x7f2cec038bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009990 tx=0x7f2cf8006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 msgr2=0x7f2d041113a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 --2- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041113a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2cf40090b0 tx=0x7f2cf4009250 comp rx=0 tx=0).stop 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 shutdown_connections 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 --2- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f2cec036740 0x7f2cec038bf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 --2- 172.21.15.7:0/3649901926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d04102590 0x7f2d041113a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 >> 172.21.15.7:0/3649901926 conn(0x7f2d040fdd50 msgr2=0x7f2d040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 shutdown_connections 2024-03-16T08:52:27.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:26.305+0000 7f2d093a1700 1 -- 172.21.15.7:0/3649901926 wait complete. 2024-03-16T08:52:28.714 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:28 smithi007 ceph-mon[29470]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:52:28.714 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:28 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/4028778057' entity='client.admin' 2024-03-16T08:52:28.721 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.928+0000 7f9eaf477700 1 Processor -- start 2024-03-16T08:52:28.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.928+0000 7f9eaf477700 1 -- start start 2024-03-16T08:52:28.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.928+0000 7f9eaf477700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8105310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:28.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.928+0000 7f9eaf477700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8105850 con 0x7f9ea8076be0 2024-03-16T08:52:28.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.928+0000 7f9ead213700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8105310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:28.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.929+0000 7f9ead213700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8105310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57714/0 (socket says 172.21.15.7:57714) 2024-03-16T08:52:28.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.929+0000 7f9ead213700 1 -- 172.21.15.7:0/2007020419 learned_addr learned my addr 172.21.15.7:0/2007020419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:28.722 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.929+0000 7f9ead213700 1 -- 172.21.15.7:0/2007020419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8105990 con 0x7f9ea8076be0 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.929+0000 7f9ead213700 1 --2- 172.21.15.7:0/2007020419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8105310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40098e0 tx=0x7f9ea4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139c0f2886de70f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.929+0000 7f9e9ffff700 1 -- 172.21.15.7:0/2007020419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ea400b860 con 0x7f9ea8076be0 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.929+0000 7f9e9ffff700 1 -- 172.21.15.7:0/2007020419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ea400be90 con 0x7f9ea8076be0 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.929+0000 7f9e9ffff700 1 -- 172.21.15.7:0/2007020419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ea40103c0 con 0x7f9ea8076be0 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.930+0000 7f9eaf477700 1 -- 172.21.15.7:0/2007020419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 msgr2=0x7f9ea8105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.930+0000 7f9eaf477700 1 --2- 172.21.15.7:0/2007020419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8105310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40098e0 tx=0x7f9ea4009250 comp rx=0 tx=0).stop 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.930+0000 7f9eaf477700 1 -- 172.21.15.7:0/2007020419 shutdown_connections 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.930+0000 7f9eaf477700 1 --2- 172.21.15.7:0/2007020419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8105310 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.930+0000 7f9eaf477700 1 -- 172.21.15.7:0/2007020419 >> 172.21.15.7:0/2007020419 conn(0x7f9ea80fcdc0 msgr2=0x7f9ea80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.930+0000 7f9eaf477700 1 -- 172.21.15.7:0/2007020419 shutdown_connections 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.930+0000 7f9eaf477700 1 -- 172.21.15.7:0/2007020419 wait complete. 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.930+0000 7f9eaf477700 1 Processor -- start 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9eaf477700 1 -- start start 2024-03-16T08:52:28.723 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9eaf477700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9eaf477700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81031a0 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9ead213700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9ead213700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57722/0 (socket says 172.21.15.7:57722) 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9ead213700 1 -- 172.21.15.7:0/4028778057 learned_addr learned my addr 172.21.15.7:0/4028778057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9ead213700 1 -- 172.21.15.7:0/4028778057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40094f0 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9ead213700 1 --2- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8102c60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40094c0 tx=0x7f9ea4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9e9dffb700 1 -- 172.21.15.7:0/4028778057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ea4010890 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9e9dffb700 1 -- 172.21.15.7:0/4028778057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ea4010ec0 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea8101210 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.931+0000 7f9e9dffb700 1 -- 172.21.15.7:0/4028778057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ea40243f0 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.932+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea8101700 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.932+0000 7f9e9dffb700 1 -- 172.21.15.7:0/4028778057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f9ea402d3f0 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.933+0000 7f9e9dffb700 1 --2- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9e940367e0 0x7f9e94038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.933+0000 7f9e9dffb700 1 -- 172.21.15.7:0/4028778057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9ea404a5b0 con 0x7f9ea8076be0 2024-03-16T08:52:28.724 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.933+0000 7f9eaca12700 1 --2- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9e940367e0 0x7f9e94038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.933+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea804bed0 con 0x7f9ea8076be0 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.933+0000 7f9eaca12700 1 --2- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9e940367e0 0x7f9e94038c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009990 tx=0x7f9e98006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:27.938+0000 7f9e9dffb700 1 -- 172.21.15.7:0/4028778057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f9ea402b8e0 con 0x7f9ea8076be0 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.130+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f9ea8106720 con 0x7f9ea8076be0 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.133+0000 7f9e9dffb700 1 -- 172.21.15.7:0/4028778057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f9ea402c350 con 0x7f9ea8076be0 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.135+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9e940367e0 msgr2=0x7f9e94038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.135+0000 7f9eaf477700 1 --2- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9e940367e0 0x7f9e94038c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009990 tx=0x7f9e98006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.135+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 msgr2=0x7f9ea8102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.135+0000 7f9eaf477700 1 --2- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8102c60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40094c0 tx=0x7f9ea4009250 comp rx=0 tx=0).stop 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.136+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 shutdown_connections 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.136+0000 7f9eaf477700 1 --2- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f9e940367e0 0x7f9e94038c90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.136+0000 7f9eaf477700 1 --2- 172.21.15.7:0/4028778057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ea8076be0 0x7f9ea8102c60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.136+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 >> 172.21.15.7:0/4028778057 conn(0x7f9ea80fcdc0 msgr2=0x7f9ea80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.136+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 shutdown_connections 2024-03-16T08:52:28.725 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:28.136+0000 7f9eaf477700 1 -- 172.21.15.7:0/4028778057 wait complete. 2024-03-16T08:52:28.726 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-03-16T08:52:29.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:29 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3029394237' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-16T08:52:30.095 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.114+0000 7fbf4034a700 1 Processor -- start 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.114+0000 7fbf4034a700 1 -- start start 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.115+0000 7fbf4034a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38101140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.115+0000 7fbf4034a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38101680 con 0x7fbf38100d30 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.115+0000 7fbf3e0e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.115+0000 7fbf3e0e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57724/0 (socket says 172.21.15.7:57724) 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.115+0000 7fbf3e0e6700 1 -- 172.21.15.7:0/3373057653 learned_addr learned my addr 172.21.15.7:0/3373057653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.115+0000 7fbf3e0e6700 1 -- 172.21.15.7:0/3373057653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf381017c0 con 0x7fbf38100d30 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.115+0000 7fbf3e0e6700 1 --2- 172.21.15.7:0/3373057653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38101140 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0098e0 tx=0x7fbf2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2614253b5109a63 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf3d0e4700 1 -- 172.21.15.7:0/3373057653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf2c00b860 con 0x7fbf38100d30 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf3d0e4700 1 -- 172.21.15.7:0/3373057653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf2c00be90 con 0x7fbf38100d30 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf3d0e4700 1 -- 172.21.15.7:0/3373057653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf2c0103c0 con 0x7fbf38100d30 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf4034a700 1 -- 172.21.15.7:0/3373057653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 msgr2=0x7fbf38101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:30.096 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf4034a700 1 --2- 172.21.15.7:0/3373057653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38101140 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0098e0 tx=0x7fbf2c009250 comp rx=0 tx=0).stop 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf4034a700 1 -- 172.21.15.7:0/3373057653 shutdown_connections 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf4034a700 1 --2- 172.21.15.7:0/3373057653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38101140 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf4034a700 1 -- 172.21.15.7:0/3373057653 >> 172.21.15.7:0/3373057653 conn(0x7fbf380fc4d0 msgr2=0x7fbf380fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf4034a700 1 -- 172.21.15.7:0/3373057653 shutdown_connections 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.116+0000 7fbf4034a700 1 -- 172.21.15.7:0/3373057653 wait complete. 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.117+0000 7fbf4034a700 1 Processor -- start 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.117+0000 7fbf4034a700 1 -- start start 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.117+0000 7fbf4034a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.117+0000 7fbf4034a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38111ad0 con 0x7fbf38100d30 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.117+0000 7fbf3e0e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.117+0000 7fbf3e0e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57726/0 (socket says 172.21.15.7:57726) 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.117+0000 7fbf3e0e6700 1 -- 172.21.15.7:0/3029394237 learned_addr learned my addr 172.21.15.7:0/3029394237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.118+0000 7fbf3e0e6700 1 -- 172.21.15.7:0/3029394237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0094f0 con 0x7fbf38100d30 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.118+0000 7fbf3e0e6700 1 --2- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38111590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c011680 tx=0x7fbf2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:30.097 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.118+0000 7fbf2affd700 1 -- 172.21.15.7:0/3029394237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf2c010890 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.118+0000 7fbf2affd700 1 -- 172.21.15.7:0/3029394237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf2c010ec0 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.118+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf38111cd0 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.118+0000 7fbf2affd700 1 -- 172.21.15.7:0/3029394237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf2c0243f0 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.118+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf38112840 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.119+0000 7fbf2affd700 1 -- 172.21.15.7:0/3029394237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbf2c02d3f0 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.119+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3805f5f0 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.119+0000 7fbf2affd700 1 --2- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fbf240367d0 0x7fbf24038c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.119+0000 7fbf2affd700 1 -- 172.21.15.7:0/3029394237 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbf2c04a370 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.119+0000 7fbf3d8e5700 1 --2- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fbf240367d0 0x7fbf24038c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.120+0000 7fbf3d8e5700 1 --2- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fbf240367d0 0x7fbf24038c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009990 tx=0x7fbf34006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.124+0000 7fbf2affd700 1 -- 172.21.15.7:0/3029394237 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fbf2c006d70 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.371+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fbf3804bed0 con 0x7fbf38100d30 2024-03-16T08:52:30.098 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.516+0000 7fbf2affd700 1 -- 172.21.15.7:0/3029394237 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fbf2c01a430 con 0x7fbf38100d30 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.517+0000 7fbf2affd700 1 -- 172.21.15.7:0/3029394237 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fbf2c02cb20 con 0x7fbf38100d30 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fbf240367d0 msgr2=0x7fbf24038c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 --2- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fbf240367d0 0x7fbf24038c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009990 tx=0x7fbf34006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 msgr2=0x7fbf38111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 --2- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38111590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c011680 tx=0x7fbf2c009250 comp rx=0 tx=0).stop 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 shutdown_connections 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 --2- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7fbf240367d0 0x7fbf24038c80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 --2- 172.21.15.7:0/3029394237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf38100d30 0x7fbf38111590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 >> 172.21.15.7:0/3029394237 conn(0x7fbf380fc4d0 msgr2=0x7fbf38105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 shutdown_connections 2024-03-16T08:52:30.099 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:29.519+0000 7fbf4034a700 1 -- 172.21.15.7:0/3029394237 wait complete. 2024-03-16T08:52:30.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:30 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3029394237' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-16T08:52:30.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:30 smithi007 ceph-mon[29470]: mgrmap e9: smithi007.uamwtp(active, since 24s) 2024-03-16T08:52:31.446 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:52:31.446 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-16T08:52:31.446 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T08:52:31.446 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "smithi007.uamwtp", 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.571+0000 7f9099ae8700 1 Processor -- start 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.571+0000 7f9099ae8700 1 -- start start 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.571+0000 7f9099ae8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f90940771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.571+0000 7f9099ae8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094079090 con 0x7f9094078c80 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.571+0000 7f90937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f90940771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.572+0000 7f90937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f90940771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57746/0 (socket says 172.21.15.7:57746) 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.572+0000 7f90937fe700 1 -- 172.21.15.7:0/2788073379 learned_addr learned my addr 172.21.15.7:0/2788073379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.572+0000 7f90937fe700 1 -- 172.21.15.7:0/2788073379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094077720 con 0x7f9094078c80 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.572+0000 7f90937fe700 1 --2- 172.21.15.7:0/2788073379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f90940771e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f90880098e0 tx=0x7f9088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95d70bae5b51262 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.572+0000 7f90927fc700 1 -- 172.21.15.7:0/2788073379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f908800b860 con 0x7f9094078c80 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.572+0000 7f90927fc700 1 -- 172.21.15.7:0/2788073379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f908800be90 con 0x7f9094078c80 2024-03-16T08:52:31.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.572+0000 7f90927fc700 1 -- 172.21.15.7:0/2788073379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90880103c0 con 0x7f9094078c80 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.573+0000 7f9099ae8700 1 -- 172.21.15.7:0/2788073379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 msgr2=0x7f90940771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.573+0000 7f9099ae8700 1 --2- 172.21.15.7:0/2788073379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f90940771e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f90880098e0 tx=0x7f9088009250 comp rx=0 tx=0).stop 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.573+0000 7f9099ae8700 1 -- 172.21.15.7:0/2788073379 shutdown_connections 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.573+0000 7f9099ae8700 1 --2- 172.21.15.7:0/2788073379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f90940771e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.573+0000 7f9099ae8700 1 -- 172.21.15.7:0/2788073379 >> 172.21.15.7:0/2788073379 conn(0x7f90940fde50 msgr2=0x7f90941002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.573+0000 7f9099ae8700 1 -- 172.21.15.7:0/2788073379 shutdown_connections 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.573+0000 7f9099ae8700 1 -- 172.21.15.7:0/2788073379 wait complete. 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.574+0000 7f9099ae8700 1 Processor -- start 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.574+0000 7f9099ae8700 1 -- start start 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.574+0000 7f9099ae8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f9094113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.574+0000 7f9099ae8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9094113d40 con 0x7f9094078c80 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.574+0000 7f90937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f9094113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.574+0000 7f90937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f9094113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57760/0 (socket says 172.21.15.7:57760) 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.574+0000 7f90937fe700 1 -- 172.21.15.7:0/2763031419 learned_addr learned my addr 172.21.15.7:0/2763031419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:31.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.574+0000 7f90937fe700 1 -- 172.21.15.7:0/2763031419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90880094f0 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.575+0000 7f90937fe700 1 --2- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f9094113800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9088006fd0 tx=0x7f9088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.575+0000 7f907bfff700 1 -- 172.21.15.7:0/2763031419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9088010630 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.575+0000 7f907bfff700 1 -- 172.21.15.7:0/2763031419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9088010c60 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.575+0000 7f907bfff700 1 -- 172.21.15.7:0/2763031419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f908801ad40 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.575+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9094113f40 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.575+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9094114ab0 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.576+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909405f5f0 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.576+0000 7f907bfff700 1 -- 172.21.15.7:0/2763031419 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f908801a5f0 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.576+0000 7f907bfff700 1 --2- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f907c036830 0x7f907c038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.576+0000 7f907bfff700 1 -- 172.21.15.7:0/2763031419 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9088049fe0 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.576+0000 7f9092ffd700 1 -- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f907c036830 msgr2=0x7f907c038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3435258711 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.576+0000 7f9092ffd700 1 --2- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f907c036830 0x7f907c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.584+0000 7f907bfff700 1 -- 172.21.15.7:0/2763031419 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f9088023950 con 0x7f9094078c80 2024-03-16T08:52:31.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.777+0000 7f9092ffd700 1 -- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f907c036830 msgr2=0x7f907c038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3435258711 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.777+0000 7f9092ffd700 1 --2- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f907c036830 0x7f907c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.836+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9094114e00 con 0x7f9094078c80 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.837+0000 7f907bfff700 1 -- 172.21.15.7:0/2763031419 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f908802a070 con 0x7f9094078c80 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f907c036830 msgr2=0x7f907c038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 --2- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f907c036830 0x7f907c038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 msgr2=0x7f9094113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 --2- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f9094113800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9088006fd0 tx=0x7f9088009250 comp rx=0 tx=0).stop 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 shutdown_connections 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 --2- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f907c036830 0x7f907c038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 --2- 172.21.15.7:0/2763031419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094078c80 0x7f9094113800 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 >> 172.21.15.7:0/2763031419 conn(0x7f90940fde50 msgr2=0x7f90940fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 shutdown_connections 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:30.839+0000 7f9099ae8700 1 -- 172.21.15.7:0/2763031419 wait complete. 2024-03-16T08:52:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-03-16T08:52:31.451 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-03-16T08:52:31.657 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:31 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/2763031419' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T08:52:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: Active manager daemon smithi007.uamwtp restarted 2024-03-16T08:52:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: Activating manager daemon smithi007.uamwtp 2024-03-16T08:52:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: osdmap e3: 0 total, 0 up, 0 in 2024-03-16T08:52:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: mgrmap e10: smithi007.uamwtp(active, starting, since 0.003113s) 2024-03-16T08:52:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:52:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T08:52:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:52:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:52:36.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:52:36.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: Manager daemon smithi007.uamwtp is now available 2024-03-16T08:52:36.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:52:36.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:36 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:52:37.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:37 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:52:37.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:37 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T08:52:37.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:37 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:37.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:37 smithi007 ceph-mon[29470]: mgrmap e11: smithi007.uamwtp(active, since 1.0059s) 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.887+0000 7f88f5df4700 1 Processor -- start 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.887+0000 7f88f5df4700 1 -- start start 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.888+0000 7f88f5df4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f01052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.888+0000 7f88f5df4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0105820 con 0x7f88f0071360 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.888+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f01052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:37.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.888+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f01052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57766/0 (socket says 172.21.15.7:57766) 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.888+0000 7f88ef7fe700 1 -- 172.21.15.7:0/2011744957 learned_addr learned my addr 172.21.15.7:0/2011744957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.888+0000 7f88ef7fe700 1 -- 172.21.15.7:0/2011744957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f0105960 con 0x7f88f0071360 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.888+0000 7f88ef7fe700 1 --2- 172.21.15.7:0/2011744957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f01052e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f88e00098e0 tx=0x7f88e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e859f9293486865 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.888+0000 7f88ee7fc700 1 -- 172.21.15.7:0/2011744957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88e000b860 con 0x7f88f0071360 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88ee7fc700 1 -- 172.21.15.7:0/2011744957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88e000be90 con 0x7f88f0071360 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88ee7fc700 1 -- 172.21.15.7:0/2011744957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88e00103c0 con 0x7f88f0071360 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88f5df4700 1 -- 172.21.15.7:0/2011744957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 msgr2=0x7f88f01052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88f5df4700 1 --2- 172.21.15.7:0/2011744957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f01052e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f88e00098e0 tx=0x7f88e0009250 comp rx=0 tx=0).stop 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88f5df4700 1 -- 172.21.15.7:0/2011744957 shutdown_connections 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88f5df4700 1 --2- 172.21.15.7:0/2011744957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f01052e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88f5df4700 1 -- 172.21.15.7:0/2011744957 >> 172.21.15.7:0/2011744957 conn(0x7f88f00fcd50 msgr2=0x7f88f00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88f5df4700 1 -- 172.21.15.7:0/2011744957 shutdown_connections 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.889+0000 7f88f5df4700 1 -- 172.21.15.7:0/2011744957 wait complete. 2024-03-16T08:52:37.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.890+0000 7f88f5df4700 1 Processor -- start 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.890+0000 7f88f5df4700 1 -- start start 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.890+0000 7f88f5df4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f0102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.890+0000 7f88f5df4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0103170 con 0x7f88f0071360 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.890+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f0102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.890+0000 7f88ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f0102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57774/0 (socket says 172.21.15.7:57774) 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.890+0000 7f88ef7fe700 1 -- 172.21.15.7:0/255509784 learned_addr learned my addr 172.21.15.7:0/255509784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.891+0000 7f88ef7fe700 1 -- 172.21.15.7:0/255509784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00094f0 con 0x7f88f0071360 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.891+0000 7f88ef7fe700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f0102c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88e00094c0 tx=0x7f88e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.891+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88e00118f0 con 0x7f88f0071360 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.891+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f01011e0 con 0x7f88f0071360 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.891+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88e0010d60 con 0x7f88f0071360 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.891+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88e001a430 con 0x7f88f0071360 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.891+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f01016d0 con 0x7f88f0071360 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.892+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f88e001a590 con 0x7f88f0071360 2024-03-16T08:52:37.867 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.892+0000 7f88d7fff700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 0x7f88d8038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.892+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 --> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88f004bed0 con 0x7f88d80367e0 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.892+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f88e004a530 con 0x7f88f0071360 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.892+0000 7f88eeffd700 1 -- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 msgr2=0x7f88d8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3435258711 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:31.892+0000 7f88eeffd700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 0x7f88d8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:32.093+0000 7f88eeffd700 1 -- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 msgr2=0x7f88d8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3435258711 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:32.093+0000 7f88eeffd700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 0x7f88d8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:32.494+0000 7f88eeffd700 1 -- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 msgr2=0x7f88d8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3435258711 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:32.494+0000 7f88eeffd700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 0x7f88d8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:33.295+0000 7f88eeffd700 1 -- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 msgr2=0x7f88d8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3435258711 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:33.295+0000 7f88eeffd700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 0x7f88d8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:34.897+0000 7f88eeffd700 1 -- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 msgr2=0x7f88d8038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3435258711 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:34.897+0000 7f88eeffd700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 0x7f88d8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:36.182+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f88e0024440 con 0x7f88f0071360 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:36.182+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 msgr2=0x7f88d8038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:52:37.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:36.182+0000 7f88d7fff700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 0x7f88d8038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.183+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f88e002d770 con 0x7f88f0071360 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.184+0000 7f88d7fff700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f88d8039ff0 0x7f88d803c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.184+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 --> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88f004bed0 con 0x7f88d8039ff0 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.184+0000 7f88eeffd700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f88d8039ff0 0x7f88d803c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.185+0000 7f88eeffd700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f88d8039ff0 0x7f88d803c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f88e40039d0 tx=0x7f88e40092f0 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.185+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mgr.14160 v2:172.21.15.7:6800/2343923853 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f88f004bed0 con 0x7f88d8039ff0 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.191+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 --> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f88f0106740 con 0x7f88d8039ff0 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.191+0000 7f88d7fff700 1 -- 172.21.15.7:0/255509784 <== mgr.14160 v2:172.21.15.7:6800/2343923853 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f88f0106740 con 0x7f88d8039ff0 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f88d8039ff0 msgr2=0x7f88d803c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f88d8039ff0 0x7f88d803c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f88e40039d0 tx=0x7f88e40092f0 comp rx=0 tx=0).stop 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 msgr2=0x7f88f0102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f0102c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88e00094c0 tx=0x7f88e0009250 comp rx=0 tx=0).stop 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 shutdown_connections 2024-03-16T08:52:37.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f88d8039ff0 0x7f88d803c3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:37.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f0071360 0x7f88f0102c30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:37.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 --2- 172.21.15.7:0/255509784 >> [v2:172.21.15.7:6800/3435258711,v1:172.21.15.7:6801/3435258711] conn(0x7f88d80367e0 0x7f88d8038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:37.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 >> 172.21.15.7:0/255509784 conn(0x7f88f00fcd50 msgr2=0x7f88f00fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:37.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 shutdown_connections 2024-03-16T08:52:37.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:37.192+0000 7f88f5df4700 1 -- 172.21.15.7:0/255509784 wait complete. 2024-03-16T08:52:37.870 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-03-16T08:52:37.870 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-03-16T08:52:38.487 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:38 smithi007 ceph-mon[29470]: [16/Mar/2024:08:52:36] ENGINE Bus STARTING 2024-03-16T08:52:38.488 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:38 smithi007 ceph-mon[29470]: [16/Mar/2024:08:52:36] ENGINE Serving on https://172.21.15.7:7150 2024-03-16T08:52:38.488 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:38 smithi007 ceph-mon[29470]: [16/Mar/2024:08:52:36] ENGINE Bus STARTED 2024-03-16T08:52:38.488 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:38 smithi007 ceph-mon[29470]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T08:52:38.488 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:38 smithi007 ceph-mon[29470]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T08:52:39.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-16T08:52:39.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.274+0000 7f9149b89700 1 Processor -- start 2024-03-16T08:52:39.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.274+0000 7f9149b89700 1 -- start start 2024-03-16T08:52:39.145 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.275+0000 7f9149b89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.275+0000 7f9149b89700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9144103080 con 0x7f9144102730 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.275+0000 7f91437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.275+0000 7f91437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60126/0 (socket says 172.21.15.7:60126) 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.275+0000 7f91437fe700 1 -- 172.21.15.7:0/679285356 learned_addr learned my addr 172.21.15.7:0/679285356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.275+0000 7f91437fe700 1 -- 172.21.15.7:0/679285356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91441031c0 con 0x7f9144102730 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.275+0000 7f91437fe700 1 --2- 172.21.15.7:0/679285356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144102b40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9134009e50 tx=0x7f9134009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac2dda673c5c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.275+0000 7f91427fc700 1 -- 172.21.15.7:0/679285356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91340092e0 con 0x7f9144102730 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f91427fc700 1 -- 172.21.15.7:0/679285356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9134012430 con 0x7f9144102730 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f91427fc700 1 -- 172.21.15.7:0/679285356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91340075a0 con 0x7f9144102730 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f9149b89700 1 -- 172.21.15.7:0/679285356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 msgr2=0x7f9144102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f9149b89700 1 --2- 172.21.15.7:0/679285356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144102b40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9134009e50 tx=0x7f9134009840 comp rx=0 tx=0).stop 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f9149b89700 1 -- 172.21.15.7:0/679285356 shutdown_connections 2024-03-16T08:52:39.146 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f9149b89700 1 --2- 172.21.15.7:0/679285356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144102b40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f9149b89700 1 -- 172.21.15.7:0/679285356 >> 172.21.15.7:0/679285356 conn(0x7f91440fdf10 msgr2=0x7f9144100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f9149b89700 1 -- 172.21.15.7:0/679285356 shutdown_connections 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.276+0000 7f9149b89700 1 -- 172.21.15.7:0/679285356 wait complete. 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f9149b89700 1 Processor -- start 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f9149b89700 1 -- start start 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f9149b89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f9149b89700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9144111bb0 con 0x7f9144102730 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f91437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f91437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144111670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60136/0 (socket says 172.21.15.7:60136) 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f91437fe700 1 -- 172.21.15.7:0/1656863527 learned_addr learned my addr 172.21.15.7:0/1656863527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f91437fe700 1 -- 172.21.15.7:0/1656863527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9134009ae0 con 0x7f9144102730 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.277+0000 7f91437fe700 1 --2- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144111670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9134009fd0 tx=0x7f9134009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.278+0000 7f913bfff700 1 -- 172.21.15.7:0/1656863527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91340092e0 con 0x7f9144102730 2024-03-16T08:52:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.278+0000 7f913bfff700 1 -- 172.21.15.7:0/1656863527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9134007ce0 con 0x7f9144102730 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.278+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9144111db0 con 0x7f9144102730 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.278+0000 7f913bfff700 1 -- 172.21.15.7:0/1656863527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9134025960 con 0x7f9144102730 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.278+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9144112920 con 0x7f9144102730 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.278+0000 7f913bfff700 1 -- 172.21.15.7:0/1656863527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f9134019050 con 0x7f9144102730 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.279+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914405f5f0 con 0x7f9144102730 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.279+0000 7f913bfff700 1 --2- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f91300367e0 0x7f9130038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.279+0000 7f913bfff700 1 -- 172.21.15.7:0/1656863527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f913404a3b0 con 0x7f9144102730 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.279+0000 7f9142ffd700 1 --2- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f91300367e0 0x7f9130038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.279+0000 7f9142ffd700 1 --2- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f91300367e0 0x7f9130038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f912c009990 tx=0x7f912c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.283+0000 7f913bfff700 1 -- 172.21.15.7:0/1656863527 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f913402d350 con 0x7f9144102730 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.485+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 --> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f91440ff2e0 con 0x7f91300367e0 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.554+0000 7f913bfff700 1 -- 172.21.15.7:0/1656863527 <== mgr.14160 v2:172.21.15.7:6800/2343923853 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f91440ff2e0 con 0x7f91300367e0 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f91300367e0 msgr2=0x7f9130038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 --2- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f91300367e0 0x7f9130038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f912c009990 tx=0x7f912c006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 msgr2=0x7f9144111670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 --2- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144111670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9134009fd0 tx=0x7f9134009840 comp rx=0 tx=0).stop 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 shutdown_connections 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 --2- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f91300367e0 0x7f9130038c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 --2- 172.21.15.7:0/1656863527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9144102730 0x7f9144111670 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 >> 172.21.15.7:0/1656863527 conn(0x7f91440fdf10 msgr2=0x7f91440feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 shutdown_connections 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:38.556+0000 7f9149b89700 1 -- 172.21.15.7:0/1656863527 wait complete. 2024-03-16T08:52:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-03-16T08:52:39.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:39 smithi007 ceph-mon[29470]: mgrmap e12: smithi007.uamwtp(active, since 2s) 2024-03-16T08:52:39.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:39 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:39.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:39 smithi007 ceph-mon[29470]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:39.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:39 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:39.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:39 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:39.405 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:39 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:41.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:41 smithi007 ceph-mon[29470]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:41.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:41 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$V9Px/d3dSV.7KNxBSZAtt.9tKUckPxTItgdjWyqjKV/97kG66Ghna", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710579160, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.632+0000 7fcf9d900700 1 Processor -- start 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.632+0000 7fcf9d900700 1 -- start start 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.632+0000 7fcf9d900700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98107740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.632+0000 7fcf9d900700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf98103130 con 0x7fcf98071300 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.632+0000 7fcf96ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98107740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.632+0000 7fcf96ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98107740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60144/0 (socket says 172.21.15.7:60144) 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.632+0000 7fcf96ffd700 1 -- 172.21.15.7:0/1721573177 learned_addr learned my addr 172.21.15.7:0/1721573177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.633+0000 7fcf96ffd700 1 -- 172.21.15.7:0/1721573177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf98107c80 con 0x7fcf98071300 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.633+0000 7fcf96ffd700 1 --2- 172.21.15.7:0/1721573177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98107740 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009e50 tx=0x7fcf90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c14a4900e69d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.633+0000 7fcf95ffb700 1 -- 172.21.15.7:0/1721573177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf900092e0 con 0x7fcf98071300 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.633+0000 7fcf95ffb700 1 -- 172.21.15.7:0/1721573177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf90012430 con 0x7fcf98071300 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.633+0000 7fcf95ffb700 1 -- 172.21.15.7:0/1721573177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf900075a0 con 0x7fcf98071300 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.633+0000 7fcf9d900700 1 -- 172.21.15.7:0/1721573177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 msgr2=0x7fcf98107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.633+0000 7fcf9d900700 1 --2- 172.21.15.7:0/1721573177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98107740 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009e50 tx=0x7fcf90009840 comp rx=0 tx=0).stop 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 -- 172.21.15.7:0/1721573177 shutdown_connections 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 --2- 172.21.15.7:0/1721573177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98107740 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 -- 172.21.15.7:0/1721573177 >> 172.21.15.7:0/1721573177 conn(0x7fcf980fcd50 msgr2=0x7fcf980ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 -- 172.21.15.7:0/1721573177 shutdown_connections 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 -- 172.21.15.7:0/1721573177 wait complete. 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 Processor -- start 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 -- start start 2024-03-16T08:52:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98113880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.634+0000 7fcf9d900700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf98113dc0 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf96ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98113880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf96ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98113880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60160/0 (socket says 172.21.15.7:60160) 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf96ffd700 1 -- 172.21.15.7:0/716844320 learned_addr learned my addr 172.21.15.7:0/716844320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf96ffd700 1 -- 172.21.15.7:0/716844320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf90009ae0 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf96ffd700 1 --2- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98113880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf90012de0 tx=0x7fcf90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf8f7fe700 1 -- 172.21.15.7:0/716844320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf900092e0 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf98113fc0 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf8f7fe700 1 -- 172.21.15.7:0/716844320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf90007ce0 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf8f7fe700 1 -- 172.21.15.7:0/716844320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf90025910 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.635+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf98114b30 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.636+0000 7fcf8f7fe700 1 -- 172.21.15.7:0/716844320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fcf90019050 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.636+0000 7fcf8f7fe700 1 --2- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fcf840367e0 0x7fcf84038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.636+0000 7fcf8f7fe700 1 -- 172.21.15.7:0/716844320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcf9004a1d0 con 0x7fcf98071300 2024-03-16T08:52:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.636+0000 7fcf967fc700 1 --2- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fcf840367e0 0x7fcf84038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.637+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9805f5f0 con 0x7fcf98071300 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.637+0000 7fcf967fc700 1 --2- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fcf840367e0 0x7fcf84038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009990 tx=0x7fcf80006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.642+0000 7fcf8f7fe700 1 -- 172.21.15.7:0/716844320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fcf90017020 con 0x7fcf98071300 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:39.871+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 --> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcf9805f410 con 0x7fcf840367e0 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.117+0000 7fcf8f7fe700 1 -- 172.21.15.7:0/716844320 <== mgr.14160 v2:172.21.15.7:6800/2343923853 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fcf9805f410 con 0x7fcf840367e0 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fcf840367e0 msgr2=0x7fcf84038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 --2- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fcf840367e0 0x7fcf84038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009990 tx=0x7fcf80006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 msgr2=0x7fcf98113880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 --2- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98113880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf90012de0 tx=0x7fcf90009840 comp rx=0 tx=0).stop 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 shutdown_connections 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 --2- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fcf840367e0 0x7fcf84038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 --2- 172.21.15.7:0/716844320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf98071300 0x7fcf98113880 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 >> 172.21.15.7:0/716844320 conn(0x7fcf980fcd50 msgr2=0x7fcf980fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 shutdown_connections 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:40.119+0000 7fcf9d900700 1 -- 172.21.15.7:0/716844320 wait complete. 2024-03-16T08:52:41.530 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-03-16T08:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-03-16T08:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.911+0000 7f5758b3e700 1 Processor -- start 2024-03-16T08:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.911+0000 7f5758b3e700 1 -- start start 2024-03-16T08:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.911+0000 7f5758b3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f5754102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.911+0000 7f5758b3e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5754103000 con 0x7f57541026b0 2024-03-16T08:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.911+0000 7f575259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f5754102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.911+0000 7f575259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f5754102ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60164/0 (socket says 172.21.15.7:60164) 2024-03-16T08:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.911+0000 7f575259c700 1 -- 172.21.15.7:0/1621538847 learned_addr learned my addr 172.21.15.7:0/1621538847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.911+0000 7f575259c700 1 -- 172.21.15.7:0/1621538847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5754103140 con 0x7f57541026b0 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f575259c700 1 --2- 172.21.15.7:0/1621538847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f5754102ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57440098e0 tx=0x7f5744009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47e58a6b3e89d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f575159a700 1 -- 172.21.15.7:0/1621538847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f574400b860 con 0x7f57541026b0 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f575159a700 1 -- 172.21.15.7:0/1621538847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f574400be90 con 0x7f57541026b0 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f575159a700 1 -- 172.21.15.7:0/1621538847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57440103c0 con 0x7f57541026b0 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f5758b3e700 1 -- 172.21.15.7:0/1621538847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 msgr2=0x7f5754102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f5758b3e700 1 --2- 172.21.15.7:0/1621538847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f5754102ac0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57440098e0 tx=0x7f5744009250 comp rx=0 tx=0).stop 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f5758b3e700 1 -- 172.21.15.7:0/1621538847 shutdown_connections 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f5758b3e700 1 --2- 172.21.15.7:0/1621538847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f5754102ac0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.912+0000 7f5758b3e700 1 -- 172.21.15.7:0/1621538847 >> 172.21.15.7:0/1621538847 conn(0x7f57540fde70 msgr2=0x7f5754100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f5758b3e700 1 -- 172.21.15.7:0/1621538847 shutdown_connections 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f5758b3e700 1 -- 172.21.15.7:0/1621538847 wait complete. 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f5758b3e700 1 Processor -- start 2024-03-16T08:52:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f5758b3e700 1 -- start start 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f5758b3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f57541115a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f5758b3e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5754111ae0 con 0x7f57541026b0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f575259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f57541115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f575259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f57541115a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60168/0 (socket says 172.21.15.7:60168) 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.913+0000 7f575259c700 1 -- 172.21.15.7:0/870945830 learned_addr learned my addr 172.21.15.7:0/870945830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.914+0000 7f575259c700 1 -- 172.21.15.7:0/870945830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57440094f0 con 0x7f57541026b0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.914+0000 7f575259c700 1 --2- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f57541115a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5744011680 tx=0x7f5744009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.914+0000 7f5742ffd700 1 -- 172.21.15.7:0/870945830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5744010890 con 0x7f57541026b0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.914+0000 7f5742ffd700 1 -- 172.21.15.7:0/870945830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5744010ec0 con 0x7f57541026b0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.914+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5754111ce0 con 0x7f57541026b0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.914+0000 7f5742ffd700 1 -- 172.21.15.7:0/870945830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57440243f0 con 0x7f57541026b0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.914+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5754112850 con 0x7f57541026b0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.915+0000 7f5742ffd700 1 -- 172.21.15.7:0/870945830 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f574402d3f0 con 0x7f57541026b0 2024-03-16T08:52:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.915+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575405f5f0 con 0x7f57541026b0 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.915+0000 7f5742ffd700 1 --2- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f573c0367e0 0x7f573c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.915+0000 7f5742ffd700 1 -- 172.21.15.7:0/870945830 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f574404a4f0 con 0x7f57541026b0 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.915+0000 7f5751d9b700 1 --2- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f573c0367e0 0x7f573c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.915+0000 7f5751d9b700 1 --2- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f573c0367e0 0x7f573c038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5748009990 tx=0x7f5748006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:41.920+0000 7f5742ffd700 1 -- 172.21.15.7:0/870945830 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f574402b8e0 con 0x7f57541026b0 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.110+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f575404bed0 con 0x7f57541026b0 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.111+0000 7f5742ffd700 1 -- 172.21.15.7:0/870945830 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f574402c350 con 0x7f57541026b0 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f573c0367e0 msgr2=0x7f573c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 --2- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f573c0367e0 0x7f573c038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5748009990 tx=0x7f5748006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 msgr2=0x7f57541115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 --2- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f57541115a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5744011680 tx=0x7f5744009250 comp rx=0 tx=0).stop 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 shutdown_connections 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 --2- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f573c0367e0 0x7f573c038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 --2- 172.21.15.7:0/870945830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57541026b0 0x7f57541115a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:42.622 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 >> 172.21.15.7:0/870945830 conn(0x7f57540fde70 msgr2=0x7f57540fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:42.622 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 shutdown_connections 2024-03-16T08:52:42.622 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:42.113+0000 7f5758b3e700 1 -- 172.21.15.7:0/870945830 wait complete. 2024-03-16T08:52:42.625 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T08:52:42.626 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-16T08:52:42.632 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T08:52:42.632 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-16T08:52:42.633 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-16T08:52:42.633 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-16T08:52:42.635 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-03-16T08:52:42.635 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:42.635 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-03-16T08:52:42.635 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-03-16T08:52:42.635 INFO:teuthology.orchestra.run.smithi007.stdout: Password: 0n1aswdc37 2024-03-16T08:52:42.635 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:42.635 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config directory 2024-03-16T08:52:42.635 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling autotune for osd_memory_target 2024-03-16T08:52:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:42 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/870945830' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-16T08:52:43.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.044+0000 7fc37487e700 1 Processor -- start 2024-03-16T08:52:43.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.044+0000 7fc37487e700 1 -- start start 2024-03-16T08:52:43.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.044+0000 7fc37487e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc370101210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:43.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.044+0000 7fc37487e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc370101750 con 0x7fc370100e00 2024-03-16T08:52:43.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.044+0000 7fc36e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc370101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:43.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.044+0000 7fc36e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc370101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49062/0 (socket says 172.21.15.7:49062) 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.044+0000 7fc36e59c700 1 -- 172.21.15.7:0/2239031368 learned_addr learned my addr 172.21.15.7:0/2239031368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.044+0000 7fc36e59c700 1 -- 172.21.15.7:0/2239031368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc370101890 con 0x7fc370100e00 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc36e59c700 1 --2- 172.21.15.7:0/2239031368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc370101210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc36000bb70 tx=0x7fc36000b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de3586079f2544c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc36d59a700 1 -- 172.21.15.7:0/2239031368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc36000dbf0 con 0x7fc370100e00 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc36d59a700 1 -- 172.21.15.7:0/2239031368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc36000dd50 con 0x7fc370100e00 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc36d59a700 1 -- 172.21.15.7:0/2239031368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc360015730 con 0x7fc370100e00 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc37487e700 1 -- 172.21.15.7:0/2239031368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 msgr2=0x7fc370101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc37487e700 1 --2- 172.21.15.7:0/2239031368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc370101210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc36000bb70 tx=0x7fc36000b4e0 comp rx=0 tx=0).stop 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc37487e700 1 -- 172.21.15.7:0/2239031368 shutdown_connections 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc37487e700 1 --2- 172.21.15.7:0/2239031368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc370101210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.045+0000 7fc37487e700 1 -- 172.21.15.7:0/2239031368 >> 172.21.15.7:0/2239031368 conn(0x7fc3700fc500 msgr2=0x7fc3700fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc37487e700 1 -- 172.21.15.7:0/2239031368 shutdown_connections 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc37487e700 1 -- 172.21.15.7:0/2239031368 wait complete. 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc37487e700 1 Processor -- start 2024-03-16T08:52:43.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc37487e700 1 -- start start 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc37487e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc37010f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc37487e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37010f940 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc36e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc37010f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc36e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc37010f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49068/0 (socket says 172.21.15.7:49068) 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.046+0000 7fc36e59c700 1 -- 172.21.15.7:0/1479174029 learned_addr learned my addr 172.21.15.7:0/1479174029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.047+0000 7fc36e59c700 1 -- 172.21.15.7:0/1479174029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36000b780 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.047+0000 7fc36e59c700 1 --2- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc37010f400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc360000c00 tx=0x7fc36000b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.047+0000 7fc35effd700 1 -- 172.21.15.7:0/1479174029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc360016730 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.047+0000 7fc35effd700 1 -- 172.21.15.7:0/1479174029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc360021440 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.047+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37010fb40 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.047+0000 7fc35effd700 1 -- 172.21.15.7:0/1479174029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3600175e0 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.047+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3701106b0 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.048+0000 7fc35effd700 1 -- 172.21.15.7:0/1479174029 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fc36001f050 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.048+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37005f5f0 con 0x7fc370100e00 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.048+0000 7fc35effd700 1 --2- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc358036880 0x7fc358038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:43.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.048+0000 7fc35effd700 1 -- 172.21.15.7:0/1479174029 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc36004c730 con 0x7fc370100e00 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.048+0000 7fc36dd9b700 1 --2- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc358036880 0x7fc358038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.048+0000 7fc36dd9b700 1 --2- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc358036880 0x7fc358038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc364009990 tx=0x7fc364006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.053+0000 7fc35effd700 1 -- 172.21.15.7:0/1479174029 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fc3600215b0 con 0x7fc370100e00 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.242+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc37005f5f0 con 0x7fc370100e00 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.243+0000 7fc35effd700 1 -- 172.21.15.7:0/1479174029 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc360014ed0 con 0x7fc370100e00 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc358036880 msgr2=0x7fc358038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 --2- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc358036880 0x7fc358038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc364009990 tx=0x7fc364006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 msgr2=0x7fc37010f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 --2- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc37010f400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc360000c00 tx=0x7fc36000b4e0 comp rx=0 tx=0).stop 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 shutdown_connections 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 --2- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc358036880 0x7fc358038d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 --2- 172.21.15.7:0/1479174029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc370100e00 0x7fc37010f400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:43.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 >> 172.21.15.7:0/1479174029 conn(0x7fc3700fc500 msgr2=0x7fc370106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:43.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 shutdown_connections 2024-03-16T08:52:43.778 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:43.245+0000 7fc37487e700 1 -- 172.21.15.7:0/1479174029 wait complete. 2024-03-16T08:52:43.894 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:43 smithi007 ceph-mon[29470]: mgrmap e13: smithi007.uamwtp(active, since 6s) 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.263+0000 7f74b287e700 1 Processor -- start 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.263+0000 7f74b287e700 1 -- start start 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.263+0000 7f74b287e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.263+0000 7f74b287e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac1030c0 con 0x7f74ac102770 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.263+0000 7f74abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.263+0000 7f74abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49082/0 (socket says 172.21.15.7:49082) 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.263+0000 7f74abfff700 1 -- 172.21.15.7:0/523779013 learned_addr learned my addr 172.21.15.7:0/523779013 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.264+0000 7f74abfff700 1 -- 172.21.15.7:0/523779013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac103200 con 0x7f74ac102770 2024-03-16T08:52:45.290 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.264+0000 7f74abfff700 1 --2- 172.21.15.7:0/523779013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac102b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f749c0098e0 tx=0x7f749c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f4478301371e54 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.264+0000 7f74aaffd700 1 -- 172.21.15.7:0/523779013 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f749c00b860 con 0x7f74ac102770 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.264+0000 7f74aaffd700 1 -- 172.21.15.7:0/523779013 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f749c00be90 con 0x7f74ac102770 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.264+0000 7f74aaffd700 1 -- 172.21.15.7:0/523779013 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f749c0103c0 con 0x7f74ac102770 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 -- 172.21.15.7:0/523779013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 msgr2=0x7f74ac102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 --2- 172.21.15.7:0/523779013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac102b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f749c0098e0 tx=0x7f749c009250 comp rx=0 tx=0).stop 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 -- 172.21.15.7:0/523779013 shutdown_connections 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 --2- 172.21.15.7:0/523779013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac102b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 -- 172.21.15.7:0/523779013 >> 172.21.15.7:0/523779013 conn(0x7f74ac0fde90 msgr2=0x7f74ac100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 -- 172.21.15.7:0/523779013 shutdown_connections 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 -- 172.21.15.7:0/523779013 wait complete. 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 Processor -- start 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.265+0000 7f74b287e700 1 -- start start 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74b287e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac1115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74b287e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac111b10 con 0x7f74ac102770 2024-03-16T08:52:45.291 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac1115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac1115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49090/0 (socket says 172.21.15.7:49090) 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74abfff700 1 -- 172.21.15.7:0/260395655 learned_addr learned my addr 172.21.15.7:0/260395655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74abfff700 1 -- 172.21.15.7:0/260395655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0094f0 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74abfff700 1 --2- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac1115d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f749c0090b0 tx=0x7f749c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74a8ff9700 1 -- 172.21.15.7:0/260395655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f749c010840 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74a8ff9700 1 -- 172.21.15.7:0/260395655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f749c010e70 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac111d10 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74a8ff9700 1 -- 172.21.15.7:0/260395655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f749c024e90 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.266+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac112880 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.267+0000 7f74a8ff9700 1 -- 172.21.15.7:0/260395655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f749c01a510 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.267+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac05f5f0 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.267+0000 7f74a8ff9700 1 --2- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f7494036830 0x7f7494038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.268+0000 7f74a8ff9700 1 -- 172.21.15.7:0/260395655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f749c04a9d0 con 0x7f74ac102770 2024-03-16T08:52:45.292 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.268+0000 7f74ab7fe700 1 --2- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f7494036830 0x7f7494038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.268+0000 7f74ab7fe700 1 --2- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f7494036830 0x7f7494038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009990 tx=0x7f74a0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.272+0000 7f74a8ff9700 1 -- 172.21.15.7:0/260395655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f749c0109a0 con 0x7f74ac102770 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.573+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f74ac04bed0 con 0x7f74ac102770 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.575+0000 7f74a8ff9700 1 -- 172.21.15.7:0/260395655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f749c02c340 con 0x7f74ac102770 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f7494036830 msgr2=0x7f7494038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 --2- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f7494036830 0x7f7494038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009990 tx=0x7f74a0006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 msgr2=0x7f74ac1115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 --2- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac1115d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f749c0090b0 tx=0x7f749c009250 comp rx=0 tx=0).stop 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 shutdown_connections 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 --2- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f7494036830 0x7f7494038ce0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 --2- 172.21.15.7:0/260395655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ac102770 0x7f74ac1115d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 >> 172.21.15.7:0/260395655 conn(0x7f74ac0fde90 msgr2=0x7f74ac0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 shutdown_connections 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-16T08:52:44.577+0000 7f74b287e700 1 -- 172.21.15.7:0/260395655 wait complete. 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:45.294 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-03-16T08:52:45.295 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:45.295 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-16T08:52:45.295 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:52:45.295 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-03-16T08:52:45.326 INFO:tasks.cephadm:Fetching config... 2024-03-16T08:52:45.327 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:52:45.327 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-16T08:52:45.343 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-16T08:52:45.343 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:52:45.343 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-16T08:52:45.397 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-16T08:52:45.397 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:52:45.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/keyring of=/dev/stdout 2024-03-16T08:52:45.464 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-16T08:52:45.464 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:52:45.464 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-16T08:52:45.520 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-16T08:52:45.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClsX2MYJ7lzPDENMN74inbHJmx0CYlNzRuWIjj54BPIwhrB9NBSmOa20N7EZeC/Fd6Cbq5IwuR6FmbvGTk2fYwJlp8GwnonVogxh8ihDPdiZ4je94jM2us+drwzms8RGoaUD7KNOLpYW9X2JC0+7nq807dbtOC7wyfS/mTuqho6r67Ckx0cCh6o8eHenpxYbluYKbQds0CrmZjzIDoA/fLlqrTIeLm+f0diIAjnPJfrpCFpJPUeMefQB1OIh9xQmRRH5qOZe2+jNqBhmsnEHnW4oe9ECQkj0RphdDQwvrvQAlZMnfQGyM6ptJjR+zxm9wSmd6TmIxFb/kt0DU55iaSGaCbtNWDkaTU83VpwnORXyLxyY0ceTx3qNnYAk+d4vXxSmVuwVpo1WURXOgK7v9clcnfAipkpkpvJp3u4k3GtMwGYFNWhxjFHS667DeiGJ+h0kgapt3vVLZ5xaFJPOmITkEY4VIaUST8N4mKnSFrwHp6MEcIOsvDH5fUwK5oKkU= ceph-4c46b77e-e372-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T08:52:45.598 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClsX2MYJ7lzPDENMN74inbHJmx0CYlNzRuWIjj54BPIwhrB9NBSmOa20N7EZeC/Fd6Cbq5IwuR6FmbvGTk2fYwJlp8GwnonVogxh8ihDPdiZ4je94jM2us+drwzms8RGoaUD7KNOLpYW9X2JC0+7nq807dbtOC7wyfS/mTuqho6r67Ckx0cCh6o8eHenpxYbluYKbQds0CrmZjzIDoA/fLlqrTIeLm+f0diIAjnPJfrpCFpJPUeMefQB1OIh9xQmRRH5qOZe2+jNqBhmsnEHnW4oe9ECQkj0RphdDQwvrvQAlZMnfQGyM6ptJjR+zxm9wSmd6TmIxFb/kt0DU55iaSGaCbtNWDkaTU83VpwnORXyLxyY0ceTx3qNnYAk+d4vXxSmVuwVpo1WURXOgK7v9clcnfAipkpkpvJp3u4k3GtMwGYFNWhxjFHS667DeiGJ+h0kgapt3vVLZ5xaFJPOmITkEY4VIaUST8N4mKnSFrwHp6MEcIOsvDH5fUwK5oKkU= ceph-4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:52:45.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClsX2MYJ7lzPDENMN74inbHJmx0CYlNzRuWIjj54BPIwhrB9NBSmOa20N7EZeC/Fd6Cbq5IwuR6FmbvGTk2fYwJlp8GwnonVogxh8ihDPdiZ4je94jM2us+drwzms8RGoaUD7KNOLpYW9X2JC0+7nq807dbtOC7wyfS/mTuqho6r67Ckx0cCh6o8eHenpxYbluYKbQds0CrmZjzIDoA/fLlqrTIeLm+f0diIAjnPJfrpCFpJPUeMefQB1OIh9xQmRRH5qOZe2+jNqBhmsnEHnW4oe9ECQkj0RphdDQwvrvQAlZMnfQGyM6ptJjR+zxm9wSmd6TmIxFb/kt0DU55iaSGaCbtNWDkaTU83VpwnORXyLxyY0ceTx3qNnYAk+d4vXxSmVuwVpo1WURXOgK7v9clcnfAipkpkpvJp3u4k3GtMwGYFNWhxjFHS667DeiGJ+h0kgapt3vVLZ5xaFJPOmITkEY4VIaUST8N4mKnSFrwHp6MEcIOsvDH5fUwK5oKkU= ceph-4c46b77e-e372-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T08:52:45.657 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClsX2MYJ7lzPDENMN74inbHJmx0CYlNzRuWIjj54BPIwhrB9NBSmOa20N7EZeC/Fd6Cbq5IwuR6FmbvGTk2fYwJlp8GwnonVogxh8ihDPdiZ4je94jM2us+drwzms8RGoaUD7KNOLpYW9X2JC0+7nq807dbtOC7wyfS/mTuqho6r67Ckx0cCh6o8eHenpxYbluYKbQds0CrmZjzIDoA/fLlqrTIeLm+f0diIAjnPJfrpCFpJPUeMefQB1OIh9xQmRRH5qOZe2+jNqBhmsnEHnW4oe9ECQkj0RphdDQwvrvQAlZMnfQGyM6ptJjR+zxm9wSmd6TmIxFb/kt0DU55iaSGaCbtNWDkaTU83VpwnORXyLxyY0ceTx3qNnYAk+d4vXxSmVuwVpo1WURXOgK7v9clcnfAipkpkpvJp3u4k3GtMwGYFNWhxjFHS667DeiGJ+h0kgapt3vVLZ5xaFJPOmITkEY4VIaUST8N4mKnSFrwHp6MEcIOsvDH5fUwK5oKkU= ceph-4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:52:45.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-16T08:52:45.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:45 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/260395655' entity='client.admin' 2024-03-16T08:52:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:52:48.414 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:48 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:48.414 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:48 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:48.414 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:48 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:52:48.414 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:48 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:48.414 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:48 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T08:52:48.415 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:48 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-16T08:52:48.415 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:48 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:52:48.415 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:48 smithi007 ceph-mon[29470]: Deploying daemon crash.smithi007 on smithi007 2024-03-16T08:52:48.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.411+0000 7f53190a2700 1 -- 172.21.15.7:0/4199116632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 msgr2=0x7f53140fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:48.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.411+0000 7f53190a2700 1 --2- 172.21.15.7:0/4199116632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 0x7f53140fd460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53040048c0 tx=0x7f5304009e30 comp rx=0 tx=0).stop 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.411+0000 7f53190a2700 1 -- 172.21.15.7:0/4199116632 shutdown_connections 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.411+0000 7f53190a2700 1 --2- 172.21.15.7:0/4199116632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 0x7f53140fd460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.411+0000 7f53190a2700 1 -- 172.21.15.7:0/4199116632 >> 172.21.15.7:0/4199116632 conn(0x7f53140f70d0 msgr2=0x7f53140f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.411+0000 7f53190a2700 1 -- 172.21.15.7:0/4199116632 shutdown_connections 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.411+0000 7f53190a2700 1 -- 172.21.15.7:0/4199116632 wait complete. 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.412+0000 7f53190a2700 1 Processor -- start 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.412+0000 7f53190a2700 1 -- start start 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.412+0000 7f53190a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 0x7f531410d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.412+0000 7f53190a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531410d600 con 0x7f53140fd050 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f5312d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 0x7f531410d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f5312d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 0x7f531410d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49108/0 (socket says 172.21.15.7:49108) 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f5312d9d700 1 -- 172.21.15.7:0/3413680613 learned_addr learned my addr 172.21.15.7:0/3413680613 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f5312d9d700 1 -- 172.21.15.7:0/3413680613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5304005040 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f5312d9d700 1 --2- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 0x7f531410d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f53040096b0 tx=0x7f5304009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f52fb7fe700 1 -- 172.21.15.7:0/3413680613 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f530400bed0 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f52fb7fe700 1 -- 172.21.15.7:0/3413680613 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f530400b5c0 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531410d800 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531410e370 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.413+0000 7f52fb7fe700 1 -- 172.21.15.7:0/3413680613 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f530401c5b0 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.414+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53000052e0 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.414+0000 7f52fb7fe700 1 -- 172.21.15.7:0/3413680613 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f530401c710 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.415+0000 7f52fb7fe700 1 --2- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f52fc0368d0 0x7f52fc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.415+0000 7f52fb7fe700 1 -- 172.21.15.7:0/3413680613 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5304049920 con 0x7f53140fd050 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.415+0000 7f531259c700 1 --2- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f52fc0368d0 0x7f52fc038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.415+0000 7f531259c700 1 --2- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f52fc0368d0 0x7f52fc038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5308009990 tx=0x7f5308006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.419+0000 7f52fb7fe700 1 -- 172.21.15.7:0/3413680613 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f5304025400 con 0x7f53140fd050 2024-03-16T08:52:48.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.629+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f5300005b10 con 0x7f53140fd050 2024-03-16T08:52:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.632+0000 7f52fb7fe700 1 -- 172.21.15.7:0/3413680613 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5304011b60 con 0x7f53140fd050 2024-03-16T08:52:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f52fc0368d0 msgr2=0x7f52fc038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 --2- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f52fc0368d0 0x7f52fc038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5308009990 tx=0x7f5308006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 msgr2=0x7f531410d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 --2- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 0x7f531410d0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f53040096b0 tx=0x7f5304009e30 comp rx=0 tx=0).stop 2024-03-16T08:52:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 shutdown_connections 2024-03-16T08:52:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 --2- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f52fc0368d0 0x7f52fc038d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 --2- 172.21.15.7:0/3413680613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53140fd050 0x7f531410d0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 >> 172.21.15.7:0/3413680613 conn(0x7f53140f70d0 msgr2=0x7f5314065070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 shutdown_connections 2024-03-16T08:52:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:48.634+0000 7f53190a2700 1 -- 172.21.15.7:0/3413680613 wait complete. 2024-03-16T08:52:49.355 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-16T08:52:49.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-16T08:52:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:52:49.920 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:49 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3413680613' entity='client.admin' 2024-03-16T08:52:52.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 -- 172.21.15.7:0/3437674724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 msgr2=0x7f8e000ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 --2- 172.21.15.7:0/3437674724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 0x7f8e000ffa30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8df80048c0 tx=0x7f8df8009e30 comp rx=0 tx=0).stop 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 -- 172.21.15.7:0/3437674724 shutdown_connections 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 --2- 172.21.15.7:0/3437674724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 0x7f8e000ffa30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 -- 172.21.15.7:0/3437674724 >> 172.21.15.7:0/3437674724 conn(0x7f8e000f70d0 msgr2=0x7f8e000f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 -- 172.21.15.7:0/3437674724 shutdown_connections 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 -- 172.21.15.7:0/3437674724 wait complete. 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 Processor -- start 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 -- start start 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 0x7f8e000fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.308+0000 7f8e07567700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000fd4f0 con 0x7f8e000fd650 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8e05303700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 0x7f8e000fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:52.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8e05303700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 0x7f8e000fcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49120/0 (socket says 172.21.15.7:49120) 2024-03-16T08:52:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8e05303700 1 -- 172.21.15.7:0/253582127 learned_addr learned my addr 172.21.15.7:0/253582127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8e05303700 1 -- 172.21.15.7:0/253582127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8005040 con 0x7f8e000fd650 2024-03-16T08:52:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8e05303700 1 --2- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 0x7f8e000fcfb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009680 tx=0x7f8df8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8dfdffb700 1 -- 172.21.15.7:0/253582127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8df800bed0 con 0x7f8e000fd650 2024-03-16T08:52:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8dfdffb700 1 -- 172.21.15.7:0/253582127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8df800b5c0 con 0x7f8e000fd650 2024-03-16T08:52:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8dfdffb700 1 -- 172.21.15.7:0/253582127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8df801c5b0 con 0x7f8e000fd650 2024-03-16T08:52:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e000fb5c0 con 0x7f8e000fd650 2024-03-16T08:52:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.309+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e000fbab0 con 0x7f8e000fd650 2024-03-16T08:52:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.310+0000 7f8dfdffb700 1 -- 172.21.15.7:0/253582127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f8df8004d30 con 0x7f8e000fd650 2024-03-16T08:52:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.310+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e00102f60 con 0x7f8e000fd650 2024-03-16T08:52:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.310+0000 7f8dfdffb700 1 --2- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f8de80368d0 0x7f8de8038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.311+0000 7f8dfdffb700 1 -- 172.21.15.7:0/253582127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8df8049a30 con 0x7f8e000fd650 2024-03-16T08:52:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.311+0000 7f8e04b02700 1 --2- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f8de80368d0 0x7f8de8038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:52.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.311+0000 7f8e04b02700 1 --2- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f8de80368d0 0x7f8de8038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009990 tx=0x7f8df4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.315+0000 7f8dfdffb700 1 -- 172.21.15.7:0/253582127 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f8df8011730 con 0x7f8e000fd650 2024-03-16T08:52:52.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.526+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 --> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e001064a0 con 0x7f8de80368d0 2024-03-16T08:52:52.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.529+0000 7f8dfdffb700 1 -- 172.21.15.7:0/253582127 <== mgr.14160 v2:172.21.15.7:6800/2343923853 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8e001064a0 con 0x7f8de80368d0 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f8de80368d0 msgr2=0x7f8de8038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 --2- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f8de80368d0 0x7f8de8038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009990 tx=0x7f8df4006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 msgr2=0x7f8e000fcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 --2- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 0x7f8e000fcfb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009680 tx=0x7f8df8009e30 comp rx=0 tx=0).stop 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 shutdown_connections 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 --2- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f8de80368d0 0x7f8de8038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 --2- 172.21.15.7:0/253582127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e000fd650 0x7f8e000fcfb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 >> 172.21.15.7:0/253582127 conn(0x7f8e000f70d0 msgr2=0x7f8e00103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:52.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 shutdown_connections 2024-03-16T08:52:52.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:52.531+0000 7f8e07567700 1 -- 172.21.15.7:0/253582127 wait complete. 2024-03-16T08:52:53.277 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi019 2024-03-16T08:52:53.277 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:52:53.278 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/etc/ceph/ceph.conf 2024-03-16T08:52:53.296 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:52:53.296 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:52:53.353 INFO:tasks.cephadm:Adding host smithi019 to orchestrator... 2024-03-16T08:52:53.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch host add smithi019 2024-03-16T08:52:53.632 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:52:53.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:53 smithi007 ceph-mon[29470]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:53.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:53 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:54 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:54 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:54 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:54 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:55.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.156+0000 7ff486d81700 1 -- 172.21.15.7:0/2384795500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 msgr2=0x7ff4800fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.156+0000 7ff486d81700 1 --2- 172.21.15.7:0/2384795500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 0x7ff4800fe4c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff4680048c0 tx=0x7ff468009e30 comp rx=0 tx=0).stop 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.156+0000 7ff486d81700 1 -- 172.21.15.7:0/2384795500 shutdown_connections 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.156+0000 7ff486d81700 1 --2- 172.21.15.7:0/2384795500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 0x7ff4800fe4c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.156+0000 7ff486d81700 1 -- 172.21.15.7:0/2384795500 >> 172.21.15.7:0/2384795500 conn(0x7ff4800f6850 msgr2=0x7ff4800f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.156+0000 7ff486d81700 1 -- 172.21.15.7:0/2384795500 shutdown_connections 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.156+0000 7ff486d81700 1 -- 172.21.15.7:0/2384795500 wait complete. 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.157+0000 7ff486d81700 1 Processor -- start 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.157+0000 7ff486d81700 1 -- start start 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.157+0000 7ff486d81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 0x7ff48010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.157+0000 7ff486d81700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48010d600 con 0x7ff4800fe0b0 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff484b1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 0x7ff48010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff484b1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 0x7ff48010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53336/0 (socket says 172.21.15.7:53336) 2024-03-16T08:52:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff484b1d700 1 -- 172.21.15.7:0/2870035105 learned_addr learned my addr 172.21.15.7:0/2870035105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:52:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff484b1d700 1 -- 172.21.15.7:0/2870035105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff468005040 con 0x7ff4800fe0b0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff484b1d700 1 --2- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 0x7ff48010d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff4680096b0 tx=0x7ff468009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff47d7fa700 1 -- 172.21.15.7:0/2870035105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff46800bed0 con 0x7ff4800fe0b0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff47d7fa700 1 -- 172.21.15.7:0/2870035105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff46800b5c0 con 0x7ff4800fe0b0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48010d800 con 0x7ff4800fe0b0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff47d7fa700 1 -- 172.21.15.7:0/2870035105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff46801c5b0 con 0x7ff4800fe0b0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.158+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48010e370 con 0x7ff4800fe0b0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.159+0000 7ff47d7fa700 1 -- 172.21.15.7:0/2870035105 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7ff46801c820 con 0x7ff4800fe0b0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.160+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff480102f80 con 0x7ff4800fe0b0 2024-03-16T08:52:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.160+0000 7ff47d7fa700 1 --2- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7ff46c036880 0x7ff46c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:52:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.160+0000 7ff47d7fa700 1 -- 172.21.15.7:0/2870035105 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff468049b30 con 0x7ff4800fe0b0 2024-03-16T08:52:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.160+0000 7ff47ffff700 1 --2- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7ff46c036880 0x7ff46c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:52:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.161+0000 7ff47ffff700 1 --2- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7ff46c036880 0x7ff46c038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff470009990 tx=0x7ff470006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:52:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.165+0000 7ff47d7fa700 1 -- 172.21.15.7:0/2870035105 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7ff468013420 con 0x7ff4800fe0b0 2024-03-16T08:52:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:55.370+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 --> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "target": ["mon-mgr", ""]}) v1 -- 0x7ff48010e5b0 con 0x7ff46c036880 2024-03-16T08:52:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:55 smithi007 ceph-mon[29470]: Deploying daemon node-exporter.smithi007 on smithi007 2024-03-16T08:52:56.899 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:56 smithi007 ceph-mon[29470]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:52:56.900 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:56 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:57.573 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi019' with addr '172.21.15.19' 2024-03-16T08:52:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.571+0000 7ff47d7fa700 1 -- 172.21.15.7:0/2870035105 <== mgr.14160 v2:172.21.15.7:6800/2343923853 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7ff48010e5b0 con 0x7ff46c036880 2024-03-16T08:52:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7ff46c036880 msgr2=0x7ff46c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 --2- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7ff46c036880 0x7ff46c038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff470009990 tx=0x7ff470006e40 comp rx=0 tx=0).stop 2024-03-16T08:52:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 msgr2=0x7ff48010d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:52:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 --2- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 0x7ff48010d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff4680096b0 tx=0x7ff468009e30 comp rx=0 tx=0).stop 2024-03-16T08:52:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 shutdown_connections 2024-03-16T08:52:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 --2- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7ff46c036880 0x7ff46c038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 --2- 172.21.15.7:0/2870035105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4800fe0b0 0x7ff48010d0c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:52:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 >> 172.21.15.7:0/2870035105 conn(0x7ff4800f6850 msgr2=0x7ff480103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:52:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 shutdown_connections 2024-03-16T08:52:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:52:57.573+0000 7ff486d81700 1 -- 172.21.15.7:0/2870035105 wait complete. 2024-03-16T08:52:57.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:57 smithi007 ceph-mon[29470]: Deploying cephadm binary to smithi019 2024-03-16T08:52:57.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:57 smithi007 ceph-mon[29470]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:52:57.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:57 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:58.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-16T08:52:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:52:59.854 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:59 smithi007 ceph-mon[29470]: Added host smithi019 2024-03-16T08:52:59.855 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:59 smithi007 ceph-mon[29470]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:52:59.855 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:59 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:59.855 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:59 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:59.855 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:59 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:52:59.855 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:52:59 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:00.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.144+0000 7f00619d6700 1 -- 172.21.15.7:0/2409543523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 msgr2=0x7f005c0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.144+0000 7f00619d6700 1 --2- 172.21.15.7:0/2409543523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 0x7f005c0fd460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0054009c60 tx=0x7f0054004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.144+0000 7f00619d6700 1 -- 172.21.15.7:0/2409543523 shutdown_connections 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.144+0000 7f00619d6700 1 --2- 172.21.15.7:0/2409543523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 0x7f005c0fd460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.144+0000 7f00619d6700 1 -- 172.21.15.7:0/2409543523 >> 172.21.15.7:0/2409543523 conn(0x7f005c0f70d0 msgr2=0x7f005c0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.144+0000 7f00619d6700 1 -- 172.21.15.7:0/2409543523 shutdown_connections 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.144+0000 7f00619d6700 1 -- 172.21.15.7:0/2409543523 wait complete. 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f00619d6700 1 Processor -- start 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f00619d6700 1 -- start start 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f00619d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 0x7f005c10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f00619d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c10d600 con 0x7f005c0fd050 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f005affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 0x7f005c10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f005affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 0x7f005c10d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53362/0 (socket says 172.21.15.7:53362) 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f005affd700 1 -- 172.21.15.7:0/3710233088 learned_addr learned my addr 172.21.15.7:0/3710233088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:53:00.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f005affd700 1 -- 172.21.15.7:0/3710233088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0054009940 con 0x7f005c0fd050 2024-03-16T08:53:00.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f005affd700 1 --2- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 0x7f005c10d0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0054004e60 tx=0x7f0054004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:00.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f00537fe700 1 -- 172.21.15.7:0/3710233088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0054012410 con 0x7f005c0fd050 2024-03-16T08:53:00.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f00537fe700 1 -- 172.21.15.7:0/3710233088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0054012a40 con 0x7f005c0fd050 2024-03-16T08:53:00.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c10d800 con 0x7f005c0fd050 2024-03-16T08:53:00.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.145+0000 7f00537fe700 1 -- 172.21.15.7:0/3710233088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f005401ba40 con 0x7f005c0fd050 2024-03-16T08:53:00.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.146+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c10e370 con 0x7f005c0fd050 2024-03-16T08:53:00.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.146+0000 7f00537fe700 1 -- 172.21.15.7:0/3710233088 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f0054019040 con 0x7f005c0fd050 2024-03-16T08:53:00.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.146+0000 7f00537fe700 1 --2- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f0048036880 0x7f0048038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:00.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.146+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005c101e70 con 0x7f005c0fd050 2024-03-16T08:53:00.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.147+0000 7f00537fe700 1 -- 172.21.15.7:0/3710233088 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0054049c20 con 0x7f005c0fd050 2024-03-16T08:53:00.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.147+0000 7f005a7fc700 1 --2- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f0048036880 0x7f0048038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:00.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.147+0000 7f005a7fc700 1 --2- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f0048036880 0x7f0048038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0044009990 tx=0x7f0044006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:00.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.151+0000 7f00537fe700 1 -- 172.21.15.7:0/3710233088 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f005401f080 con 0x7f005c0fd050 2024-03-16T08:53:00.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.350+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 --> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f005c065780 con 0x7f0048036880 2024-03-16T08:53:00.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.351+0000 7f00537fe700 1 -- 172.21.15.7:0/3710233088 <== mgr.14160 v2:172.21.15.7:6800/2343923853 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7f005c065780 con 0x7f0048036880 2024-03-16T08:53:00.353 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:53:00.353 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}] 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f0048036880 msgr2=0x7f0048038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 --2- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f0048036880 0x7f0048038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0044009990 tx=0x7f0044006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 msgr2=0x7f005c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 --2- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 0x7f005c10d0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0054004e60 tx=0x7f0054004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 shutdown_connections 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 --2- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f0048036880 0x7f0048038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 --2- 172.21.15.7:0/3710233088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005c0fd050 0x7f005c10d0c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 >> 172.21.15.7:0/3710233088 conn(0x7f005c0f70d0 msgr2=0x7f005c065070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 shutdown_connections 2024-03-16T08:53:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:00.353+0000 7f00619d6700 1 -- 172.21.15.7:0/3710233088 wait complete. 2024-03-16T08:53:00.889 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:00 smithi007 ceph-mon[29470]: Deploying daemon alertmanager.smithi007 on smithi007 2024-03-16T08:53:01.392 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-16T08:53:01.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-16T08:53:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:53:01.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:01 smithi007 ceph-mon[29470]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:01.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:01 smithi007 ceph-mon[29470]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:53:01.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:01 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:03.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:03 smithi007 ceph-mon[29470]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.696+0000 7f126fb25700 1 -- 172.21.15.7:0/2487780704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 msgr2=0x7f12680fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.696+0000 7f126fb25700 1 --2- 172.21.15.7:0/2487780704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 0x7f12680fce50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f12640048c0 tx=0x7f1264009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.696+0000 7f126fb25700 1 -- 172.21.15.7:0/2487780704 shutdown_connections 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.696+0000 7f126fb25700 1 --2- 172.21.15.7:0/2487780704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 0x7f12680fce50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.696+0000 7f126fb25700 1 -- 172.21.15.7:0/2487780704 >> 172.21.15.7:0/2487780704 conn(0x7f12680f81c0 msgr2=0x7f12680fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126fb25700 1 -- 172.21.15.7:0/2487780704 shutdown_connections 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126fb25700 1 -- 172.21.15.7:0/2487780704 wait complete. 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126fb25700 1 Processor -- start 2024-03-16T08:53:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126fb25700 1 -- start start 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126fb25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 0x7f126810d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126fb25700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126810d600 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126d8c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 0x7f126810d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126d8c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 0x7f126810d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35754/0 (socket says 172.21.15.7:35754) 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.697+0000 7f126d8c1700 1 -- 172.21.15.7:0/3952343520 learned_addr learned my addr 172.21.15.7:0/3952343520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.698+0000 7f126d8c1700 1 -- 172.21.15.7:0/3952343520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1264005040 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.698+0000 7f126d8c1700 1 --2- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 0x7f126810d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12640096b0 tx=0x7f1264009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.698+0000 7f125e7fc700 1 -- 172.21.15.7:0/3952343520 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f126400bed0 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.698+0000 7f125e7fc700 1 -- 172.21.15.7:0/3952343520 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f126400b5c0 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.698+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126810d800 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.698+0000 7f125e7fc700 1 -- 172.21.15.7:0/3952343520 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f126401c5b0 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.698+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126810e370 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.699+0000 7f125e7fc700 1 -- 172.21.15.7:0/3952343520 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f1264013420 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.699+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1268102f50 con 0x7f12680fca40 2024-03-16T08:53:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.699+0000 7f125e7fc700 1 --2- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f1254036880 0x7f1254038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.699+0000 7f125e7fc700 1 -- 172.21.15.7:0/3952343520 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1264049b00 con 0x7f12680fca40 2024-03-16T08:53:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.699+0000 7f126d0c0700 1 --2- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f1254036880 0x7f1254038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.700+0000 7f126d0c0700 1 --2- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f1254036880 0x7f1254038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1258009990 tx=0x7f1258006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.704+0000 7f125e7fc700 1 -- 172.21.15.7:0/3952343520 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f1264025400 con 0x7f12680fca40 2024-03-16T08:53:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:03.930+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f126810ed80 con 0x7f12680fca40 2024-03-16T08:53:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.572+0000 7f125e7fc700 1 -- 172.21.15.7:0/3952343520 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1264004d30 con 0x7f12680fca40 2024-03-16T08:53:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f1254036880 msgr2=0x7f1254038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 --2- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f1254036880 0x7f1254038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1258009990 tx=0x7f1258006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 msgr2=0x7f126810d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 --2- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 0x7f126810d0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12640096b0 tx=0x7f1264009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 shutdown_connections 2024-03-16T08:53:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 --2- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f1254036880 0x7f1254038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 --2- 172.21.15.7:0/3952343520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12680fca40 0x7f126810d0c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 >> 172.21.15.7:0/3952343520 conn(0x7f12680f81c0 msgr2=0x7f1268103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 shutdown_connections 2024-03-16T08:53:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:53:04.574+0000 7f126fb25700 1 -- 172.21.15.7:0/3952343520 wait complete. 2024-03-16T08:53:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-03-16T08:53:04.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:04 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3952343520' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T08:53:05.773 INFO:tasks.cephadm:Adding mon.smithi007 on smithi007 2024-03-16T08:53:05.773 INFO:tasks.cephadm:Adding mon.smithi019 on smithi019 2024-03-16T08:53:05.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch apply mon '2;smithi007:172.21.15.7=smithi007;smithi019:172.21.15.19=smithi019' 2024-03-16T08:53:05.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:05 smithi007 ceph-mon[29470]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:05.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:05 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3952343520' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T08:53:05.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:05 smithi007 ceph-mon[29470]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:53:06.028 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:07.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.496+0000 7f56895fe700 1 -- 172.21.15.19:0/2736293796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 msgr2=0x7f56840ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:07.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.496+0000 7f56895fe700 1 --2- 172.21.15.19:0/2736293796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 0x7f56840ff710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5674009c60 tx=0x7f5674004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:07.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.496+0000 7f56895fe700 1 -- 172.21.15.19:0/2736293796 shutdown_connections 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.496+0000 7f56895fe700 1 --2- 172.21.15.19:0/2736293796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 0x7f56840ff710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.496+0000 7f56895fe700 1 -- 172.21.15.19:0/2736293796 >> 172.21.15.19:0/2736293796 conn(0x7f56840f70c0 msgr2=0x7f56840f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.496+0000 7f56895fe700 1 -- 172.21.15.19:0/2736293796 shutdown_connections 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.496+0000 7f56895fe700 1 -- 172.21.15.19:0/2736293796 wait complete. 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f56895fe700 1 Processor -- start 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f56895fe700 1 -- start start 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f56895fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 0x7f568410d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f56895fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568410d630 con 0x7f5684066920 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f5682ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 0x7f568410d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f5682ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 0x7f568410d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:44472/0 (socket says 172.21.15.19:44472) 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f5682ffd700 1 -- 172.21.15.19:0/3845707924 learned_addr learned my addr 172.21.15.19:0/3845707924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f5682ffd700 1 -- 172.21.15.19:0/3845707924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5674009940 con 0x7f5684066920 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f5682ffd700 1 --2- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 0x7f568410d0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56740097b0 tx=0x7f5674004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:07.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f567b7fe700 1 -- 172.21.15.19:0/3845707924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5674012410 con 0x7f5684066920 2024-03-16T08:53:07.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f567b7fe700 1 -- 172.21.15.19:0/3845707924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5674012a40 con 0x7f5684066920 2024-03-16T08:53:07.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.497+0000 7f567b7fe700 1 -- 172.21.15.19:0/3845707924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f567401b980 con 0x7f5684066920 2024-03-16T08:53:07.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.498+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568410d830 con 0x7f5684066920 2024-03-16T08:53:07.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.498+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568410e3a0 con 0x7f5684066920 2024-03-16T08:53:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.498+0000 7f567b7fe700 1 -- 172.21.15.19:0/3845707924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f5674019040 con 0x7f5684066920 2024-03-16T08:53:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.498+0000 7f567b7fe700 1 --2- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f5670036880 0x7f5670038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.498+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684102f60 con 0x7f5684066920 2024-03-16T08:53:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.499+0000 7f567b7fe700 1 -- 172.21.15.19:0/3845707924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5674049d30 con 0x7f5684066920 2024-03-16T08:53:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.499+0000 7f56827fc700 1 --2- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f5670036880 0x7f5670038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.499+0000 7f56827fc700 1 --2- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f5670036880 0x7f5670038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f566c009990 tx=0x7f566c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:07.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.504+0000 7f567b7fe700 1 -- 172.21.15.19:0/3845707924 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f567401f020 con 0x7f5684066920 2024-03-16T08:53:07.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.704+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 --> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi007:172.21.15.7=smithi007;smithi019:172.21.15.19=smithi019", "target": ["mon-mgr", ""]}) v1 -- 0x7f568410e690 con 0x7f5670036880 2024-03-16T08:53:07.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.710+0000 7f567b7fe700 1 -- 172.21.15.19:0/3845707924 <== mgr.14160 v2:172.21.15.7:6800/2343923853 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f568410e690 con 0x7f5670036880 2024-03-16T08:53:07.712 INFO:teuthology.orchestra.run.smithi019.stdout:Scheduled mon update... 2024-03-16T08:53:07.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f5670036880 msgr2=0x7f5670038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:07.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 --2- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f5670036880 0x7f5670038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f566c009990 tx=0x7f566c006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 msgr2=0x7f568410d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 --2- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 0x7f568410d0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56740097b0 tx=0x7f5674004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 shutdown_connections 2024-03-16T08:53:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 --2- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f5670036880 0x7f5670038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 --2- 172.21.15.19:0/3845707924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5684066920 0x7f568410d0f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 >> 172.21.15.19:0/3845707924 conn(0x7f56840f70c0 msgr2=0x7f5684103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 shutdown_connections 2024-03-16T08:53:07.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:07.712+0000 7f56895fe700 1 -- 172.21.15.19:0/3845707924 wait complete. 2024-03-16T08:53:07.747 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:07 smithi007 ceph-mon[29470]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:08.353 DEBUG:teuthology.orchestra.run.smithi019:mon.smithi019> sudo journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi019.service 2024-03-16T08:53:08.355 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:08.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:08.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:08 smithi007 ceph-mon[29470]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi007:172.21.15.7=smithi007;smithi019:172.21.15.19=smithi019", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:53:08.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:08 smithi007 ceph-mon[29470]: Saving service mon spec with placement smithi007:172.21.15.7=smithi007;smithi019:172.21.15.19=smithi019;count:2 2024-03-16T08:53:08.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:08 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:08.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:08 smithi007 ceph-mon[29470]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.019+0000 7f339e5de700 1 -- 172.21.15.19:0/600801179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 msgr2=0x7f33980fb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.019+0000 7f339e5de700 1 --2- 172.21.15.19:0/600801179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 0x7f33980fb4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33880048c0 tx=0x7f3388009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.019+0000 7f339e5de700 1 -- 172.21.15.19:0/600801179 shutdown_connections 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.019+0000 7f339e5de700 1 --2- 172.21.15.19:0/600801179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 0x7f33980fb4f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.019+0000 7f339e5de700 1 -- 172.21.15.19:0/600801179 >> 172.21.15.19:0/600801179 conn(0x7f33980f6840 msgr2=0x7f33980f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.019+0000 7f339e5de700 1 -- 172.21.15.19:0/600801179 shutdown_connections 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.019+0000 7f339e5de700 1 -- 172.21.15.19:0/600801179 wait complete. 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.019+0000 7f339e5de700 1 Processor -- start 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f339e5de700 1 -- start start 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f339e5de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 0x7f339810f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f339e5de700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339810f820 con 0x7f33980fb0e0 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f3397fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 0x7f339810f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f3397fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 0x7f339810f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:44486/0 (socket says 172.21.15.19:44486) 2024-03-16T08:53:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f3397fff700 1 -- 172.21.15.19:0/3945006792 learned_addr learned my addr 172.21.15.19:0/3945006792 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f3397fff700 1 -- 172.21.15.19:0/3945006792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388005040 con 0x7f33980fb0e0 2024-03-16T08:53:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f3397fff700 1 --2- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 0x7f339810f2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3388009680 tx=0x7f3388009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.020+0000 7f3394ff9700 1 -- 172.21.15.19:0/3945006792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f338800b5c0 con 0x7f33980fb0e0 2024-03-16T08:53:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.021+0000 7f3394ff9700 1 -- 172.21.15.19:0/3945006792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f338800be70 con 0x7f33980fb0e0 2024-03-16T08:53:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.021+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339810fa20 con 0x7f33980fb0e0 2024-03-16T08:53:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.021+0000 7f3394ff9700 1 -- 172.21.15.19:0/3945006792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33880135a0 con 0x7f33980fb0e0 2024-03-16T08:53:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.021+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3398110590 con 0x7f33980fb0e0 2024-03-16T08:53:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.022+0000 7f3394ff9700 1 -- 172.21.15.19:0/3945006792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f338801c430 con 0x7f33980fb0e0 2024-03-16T08:53:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.022+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33981015f0 con 0x7f33980fb0e0 2024-03-16T08:53:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.022+0000 7f3394ff9700 1 --2- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f33800368d0 0x7f3380038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.022+0000 7f3394ff9700 1 -- 172.21.15.19:0/3945006792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f33880493a0 con 0x7f33980fb0e0 2024-03-16T08:53:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.022+0000 7f33977fe700 1 --2- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f33800368d0 0x7f3380038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.022+0000 7f33977fe700 1 --2- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f33800368d0 0x7f3380038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f338c009990 tx=0x7f338c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.027+0000 7f3394ff9700 1 -- 172.21.15.19:0/3945006792 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f33880117e0 con 0x7f33980fb0e0 2024-03-16T08:53:10.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.297+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f33981107d0 con 0x7f33980fb0e0 2024-03-16T08:53:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.298+0000 7f3394ff9700 1 -- 172.21.15.19:0/3945006792 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f3388021070 con 0x7f33980fb0e0 2024-03-16T08:53:10.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:10.300 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:10.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f33800368d0 msgr2=0x7f3380038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:10.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 --2- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f33800368d0 0x7f3380038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f338c009990 tx=0x7f338c006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:10.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 msgr2=0x7f339810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:10.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 --2- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 0x7f339810f2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3388009680 tx=0x7f3388009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:10.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 shutdown_connections 2024-03-16T08:53:10.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 --2- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f33800368d0 0x7f3380038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 --2- 172.21.15.19:0/3945006792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33980fb0e0 0x7f339810f2e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 >> 172.21.15.19:0/3945006792 conn(0x7f33980f6840 msgr2=0x7f33980f6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 shutdown_connections 2024-03-16T08:53:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:10.300+0000 7f339e5de700 1 -- 172.21.15.19:0/3945006792 wait complete. 2024-03-16T08:53:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-16T08:53:10.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-16T08:53:10.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:10.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: Deploying daemon grafana.smithi007 on smithi007 2024-03-16T08:53:10.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:10 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3945006792' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:11.856 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:11.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:11 smithi007 ceph-mon[29470]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:11 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:12.111 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:12.156 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 -- 172.21.15.19:0/2575007118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 msgr2=0x7fe4fc0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 --2- 172.21.15.19:0/2575007118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 0x7fe4fc0fd460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0048c0 tx=0x7fe4ec009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 -- 172.21.15.19:0/2575007118 shutdown_connections 2024-03-16T08:53:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 --2- 172.21.15.19:0/2575007118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 0x7fe4fc0fd460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 -- 172.21.15.19:0/2575007118 >> 172.21.15.19:0/2575007118 conn(0x7fe4fc0f70d0 msgr2=0x7fe4fc0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 -- 172.21.15.19:0/2575007118 shutdown_connections 2024-03-16T08:53:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 -- 172.21.15.19:0/2575007118 wait complete. 2024-03-16T08:53:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 Processor -- start 2024-03-16T08:53:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.657+0000 7fe501137700 1 -- start start 2024-03-16T08:53:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe501137700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 0x7fe4fc065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe501137700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc10fdc0 con 0x7fe4fc0fd050 2024-03-16T08:53:13.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe4fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 0x7fe4fc065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe4fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 0x7fe4fc065bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:60902/0 (socket says 172.21.15.19:60902) 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe4fad9d700 1 -- 172.21.15.19:0/3734979584 learned_addr learned my addr 172.21.15.19:0/3734979584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe4fad9d700 1 -- 172.21.15.19:0/3734979584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec005040 con 0x7fe4fc0fd050 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe4fad9d700 1 --2- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 0x7fe4fc065bf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009680 tx=0x7fe4ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe4e37fe700 1 -- 172.21.15.19:0/3734979584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec00b5c0 con 0x7fe4fc0fd050 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc10ff00 con 0x7fe4fc0fd050 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe4e37fe700 1 -- 172.21.15.19:0/3734979584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4ec00be70 con 0x7fe4fc0fd050 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe4e37fe700 1 -- 172.21.15.19:0/3734979584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4ec0135a0 con 0x7fe4fc0fd050 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.658+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc1103f0 con 0x7fe4fc0fd050 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.659+0000 7fe4e37fe700 1 -- 172.21.15.19:0/3734979584 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fe4ec004d30 con 0x7fe4fc0fd050 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.659+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc105920 con 0x7fe4fc0fd050 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.659+0000 7fe4e37fe700 1 --2- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fe4e4036880 0x7fe4e4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.660+0000 7fe4e37fe700 1 -- 172.21.15.19:0/3734979584 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe4ec049450 con 0x7fe4fc0fd050 2024-03-16T08:53:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.660+0000 7fe4fa59c700 1 --2- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fe4e4036880 0x7fe4e4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:13.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.660+0000 7fe4fa59c700 1 --2- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fe4e4036880 0x7fe4e4038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009990 tx=0x7fe4f0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.665+0000 7fe4e37fe700 1 -- 172.21.15.19:0/3734979584 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fe4ec011960 con 0x7fe4fc0fd050 2024-03-16T08:53:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.935+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe4fc101e70 con 0x7fe4fc0fd050 2024-03-16T08:53:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.935+0000 7fe4e37fe700 1 -- 172.21.15.19:0/3734979584 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fe4ec021020 con 0x7fe4fc0fd050 2024-03-16T08:53:13.937 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:13.937 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fe4e4036880 msgr2=0x7fe4e4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 --2- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fe4e4036880 0x7fe4e4038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009990 tx=0x7fe4f0006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 msgr2=0x7fe4fc065bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 --2- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 0x7fe4fc065bf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009680 tx=0x7fe4ec009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 shutdown_connections 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 --2- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fe4e4036880 0x7fe4e4038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 --2- 172.21.15.19:0/3734979584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc0fd050 0x7fe4fc065bf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 >> 172.21.15.19:0/3734979584 conn(0x7fe4fc0f70d0 msgr2=0x7fe4fc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 shutdown_connections 2024-03-16T08:53:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:13.938+0000 7fe501137700 1 -- 172.21.15.19:0/3734979584 wait complete. 2024-03-16T08:53:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:13.997 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:13 smithi007 ceph-mon[29470]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:14.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:14 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3734979584' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:15.520 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:15.521 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:15.777 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:15.822 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:15 smithi007 ceph-mon[29470]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.210+0000 7f4e5c15e700 1 -- 172.21.15.19:0/1249327489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 msgr2=0x7f4e540fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.210+0000 7f4e5c15e700 1 --2- 172.21.15.19:0/1249327489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 0x7f4e540fd460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009c60 tx=0x7f4e50004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 -- 172.21.15.19:0/1249327489 shutdown_connections 2024-03-16T08:53:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 --2- 172.21.15.19:0/1249327489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 0x7f4e540fd460 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 -- 172.21.15.19:0/1249327489 >> 172.21.15.19:0/1249327489 conn(0x7f4e540f70d0 msgr2=0x7f4e540f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 -- 172.21.15.19:0/1249327489 shutdown_connections 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 -- 172.21.15.19:0/1249327489 wait complete. 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 Processor -- start 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 -- start start 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 0x7f4e54065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e5c15e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5410fde0 con 0x7f4e540fd050 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e59efa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 0x7f4e54065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e59efa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 0x7f4e54065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:60918/0 (socket says 172.21.15.19:60918) 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.211+0000 7f4e59efa700 1 -- 172.21.15.19:0/2453944803 learned_addr learned my addr 172.21.15.19:0/2453944803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.212+0000 7f4e59efa700 1 -- 172.21.15.19:0/2453944803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e50009940 con 0x7f4e540fd050 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.212+0000 7f4e59efa700 1 --2- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 0x7f4e54065bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009500 tx=0x7f4e50004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:17.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.212+0000 7f4e4a7fc700 1 -- 172.21.15.19:0/2453944803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e50007c80 con 0x7f4e540fd050 2024-03-16T08:53:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.212+0000 7f4e4a7fc700 1 -- 172.21.15.19:0/2453944803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e5001d910 con 0x7f4e540fd050 2024-03-16T08:53:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.212+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5410ff20 con 0x7f4e540fd050 2024-03-16T08:53:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.212+0000 7f4e4a7fc700 1 -- 172.21.15.19:0/2453944803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e500129d0 con 0x7f4e540fd050 2024-03-16T08:53:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.212+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e54110410 con 0x7f4e540fd050 2024-03-16T08:53:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.213+0000 7f4e4a7fc700 1 -- 172.21.15.19:0/2453944803 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f4e50019040 con 0x7f4e540fd050 2024-03-16T08:53:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.213+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e54105920 con 0x7f4e540fd050 2024-03-16T08:53:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.213+0000 7f4e4a7fc700 1 --2- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f4e40036880 0x7f4e40038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.213+0000 7f4e4a7fc700 1 -- 172.21.15.19:0/2453944803 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4e5004a3f0 con 0x7f4e540fd050 2024-03-16T08:53:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.213+0000 7f4e596f9700 1 --2- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f4e40036880 0x7f4e40038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.214+0000 7f4e596f9700 1 --2- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f4e40036880 0x7f4e40038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009990 tx=0x7f4e44006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:17.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.218+0000 7f4e4a7fc700 1 -- 172.21.15.19:0/2453944803 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f4e5001da80 con 0x7f4e540fd050 2024-03-16T08:53:17.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.486+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4e5405f5f0 con 0x7f4e540fd050 2024-03-16T08:53:17.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.487+0000 7f4e4a7fc700 1 -- 172.21.15.19:0/2453944803 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f4e50017020 con 0x7f4e540fd050 2024-03-16T08:53:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:17.489 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f4e40036880 msgr2=0x7f4e40038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 --2- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f4e40036880 0x7f4e40038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009990 tx=0x7f4e44006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 msgr2=0x7f4e54065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 --2- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 0x7f4e54065bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009500 tx=0x7f4e50004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 shutdown_connections 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 --2- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f4e40036880 0x7f4e40038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 --2- 172.21.15.19:0/2453944803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e540fd050 0x7f4e54065bd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 >> 172.21.15.19:0/2453944803 conn(0x7f4e540f70d0 msgr2=0x7f4e54102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 shutdown_connections 2024-03-16T08:53:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:17.489+0000 7f4e5c15e700 1 -- 172.21.15.19:0/2453944803 wait complete. 2024-03-16T08:53:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:17 smithi007 ceph-mon[29470]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:17 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/2453944803' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:18.939 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:18.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:19.197 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:19.244 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:19.797 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:19 smithi007 ceph-mon[29470]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:20.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.613+0000 7f2691c89700 1 -- 172.21.15.19:0/1875544602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 msgr2=0x7f268c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:20.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.613+0000 7f2691c89700 1 --2- 172.21.15.19:0/1875544602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 0x7f268c0ff6b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f267c0048c0 tx=0x7f267c009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:20.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.613+0000 7f2691c89700 1 -- 172.21.15.19:0/1875544602 shutdown_connections 2024-03-16T08:53:20.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.613+0000 7f2691c89700 1 --2- 172.21.15.19:0/1875544602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 0x7f268c0ff6b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.613+0000 7f2691c89700 1 -- 172.21.15.19:0/1875544602 >> 172.21.15.19:0/1875544602 conn(0x7f268c0f70c0 msgr2=0x7f268c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.613+0000 7f2691c89700 1 -- 172.21.15.19:0/1875544602 shutdown_connections 2024-03-16T08:53:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.613+0000 7f2691c89700 1 -- 172.21.15.19:0/1875544602 wait complete. 2024-03-16T08:53:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.614+0000 7f2691c89700 1 Processor -- start 2024-03-16T08:53:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.614+0000 7f2691c89700 1 -- start start 2024-03-16T08:53:20.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.614+0000 7f2691c89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 0x7f268c10d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:20.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.614+0000 7f2691c89700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c10d610 con 0x7f268c066980 2024-03-16T08:53:20.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.614+0000 7f268b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 0x7f268c10d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:20.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.614+0000 7f268b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 0x7f268c10d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:60940/0 (socket says 172.21.15.19:60940) 2024-03-16T08:53:20.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.614+0000 7f268b7fe700 1 -- 172.21.15.19:0/3761818901 learned_addr learned my addr 172.21.15.19:0/3761818901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:20.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.615+0000 7f268b7fe700 1 -- 172.21.15.19:0/3761818901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c005040 con 0x7f268c066980 2024-03-16T08:53:20.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.615+0000 7f268b7fe700 1 --2- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 0x7f268c10d0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f267c00b900 tx=0x7f267c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:20.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.615+0000 7f2673fff700 1 -- 172.21.15.19:0/3761818901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f267c00b5c0 con 0x7f268c066980 2024-03-16T08:53:20.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.615+0000 7f2673fff700 1 -- 172.21.15.19:0/3761818901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f267c00be70 con 0x7f268c066980 2024-03-16T08:53:20.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.615+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c10d810 con 0x7f268c066980 2024-03-16T08:53:20.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.615+0000 7f2673fff700 1 -- 172.21.15.19:0/3761818901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f267c01c590 con 0x7f268c066980 2024-03-16T08:53:20.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.615+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c10e380 con 0x7f268c066980 2024-03-16T08:53:20.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.616+0000 7f2673fff700 1 -- 172.21.15.19:0/3761818901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7f267c004d30 con 0x7f268c066980 2024-03-16T08:53:20.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.616+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c102f60 con 0x7f268c066980 2024-03-16T08:53:20.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.616+0000 7f2673fff700 1 --2- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f2674036880 0x7f2674038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:20.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.616+0000 7f2673fff700 1 -- 172.21.15.19:0/3761818901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f267c049390 con 0x7f268c066980 2024-03-16T08:53:20.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.616+0000 7f268affd700 1 --2- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f2674036880 0x7f2674038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:20.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.617+0000 7f268affd700 1 --2- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f2674036880 0x7f2674038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2680009990 tx=0x7f2680006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:20.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.621+0000 7f2673fff700 1 -- 172.21.15.19:0/3761818901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f267c01b350 con 0x7f268c066980 2024-03-16T08:53:20.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.894+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f268c10ee50 con 0x7f268c066980 2024-03-16T08:53:20.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.895+0000 7f2673fff700 1 -- 172.21.15.19:0/3761818901 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f267c021030 con 0x7f268c066980 2024-03-16T08:53:20.897 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:20.897 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:20.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f2674036880 msgr2=0x7f2674038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 --2- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f2674036880 0x7f2674038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2680009990 tx=0x7f2680006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 msgr2=0x7f268c10d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 --2- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 0x7f268c10d0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f267c00b900 tx=0x7f267c009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 shutdown_connections 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 --2- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f2674036880 0x7f2674038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 --2- 172.21.15.19:0/3761818901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f268c066980 0x7f268c10d0d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 >> 172.21.15.19:0/3761818901 conn(0x7f268c0f70c0 msgr2=0x7f268c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 shutdown_connections 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:20.897+0000 7f2691c89700 1 -- 172.21.15.19:0/3761818901 wait complete. 2024-03-16T08:53:20.899 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:21.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:21 smithi007 ceph-mon[29470]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:21.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:21 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3761818901' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:22.503 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:22.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:22.758 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:23.653 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:23 smithi007 ceph-mon[29470]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:23.653 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:23 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:23.653 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:23 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:23.653 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:23 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:23.653 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:23 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:23.654 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:23 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:23.654 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:23 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:23.654 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:23 smithi007 ceph-mon[29470]: Deploying daemon prometheus.smithi007 on smithi007 2024-03-16T08:53:24.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.184+0000 7fd25c77e700 1 -- 172.21.15.19:0/2248835687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 msgr2=0x7fd2540fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.184+0000 7fd25c77e700 1 --2- 172.21.15.19:0/2248835687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 0x7fd2540fd460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009c60 tx=0x7fd24c004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.184+0000 7fd25c77e700 1 -- 172.21.15.19:0/2248835687 shutdown_connections 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.184+0000 7fd25c77e700 1 --2- 172.21.15.19:0/2248835687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 0x7fd2540fd460 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.184+0000 7fd25c77e700 1 -- 172.21.15.19:0/2248835687 >> 172.21.15.19:0/2248835687 conn(0x7fd2540f70d0 msgr2=0x7fd2540f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.184+0000 7fd25c77e700 1 -- 172.21.15.19:0/2248835687 shutdown_connections 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.184+0000 7fd25c77e700 1 -- 172.21.15.19:0/2248835687 wait complete. 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25c77e700 1 Processor -- start 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25c77e700 1 -- start start 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25c77e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 0x7fd254065c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25c77e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25410fd80 con 0x7fd2540fd050 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25a51a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 0x7fd254065c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25a51a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 0x7fd254065c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:58636/0 (socket says 172.21.15.19:58636) 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25a51a700 1 -- 172.21.15.19:0/3328785714 learned_addr learned my addr 172.21.15.19:0/3328785714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25a51a700 1 -- 172.21.15.19:0/3328785714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c009940 con 0x7fd2540fd050 2024-03-16T08:53:24.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd25a51a700 1 --2- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 0x7fd254065c30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009500 tx=0x7fd24c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:24.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd252ffd700 1 -- 172.21.15.19:0/3328785714 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd24c007c80 con 0x7fd2540fd050 2024-03-16T08:53:24.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.185+0000 7fd252ffd700 1 -- 172.21.15.19:0/3328785714 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd24c01d910 con 0x7fd2540fd050 2024-03-16T08:53:24.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.186+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25410ff20 con 0x7fd2540fd050 2024-03-16T08:53:24.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.186+0000 7fd252ffd700 1 -- 172.21.15.19:0/3328785714 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd24c0129d0 con 0x7fd2540fd050 2024-03-16T08:53:24.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.186+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2541103b0 con 0x7fd2540fd050 2024-03-16T08:53:24.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.187+0000 7fd252ffd700 1 -- 172.21.15.19:0/3328785714 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fd24c019040 con 0x7fd2540fd050 2024-03-16T08:53:24.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.187+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd254105920 con 0x7fd2540fd050 2024-03-16T08:53:24.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.187+0000 7fd252ffd700 1 --2- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fd23c036880 0x7fd23c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:24.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.187+0000 7fd252ffd700 1 -- 172.21.15.19:0/3328785714 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd24c04a280 con 0x7fd2540fd050 2024-03-16T08:53:24.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.187+0000 7fd259d19700 1 --2- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fd23c036880 0x7fd23c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:24.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.187+0000 7fd259d19700 1 --2- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fd23c036880 0x7fd23c038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd248009990 tx=0x7fd248006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:24.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.192+0000 7fd252ffd700 1 -- 172.21.15.19:0/3328785714 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fd24c01da80 con 0x7fd2540fd050 2024-03-16T08:53:24.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.460+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd25402b2a0 con 0x7fd2540fd050 2024-03-16T08:53:24.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.461+0000 7fd252ffd700 1 -- 172.21.15.19:0/3328785714 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fd24c017020 con 0x7fd2540fd050 2024-03-16T08:53:24.463 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:24.463 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fd23c036880 msgr2=0x7fd23c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 --2- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fd23c036880 0x7fd23c038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd248009990 tx=0x7fd248006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 msgr2=0x7fd254065c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 --2- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 0x7fd254065c30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009500 tx=0x7fd24c004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 shutdown_connections 2024-03-16T08:53:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 --2- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fd23c036880 0x7fd23c038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:24.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 --2- 172.21.15.19:0/3328785714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2540fd050 0x7fd254065c30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:24.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 >> 172.21.15.19:0/3328785714 conn(0x7fd2540f70d0 msgr2=0x7fd254102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:24.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 shutdown_connections 2024-03-16T08:53:24.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:24.463+0000 7fd25c77e700 1 -- 172.21.15.19:0/3328785714 wait complete. 2024-03-16T08:53:24.466 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:25.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:25 smithi007 ceph-mon[29470]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:25.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:25 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3328785714' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:26.040 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:26.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:26.342 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:27.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:27 smithi007 ceph-mon[29470]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:27.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:27 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.811+0000 7fdc844cb700 1 -- 172.21.15.19:0/3895320915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 msgr2=0x7fdc7c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.811+0000 7fdc844cb700 1 --2- 172.21.15.19:0/3895320915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 0x7fdc7c0fce70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdc780048c0 tx=0x7fdc78009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.811+0000 7fdc844cb700 1 -- 172.21.15.19:0/3895320915 shutdown_connections 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.811+0000 7fdc844cb700 1 --2- 172.21.15.19:0/3895320915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 0x7fdc7c0fce70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.811+0000 7fdc844cb700 1 -- 172.21.15.19:0/3895320915 >> 172.21.15.19:0/3895320915 conn(0x7fdc7c0f81c0 msgr2=0x7fdc7c0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.811+0000 7fdc844cb700 1 -- 172.21.15.19:0/3895320915 shutdown_connections 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.811+0000 7fdc844cb700 1 -- 172.21.15.19:0/3895320915 wait complete. 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.812+0000 7fdc844cb700 1 Processor -- start 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.812+0000 7fdc844cb700 1 -- start start 2024-03-16T08:53:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.812+0000 7fdc844cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 0x7fdc7c10f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.812+0000 7fdc844cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c10f7c0 con 0x7fdc7c0fca60 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.812+0000 7fdc82267700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 0x7fdc7c10f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.812+0000 7fdc82267700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 0x7fdc7c10f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:58662/0 (socket says 172.21.15.19:58662) 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.812+0000 7fdc82267700 1 -- 172.21.15.19:0/3824863085 learned_addr learned my addr 172.21.15.19:0/3824863085 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.812+0000 7fdc82267700 1 -- 172.21.15.19:0/3824863085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78005040 con 0x7fdc7c0fca60 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.813+0000 7fdc82267700 1 --2- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 0x7fdc7c10f280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc780096b0 tx=0x7fdc78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.813+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3824863085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc7800bed0 con 0x7fdc7c0fca60 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.813+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c10f9c0 con 0x7fdc7c0fca60 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.813+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3824863085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc7800b5c0 con 0x7fdc7c0fca60 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.813+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3824863085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc7801c5b0 con 0x7fdc7c0fca60 2024-03-16T08:53:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.813+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c110530 con 0x7fdc7c0fca60 2024-03-16T08:53:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.814+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3824863085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fdc78013420 con 0x7fdc7c0fca60 2024-03-16T08:53:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.814+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c102f70 con 0x7fdc7c0fca60 2024-03-16T08:53:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.814+0000 7fdc72ffd700 1 --2- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fdc680368d0 0x7fdc68038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.814+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3824863085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdc78049a60 con 0x7fdc7c0fca60 2024-03-16T08:53:27.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.814+0000 7fdc81a66700 1 --2- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fdc680368d0 0x7fdc68038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:27.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.814+0000 7fdc81a66700 1 --2- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fdc680368d0 0x7fdc68038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009990 tx=0x7fdc6c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:27.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:27.823+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3824863085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fdc78013a80 con 0x7fdc7c0fca60 2024-03-16T08:53:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.090+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdc7c110770 con 0x7fdc7c0fca60 2024-03-16T08:53:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.091+0000 7fdc72ffd700 1 -- 172.21.15.19:0/3824863085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fdc78011370 con 0x7fdc7c0fca60 2024-03-16T08:53:28.093 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:28.093 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fdc680368d0 msgr2=0x7fdc68038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 --2- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fdc680368d0 0x7fdc68038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009990 tx=0x7fdc6c006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 msgr2=0x7fdc7c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 --2- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 0x7fdc7c10f280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc780096b0 tx=0x7fdc78009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 shutdown_connections 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 --2- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fdc680368d0 0x7fdc68038d80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 --2- 172.21.15.19:0/3824863085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c0fca60 0x7fdc7c10f280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 >> 172.21.15.19:0/3824863085 conn(0x7fdc7c0f81c0 msgr2=0x7fdc7c0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 shutdown_connections 2024-03-16T08:53:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:28.093+0000 7fdc844cb700 1 -- 172.21.15.19:0/3824863085 wait complete. 2024-03-16T08:53:28.096 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:28 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3824863085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:29.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:29 smithi007 ceph-mon[29470]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:29.653 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:29.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 -- 172.21.15.19:0/11803735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 msgr2=0x7fef240fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 --2- 172.21.15.19:0/11803735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 0x7fef240fe4c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fef14009c60 tx=0x7fef14004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 -- 172.21.15.19:0/11803735 shutdown_connections 2024-03-16T08:53:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 --2- 172.21.15.19:0/11803735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 0x7fef240fe4c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 -- 172.21.15.19:0/11803735 >> 172.21.15.19:0/11803735 conn(0x7fef240f6850 msgr2=0x7fef240f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 -- 172.21.15.19:0/11803735 shutdown_connections 2024-03-16T08:53:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 -- 172.21.15.19:0/11803735 wait complete. 2024-03-16T08:53:31.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 Processor -- start 2024-03-16T08:53:31.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.340+0000 7fef29d18700 1 -- start start 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef29d18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 0x7fef2410f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef29d18700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2410f7f0 con 0x7fef240fe0b0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef237fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 0x7fef2410f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef237fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 0x7fef2410f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:38616/0 (socket says 172.21.15.19:38616) 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef237fe700 1 -- 172.21.15.19:0/927125757 learned_addr learned my addr 172.21.15.19:0/927125757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef237fe700 1 -- 172.21.15.19:0/927125757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef14009940 con 0x7fef240fe0b0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef237fe700 1 --2- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 0x7fef2410f2b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef14004e60 tx=0x7fef14004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef0bfff700 1 -- 172.21.15.19:0/927125757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef14012410 con 0x7fef240fe0b0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef0bfff700 1 -- 172.21.15.19:0/927125757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef14012a40 con 0x7fef240fe0b0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2410f9f0 con 0x7fef240fe0b0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.341+0000 7fef0bfff700 1 -- 172.21.15.19:0/927125757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef1401ba40 con 0x7fef240fe0b0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.342+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef24110560 con 0x7fef240fe0b0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.342+0000 7fef0bfff700 1 -- 172.21.15.19:0/927125757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 43423+0+0 (secure 0 0 0) 0x7fef14019040 con 0x7fef240fe0b0 2024-03-16T08:53:31.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.342+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef24102f80 con 0x7fef240fe0b0 2024-03-16T08:53:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.343+0000 7fef0bfff700 1 --2- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fef0c036880 0x7fef0c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.343+0000 7fef0bfff700 1 -- 172.21.15.19:0/927125757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fef14049ce0 con 0x7fef240fe0b0 2024-03-16T08:53:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.343+0000 7fef22ffd700 1 --2- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fef0c036880 0x7fef0c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:31.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.343+0000 7fef22ffd700 1 --2- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fef0c036880 0x7fef0c038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef18009990 tx=0x7fef18006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:31.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.348+0000 7fef0bfff700 1 -- 172.21.15.19:0/927125757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fef14022b90 con 0x7fef240fe0b0 2024-03-16T08:53:31.374 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:31 smithi007 ceph-mon[29470]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:31.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.619+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fef241107a0 con 0x7fef240fe0b0 2024-03-16T08:53:31.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.620+0000 7fef0bfff700 1 -- 172.21.15.19:0/927125757 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fef140172f0 con 0x7fef240fe0b0 2024-03-16T08:53:31.622 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:31.622 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fef0c036880 msgr2=0x7fef0c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 --2- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fef0c036880 0x7fef0c038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef18009990 tx=0x7fef18006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 msgr2=0x7fef2410f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 --2- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 0x7fef2410f2b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef14004e60 tx=0x7fef14004d10 comp rx=0 tx=0).stop 2024-03-16T08:53:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 shutdown_connections 2024-03-16T08:53:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 --2- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fef0c036880 0x7fef0c038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 --2- 172.21.15.19:0/927125757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef240fe0b0 0x7fef2410f2b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 >> 172.21.15.19:0/927125757 conn(0x7fef240f6850 msgr2=0x7fef240f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 shutdown_connections 2024-03-16T08:53:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:31.622+0000 7fef29d18700 1 -- 172.21.15.19:0/927125757 wait complete. 2024-03-16T08:53:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:32.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:32 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/927125757' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:33.124 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:33.125 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:33.378 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:33.424 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:33.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:33 smithi007 ceph-mon[29470]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:34.877 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:34 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:34.877 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:34 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:34.877 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:34 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:34.877 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:34 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-16T08:53:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.950+0000 7fc6932cb700 1 -- 172.21.15.19:0/1071479007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 msgr2=0x7fc68c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.950+0000 7fc6932cb700 1 --2- 172.21.15.19:0/1071479007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 0x7fc68c0fce70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6880048c0 tx=0x7fc688009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.950+0000 7fc6932cb700 1 -- 172.21.15.19:0/1071479007 shutdown_connections 2024-03-16T08:53:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.950+0000 7fc6932cb700 1 --2- 172.21.15.19:0/1071479007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 0x7fc68c0fce70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.950+0000 7fc6932cb700 1 -- 172.21.15.19:0/1071479007 >> 172.21.15.19:0/1071479007 conn(0x7fc68c0f81c0 msgr2=0x7fc68c0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.950+0000 7fc6932cb700 1 -- 172.21.15.19:0/1071479007 shutdown_connections 2024-03-16T08:53:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.950+0000 7fc6932cb700 1 -- 172.21.15.19:0/1071479007 wait complete. 2024-03-16T08:53:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.951+0000 7fc6932cb700 1 Processor -- start 2024-03-16T08:53:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.951+0000 7fc6932cb700 1 -- start start 2024-03-16T08:53:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.951+0000 7fc6932cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 0x7fc68c10d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.951+0000 7fc6932cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c10d5d0 con 0x7fc68c0fca60 2024-03-16T08:53:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.951+0000 7fc691067700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 0x7fc68c10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.951+0000 7fc691067700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 0x7fc68c10d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:38624/0 (socket says 172.21.15.19:38624) 2024-03-16T08:53:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.951+0000 7fc691067700 1 -- 172.21.15.19:0/2094589914 learned_addr learned my addr 172.21.15.19:0/2094589914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.952+0000 7fc691067700 1 -- 172.21.15.19:0/2094589914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc688005040 con 0x7fc68c0fca60 2024-03-16T08:53:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.952+0000 7fc691067700 1 --2- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 0x7fc68c10d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6880096b0 tx=0x7fc688009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.952+0000 7fc681ffb700 1 -- 172.21.15.19:0/2094589914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc68800bed0 con 0x7fc68c0fca60 2024-03-16T08:53:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.952+0000 7fc681ffb700 1 -- 172.21.15.19:0/2094589914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc68800b5c0 con 0x7fc68c0fca60 2024-03-16T08:53:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.952+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68c10d7d0 con 0x7fc68c0fca60 2024-03-16T08:53:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.952+0000 7fc681ffb700 1 -- 172.21.15.19:0/2094589914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc68801c5b0 con 0x7fc68c0fca60 2024-03-16T08:53:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.952+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c10e340 con 0x7fc68c0fca60 2024-03-16T08:53:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.953+0000 7fc681ffb700 1 -- 172.21.15.19:0/2094589914 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 14) v1 ==== 43437+0+0 (secure 0 0 0) 0x7fc688013420 con 0x7fc68c0fca60 2024-03-16T08:53:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.953+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68c102f70 con 0x7fc68c0fca60 2024-03-16T08:53:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.953+0000 7fc681ffb700 1 --2- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc678036920 0x7fc678038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.954+0000 7fc681ffb700 1 -- 172.21.15.19:0/2094589914 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc688049a30 con 0x7fc68c0fca60 2024-03-16T08:53:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.954+0000 7fc690866700 1 -- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc678036920 msgr2=0x7fc678038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2343923853 2024-03-16T08:53:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.954+0000 7fc690866700 1 --2- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc678036920 0x7fc678038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:53:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:34.958+0000 7fc681ffb700 1 -- 172.21.15.19:0/2094589914 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fc688025400 con 0x7fc68c0fca60 2024-03-16T08:53:35.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.155+0000 7fc690866700 1 -- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc678036920 msgr2=0x7fc678038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2343923853 2024-03-16T08:53:35.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.155+0000 7fc690866700 1 --2- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc678036920 0x7fc678038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:53:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.226+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc68c10ed50 con 0x7fc68c0fca60 2024-03-16T08:53:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.226+0000 7fc681ffb700 1 -- 172.21.15.19:0/2094589914 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fc688004d30 con 0x7fc68c0fca60 2024-03-16T08:53:35.228 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:35.228 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc678036920 msgr2=0x7fc678038dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:53:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 --2- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc678036920 0x7fc678038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 msgr2=0x7fc68c10d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 --2- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 0x7fc68c10d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6880096b0 tx=0x7fc688009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 shutdown_connections 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 --2- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7fc678036920 0x7fc678038dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 --2- 172.21.15.19:0/2094589914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc68c0fca60 0x7fc68c10d090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 >> 172.21.15.19:0/2094589914 conn(0x7fc68c0f81c0 msgr2=0x7fc68c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.228+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 shutdown_connections 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:35.229+0000 7fc6932cb700 1 -- 172.21.15.19:0/2094589914 wait complete. 2024-03-16T08:53:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:35.895 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:35 smithi007 ceph-mon[29470]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:53:35.895 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:35 smithi007 ceph-mon[29470]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-16T08:53:35.895 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:35 smithi007 ceph-mon[29470]: mgrmap e14: smithi007.uamwtp(active, since 58s) 2024-03-16T08:53:35.895 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:35 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/2094589914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:36.830 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:36.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:37.087 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:38.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.513+0000 7f74a965e700 1 -- 172.21.15.19:0/239246531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 msgr2=0x7f74a40fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:38.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.513+0000 7f74a965e700 1 --2- 172.21.15.19:0/239246531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 0x7f74a40fce50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f749c0048c0 tx=0x7f749c009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:38.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.513+0000 7f74a965e700 1 -- 172.21.15.19:0/239246531 shutdown_connections 2024-03-16T08:53:38.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.513+0000 7f74a965e700 1 --2- 172.21.15.19:0/239246531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 0x7f74a40fce50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:38.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.513+0000 7f74a965e700 1 -- 172.21.15.19:0/239246531 >> 172.21.15.19:0/239246531 conn(0x7f74a40f81c0 msgr2=0x7f74a40fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:38.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.513+0000 7f74a965e700 1 -- 172.21.15.19:0/239246531 shutdown_connections 2024-03-16T08:53:38.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.513+0000 7f74a965e700 1 -- 172.21.15.19:0/239246531 wait complete. 2024-03-16T08:53:38.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.514+0000 7f74a965e700 1 Processor -- start 2024-03-16T08:53:38.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.514+0000 7f74a965e700 1 -- start start 2024-03-16T08:53:38.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.514+0000 7f74a965e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 0x7f74a4071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:38.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.514+0000 7f74a965e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40722a0 con 0x7f74a40fca40 2024-03-16T08:53:38.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.514+0000 7f74a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 0x7f74a4071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:38.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.514+0000 7f74a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 0x7f74a4071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:38642/0 (socket says 172.21.15.19:38642) 2024-03-16T08:53:38.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.514+0000 7f74a2ffd700 1 -- 172.21.15.19:0/3353992565 learned_addr learned my addr 172.21.15.19:0/3353992565 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:38.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.515+0000 7f74a2ffd700 1 -- 172.21.15.19:0/3353992565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c005040 con 0x7f74a40fca40 2024-03-16T08:53:38.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.515+0000 7f74a2ffd700 1 --2- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 0x7f74a4071d60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f749c0096b0 tx=0x7f749c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:38.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.515+0000 7f749b7fe700 1 -- 172.21.15.19:0/3353992565 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f749c00bed0 con 0x7f74a40fca40 2024-03-16T08:53:38.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.515+0000 7f749b7fe700 1 -- 172.21.15.19:0/3353992565 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f749c00b5c0 con 0x7f74a40fca40 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.515+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a4070370 con 0x7f74a40fca40 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.515+0000 7f749b7fe700 1 -- 172.21.15.19:0/3353992565 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f749c01c5b0 con 0x7f74a40fca40 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.515+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a4070860 con 0x7f74a40fca40 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.516+0000 7f749b7fe700 1 -- 172.21.15.19:0/3353992565 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 14) v1 ==== 43437+0+0 (secure 0 0 0) 0x7f749c013420 con 0x7f74a40fca40 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.516+0000 7f749b7fe700 1 --2- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f74900368d0 0x7f7490038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.516+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a405f5f0 con 0x7f74a40fca40 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.516+0000 7f74a27fc700 1 -- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f74900368d0 msgr2=0x7f7490038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2343923853 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.516+0000 7f749b7fe700 1 -- 172.21.15.19:0/3353992565 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f749c049b30 con 0x7f74a40fca40 2024-03-16T08:53:38.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.516+0000 7f74a27fc700 1 --2- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f74900368d0 0x7f7490038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:53:38.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.521+0000 7f749b7fe700 1 -- 172.21.15.19:0/3353992565 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f749c025400 con 0x7f74a40fca40 2024-03-16T08:53:38.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.717+0000 7f74a27fc700 1 -- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f74900368d0 msgr2=0x7f7490038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2343923853 2024-03-16T08:53:38.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.717+0000 7f74a27fc700 1 --2- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f74900368d0 0x7f7490038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T08:53:38.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.793+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f74a404bed0 con 0x7f74a40fca40 2024-03-16T08:53:38.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.794+0000 7f749b7fe700 1 -- 172.21.15.19:0/3353992565 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f749c02bb10 con 0x7f74a40fca40 2024-03-16T08:53:38.796 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:38.796 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f74900368d0 msgr2=0x7f7490038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 --2- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f74900368d0 0x7f7490038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 msgr2=0x7f74a4071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 --2- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 0x7f74a4071d60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f749c0096b0 tx=0x7f749c009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 shutdown_connections 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 --2- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:6800/2343923853,v1:172.21.15.7:6801/2343923853] conn(0x7f74900368d0 0x7f7490038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 --2- 172.21.15.19:0/3353992565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40fca40 0x7f74a4071d60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 >> 172.21.15.19:0/3353992565 conn(0x7f74a40f81c0 msgr2=0x7f74a4103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:38.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 shutdown_connections 2024-03-16T08:53:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:38.796+0000 7f74a965e700 1 -- 172.21.15.19:0/3353992565 wait complete. 2024-03-16T08:53:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:38.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:38 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3353992565' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:40.315 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:40.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:40.570 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: Active manager daemon smithi007.uamwtp restarted 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: Activating manager daemon smithi007.uamwtp 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: osdmap e5: 0 total, 0 up, 0 in 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: mgrmap e15: smithi007.uamwtp(active, starting, since 0.00314663s) 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:53:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:53:41.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: Manager daemon smithi007.uamwtp is now available 2024-03-16T08:53:41.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:53:41.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:53:41.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:41.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 -- 172.21.15.19:0/1510150915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 msgr2=0x7f81840fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 --2- 172.21.15.19:0/1510150915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 0x7f81840fce90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f81800048c0 tx=0x7f8180009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 -- 172.21.15.19:0/1510150915 shutdown_connections 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 --2- 172.21.15.19:0/1510150915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 0x7f81840fce90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 -- 172.21.15.19:0/1510150915 >> 172.21.15.19:0/1510150915 conn(0x7f81840f8220 msgr2=0x7f81840fa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 -- 172.21.15.19:0/1510150915 shutdown_connections 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 -- 172.21.15.19:0/1510150915 wait complete. 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 Processor -- start 2024-03-16T08:53:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 -- start start 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 0x7f8184071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.069+0000 7f818b739700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840722a0 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f81894d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 0x7f8184071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f81894d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 0x7f8184071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:51990/0 (socket says 172.21.15.19:51990) 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f81894d5700 1 -- 172.21.15.19:0/2963614822 learned_addr learned my addr 172.21.15.19:0/2963614822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f81894d5700 1 -- 172.21.15.19:0/2963614822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180005040 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f81894d5700 1 --2- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 0x7f8184071d60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f81800096b0 tx=0x7f8180009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f8179ffb700 1 -- 172.21.15.19:0/2963614822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f818000bed0 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f8179ffb700 1 -- 172.21.15.19:0/2963614822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f818000b5c0 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f8179ffb700 1 -- 172.21.15.19:0/2963614822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f818001c5b0 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.070+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8184070370 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.071+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8184070860 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.071+0000 7f8179ffb700 1 -- 172.21.15.19:0/2963614822 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) v1 ==== 43162+0+0 (secure 0 0 0) 0x7f8180013a80 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.071+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818405f5f0 con 0x7f81840fca80 2024-03-16T08:53:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.072+0000 7f8179ffb700 1 -- 172.21.15.19:0/2963614822 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8180049790 con 0x7f81840fca80 2024-03-16T08:53:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.076+0000 7f8179ffb700 1 -- 172.21.15.19:0/2963614822 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f8180011960 con 0x7f81840fca80 2024-03-16T08:53:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.316+0000 7f8179ffb700 1 -- 172.21.15.19:0/2963614822 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7f8180025400 con 0x7f81840fca80 2024-03-16T08:53:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.316+0000 7f8179ffb700 1 --2- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8170037880 0x7f8170039c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.316+0000 7f8188cd4700 1 --2- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8170037880 0x7f8170039c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.317+0000 7f8188cd4700 1 --2- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8170037880 0x7f8170039c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8174009990 tx=0x7f8174006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.356+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f818404bed0 con 0x7f81840fca80 2024-03-16T08:53:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.356+0000 7f8179ffb700 1 -- 172.21.15.19:0/2963614822 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f818002b420 con 0x7f81840fca80 2024-03-16T08:53:42.358 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:42.358 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.358+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8170037880 msgr2=0x7f8170039c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.358+0000 7f818b739700 1 --2- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8170037880 0x7f8170039c60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8174009990 tx=0x7f8174006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.358+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 msgr2=0x7f8184071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.358+0000 7f818b739700 1 --2- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 0x7f8184071d60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f81800096b0 tx=0x7f8180009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.358+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 shutdown_connections 2024-03-16T08:53:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.358+0000 7f818b739700 1 --2- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8170037880 0x7f8170039c60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.358+0000 7f818b739700 1 --2- 172.21.15.19:0/2963614822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840fca80 0x7f8184071d60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.358+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 >> 172.21.15.19:0/2963614822 conn(0x7f81840f8220 msgr2=0x7f8184103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.359+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 shutdown_connections 2024-03-16T08:53:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:42.359+0000 7f818b739700 1 -- 172.21.15.19:0/2963614822 wait complete. 2024-03-16T08:53:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:53:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T08:53:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:42 smithi007 ceph-mon[29470]: mgrmap e16: smithi007.uamwtp(active, since 1.00579s) 2024-03-16T08:53:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:42 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/2963614822' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:43.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:43 smithi007 ceph-mon[29470]: [16/Mar/2024:08:53:42] ENGINE Bus STARTING 2024-03-16T08:53:43.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:43 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:43.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:43 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:43.949 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:43.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:44.373 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:44.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:44 smithi007 ceph-mon[29470]: [16/Mar/2024:08:53:42] ENGINE Serving on https://172.21.15.7:7150 2024-03-16T08:53:44.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:44 smithi007 ceph-mon[29470]: [16/Mar/2024:08:53:42] ENGINE Bus STARTED 2024-03-16T08:53:44.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:44 smithi007 ceph-mon[29470]: mgrmap e17: smithi007.uamwtp(active, since 2s) 2024-03-16T08:53:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T08:53:46.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.852+0000 7fa9a1308700 1 -- 172.21.15.19:0/3642481079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 msgr2=0x7fa99c0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:46.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.852+0000 7fa9a1308700 1 --2- 172.21.15.19:0/3642481079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 0x7fa99c0fbd70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0048c0 tx=0x7fa98c009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:46.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.852+0000 7fa9a1308700 1 -- 172.21.15.19:0/3642481079 shutdown_connections 2024-03-16T08:53:46.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.852+0000 7fa9a1308700 1 --2- 172.21.15.19:0/3642481079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 0x7fa99c0fbd70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:46.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.852+0000 7fa9a1308700 1 -- 172.21.15.19:0/3642481079 >> 172.21.15.19:0/3642481079 conn(0x7fa99c0f70c0 msgr2=0x7fa99c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:46.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.852+0000 7fa9a1308700 1 -- 172.21.15.19:0/3642481079 shutdown_connections 2024-03-16T08:53:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.852+0000 7fa9a1308700 1 -- 172.21.15.19:0/3642481079 wait complete. 2024-03-16T08:53:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.853+0000 7fa9a1308700 1 Processor -- start 2024-03-16T08:53:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.853+0000 7fa9a1308700 1 -- start start 2024-03-16T08:53:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.853+0000 7fa9a1308700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 0x7fa99c10f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.853+0000 7fa9a1308700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c10f7c0 con 0x7fa99c0fb960 2024-03-16T08:53:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.853+0000 7fa99affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 0x7fa99c10f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa99affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 0x7fa99c10f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:52014/0 (socket says 172.21.15.19:52014) 2024-03-16T08:53:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa99affd700 1 -- 172.21.15.19:0/423960790 learned_addr learned my addr 172.21.15.19:0/423960790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:46.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa99affd700 1 -- 172.21.15.19:0/423960790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c005040 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa99affd700 1 --2- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 0x7fa99c10f280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0096b0 tx=0x7fa98c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa9837fe700 1 -- 172.21.15.19:0/423960790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa98c00bed0 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa9837fe700 1 -- 172.21.15.19:0/423960790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa98c00b5c0 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c10f9c0 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa9837fe700 1 -- 172.21.15.19:0/423960790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa98c01c5b0 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.854+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c110530 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.855+0000 7fa9837fe700 1 -- 172.21.15.19:0/423960790 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7fa98c01c820 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.855+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c101e70 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.855+0000 7fa9837fe700 1 --2- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa984036970 0x7fa984038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.855+0000 7fa9837fe700 1 -- 172.21.15.19:0/423960790 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa98c049d90 con 0x7fa99c0fb960 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.855+0000 7fa99a7fc700 1 --2- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa984036970 0x7fa984038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:46.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.856+0000 7fa99a7fc700 1 --2- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa984036970 0x7fa984038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa990009990 tx=0x7fa990006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:46.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:46.860+0000 7fa9837fe700 1 -- 172.21.15.19:0/423960790 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fa98c04f020 con 0x7fa99c0fb960 2024-03-16T08:53:47.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.129+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa99c110770 con 0x7fa99c0fb960 2024-03-16T08:53:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.129+0000 7fa9837fe700 1 -- 172.21.15.19:0/423960790 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fa98c011d40 con 0x7fa99c0fb960 2024-03-16T08:53:47.131 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:47.131 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:47.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa984036970 msgr2=0x7fa984038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:47.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 --2- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa984036970 0x7fa984038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa990009990 tx=0x7fa990006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 msgr2=0x7fa99c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 --2- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 0x7fa99c10f280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0096b0 tx=0x7fa98c009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 shutdown_connections 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 --2- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa984036970 0x7fa984038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 --2- 172.21.15.19:0/423960790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa99c0fb960 0x7fa99c10f280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 >> 172.21.15.19:0/423960790 conn(0x7fa99c0f70c0 msgr2=0x7fa99c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.131+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 shutdown_connections 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:47.132+0000 7fa9a1308700 1 -- 172.21.15.19:0/423960790 wait complete. 2024-03-16T08:53:47.134 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:53:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:48 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/423960790' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:53:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:53:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:53:48.688 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:48.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:48.935 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:53:49.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T08:53:49.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T08:53:49.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:53:49.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:53:49.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:53:49.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:53:49.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:49.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:49.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:49.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:49.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:49.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T08:53:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:50 smithi007 ceph-mon[29470]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:53:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:50 smithi007 ceph-mon[29470]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:53:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:50 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-16T08:53:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:50 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:53:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:50 smithi007 ceph-mon[29470]: Deploying daemon crash.smithi019 on smithi019 2024-03-16T08:53:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.532+0000 7f4ee3023700 1 -- 172.21.15.19:0/593803471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 msgr2=0x7f4edc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.532+0000 7f4ee3023700 1 --2- 172.21.15.19:0/593803471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 0x7f4edc0706b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00048c0 tx=0x7f4ed0009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.532+0000 7f4ee3023700 1 -- 172.21.15.19:0/593803471 shutdown_connections 2024-03-16T08:53:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.532+0000 7f4ee3023700 1 --2- 172.21.15.19:0/593803471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 0x7f4edc0706b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.532+0000 7f4ee3023700 1 -- 172.21.15.19:0/593803471 >> 172.21.15.19:0/593803471 conn(0x7f4edc0f85e0 msgr2=0x7f4edc0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.532+0000 7f4ee3023700 1 -- 172.21.15.19:0/593803471 shutdown_connections 2024-03-16T08:53:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.532+0000 7f4ee3023700 1 -- 172.21.15.19:0/593803471 wait complete. 2024-03-16T08:53:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.533+0000 7f4ee3023700 1 Processor -- start 2024-03-16T08:53:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.533+0000 7f4ee3023700 1 -- start start 2024-03-16T08:53:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.533+0000 7f4ee3023700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 0x7f4edc1153c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.533+0000 7f4ee3023700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc115900 con 0x7f4edc0702e0 2024-03-16T08:53:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.533+0000 7f4ee0dbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 0x7f4edc1153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.533+0000 7f4ee0dbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 0x7f4edc1153c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:52024/0 (socket says 172.21.15.19:52024) 2024-03-16T08:53:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.533+0000 7f4ee0dbf700 1 -- 172.21.15.19:0/3251877432 learned_addr learned my addr 172.21.15.19:0/3251877432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:50.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.534+0000 7f4ee0dbf700 1 -- 172.21.15.19:0/3251877432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed0005040 con 0x7f4edc0702e0 2024-03-16T08:53:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.534+0000 7f4ee0dbf700 1 --2- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 0x7f4edc1153c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000b820 tx=0x7f4ed0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.534+0000 7f4ed97fa700 1 -- 172.21.15.19:0/3251877432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ed00093c0 con 0x7f4edc0702e0 2024-03-16T08:53:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.534+0000 7f4ed97fa700 1 -- 172.21.15.19:0/3251877432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ed0013420 con 0x7f4edc0702e0 2024-03-16T08:53:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.534+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc115b00 con 0x7f4edc0702e0 2024-03-16T08:53:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.534+0000 7f4ed97fa700 1 -- 172.21.15.19:0/3251877432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ed001c5b0 con 0x7f4edc0702e0 2024-03-16T08:53:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.534+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc115f70 con 0x7f4edc0702e0 2024-03-16T08:53:50.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.535+0000 7f4ed97fa700 1 -- 172.21.15.19:0/3251877432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7f4ed0009520 con 0x7f4edc0702e0 2024-03-16T08:53:50.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.535+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc071a20 con 0x7f4edc0702e0 2024-03-16T08:53:50.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.535+0000 7f4ed97fa700 1 --2- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f4ecc036a00 0x7f4ecc038eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:50.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.535+0000 7f4ed97fa700 1 -- 172.21.15.19:0/3251877432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f4ed0049b10 con 0x7f4edc0702e0 2024-03-16T08:53:50.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.535+0000 7f4edbfff700 1 --2- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f4ecc036a00 0x7f4ecc038eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:50.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.536+0000 7f4edbfff700 1 --2- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f4ecc036a00 0x7f4ecc038eb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009990 tx=0x7f4ec8006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.540+0000 7f4ed97fa700 1 -- 172.21.15.19:0/3251877432 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f4ed0011550 con 0x7f4edc0702e0 2024-03-16T08:53:50.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.811+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4edc05f5f0 con 0x7f4edc0702e0 2024-03-16T08:53:50.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.812+0000 7f4ed97fa700 1 -- 172.21.15.19:0/3251877432 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f4ed0011370 con 0x7f4edc0702e0 2024-03-16T08:53:50.814 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:50.814 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f4ecc036a00 msgr2=0x7f4ecc038eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 --2- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f4ecc036a00 0x7f4ecc038eb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009990 tx=0x7f4ec8006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 msgr2=0x7f4edc1153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 --2- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 0x7f4edc1153c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000b820 tx=0x7f4ed0009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 shutdown_connections 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 --2- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f4ecc036a00 0x7f4ecc038eb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 --2- 172.21.15.19:0/3251877432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4edc0702e0 0x7f4edc1153c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 >> 172.21.15.19:0/3251877432 conn(0x7f4edc0f85e0 msgr2=0x7f4edc105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 shutdown_connections 2024-03-16T08:53:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:50.814+0000 7f4ee3023700 1 -- 172.21.15.19:0/3251877432 wait complete. 2024-03-16T08:53:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:51.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:51 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3251877432' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:52.660 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:52.660 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:52.933 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 -- 172.21.15.19:0/3089331835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 msgr2=0x7fe400102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 --2- 172.21.15.19:0/3089331835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 0x7fe400102eb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40048c0 tx=0x7fe3f4009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 -- 172.21.15.19:0/3089331835 shutdown_connections 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 --2- 172.21.15.19:0/3089331835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 0x7fe400102eb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 -- 172.21.15.19:0/3089331835 >> 172.21.15.19:0/3089331835 conn(0x7fe4000f6c30 msgr2=0x7fe4000f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 -- 172.21.15.19:0/3089331835 shutdown_connections 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 -- 172.21.15.19:0/3089331835 wait complete. 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 Processor -- start 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.845+0000 7fe40707f700 1 -- start start 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.846+0000 7fe40707f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 0x7fe400113080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.846+0000 7fe40707f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001135c0 con 0x7fe400102ae0 2024-03-16T08:53:54.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.846+0000 7fe404e1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 0x7fe400113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:54.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.846+0000 7fe404e1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 0x7fe400113080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:40470/0 (socket says 172.21.15.19:40470) 2024-03-16T08:53:54.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.846+0000 7fe404e1b700 1 -- 172.21.15.19:0/4172644370 learned_addr learned my addr 172.21.15.19:0/4172644370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:54.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.846+0000 7fe404e1b700 1 -- 172.21.15.19:0/4172644370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f4005040 con 0x7fe400102ae0 2024-03-16T08:53:54.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.846+0000 7fe404e1b700 1 --2- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 0x7fe400113080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400b820 tx=0x7fe3f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:54.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.847+0000 7fe3fd7fa700 1 -- 172.21.15.19:0/4172644370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3f40093f0 con 0x7fe400102ae0 2024-03-16T08:53:54.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.847+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe400113820 con 0x7fe400102ae0 2024-03-16T08:53:54.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.847+0000 7fe3fd7fa700 1 -- 172.21.15.19:0/4172644370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3f4013420 con 0x7fe400102ae0 2024-03-16T08:53:54.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.847+0000 7fe3fd7fa700 1 -- 172.21.15.19:0/4172644370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3f401c570 con 0x7fe400102ae0 2024-03-16T08:53:54.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.847+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe400113c90 con 0x7fe400102ae0 2024-03-16T08:53:54.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.848+0000 7fe3fd7fa700 1 -- 172.21.15.19:0/4172644370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7fe3f401c830 con 0x7fe400102ae0 2024-03-16T08:53:54.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.848+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4000fe2a0 con 0x7fe400102ae0 2024-03-16T08:53:54.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.848+0000 7fe3fd7fa700 1 --2- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fe3f0036970 0x7fe3f0038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:54.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.848+0000 7fe3fd7fa700 1 -- 172.21.15.19:0/4172644370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe3f4049bf0 con 0x7fe400102ae0 2024-03-16T08:53:54.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.848+0000 7fe3fffff700 1 --2- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fe3f0036970 0x7fe3f0038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:54.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.849+0000 7fe3fffff700 1 --2- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fe3f0036970 0x7fe3f0038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009990 tx=0x7fe3ec006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:54.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:54.853+0000 7fe3fd7fa700 1 -- 172.21.15.19:0/4172644370 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fe3f4023300 con 0x7fe400102ae0 2024-03-16T08:53:55.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.187+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe40005f5f0 con 0x7fe400102ae0 2024-03-16T08:53:55.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.187+0000 7fe3fd7fa700 1 -- 172.21.15.19:0/4172644370 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fe3f4011b10 con 0x7fe400102ae0 2024-03-16T08:53:55.190 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:53:55.190 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:53:55.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fe3f0036970 msgr2=0x7fe3f0038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:55.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 --2- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fe3f0036970 0x7fe3f0038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009990 tx=0x7fe3ec006e40 comp rx=0 tx=0).stop 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 msgr2=0x7fe400113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 --2- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 0x7fe400113080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400b820 tx=0x7fe3f4009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 shutdown_connections 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 --2- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fe3f0036970 0x7fe3f0038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 --2- 172.21.15.19:0/4172644370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe400102ae0 0x7fe400113080 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 >> 172.21.15.19:0/4172644370 conn(0x7fe4000f6c30 msgr2=0x7fe4000f8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.189+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 shutdown_connections 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:55.190+0000 7fe40707f700 1 -- 172.21.15.19:0/4172644370 wait complete. 2024-03-16T08:53:55.192 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:53:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:55 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:55 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:55 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:55 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:55 smithi007 ceph-mon[29470]: Deploying daemon node-exporter.smithi019 on smithi019 2024-03-16T08:53:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:55 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/4172644370' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:53:56.727 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:53:56.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:53:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:53:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:53:57.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:53:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.994+0000 7fdbed543700 1 -- 172.21.15.19:0/700728758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 msgr2=0x7fdbe8102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:53:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.994+0000 7fdbed543700 1 --2- 172.21.15.19:0/700728758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 0x7fdbe8102eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80048c0 tx=0x7fdbd8009e30 comp rx=0 tx=0).stop 2024-03-16T08:53:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.994+0000 7fdbed543700 1 -- 172.21.15.19:0/700728758 shutdown_connections 2024-03-16T08:53:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.994+0000 7fdbed543700 1 --2- 172.21.15.19:0/700728758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 0x7fdbe8102eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:53:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.994+0000 7fdbed543700 1 -- 172.21.15.19:0/700728758 >> 172.21.15.19:0/700728758 conn(0x7fdbe80f85e0 msgr2=0x7fdbe80fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:53:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.994+0000 7fdbed543700 1 -- 172.21.15.19:0/700728758 shutdown_connections 2024-03-16T08:53:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.994+0000 7fdbed543700 1 -- 172.21.15.19:0/700728758 wait complete. 2024-03-16T08:53:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.995+0000 7fdbed543700 1 Processor -- start 2024-03-16T08:53:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.995+0000 7fdbed543700 1 -- start start 2024-03-16T08:53:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.995+0000 7fdbed543700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 0x7fdbe8113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:53:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.995+0000 7fdbed543700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81136a0 con 0x7fdbe8102ae0 2024-03-16T08:53:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.995+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 0x7fdbe8113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:53:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.995+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 0x7fdbe8113160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:40486/0 (socket says 172.21.15.19:40486) 2024-03-16T08:53:59.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.995+0000 7fdbe6ffd700 1 -- 172.21.15.19:0/3062920567 learned_addr learned my addr 172.21.15.19:0/3062920567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:53:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.996+0000 7fdbe6ffd700 1 -- 172.21.15.19:0/3062920567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd8005040 con 0x7fdbe8102ae0 2024-03-16T08:53:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.996+0000 7fdbe6ffd700 1 --2- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 0x7fdbe8113160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800b820 tx=0x7fdbd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:53:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.996+0000 7fdbcf7fe700 1 -- 172.21.15.19:0/3062920567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbd80093c0 con 0x7fdbe8102ae0 2024-03-16T08:53:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.996+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe81138a0 con 0x7fdbe8102ae0 2024-03-16T08:53:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.996+0000 7fdbcf7fe700 1 -- 172.21.15.19:0/3062920567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbd8013420 con 0x7fdbe8102ae0 2024-03-16T08:53:59.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.996+0000 7fdbcf7fe700 1 -- 172.21.15.19:0/3062920567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbd801c5b0 con 0x7fdbe8102ae0 2024-03-16T08:54:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.996+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe8113d90 con 0x7fdbe8102ae0 2024-03-16T08:54:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.997+0000 7fdbcf7fe700 1 -- 172.21.15.19:0/3062920567 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7fdbd8013a90 con 0x7fdbe8102ae0 2024-03-16T08:54:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.997+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe80fe1e0 con 0x7fdbe8102ae0 2024-03-16T08:54:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.997+0000 7fdbcf7fe700 1 --2- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdbd0036970 0x7fdbd0038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.997+0000 7fdbcf7fe700 1 -- 172.21.15.19:0/3062920567 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fdbd8049b20 con 0x7fdbe8102ae0 2024-03-16T08:54:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.997+0000 7fdbe67fc700 1 --2- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdbd0036970 0x7fdbd0038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:00.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:53:59.998+0000 7fdbe67fc700 1 --2- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdbd0036970 0x7fdbd0038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009990 tx=0x7fdbdc006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.002+0000 7fdbcf7fe700 1 -- 172.21.15.19:0/3062920567 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fdbd80098a0 con 0x7fdbe8102ae0 2024-03-16T08:54:00.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:00.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:00.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:00.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:00.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:54:00.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T08:54:00.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:54:00.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:00.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:53:59 smithi007 ceph-mon[29470]: Deploying daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T08:54:00.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.276+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdbe805f5f0 con 0x7fdbe8102ae0 2024-03-16T08:54:00.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.277+0000 7fdbcf7fe700 1 -- 172.21.15.19:0/3062920567 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7fdbd8011370 con 0x7fdbe8102ae0 2024-03-16T08:54:00.279 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:54:00.279 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:54:00.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdbd0036970 msgr2=0x7fdbd0038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:00.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 --2- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdbd0036970 0x7fdbd0038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009990 tx=0x7fdbdc006e40 comp rx=0 tx=0).stop 2024-03-16T08:54:00.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 msgr2=0x7fdbe8113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:00.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 --2- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 0x7fdbe8113160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800b820 tx=0x7fdbd8009e30 comp rx=0 tx=0).stop 2024-03-16T08:54:00.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 shutdown_connections 2024-03-16T08:54:00.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 --2- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdbd0036970 0x7fdbd0038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:00.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 --2- 172.21.15.19:0/3062920567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8102ae0 0x7fdbe8113160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:00.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 >> 172.21.15.19:0/3062920567 conn(0x7fdbe80f85e0 msgr2=0x7fdbe80f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:00.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 shutdown_connections 2024-03-16T08:54:00.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:00.279+0000 7fdbed543700 1 -- 172.21.15.19:0/3062920567 wait complete. 2024-03-16T08:54:00.282 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:54:01.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:00 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3062920567' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:01.883 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:54:01.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:54:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:54:02.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:02 smithi007 ceph-mon[29470]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:02.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:02.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:02.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:02.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:02.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:02.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:54:02.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:03.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:03 smithi007 ceph-mon[29470]: Deploying daemon mon.smithi019 on smithi019 2024-03-16T08:54:04.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.409+0000 7f008a5cb700 1 -- 172.21.15.19:0/1394013947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 msgr2=0x7f0084102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:04.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.409+0000 7f008a5cb700 1 --2- 172.21.15.19:0/1394013947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 0x7f0084102fa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f00740048c0 tx=0x7f0074009e30 comp rx=0 tx=0).stop 2024-03-16T08:54:04.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.409+0000 7f008a5cb700 1 -- 172.21.15.19:0/1394013947 shutdown_connections 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.409+0000 7f008a5cb700 1 --2- 172.21.15.19:0/1394013947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 0x7f0084102fa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.409+0000 7f008a5cb700 1 -- 172.21.15.19:0/1394013947 >> 172.21.15.19:0/1394013947 conn(0x7f00840f85e0 msgr2=0x7f00840fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.409+0000 7f008a5cb700 1 -- 172.21.15.19:0/1394013947 shutdown_connections 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.409+0000 7f008a5cb700 1 -- 172.21.15.19:0/1394013947 wait complete. 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.410+0000 7f008a5cb700 1 Processor -- start 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.410+0000 7f008a5cb700 1 -- start start 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.410+0000 7f008a5cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 0x7f0084113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.410+0000 7f008a5cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0084113790 con 0x7f0084102bd0 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.410+0000 7f0083fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 0x7f0084113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.410+0000 7f0083fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 0x7f0084113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.19:34320/0 (socket says 172.21.15.19:34320) 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.410+0000 7f0083fff700 1 -- 172.21.15.19:0/3794296682 learned_addr learned my addr 172.21.15.19:0/3794296682 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.411+0000 7f0083fff700 1 -- 172.21.15.19:0/3794296682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074005040 con 0x7f0084102bd0 2024-03-16T08:54:04.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.411+0000 7f0083fff700 1 --2- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 0x7f0084113250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f007400b820 tx=0x7f0074009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.411+0000 7f0080ff9700 1 -- 172.21.15.19:0/3794296682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00740093c0 con 0x7f0084102bd0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.411+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0084113990 con 0x7f0084102bd0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.411+0000 7f0080ff9700 1 -- 172.21.15.19:0/3794296682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0074013420 con 0x7f0084102bd0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.411+0000 7f0080ff9700 1 -- 172.21.15.19:0/3794296682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f007401c5b0 con 0x7f0084102bd0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.411+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0084113e80 con 0x7f0084102bd0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.412+0000 7f0080ff9700 1 -- 172.21.15.19:0/3794296682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 43479+0+0 (secure 0 0 0) 0x7f007401c7f0 con 0x7f0084102bd0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.412+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008405f5f0 con 0x7f0084102bd0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.412+0000 7f0080ff9700 1 --2- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f006c036970 0x7f006c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.412+0000 7f0080ff9700 1 -- 172.21.15.19:0/3794296682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0074049ab0 con 0x7f0084102bd0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.412+0000 7f00837fe700 1 --2- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f006c036970 0x7f006c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:04.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.413+0000 7f00837fe700 1 --2- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f006c036970 0x7f006c038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0078009990 tx=0x7f0078006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:04.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.417+0000 7f0080ff9700 1 -- 172.21.15.19:0/3794296682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f0074023300 con 0x7f0084102bd0 2024-03-16T08:54:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.691+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f008410e8a0 con 0x7f0084102bd0 2024-03-16T08:54:04.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.692+0000 7f0080ff9700 1 -- 172.21.15.19:0/3794296682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f0074011b60 con 0x7f0084102bd0 2024-03-16T08:54:04.695 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:54:04.695 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:51:33.787188Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T08:54:04.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f006c036970 msgr2=0x7f006c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:04.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 --2- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f006c036970 0x7f006c038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0078009990 tx=0x7f0078006e40 comp rx=0 tx=0).stop 2024-03-16T08:54:04.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 msgr2=0x7f0084113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:04.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 --2- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 0x7f0084113250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f007400b820 tx=0x7f0074009e30 comp rx=0 tx=0).stop 2024-03-16T08:54:04.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 shutdown_connections 2024-03-16T08:54:04.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 --2- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f006c036970 0x7f006c038e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:04.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 --2- 172.21.15.19:0/3794296682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0084102bd0 0x7f0084113250 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 >> 172.21.15.19:0/3794296682 conn(0x7f00840f85e0 msgr2=0x7f00840f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 shutdown_connections 2024-03-16T08:54:04.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:04.694+0000 7f008a5cb700 1 -- 172.21.15.19:0/3794296682 wait complete. 2024-03-16T08:54:04.699 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 1 2024-03-16T08:54:04.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:04 smithi007 ceph-mon[29470]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:05 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3794296682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:06.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:06 smithi007 ceph-mon[29470]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:06.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:06 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:06.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 podman[36384]: 2024-03-16 08:54:06.767995499 +0000 UTC m=+0.123453912 container create bfd7571d5d35dd4f94bf7366ec5c9fb2c78451736bb325bb68cb52449e011d2b (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-1df2724, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-16T08:54:07.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 podman[36384]: 2024-03-16 08:54:06.962159608 +0000 UTC m=+0.317618037 container init bfd7571d5d35dd4f94bf7366ec5c9fb2c78451736bb325bb68cb52449e011d2b (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=quincy-1df2724, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-16T08:54:07.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 podman[36384]: 2024-03-16 08:54:06.965256751 +0000 UTC m=+0.320715164 container start bfd7571d5d35dd4f94bf7366ec5c9fb2c78451736bb325bb68cb52449e011d2b (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, RELEASE=quincy-1df2724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T08:54:07.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T08:54:07.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable), process ceph-mon, pid 2 2024-03-16T08:54:07.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: pidfile_write: ignore empty --pid-file 2024-03-16T08:54:07.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: load: jerasure load: lrc 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: RocksDB version: 6.15.5 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Compile date Mar 15 2024 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: DB SUMMARY 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: DB Session ID: UWL5KMRA5GRGZXCJ2PQK 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: CURRENT file: CURRENT 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi019/store.db dir, Total Num: 0, files: 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi019/store.db: 000004.log size: 511 ; 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.error_if_exists: 0 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.create_if_missing: 0 2024-03-16T08:54:07.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.paranoid_checks: 1 2024-03-16T08:54:07.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T08:54:07.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.env: 0x561101bbaa80 2024-03-16T08:54:07.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.fs: Posix File System 2024-03-16T08:54:07.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.info_log: 0x561103431c00 2024-03-16T08:54:07.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T08:54:07.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.statistics: (nil) 2024-03-16T08:54:07.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.use_fsync: 0 2024-03-16T08:54:07.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_log_file_size: 0 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.allow_fallocate: 1 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.use_direct_reads: 0 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.db_log_dir: 2024-03-16T08:54:07.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi019/store.db 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.write_buffer_manager: 0x5611041a6ab0 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T08:54:07.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.unordered_write: 0 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.row_cache: None 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.wal_filter: None 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T08:54:07.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.preserve_deletes: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.two_write_queues: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.atomic_flush: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.log_readahead_size: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T08:54:07.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_background_jobs: 2 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_background_compactions: -1 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_subcompactions: 1 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_open_files: -1 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T08:54:07.221 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_background_flushes: -1 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Compression algorithms supported: 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: kZSTD supported: 0 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: kXpressCompression supported: 0 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: kZlibCompression supported: 1 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: kSnappyCompression supported: 1 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: kLZ4Compression supported: 1 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: kBZip2Compression supported: 0 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi019/store.db/MANIFEST-000003 2024-03-16T08:54:07.222 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.merge_operator: 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_filter: None 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561103486f58) 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-03-16T08:54:07.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_type: 0 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_block_index_type: 0 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_shortening: 1 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: hash_index_allow_collision: 1 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: checksum: 1 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: no_block_cache: 0 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache: 0x5611034b9090 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_options: 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: capacity : 536870912 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: num_shard_bits : 4 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: strict_capacity_limit : 0 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-03-16T08:54:07.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_compressed: (nil) 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: persistent_cache: (nil) 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_size: 4096 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_size_deviation: 10 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_restart_interval: 16 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_block_restart_interval: 1 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: metadata_block_size: 4096 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: partition_filters: 0 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: use_delta_encoding: 1 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: whole_key_filtering: 1 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: verify_compression: 0 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: format_version: 4 2024-03-16T08:54:07.225 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: enable_index_compression: 1 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_align: 0 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compression: NoCompression 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.num_levels: 7 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T08:54:07.226 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T08:54:07.227 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T08:54:07.228 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.arena_block_size: 4194304 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T08:54:07.229 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.table_properties_collectors: 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.inplace_update_support: 0 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.bloom_locality: 0 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.max_successive_merges: 0 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T08:54:07.230 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.ttl: 2592000 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.enable_blob_files: false 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.min_blob_size: 0 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi019/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579246991559, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-03-16T08:54:07.231 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579246991899, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1710579246, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "19cd7237-d486-4995-b65b-43ae3fe1c1ba", "db_session_id": "UWL5KMRA5GRGZXCJ2PQK"}} 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579246992312, "job": 1, "event": "recovery_finished"} 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi019/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x56110344d180 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: DB pointer 0x561103522000 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** DB Stats ** 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T08:54:07.232 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-16T08:54:07.233 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:54:07.234 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T08:54:07.235 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-03-16T08:54:07.236 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T08:54:07.237 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: mon.smithi019 does not exist in monmap, will attempt to join an existing cluster 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: using public_addr v2:172.21.15.19:0/0 -> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: starting mon.smithi019 rank -1 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi019 fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:06 smithi019 ceph-mon[36407]: mon.smithi019@-1(???) e0 preinit fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).mds e1 new map 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).mds e1 print_map 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: e1 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: legacy client fscid: -1 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: No filesystems configured 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-16T08:54:07.238 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.239 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying daemon node-exporter.smithi007 on smithi007 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying cephadm binary to smithi019 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Added host smithi019 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying daemon alertmanager.smithi007 on smithi007 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:54:07.240 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3952343520' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3952343520' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi007:172.21.15.7=smithi007;smithi019:172.21.15.19=smithi019", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Saving service mon spec with placement smithi007:172.21.15.7=smithi007;smithi019:172.21.15.19=smithi019;count:2 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.241 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying daemon grafana.smithi007 on smithi007 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3945006792' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3734979584' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.242 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/2453944803' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3761818901' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying daemon prometheus.smithi007 on smithi007 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.243 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3328785714' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3824863085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/927125757' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14160 172.21.15.7:0/1194604587' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-16T08:54:07.244 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mgrmap e14: smithi007.uamwtp(active, since 58s) 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/2094589914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3353992565' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Active manager daemon smithi007.uamwtp restarted 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Activating manager daemon smithi007.uamwtp 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: osdmap e5: 0 total, 0 up, 0 in 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mgrmap e15: smithi007.uamwtp(active, starting, since 0.00314663s) 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Manager daemon smithi007.uamwtp is now available 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.245 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mgrmap e16: smithi007.uamwtp(active, since 1.00579s) 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/2963614822' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: [16/Mar/2024:08:53:42] ENGINE Bus STARTING 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: [16/Mar/2024:08:53:42] ENGINE Serving on https://172.21.15.7:7150 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: [16/Mar/2024:08:53:42] ENGINE Bus STARTED 2024-03-16T08:54:07.246 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mgrmap e17: smithi007.uamwtp(active, since 2s) 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/423960790' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:07.247 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:54:07.248 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying daemon crash.smithi019 on smithi019 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3251877432' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying daemon node-exporter.smithi019 on smithi019 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/4172644370' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.249 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3062920567' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.250 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: Deploying daemon mon.smithi019 on smithi019 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3794296682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 ceph-mon[36407]: mon.smithi019@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 bash[36384]: bfd7571d5d35dd4f94bf7366ec5c9fb2c78451736bb325bb68cb52449e011d2b 2024-03-16T08:54:07.251 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:07 smithi019 systemd[1]: Started Ceph mon.smithi019 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T08:54:07.585 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T08:54:07.586 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T08:54:07.844 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi019/config 2024-03-16T08:54:12.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.393+0000 7fdfed84b700 1 -- 172.21.15.19:0/2486534545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 msgr2=0x7fdfd0005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.393+0000 7fdfed84b700 1 --2- 172.21.15.19:0/2486534545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 0x7fdfd0005a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80fb9b0 tx=0x7fdfdc00b040 comp rx=0 tx=0).stop 2024-03-16T08:54:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.394+0000 7fdfed84b700 1 -- 172.21.15.19:0/2486534545 shutdown_connections 2024-03-16T08:54:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.394+0000 7fdfed84b700 1 --2- 172.21.15.19:0/2486534545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 0x7fdfd0005a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.394+0000 7fdfed84b700 1 -- 172.21.15.19:0/2486534545 >> 172.21.15.19:0/2486534545 conn(0x7fdfe80f75e0 msgr2=0x7fdfe80f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.394+0000 7fdfed84b700 1 -- 172.21.15.19:0/2486534545 shutdown_connections 2024-03-16T08:54:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.394+0000 7fdfed84b700 1 -- 172.21.15.19:0/2486534545 wait complete. 2024-03-16T08:54:12.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.394+0000 7fdfed84b700 1 Processor -- start 2024-03-16T08:54:12.398 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:54:12.398 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: mon.smithi007 calling monitor election 2024-03-16T08:54:12.398 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.398 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: mon.smithi019 calling monitor election 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/4061216174' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: mon.smithi007 is new leader, mons smithi007,smithi019 in quorum (ranks 0,1) 2024-03-16T08:54:12.399 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: monmap e2: 2 mons at {smithi007=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: fsmap 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: osdmap e5: 0 total, 0 up, 0 in 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: mgrmap e17: smithi007.uamwtp(active, since 30s) 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/4061216174' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: overall HEALTH_OK 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/4061216174' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/4061216174' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.400 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:12 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfed84b700 1 -- start start 2024-03-16T08:54:12.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfed84b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 0x7fdfe80fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:12.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfed84b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 0x7fdfe80fe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfed84b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80ff4f0 con 0x7fdfd0003580 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfed84b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80ff660 con 0x7fdfe80fdbd0 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 0x7fdfe80fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe67fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 0x7fdfe80fe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60726/0 (socket says 172.21.15.19:60726) 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe67fc700 1 -- 172.21.15.19:0/1625622321 learned_addr learned my addr 172.21.15.19:0/1625622321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe6ffd700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 0x7fdfe80fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe67fc700 1 -- 172.21.15.19:0/1625622321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 msgr2=0x7fdfe80fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe67fc700 1 -- 172.21.15.19:0/1625622321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 msgr2=0x7fdfe80fe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe67fc700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 0x7fdfe80fe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe67fc700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 0x7fdfe80fe040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe6ffd700 1 -- 172.21.15.19:0/1625622321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 msgr2=0x7fdfe80fe040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe6ffd700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 0x7fdfe80fe040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:12.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.395+0000 7fdfe6ffd700 1 -- 172.21.15.19:0/1625622321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc005040 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.396+0000 7fdfe6ffd700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 0x7fdfe80fd690 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009ce0 tx=0x7fdfdc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.396+0000 7fdfcf7fe700 1 -- 172.21.15.19:0/1625622321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfdc01b070 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.396+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe811dd80 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.396+0000 7fdfcf7fe700 1 -- 172.21.15.19:0/1625622321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfdc016d10 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.396+0000 7fdfcf7fe700 1 -- 172.21.15.19:0/1625622321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfdc01f670 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.396+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe811e1d0 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.397+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe80fd1c0 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.397+0000 7fdfcf7fe700 1 -- 172.21.15.19:0/1625622321 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7fdfdc020440 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.398+0000 7fdfcf7fe700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdfd4069320 0x7fdfd406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.398+0000 7fdfcf7fe700 1 -- 172.21.15.19:0/1625622321 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fdfdc087fe0 con 0x7fdfd0003580 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.398+0000 7fdfe67fc700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdfd4069320 0x7fdfd406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:12.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.398+0000 7fdfe67fc700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdfd4069320 0x7fdfd406b7d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8006fd0 tx=0x7fdfd8008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:12.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.403+0000 7fdfcf7fe700 1 -- 172.21.15.19:0/1625622321 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fdfdc0552f0 con 0x7fdfd0003580 2024-03-16T08:54:12.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:54:12.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: mon.smithi007 calling monitor election 2024-03-16T08:54:12.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: mon.smithi019 calling monitor election 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/4061216174' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: mon.smithi007 is new leader, mons smithi007,smithi019 in quorum (ranks 0,1) 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: monmap e2: 2 mons at {smithi007=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],smithi019=[v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: fsmap 2024-03-16T08:54:12.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: osdmap e5: 0 total, 0 up, 0 in 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: mgrmap e17: smithi007.uamwtp(active, since 30s) 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/4061216174' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: overall HEALTH_OK 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/4061216174' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/4061216174' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:12 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:12.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.709+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdfe80fed80 con 0x7fdfd0003580 2024-03-16T08:54:12.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.710+0000 7fdfcf7fe700 1 -- 172.21.15.19:0/1625622321 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1009 (secure 0 0 0) 0x7fdfdc058910 con 0x7fdfd0003580 2024-03-16T08:54:12.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-16T08:54:12.712 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","modified":"2024-03-16T08:54:07.017918Z","created":"2024-03-16T08:51:33.787188Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi007","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi019","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T08:54:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdfd4069320 msgr2=0x7fdfd406b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdfd4069320 0x7fdfd406b7d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8006fd0 tx=0x7fdfd8008040 comp rx=0 tx=0).stop 2024-03-16T08:54:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 msgr2=0x7fdfe80fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 0x7fdfe80fd690 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009ce0 tx=0x7fdfdc00b040 comp rx=0 tx=0).stop 2024-03-16T08:54:12.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 shutdown_connections 2024-03-16T08:54:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fdfd4069320 0x7fdfd406b7d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfd0003580 0x7fdfe80fd690 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 --2- 172.21.15.19:0/1625622321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe80fdbd0 0x7fdfe80fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 >> 172.21.15.19:0/1625622321 conn(0x7fdfe80f75e0 msgr2=0x7fdfe80663d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 shutdown_connections 2024-03-16T08:54:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:54:12.712+0000 7fdfed84b700 1 -- 172.21.15.19:0/1625622321 wait complete. 2024-03-16T08:54:12.715 INFO:teuthology.orchestra.run.smithi019.stderr:dumped monmap epoch 2 2024-03-16T08:54:13.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:13 smithi019 ceph-mon[36407]: mgrmap e18: smithi007.uamwtp(active, since 30s), standbys: smithi019.iuqxhi 2024-03-16T08:54:13.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:13 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.iuqxhi", "id": "smithi019.iuqxhi"}]: dispatch 2024-03-16T08:54:13.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:13 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:54:13.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:13 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/1625622321' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:13.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:13 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:13.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:13 smithi007 ceph-mon[29470]: mgrmap e18: smithi007.uamwtp(active, since 30s), standbys: smithi019.iuqxhi 2024-03-16T08:54:13.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:13 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.iuqxhi", "id": "smithi019.iuqxhi"}]: dispatch 2024-03-16T08:54:13.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:13 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:54:13.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:13 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/1625622321' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T08:54:13.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:13 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:54:13.836 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-16T08:54:13.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-16T08:54:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:54:14.399 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:14 smithi007 ceph-mon[29470]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:14.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:14 smithi019 ceph-mon[36407]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 -- 172.21.15.7:0/1057874218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218102ae0 msgr2=0x7ff218102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 --2- 172.21.15.7:0/1057874218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218102ae0 0x7ff218102eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2100048c0 tx=0x7ff210009e30 comp rx=0 tx=0).stop 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 -- 172.21.15.7:0/1057874218 shutdown_connections 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 --2- 172.21.15.7:0/1057874218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218102ae0 0x7ff218102eb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 -- 172.21.15.7:0/1057874218 >> 172.21.15.7:0/1057874218 conn(0x7ff2180f85e0 msgr2=0x7ff2180fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 -- 172.21.15.7:0/1057874218 shutdown_connections 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 -- 172.21.15.7:0/1057874218 wait complete. 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 Processor -- start 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.663+0000 7ff21cff0700 1 -- start start 2024-03-16T08:54:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff21cff0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff218102ae0 0x7ff2180702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff21cff0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218071f60 0x7ff218070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff21cff0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218070ee0 con 0x7ff218071f60 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff21cff0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218071020 con 0x7ff218102ae0 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff215d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218071f60 0x7ff218070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff215d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218071f60 0x7ff218070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56624/0 (socket says 172.21.15.7:56624) 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff215d9b700 1 -- 172.21.15.7:0/3913785030 learned_addr learned my addr 172.21.15.7:0/3913785030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff21659c700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff218102ae0 0x7ff2180702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff215d9b700 1 -- 172.21.15.7:0/3913785030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff218102ae0 msgr2=0x7ff2180702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff215d9b700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff218102ae0 0x7ff2180702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff215d9b700 1 -- 172.21.15.7:0/3913785030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff210005040 con 0x7ff218071f60 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff21659c700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff218102ae0 0x7ff2180702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff215d9b700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218071f60 0x7ff218070800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b790 tx=0x7ff200009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff20effd700 1 -- 172.21.15.7:0/3913785030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff200010870 con 0x7ff218071f60 2024-03-16T08:54:15.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21811dd40 con 0x7ff218071f60 2024-03-16T08:54:15.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff20effd700 1 -- 172.21.15.7:0/3913785030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff20000f3e0 con 0x7ff218071f60 2024-03-16T08:54:15.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff20effd700 1 -- 172.21.15.7:0/3913785030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff200010ea0 con 0x7ff218071f60 2024-03-16T08:54:15.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.664+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21811e250 con 0x7ff218071f60 2024-03-16T08:54:15.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.665+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff218101d50 con 0x7ff218071f60 2024-03-16T08:54:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.666+0000 7ff20effd700 1 -- 172.21.15.7:0/3913785030 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7ff20000f550 con 0x7ff218071f60 2024-03-16T08:54:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.666+0000 7ff20effd700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7ff204069360 0x7ff20406b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.666+0000 7ff20effd700 1 -- 172.21.15.7:0/3913785030 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff2000866d0 con 0x7ff218071f60 2024-03-16T08:54:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.666+0000 7ff21659c700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7ff204069360 0x7ff20406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.667+0000 7ff21659c700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7ff204069360 0x7ff20406b810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff210007f50 tx=0x7ff210009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.671+0000 7ff20effd700 1 -- 172.21.15.7:0/3913785030 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7ff2000574f0 con 0x7ff218071f60 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:54:15.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:54:15.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.865+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff2180fe1e0 con 0x7ff218071f60 2024-03-16T08:54:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.866+0000 7ff20effd700 1 -- 172.21.15.7:0/3913785030 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+221 (secure 0 0 0) 0x7ff2000140f0 con 0x7ff218071f60 2024-03-16T08:54:15.868 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:54:15.868 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-03-16T08:54:15.868 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T08:54:15.868 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2024-03-16T08:54:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7ff204069360 msgr2=0x7ff20406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7ff204069360 0x7ff20406b810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff210007f50 tx=0x7ff210009e30 comp rx=0 tx=0).stop 2024-03-16T08:54:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218071f60 msgr2=0x7ff218070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218071f60 0x7ff218070800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff20000b790 tx=0x7ff200009250 comp rx=0 tx=0).stop 2024-03-16T08:54:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 shutdown_connections 2024-03-16T08:54:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7ff204069360 0x7ff20406b810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff218102ae0 0x7ff2180702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 --2- 172.21.15.7:0/3913785030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff218071f60 0x7ff218070800 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 >> 172.21.15.7:0/3913785030 conn(0x7ff2180f85e0 msgr2=0x7ff2180f9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 shutdown_connections 2024-03-16T08:54:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:15.868+0000 7ff21cff0700 1 -- 172.21.15.7:0/3913785030 wait complete. 2024-03-16T08:54:15.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:54:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:16.496 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-16T08:54:16.496 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:54:16.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T08:54:16.526 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:54:16.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:54:16.593 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:54:16.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T08:54:16.630 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:54:16.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T08:54:16.700 INFO:tasks.cephadm:Deploying OSDs... 2024-03-16T08:54:16.700 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:54:16.700 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T08:54:16.715 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T08:54:16.715 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-03-16T08:54:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:16 smithi007 ceph-mon[29470]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:54:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:16 smithi007 ceph-mon[29470]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:54:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:16 smithi007 ceph-mon[29470]: Reconfiguring mon.smithi007 (unknown last config time)... 2024-03-16T08:54:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:16 smithi007 ceph-mon[29470]: Reconfiguring daemon mon.smithi007 on smithi007 2024-03-16T08:54:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:16 smithi007 ceph-mon[29470]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:16 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3913785030' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:16 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T08:54:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:54:16.762 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-16T08:54:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:54:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:54:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-16 08:53:33.006189297 +0000 2024-03-16T08:54:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-16 08:47:56.953471477 +0000 2024-03-16T08:54:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-16 08:47:56.953471477 +0000 2024-03-16T08:54:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-16 08:47:56.953471477 +0000 2024-03-16T08:54:16.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T08:54:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-16T08:54:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-16T08:54:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.9584e-05 s, 5.1 MB/s 2024-03-16T08:54:16.830 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T08:54:16.877 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:16 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:54:16.877 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:16 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T08:54:16.877 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:16 smithi019 ceph-mon[36407]: Reconfiguring mon.smithi007 (unknown last config time)... 2024-03-16T08:54:16.877 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:16 smithi019 ceph-mon[36407]: Reconfiguring daemon mon.smithi007 on smithi007 2024-03-16T08:54:16.877 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:16 smithi019 ceph-mon[36407]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:16.877 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:16 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3913785030' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:16.877 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:16 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:16.887 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-03-16T08:54:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T08:54:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:54:16.941 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-16T08:54:16.941 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:54:16.941 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:54:16.941 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-16 08:53:33.006189297 +0000 2024-03-16T08:54:16.942 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-16 08:47:57.283466780 +0000 2024-03-16T08:54:16.942 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-16 08:47:57.283466780 +0000 2024-03-16T08:54:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-16 08:47:57.283466780 +0000 2024-03-16T08:54:16.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T08:54:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-16T08:54:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-16T08:54:17.007 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000102872 s, 5.0 MB/s 2024-03-16T08:54:17.008 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T08:54:17.063 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-03-16T08:54:17.118 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T08:54:17.118 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:54:17.118 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-16T08:54:17.118 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:54:17.118 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:54:17.118 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-16 08:53:33.007189283 +0000 2024-03-16T08:54:17.118 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-16 08:47:57.610462125 +0000 2024-03-16T08:54:17.119 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-16 08:47:57.610462125 +0000 2024-03-16T08:54:17.119 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-16 08:47:57.610462125 +0000 2024-03-16T08:54:17.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T08:54:17.184 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-16T08:54:17.184 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-16T08:54:17.184 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 8.5037e-05 s, 6.0 MB/s 2024-03-16T08:54:17.185 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T08:54:17.240 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-16 08:53:33.007189283 +0000 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-16 08:47:57.955457215 +0000 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-16 08:47:57.955457215 +0000 2024-03-16T08:54:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-16 08:47:57.955457215 +0000 2024-03-16T08:54:17.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T08:54:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-16T08:54:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-16T08:54:17.359 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.3462e-05 s, 5.5 MB/s 2024-03-16T08:54:17.361 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T08:54:17.415 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:54:17.415 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T08:54:17.431 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T08:54:17.431 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-03-16T08:54:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T08:54:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:54:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-16T08:54:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:54:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:54:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-16 08:54:06.340271704 +0000 2024-03-16T08:54:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-16 08:47:56.973496933 +0000 2024-03-16T08:54:17.488 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-16 08:47:56.973496933 +0000 2024-03-16T08:54:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-16 08:47:56.973496933 +0000 2024-03-16T08:54:17.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T08:54:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-16T08:54:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-16T08:54:17.552 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000207542 s, 2.5 MB/s 2024-03-16T08:54:17.553 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T08:54:17.608 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-03-16T08:54:17.663 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T08:54:17.664 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:54:17.664 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-16T08:54:17.664 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:54:17.664 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:54:17.664 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-16 08:54:06.340271704 +0000 2024-03-16T08:54:17.665 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-16 08:47:57.328493757 +0000 2024-03-16T08:54:17.665 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-16 08:47:57.328493757 +0000 2024-03-16T08:54:17.665 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-16 08:47:57.328493757 +0000 2024-03-16T08:54:17.665 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T08:54:17.728 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-16T08:54:17.729 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-16T08:54:17.729 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000192193 s, 2.7 MB/s 2024-03-16T08:54:17.730 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T08:54:17.784 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-03-16T08:54:17.838 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T08:54:17.838 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:54:17.838 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-16T08:54:17.838 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:54:17.838 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:54:17.839 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-16 08:54:06.340271704 +0000 2024-03-16T08:54:17.839 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-16 08:47:57.661490778 +0000 2024-03-16T08:54:17.839 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-16 08:47:57.661490778 +0000 2024-03-16T08:54:17.839 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-16 08:47:57.661490778 +0000 2024-03-16T08:54:17.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T08:54:17.903 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-16T08:54:17.904 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-16T08:54:17.904 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000208356 s, 2.5 MB/s 2024-03-16T08:54:17.905 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T08:54:17.960 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-16 08:54:06.341271695 +0000 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-16 08:47:58.014487620 +0000 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-16 08:47:58.014487620 +0000 2024-03-16T08:54:18.015 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-16 08:47:58.014487620 +0000 2024-03-16T08:54:18.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T08:54:18.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-16T08:54:18.079 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-16T08:54:18.079 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000166595 s, 3.1 MB/s 2024-03-16T08:54:18.080 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T08:54:18.135 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-03-16T08:54:18.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T08:54:18.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:18 smithi007 ceph-mon[29470]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:18.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:18 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:18.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:18 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:18.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:18 smithi007 ceph-mon[29470]: Reconfiguring mgr.smithi007.uamwtp (unknown last config time)... 2024-03-16T08:54:18.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:18 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.uamwtp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:54:18.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:18 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:54:18.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:18 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:18.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:18 smithi007 ceph-mon[29470]: Reconfiguring daemon mgr.smithi007.uamwtp on smithi007 2024-03-16T08:54:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:54:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:18 smithi019 ceph-mon[36407]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:18 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:18 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:18 smithi019 ceph-mon[36407]: Reconfiguring mgr.smithi007.uamwtp (unknown last config time)... 2024-03-16T08:54:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:18 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.uamwtp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:54:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:18 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:54:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:18 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:18 smithi019 ceph-mon[36407]: Reconfiguring daemon mgr.smithi007.uamwtp on smithi007 2024-03-16T08:54:19.857 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:19.858 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:19.858 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:19 smithi007 ceph-mon[29470]: Reconfiguring crash.smithi007 (monmap changed)... 2024-03-16T08:54:19.858 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T08:54:19.858 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:19.858 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:19 smithi007 ceph-mon[29470]: Reconfiguring daemon crash.smithi007 on smithi007 2024-03-16T08:54:19.858 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:19 smithi007 ceph-mon[29470]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:20.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:19 smithi019 ceph-mon[36407]: Reconfiguring crash.smithi007 (monmap changed)... 2024-03-16T08:54:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T08:54:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:19 smithi019 ceph-mon[36407]: Reconfiguring daemon crash.smithi007 on smithi007 2024-03-16T08:54:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:19 smithi019 ceph-mon[36407]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T08:54:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T08:54:20.910 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-16T08:54:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-16T08:54:20.910 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00820171 s, 1.3 GB/s 2024-03-16T08:54:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-16T08:54:21.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-03-16T08:54:22.178 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:54:22.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:22 smithi007 ceph-mon[29470]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:22.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:22 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:22.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:22 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:22.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:22 smithi019 ceph-mon[36407]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:22.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:22 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:22.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:22 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:23.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:23 smithi007 ceph-mon[29470]: Reconfiguring alertmanager.smithi007 (dependencies changed)... 2024-03-16T08:54:23.934 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:23 smithi007 ceph-mon[29470]: Reconfiguring daemon alertmanager.smithi007 on smithi007 2024-03-16T08:54:23.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:23 smithi019 ceph-mon[36407]: Reconfiguring alertmanager.smithi007 (dependencies changed)... 2024-03-16T08:54:23.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:23 smithi019 ceph-mon[36407]: Reconfiguring daemon alertmanager.smithi007 on smithi007 2024-03-16T08:54:24.784 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:24 smithi007 ceph-mon[29470]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:24.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:24 smithi019 ceph-mon[36407]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 -- 172.21.15.7:0/4017206488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fce70 msgr2=0x7f989c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 --2- 172.21.15.7:0/4017206488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fce70 0x7f989c0fd280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9894004940 tx=0x7f9894009e30 comp rx=0 tx=0).stop 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 -- 172.21.15.7:0/4017206488 shutdown_connections 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 --2- 172.21.15.7:0/4017206488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c0fdfe0 0x7f989c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 --2- 172.21.15.7:0/4017206488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fce70 0x7f989c0fd280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 -- 172.21.15.7:0/4017206488 >> 172.21.15.7:0/4017206488 conn(0x7f989c0f8550 msgr2=0x7f989c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 -- 172.21.15.7:0/4017206488 shutdown_connections 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 -- 172.21.15.7:0/4017206488 wait complete. 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 Processor -- start 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.222+0000 7f98a4283700 1 -- start start 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a4283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c0fce70 0x7f989c10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a4283700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fdfe0 0x7f989c10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a4283700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c10e4f0 con 0x7f989c0fdfe0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a4283700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c112e90 con 0x7f989c0fce70 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a181e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fdfe0 0x7f989c10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a181e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fdfe0 0x7f989c10de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53710/0 (socket says 172.21.15.7:53710) 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a181e700 1 -- 172.21.15.7:0/3483119728 learned_addr learned my addr 172.21.15.7:0/3483119728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a201f700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c0fce70 0x7f989c10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a181e700 1 -- 172.21.15.7:0/3483119728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c0fce70 msgr2=0x7f989c10d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a181e700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c0fce70 0x7f989c10d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a181e700 1 -- 172.21.15.7:0/3483119728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894005040 con 0x7f989c0fdfe0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a181e700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fdfe0 0x7f989c10de60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f989000b790 tx=0x7f9890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f989affd700 1 -- 172.21.15.7:0/3483119728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f989000f6f0 con 0x7f989c0fdfe0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f989affd700 1 -- 172.21.15.7:0/3483119728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f989000fd20 con 0x7f989c0fdfe0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c1130f0 con 0x7f989c0fdfe0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f989affd700 1 -- 172.21.15.7:0/3483119728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98900103a0 con 0x7f989c0fdfe0 2024-03-16T08:54:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.223+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c113660 con 0x7f989c0fdfe0 2024-03-16T08:54:26.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.224+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989c04c710 con 0x7f989c0fdfe0 2024-03-16T08:54:26.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.225+0000 7f989affd700 1 -- 172.21.15.7:0/3483119728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7f9890011400 con 0x7f989c0fdfe0 2024-03-16T08:54:26.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.225+0000 7f989affd700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9884069210 0x7f988406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:54:26.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.225+0000 7f989affd700 1 -- 172.21.15.7:0/3483119728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9890085d80 con 0x7f989c0fdfe0 2024-03-16T08:54:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.225+0000 7f98a201f700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9884069210 0x7f988406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:54:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.226+0000 7f98a201f700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9884069210 0x7f988406b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f989400b970 tx=0x7f9894009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:54:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.230+0000 7f989affd700 1 -- 172.21.15.7:0/3483119728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f9890056cc0 con 0x7f989c0fdfe0 2024-03-16T08:54:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:26.443+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f989c1022b0 con 0x7f9884069210 2024-03-16T08:54:26.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:26 smithi007 ceph-mon[29470]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:26.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:26.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:54:26.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:54:26.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:26.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:26 smithi019 ceph-mon[36407]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:54:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:54:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:27.709 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:27 smithi007 ceph-mon[29470]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:54:27.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:27 smithi019 ceph-mon[36407]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:54:28.581 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:28 smithi007 ceph-mon[29470]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:28.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:28 smithi019 ceph-mon[36407]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:30.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:30 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:30.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:30 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:30.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:30 smithi019 ceph-mon[36407]: Reconfiguring grafana.smithi007 (dependencies changed)... 2024-03-16T08:54:30.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:30 smithi019 ceph-mon[36407]: Reconfiguring daemon grafana.smithi007 on smithi007 2024-03-16T08:54:30.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:30 smithi019 ceph-mon[36407]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:30.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:30 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:30.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:30 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:30.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:30 smithi007 ceph-mon[29470]: Reconfiguring grafana.smithi007 (dependencies changed)... 2024-03-16T08:54:30.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:30 smithi007 ceph-mon[29470]: Reconfiguring daemon grafana.smithi007 on smithi007 2024-03-16T08:54:30.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:30 smithi007 ceph-mon[29470]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:31.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:31 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/901787572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8f96044-56c4-4825-8138-629e5352a8cf"}]: dispatch 2024-03-16T08:54:31.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:31 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/901787572' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8f96044-56c4-4825-8138-629e5352a8cf"}]': finished 2024-03-16T08:54:31.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:31 smithi019 ceph-mon[36407]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T08:54:31.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:31 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:31.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:31 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/2674702007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:54:31.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:31 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/901787572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8f96044-56c4-4825-8138-629e5352a8cf"}]: dispatch 2024-03-16T08:54:31.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:31 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/901787572' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8f96044-56c4-4825-8138-629e5352a8cf"}]': finished 2024-03-16T08:54:31.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:31 smithi007 ceph-mon[29470]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T08:54:31.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:31 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:31.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:31 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/2674702007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:54:32.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:32 smithi019 ceph-mon[36407]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:32.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:32 smithi007 ceph-mon[29470]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:34.835 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:34 smithi007 ceph-mon[29470]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:34.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:34 smithi019 ceph-mon[36407]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:36.847 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:36 smithi007 ceph-mon[29470]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:36.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:36 smithi019 ceph-mon[36407]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:38.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:38 smithi007 ceph-mon[29470]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:38.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:38 smithi019 ceph-mon[36407]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:40.114 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:40.114 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:40.114 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:40 smithi007 ceph-mon[29470]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-03-16T08:54:40.114 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:40 smithi007 ceph-mon[29470]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-03-16T08:54:40.114 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:40 smithi007 ceph-mon[29470]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:40.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:40 smithi019 ceph-mon[36407]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-03-16T08:54:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:40 smithi019 ceph-mon[36407]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-03-16T08:54:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:40 smithi019 ceph-mon[36407]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T08:54:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:41 smithi007 ceph-mon[29470]: Deploying daemon osd.0 on smithi007 2024-03-16T08:54:41.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T08:54:41.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:41.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:41 smithi019 ceph-mon[36407]: Deploying daemon osd.0 on smithi007 2024-03-16T08:54:42.371 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:42 smithi007 ceph-mon[29470]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:42.371 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:42.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:42 smithi019 ceph-mon[36407]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:42.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:44.631 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:44 smithi007 ceph-mon[29470]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:44.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:44 smithi019 ceph-mon[36407]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:45.895 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:45.896 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:45.896 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:45 smithi007 ceph-mon[29470]: Reconfiguring crash.smithi019 (monmap changed)... 2024-03-16T08:54:45.896 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T08:54:45.896 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:45.896 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:45 smithi007 ceph-mon[29470]: Reconfiguring daemon crash.smithi019 on smithi019 2024-03-16T08:54:45.896 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:45 smithi007 ceph-mon[29470]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:46.054 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:46.054 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:46.054 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:45 smithi019 ceph-mon[36407]: Reconfiguring crash.smithi019 (monmap changed)... 2024-03-16T08:54:46.054 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T08:54:46.054 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:46.055 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:45 smithi019 ceph-mon[36407]: Reconfiguring daemon crash.smithi019 on smithi019 2024-03-16T08:54:46.055 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:45 smithi019 ceph-mon[36407]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:47.186 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:47.186 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:47.187 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:47 smithi007 ceph-mon[29470]: Reconfiguring mgr.smithi019.iuqxhi (monmap changed)... 2024-03-16T08:54:47.187 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:54:47.187 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:54:47.187 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:47 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:47.187 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:47 smithi007 ceph-mon[29470]: Reconfiguring daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T08:54:47.401 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:47 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:47.401 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:47 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:47.402 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:47 smithi019 ceph-mon[36407]: Reconfiguring mgr.smithi019.iuqxhi (monmap changed)... 2024-03-16T08:54:47.402 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:47 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T08:54:47.402 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:47 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T08:54:47.402 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:47 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:47.402 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:47 smithi019 ceph-mon[36407]: Reconfiguring daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T08:54:48.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:48 smithi019 ceph-mon[36407]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:48.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:48 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:48.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:48 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:48.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:48 smithi019 ceph-mon[36407]: Reconfiguring mon.smithi019 (monmap changed)... 2024-03-16T08:54:48.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:48 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:54:48.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:48 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:54:48.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:48 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:48.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:48 smithi019 ceph-mon[36407]: Reconfiguring daemon mon.smithi019 on smithi019 2024-03-16T08:54:48.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:48 smithi007 ceph-mon[29470]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:48.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:48.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:48.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:48 smithi007 ceph-mon[29470]: Reconfiguring mon.smithi019 (monmap changed)... 2024-03-16T08:54:48.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T08:54:48.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T08:54:48.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:54:48.727 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:48 smithi007 ceph-mon[29470]: Reconfiguring daemon mon.smithi019 on smithi019 2024-03-16T08:54:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T08:54:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T08:54:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi007.front.sepia.ceph.com:9093"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi007.front.sepia.ceph.com:9093"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi007.front.sepia.ceph.com:3000"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi007.front.sepia.ceph.com:3000"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi007.front.sepia.ceph.com:9095"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi007.front.sepia.ceph.com:9095"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:54:49.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:49 smithi007 ceph-mon[29470]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:50.145 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:50.145 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:50.145 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T08:54:50.145 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T08:54:50.145 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi007.front.sepia.ceph.com:9093"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi007.front.sepia.ceph.com:9093"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi007.front.sepia.ceph.com:3000"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi007.front.sepia.ceph.com:3000"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi007.front.sepia.ceph.com:9095"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi007.front.sepia.ceph.com:9095"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:54:50.146 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:49 smithi019 ceph-mon[36407]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:51.045 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:50 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:51.045 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:50 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:51.045 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:50 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:51.045 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:50 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:51.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:50 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:51.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:50 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:51.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:50 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:51.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:50 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:52.813 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:52 smithi007 ceph-mon[29470]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:52.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:52 smithi019 ceph-mon[36407]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.923+0000 7f989affd700 1 -- 172.21.15.7:0/3483119728 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f989c1022b0 con 0x7f9884069210 2024-03-16T08:54:53.925 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-03-16T08:54:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9884069210 msgr2=0x7f988406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9884069210 0x7f988406b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f989400b970 tx=0x7f9894009e30 comp rx=0 tx=0).stop 2024-03-16T08:54:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fdfe0 msgr2=0x7f989c10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:54:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fdfe0 0x7f989c10de60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f989000b790 tx=0x7f9890009250 comp rx=0 tx=0).stop 2024-03-16T08:54:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 shutdown_connections 2024-03-16T08:54:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9884069210 0x7f988406b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f989c0fce70 0x7f989c10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:53.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 --2- 172.21.15.7:0/3483119728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f989c0fdfe0 0x7f989c10de60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:54:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 >> 172.21.15.7:0/3483119728 conn(0x7f989c0f8550 msgr2=0x7f989c0f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:54:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 shutdown_connections 2024-03-16T08:54:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:54:53.925+0000 7f98a4283700 1 -- 172.21.15.7:0/3483119728 wait complete. 2024-03-16T08:54:54.739 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.0.service 2024-03-16T08:54:54.741 INFO:tasks.cephadm:Deploying osd.1 on smithi007 with /dev/vg_nvme/lv_3... 2024-03-16T08:54:54.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T08:54:54.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:54 smithi007 ceph-mon[29470]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:54.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:54 smithi007 ceph-mon[29470]: from='osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T08:54:54.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:54 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:54.806 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:54 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:54.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:54 smithi019 ceph-mon[36407]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:54.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:54 smithi019 ceph-mon[36407]: from='osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T08:54:54.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:54 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:54.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:54 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:55.139 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:54:55.843 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:55 smithi019 ceph-mon[36407]: from='osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T08:54:55.843 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:55 smithi019 ceph-mon[36407]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T08:54:55.843 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:55 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:55.843 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:55 smithi019 ceph-mon[36407]: from='osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T08:54:55.844 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:55 smithi019 ceph-mon[36407]: from='osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-16T08:54:55.844 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:55 smithi019 ceph-mon[36407]: osdmap e8: 1 total, 0 up, 1 in 2024-03-16T08:54:55.844 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:55 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:55.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-mon[29470]: from='osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T08:54:55.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-mon[29470]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T08:54:55.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:55.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-mon[29470]: from='osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T08:54:55.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-mon[29470]: from='osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-16T08:54:55.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-mon[29470]: osdmap e8: 1 total, 0 up, 1 in 2024-03-16T08:54:55.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:55.965 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[48134]: 2024-03-16T08:54:55.545+0000 7feb13b59700 -1 osd.0 0 waiting for initial osdmap 2024-03-16T08:54:55.965 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:54:55 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[48134]: 2024-03-16T08:54:55.548+0000 7feb0b94e700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:54:56.920 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:56 smithi007 ceph-mon[29470]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:56.920 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:56.920 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:56.920 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:56.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:56 smithi019 ceph-mon[36407]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:56.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:56.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:54:56.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:57 smithi007 ceph-mon[29470]: purged_snaps scrub starts 2024-03-16T08:54:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:57 smithi007 ceph-mon[29470]: purged_snaps scrub ok 2024-03-16T08:54:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:57 smithi007 ceph-mon[29470]: osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] boot 2024-03-16T08:54:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:57 smithi007 ceph-mon[29470]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T08:54:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:57.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:57 smithi019 ceph-mon[36407]: purged_snaps scrub starts 2024-03-16T08:54:57.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:57 smithi019 ceph-mon[36407]: purged_snaps scrub ok 2024-03-16T08:54:57.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:57 smithi019 ceph-mon[36407]: osd.0 [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] boot 2024-03-16T08:54:57.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:57 smithi019 ceph-mon[36407]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T08:54:57.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T08:54:57.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:57.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:54:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T08:54:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T08:54:58.039 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-16T08:54:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-16T08:54:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852024 s, 1.2 GB/s 2024-03-16T08:54:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-16T08:54:58.669 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:58 smithi007 ceph-mon[29470]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:58.669 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:54:58 smithi007 ceph-mon[29470]: osdmap e10: 1 total, 1 up, 1 in 2024-03-16T08:54:58.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:58 smithi019 ceph-mon[36407]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T08:54:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:54:58 smithi019 ceph-mon[36407]: osdmap e10: 1 total, 1 up, 1 in 2024-03-16T08:54:59.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-03-16T08:55:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:55:00.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:00 smithi019 ceph-mon[36407]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:00.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:00 smithi007 ceph-mon[29470]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.354+0000 7fd20fb65700 1 -- 172.21.15.7:0/3797938149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fcee0 msgr2=0x7fd2080fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.354+0000 7fd20fb65700 1 --2- 172.21.15.7:0/3797938149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fcee0 0x7fd2080fd310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd204004940 tx=0x7fd204009e30 comp rx=0 tx=0).stop 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.354+0000 7fd20fb65700 1 -- 172.21.15.7:0/3797938149 shutdown_connections 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.354+0000 7fd20fb65700 1 --2- 172.21.15.7:0/3797938149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2080fe0d0 0x7fd2080fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.354+0000 7fd20fb65700 1 --2- 172.21.15.7:0/3797938149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fcee0 0x7fd2080fd310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.354+0000 7fd20fb65700 1 -- 172.21.15.7:0/3797938149 >> 172.21.15.7:0/3797938149 conn(0x7fd2080f8560 msgr2=0x7fd2080fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.354+0000 7fd20fb65700 1 -- 172.21.15.7:0/3797938149 shutdown_connections 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.354+0000 7fd20fb65700 1 -- 172.21.15.7:0/3797938149 wait complete. 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20fb65700 1 Processor -- start 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20fb65700 1 -- start start 2024-03-16T08:55:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20fb65700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2080fcee0 0x7fd20810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20fb65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fe0d0 0x7fd20810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20fb65700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20810e5a0 con 0x7fd2080fe0d0 2024-03-16T08:55:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20fb65700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20810edf0 con 0x7fd2080fcee0 2024-03-16T08:55:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20d100700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fe0d0 0x7fd20810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20d100700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fe0d0 0x7fd20810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47852/0 (socket says 172.21.15.7:47852) 2024-03-16T08:55:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20d901700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2080fcee0 0x7fd20810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20d100700 1 -- 172.21.15.7:0/945525386 learned_addr learned my addr 172.21.15.7:0/945525386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:55:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20d100700 1 -- 172.21.15.7:0/945525386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2080fcee0 msgr2=0x7fd20810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20d100700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2080fcee0 0x7fd20810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20d100700 1 -- 172.21.15.7:0/945525386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204005040 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.355+0000 7fd20d100700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fe0d0 0x7fd20810dfa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8002a50 tx=0x7fd1f80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.356+0000 7fd1fe7fc700 1 -- 172.21.15.7:0/945525386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1f8007cc0 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.356+0000 7fd1fe7fc700 1 -- 172.21.15.7:0/945525386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1f800f040 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.356+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20810f0d0 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.356+0000 7fd1fe7fc700 1 -- 172.21.15.7:0/945525386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1f800da30 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.356+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20810f5c0 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.357+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20805f5f0 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.357+0000 7fd1fe7fc700 1 -- 172.21.15.7:0/945525386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7fd1f800db90 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.357+0000 7fd1fe7fc700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1f4069210 0x7fd1f406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.358+0000 7fd1fe7fc700 1 -- 172.21.15.7:0/945525386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fd1f8086090 con 0x7fd2080fe0d0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.358+0000 7fd20d901700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1f4069210 0x7fd1f406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.358+0000 7fd20d901700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1f4069210 0x7fd1f406b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd204004890 tx=0x7fd204009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:55:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.364+0000 7fd1fe7fc700 1 -- 172.21.15.7:0/945525386 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fd1f8056c50 con 0x7fd2080fe0d0 2024-03-16T08:55:02.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: Detected new or changed devices on smithi007 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: Adjusting osd_memory_target on smithi007 to 12869M 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:02.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:02.579+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd20810f940 con 0x7fd1f4069210 2024-03-16T08:55:02.717 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: Detected new or changed devices on smithi007 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: Adjusting osd_memory_target on smithi007 to 12869M 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:02.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:03.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:03 smithi007 ceph-mon[29470]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:55:03.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:03 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:55:03.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:03 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:55:03.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:03 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:03 smithi019 ceph-mon[36407]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:55:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:03 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:55:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:03 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:55:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:03 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:04.599 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:04 smithi007 ceph-mon[29470]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:04.599 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:04 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:04.599 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:04 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:04.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:04 smithi019 ceph-mon[36407]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:04.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:04 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:04.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:04 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:06.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:06 smithi019 ceph-mon[36407]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:06.747 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:06 smithi007 ceph-mon[29470]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/178564796' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3fcdd0c-0016-4781-803b-bbca53b4517b"}]: dispatch 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/178564796' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3fcdd0c-0016-4781-803b-bbca53b4517b"}]': finished 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3165294917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:07.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/178564796' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3fcdd0c-0016-4781-803b-bbca53b4517b"}]: dispatch 2024-03-16T08:55:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/178564796' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3fcdd0c-0016-4781-803b-bbca53b4517b"}]': finished 2024-03-16T08:55:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T08:55:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3165294917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:55:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:07.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:07.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:07.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:07 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:08.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:08 smithi019 ceph-mon[36407]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:08.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:08 smithi007 ceph-mon[29470]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:10 smithi007 ceph-mon[29470]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:10.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:10 smithi019 ceph-mon[36407]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:11.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:11 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:55:11.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:11 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:55:12.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:12 smithi007 ceph-mon[29470]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:12.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:12 smithi019 ceph-mon[36407]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:14 smithi007 ceph-mon[29470]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:14 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T08:55:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:14 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:14.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:14 smithi019 ceph-mon[36407]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:14.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:14 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T08:55:14.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:14 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:15 smithi007 ceph-mon[29470]: Deploying daemon osd.1 on smithi007 2024-03-16T08:55:15.841 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:15 smithi019 ceph-mon[36407]: Deploying daemon osd.1 on smithi007 2024-03-16T08:55:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:16 smithi007 ceph-mon[29470]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:16.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:16 smithi019 ceph-mon[36407]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:18.852 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:18 smithi007 ceph-mon[29470]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:18.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:18 smithi019 ceph-mon[36407]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:20.351 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:20.351 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:20.351 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:20.351 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:19 smithi007 ceph-mon[29470]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:20.383 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:20.383 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:20.383 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:20.383 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:19 smithi019 ceph-mon[36407]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:21.987 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:21 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:21.988 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:21 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:21.988 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:21 smithi007 ceph-mon[29470]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:22.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.034+0000 7fd1fe7fc700 1 -- 172.21.15.7:0/945525386 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd20810f940 con 0x7fd1f4069210 2024-03-16T08:55:22.036 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 1 on host 'smithi007' 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.036+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1f4069210 msgr2=0x7fd1f406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.036+0000 7fd20fb65700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1f4069210 0x7fd1f406b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd204004890 tx=0x7fd204009e30 comp rx=0 tx=0).stop 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.036+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fe0d0 msgr2=0x7fd20810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.036+0000 7fd20fb65700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fe0d0 0x7fd20810dfa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8002a50 tx=0x7fd1f80046b0 comp rx=0 tx=0).stop 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.037+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 shutdown_connections 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.037+0000 7fd20fb65700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1f4069210 0x7fd1f406b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.037+0000 7fd20fb65700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2080fcee0 0x7fd20810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.037+0000 7fd20fb65700 1 --2- 172.21.15.7:0/945525386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2080fe0d0 0x7fd20810dfa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.037+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 >> 172.21.15.7:0/945525386 conn(0x7fd2080f8560 msgr2=0x7fd2080f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.037+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 shutdown_connections 2024-03-16T08:55:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:22.037+0000 7fd20fb65700 1 -- 172.21.15.7:0/945525386 wait complete. 2024-03-16T08:55:22.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:21 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:22.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:21 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:22.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:21 smithi019 ceph-mon[36407]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:22.603 DEBUG:teuthology.orchestra.run.smithi007:osd.1> sudo journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.1.service 2024-03-16T08:55:22.605 INFO:tasks.cephadm:Deploying osd.2 on smithi007 with /dev/vg_nvme/lv_2... 2024-03-16T08:55:22.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T08:55:23.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:23 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:23.161 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:23 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:23.161 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:23 smithi007 ceph-mon[29470]: from='osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T08:55:23.161 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:22 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[55550]: 2024-03-16T08:55:22.863+0000 7f2cfbf1d540 -1 osd.1 0 log_to_monitors true 2024-03-16T08:55:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:55:23.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:23 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:23.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:23 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:23.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:23 smithi019 ceph-mon[36407]: from='osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T08:55:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:24 smithi007 ceph-mon[29470]: from='osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T08:55:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:24 smithi007 ceph-mon[29470]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T08:55:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:24 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:24 smithi007 ceph-mon[29470]: from='osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T08:55:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:24 smithi007 ceph-mon[29470]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:24.248 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:24 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[55550]: 2024-03-16T08:55:24.027+0000 7f2cf26ef700 -1 osd.1 0 waiting for initial osdmap 2024-03-16T08:55:24.249 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:24 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[55550]: 2024-03-16T08:55:24.031+0000 7f2cef4ee700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:55:24.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:24 smithi019 ceph-mon[36407]: from='osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T08:55:24.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:24 smithi019 ceph-mon[36407]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T08:55:24.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:24 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:24.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:24 smithi019 ceph-mon[36407]: from='osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T08:55:24.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:24 smithi019 ceph-mon[36407]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:25.280 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:25 smithi007 ceph-mon[29470]: from='osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-16T08:55:25.280 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:25 smithi007 ceph-mon[29470]: osdmap e13: 2 total, 1 up, 2 in 2024-03-16T08:55:25.280 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:25 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:25.280 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:25 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:25 smithi019 ceph-mon[36407]: from='osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-16T08:55:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:25 smithi019 ceph-mon[36407]: osdmap e13: 2 total, 1 up, 2 in 2024-03-16T08:55:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:25 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:25 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T08:55:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T08:55:26.075 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-16T08:55:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-16T08:55:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00981771 s, 1.1 GB/s 2024-03-16T08:55:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-16T08:55:26.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:26 smithi019 ceph-mon[36407]: purged_snaps scrub starts 2024-03-16T08:55:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:26 smithi019 ceph-mon[36407]: purged_snaps scrub ok 2024-03-16T08:55:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:26 smithi019 ceph-mon[36407]: osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] boot 2024-03-16T08:55:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:26 smithi019 ceph-mon[36407]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T08:55:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:26 smithi019 ceph-mon[36407]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:26 smithi007 ceph-mon[29470]: purged_snaps scrub starts 2024-03-16T08:55:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:26 smithi007 ceph-mon[29470]: purged_snaps scrub ok 2024-03-16T08:55:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:26 smithi007 ceph-mon[29470]: osd.1 [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] boot 2024-03-16T08:55:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:26 smithi007 ceph-mon[29470]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T08:55:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T08:55:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:26 smithi007 ceph-mon[29470]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-16T08:55:26.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-03-16T08:55:27.051 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:55:27.304 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:27 smithi007 ceph-mon[29470]: osdmap e15: 2 total, 2 up, 2 in 2024-03-16T08:55:27.304 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:27 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:55:27.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:27 smithi019 ceph-mon[36407]: osdmap e15: 2 total, 2 up, 2 in 2024-03-16T08:55:27.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:27 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:55:28.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:28 smithi007 ceph-mon[29470]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:28.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:28 smithi019 ceph-mon[36407]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:29.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 -- 172.21.15.7:0/4215630601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 msgr2=0x7fb0b00faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 --2- 172.21.15.7:0/4215630601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 0x7fb0b00faee0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0004940 tx=0x7fb0a0009e30 comp rx=0 tx=0).stop 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 -- 172.21.15.7:0/4215630601 shutdown_connections 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 --2- 172.21.15.7:0/4215630601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 0x7fb0b00faee0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 --2- 172.21.15.7:0/4215630601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b00fe450 0x7fb0b00fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 -- 172.21.15.7:0/4215630601 >> 172.21.15.7:0/4215630601 conn(0x7fb0b00f6c10 msgr2=0x7fb0b00f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 -- 172.21.15.7:0/4215630601 shutdown_connections 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 -- 172.21.15.7:0/4215630601 wait complete. 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 Processor -- start 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 -- start start 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b00fe450 0x7fb0b010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 0x7fb0b010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b010e490 con 0x7fb0b00feda0 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b7d58700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b010e600 con 0x7fb0b00fe450 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.190+0000 7fb0b52f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 0x7fb0b010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b52f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 0x7fb0b010de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32834/0 (socket says 172.21.15.7:32834) 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b52f3700 1 -- 172.21.15.7:0/2651303300 learned_addr learned my addr 172.21.15.7:0/2651303300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b5af4700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b00fe450 0x7fb0b010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b52f3700 1 -- 172.21.15.7:0/2651303300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b00fe450 msgr2=0x7fb0b010d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b52f3700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b00fe450 0x7fb0b010d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b52f3700 1 -- 172.21.15.7:0/2651303300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0005040 con 0x7fb0b00feda0 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b52f3700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 0x7fb0b010de90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000b3b0 tx=0x7fb0a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0a67fc700 1 -- 172.21.15.7:0/2651303300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0a000e040 con 0x7fb0b00feda0 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0a67fc700 1 -- 172.21.15.7:0/2651303300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0a000bbf0 con 0x7fb0b00feda0 2024-03-16T08:55:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0a67fc700 1 -- 172.21.15.7:0/2651303300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0a001c630 con 0x7fb0b00feda0 2024-03-16T08:55:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b0113000 con 0x7fb0b00feda0 2024-03-16T08:55:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.191+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b01135c0 con 0x7fb0b00feda0 2024-03-16T08:55:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.192+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b004c710 con 0x7fb0b00feda0 2024-03-16T08:55:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.193+0000 7fb0a67fc700 1 -- 172.21.15.7:0/2651303300 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7fb0a000bd60 con 0x7fb0b00feda0 2024-03-16T08:55:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.193+0000 7fb0a67fc700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb09c069210 0x7fb09c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.193+0000 7fb0b5af4700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb09c069210 0x7fb09c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.193+0000 7fb0a67fc700 1 -- 172.21.15.7:0/2651303300 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7fb0a0087070 con 0x7fb0b00feda0 2024-03-16T08:55:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.193+0000 7fb0b5af4700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb09c069210 0x7fb09c06b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac004f40 tx=0x7fb0ac00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:55:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.198+0000 7fb0a67fc700 1 -- 172.21.15.7:0/2651303300 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fb0a0057ba0 con 0x7fb0b00feda0 2024-03-16T08:55:29.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:29.406+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0b0102520 con 0x7fb09c069210 2024-03-16T08:55:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:29 smithi007 ceph-mon[29470]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:29 smithi007 ceph-mon[29470]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:55:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:55:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:55:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:30.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:30.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:30.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:29 smithi019 ceph-mon[36407]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:30.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:29 smithi019 ceph-mon[36407]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:55:30.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:55:30.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:55:30.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:32.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:32.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:32.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:32.843 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:32.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:32.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:32.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:32.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:32.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:32.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:32 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:32 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:33 smithi019 ceph-mon[36407]: Detected new or changed devices on smithi007 2024-03-16T08:55:33.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:33 smithi019 ceph-mon[36407]: Adjusting osd_memory_target on smithi007 to 6434M 2024-03-16T08:55:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:33 smithi007 ceph-mon[29470]: Detected new or changed devices on smithi007 2024-03-16T08:55:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:33 smithi007 ceph-mon[29470]: Adjusting osd_memory_target on smithi007 to 6434M 2024-03-16T08:55:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:34 smithi019 ceph-mon[36407]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:34 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/806296577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f400784-c979-4f41-bcd2-a52737e2e65e"}]: dispatch 2024-03-16T08:55:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:34 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/806296577' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f400784-c979-4f41-bcd2-a52737e2e65e"}]': finished 2024-03-16T08:55:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:34 smithi019 ceph-mon[36407]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T08:55:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:34 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:34 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/4294862448' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:55:34.997 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:34 smithi007 ceph-mon[29470]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:34 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/806296577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f400784-c979-4f41-bcd2-a52737e2e65e"}]: dispatch 2024-03-16T08:55:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:34 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/806296577' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f400784-c979-4f41-bcd2-a52737e2e65e"}]': finished 2024-03-16T08:55:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:34 smithi007 ceph-mon[29470]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T08:55:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:34 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:34 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/4294862448' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:55:36.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:36 smithi019 ceph-mon[36407]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:36.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:36 smithi007 ceph-mon[29470]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:37 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:37 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:37 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:37 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:37.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:37 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:37.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:37 smithi007 ceph-mon[29470]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:38.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:37 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:38.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:37 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:38.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:37 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:38.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:37 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:38.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:37 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:38.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:37 smithi019 ceph-mon[36407]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:40.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:40 smithi019 ceph-mon[36407]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:40 smithi007 ceph-mon[29470]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:41.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T08:55:41.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:41.491 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:55:41.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T08:55:41.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:41.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:55:42.744 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:42 smithi007 ceph-mon[29470]: Deploying daemon osd.2 on smithi007 2024-03-16T08:55:42.744 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:42 smithi007 ceph-mon[29470]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:42.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:42 smithi019 ceph-mon[36407]: Deploying daemon osd.2 on smithi007 2024-03-16T08:55:42.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:42 smithi019 ceph-mon[36407]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:44.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:44 smithi019 ceph-mon[36407]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:44.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:44 smithi007 ceph-mon[29470]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:46 smithi007 ceph-mon[29470]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:46 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:46 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:46 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:46.876 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:46 smithi019 ceph-mon[36407]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:46.876 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:46 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:46.876 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:46 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:46.876 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:46 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:48 smithi007 ceph-mon[29470]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:48.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.923+0000 7fb0a67fc700 1 -- 172.21.15.7:0/2651303300 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb0b0102520 con 0x7fb09c069210 2024-03-16T08:55:48.926 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 2 on host 'smithi007' 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb09c069210 msgr2=0x7fb09c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb09c069210 0x7fb09c06b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac004f40 tx=0x7fb0ac00b040 comp rx=0 tx=0).stop 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 msgr2=0x7fb0b010de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 0x7fb0b010de90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000b3b0 tx=0x7fb0a0009e30 comp rx=0 tx=0).stop 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 shutdown_connections 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb09c069210 0x7fb09c06b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0b00fe450 0x7fb0b010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 --2- 172.21.15.7:0/2651303300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b00feda0 0x7fb0b010de90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 >> 172.21.15.7:0/2651303300 conn(0x7fb0b00f6c10 msgr2=0x7fb0b00f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 shutdown_connections 2024-03-16T08:55:48.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:55:48.925+0000 7fb0b7d58700 1 -- 172.21.15.7:0/2651303300 wait complete. 2024-03-16T08:55:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:48 smithi019 ceph-mon[36407]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:49.894 DEBUG:teuthology.orchestra.run.smithi007:osd.2> sudo journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.2.service 2024-03-16T08:55:49.897 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_4... 2024-03-16T08:55:49.897 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T08:55:50.164 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi019/config 2024-03-16T08:55:50.207 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:50.208 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:50.208 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:49 smithi007 ceph-mon[29470]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:50.208 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:49 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[62905]: 2024-03-16T08:55:49.949+0000 7f900aace540 -1 osd.2 0 log_to_monitors true 2024-03-16T08:55:50.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:50.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:50.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:49 smithi019 ceph-mon[36407]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:51.200 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:50 smithi007 ceph-mon[29470]: from='osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T08:55:51.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:50 smithi019 ceph-mon[36407]: from='osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T08:55:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:51 smithi007 ceph-mon[29470]: from='osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T08:55:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:51 smithi007 ceph-mon[29470]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T08:55:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:51 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:51 smithi007 ceph-mon[29470]: from='osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T08:55:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:51 smithi007 ceph-mon[29470]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:51.999 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:51 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[62905]: 2024-03-16T08:55:51.902+0000 7f9002aa3700 -1 osd.2 0 waiting for initial osdmap 2024-03-16T08:55:51.999 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:51 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[62905]: 2024-03-16T08:55:51.907+0000 7f8ffd89e700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:55:52.125 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T08:55:52.125 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T08:55:52.125 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-03-16T08:55:52.126 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-03-16T08:55:52.126 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014479 s, 724 MB/s 2024-03-16T08:55:52.126 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-03-16T08:55:52.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:51 smithi019 ceph-mon[36407]: from='osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T08:55:52.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:51 smithi019 ceph-mon[36407]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T08:55:52.173 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:51 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:52.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:51 smithi019 ceph-mon[36407]: from='osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T08:55:52.174 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:51 smithi019 ceph-mon[36407]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:52.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-03-16T08:55:52.846 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi019/config 2024-03-16T08:55:53.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:52 smithi019 ceph-mon[36407]: from='osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-16T08:55:53.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:52 smithi019 ceph-mon[36407]: osdmap e18: 3 total, 2 up, 3 in 2024-03-16T08:55:53.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:52 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:53.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:52 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:53.163 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:52 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:53.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:52 smithi007 ceph-mon[29470]: from='osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-16T08:55:53.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:52 smithi007 ceph-mon[29470]: osdmap e18: 3 total, 2 up, 3 in 2024-03-16T08:55:53.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:52 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:53.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:52 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:53.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:52 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:54.067 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: purged_snaps scrub starts 2024-03-16T08:55:54.067 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: purged_snaps scrub ok 2024-03-16T08:55:54.067 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] boot 2024-03-16T08:55:54.067 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T08:55:54.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:54.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:54.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:54.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:54.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-16T08:55:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: purged_snaps scrub starts 2024-03-16T08:55:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: purged_snaps scrub ok 2024-03-16T08:55:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: osd.2 [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] boot 2024-03-16T08:55:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T08:55:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T08:55:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-16T08:55:54.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-16T08:55:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.278+0000 7f26e18d1700 1 -- 172.21.15.19:0/479501812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fbcd0 msgr2=0x7f26dc0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.278+0000 7f26e18d1700 1 --2- 172.21.15.19:0/479501812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fbcd0 0x7f26dc0fc0e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009ce0 tx=0x7f26d4004d10 comp rx=0 tx=0).stop 2024-03-16T08:55:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.278+0000 7f26e18d1700 1 -- 172.21.15.19:0/479501812 shutdown_connections 2024-03-16T08:55:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.278+0000 7f26e18d1700 1 --2- 172.21.15.19:0/479501812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0fcf80 0x7f26dc0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.278+0000 7f26e18d1700 1 --2- 172.21.15.19:0/479501812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fbcd0 0x7f26dc0fc0e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.278+0000 7f26e18d1700 1 -- 172.21.15.19:0/479501812 >> 172.21.15.19:0/479501812 conn(0x7f26dc0f7430 msgr2=0x7f26dc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.278+0000 7f26e18d1700 1 -- 172.21.15.19:0/479501812 shutdown_connections 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.278+0000 7f26e18d1700 1 -- 172.21.15.19:0/479501812 wait complete. 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26e18d1700 1 Processor -- start 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26e18d1700 1 -- start start 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26e18d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0fbcd0 0x7f26dc10dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26e18d1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fcf80 0x7f26dc10e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26e18d1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc10e740 con 0x7f26dc0fbcd0 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26e18d1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc10ef40 con 0x7f26dc0fcf80 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26da7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fcf80 0x7f26dc10e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26da7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fcf80 0x7f26dc10e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38180/0 (socket says 172.21.15.19:38180) 2024-03-16T08:55:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26da7fc700 1 -- 172.21.15.19:0/3469754440 learned_addr learned my addr 172.21.15.19:0/3469754440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:55:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26daffd700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0fbcd0 0x7f26dc10dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26da7fc700 1 -- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0fbcd0 msgr2=0x7f26dc10dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:55:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26da7fc700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0fbcd0 0x7f26dc10dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:55:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.279+0000 7f26da7fc700 1 -- 172.21.15.19:0/3469754440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d4009940 con 0x7f26dc0fcf80 2024-03-16T08:55:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.280+0000 7f26da7fc700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fcf80 0x7f26dc10e140 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f26c4002a50 tx=0x7f26c40046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:55:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.280+0000 7f26d37fe700 1 -- 172.21.15.19:0/3469754440 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c4007cc0 con 0x7f26dc0fcf80 2024-03-16T08:55:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.280+0000 7f26d37fe700 1 -- 172.21.15.19:0/3469754440 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26c400f040 con 0x7f26dc0fcf80 2024-03-16T08:55:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.280+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc10f220 con 0x7f26dc0fcf80 2024-03-16T08:55:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.280+0000 7f26d37fe700 1 -- 172.21.15.19:0/3469754440 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c400da30 con 0x7f26dc0fcf80 2024-03-16T08:55:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.280+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc10f760 con 0x7f26dc0fcf80 2024-03-16T08:55:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.281+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc101b10 con 0x7f26dc0fcf80 2024-03-16T08:55:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.281+0000 7f26d37fe700 1 -- 172.21.15.19:0/3469754440 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) v1 ==== 86563+0+0 (secure 0 0 0) 0x7f26c400dbb0 con 0x7f26dc0fcf80 2024-03-16T08:55:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.281+0000 7f26d37fe700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f26c8069210 0x7f26c806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:55:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.282+0000 7f26d37fe700 1 -- 172.21.15.19:0/3469754440 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f26c4086610 con 0x7f26dc0fcf80 2024-03-16T08:55:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.282+0000 7f26daffd700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f26c8069210 0x7f26c806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:55:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.282+0000 7f26daffd700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f26c8069210 0x7f26c806b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26d4015040 tx=0x7f26d4004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:55:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.286+0000 7f26d37fe700 1 -- 172.21.15.19:0/3469754440 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f26c4056c50 con 0x7f26dc0fcf80 2024-03-16T08:55:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:54.484+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f26dc10f980 con 0x7f26c8069210 2024-03-16T08:55:54.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:54 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-16T08:55:54.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:54 smithi007 ceph-mon[29470]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T08:55:54.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:54 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:55:54.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:54 smithi007 ceph-mon[29470]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:55:54.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:54 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:55:54.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:54 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:55:54.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:54 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:55.189 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:54 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-16T08:55:55.189 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:54 smithi019 ceph-mon[36407]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T08:55:55.189 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:54 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T08:55:55.189 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:54 smithi019 ceph-mon[36407]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:55:55.189 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:54 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:55:55.189 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:54 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:55:55.189 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:54 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:55.594 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[65484]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:55:55.594 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[65484]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:55.594 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[65484]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:55.594 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[65484]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:55.872 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[67450]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:55:55.872 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[67450]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:55.873 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[67450]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:55.873 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[67450]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:56.070 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:55 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T08:55:56.070 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:55 smithi019 ceph-mon[36407]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T08:55:56.070 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:55 smithi019 ceph-mon[36407]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:55:56.071 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:55 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:56.139 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:55 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T08:55:56.139 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:55 smithi007 ceph-mon[29470]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T08:55:56.139 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:55 smithi007 ceph-mon[29470]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:55:56.140 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:55 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:56.140 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[67456]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:55:56.140 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[67456]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:56.140 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[67456]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:56.140 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:55 smithi007 sudo[67456]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:56.396 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67459]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:55:56.396 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67459]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:56.396 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67459]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:56.397 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67459]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:56.665 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67462]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:55:56.665 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:56.665 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67462]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:56.666 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67462]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:56.945 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67563]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:55:56.945 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67563]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:56.945 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67563]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:56.946 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67563]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:57.215 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67653]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:55:57.304 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67653]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:57.304 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67653]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:57.304 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:56 smithi007 sudo[67653]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:55:57.229+0000 7f26d37fe700 1 -- 172.21.15.19:0/3469754440 <== mon.1 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f26c4053590 con 0x7f26dc0fcf80 2024-03-16T08:55:57.483 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T08:55:57.483 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:55:57.483 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:57.483 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:57.484 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 ceph-mon[36407]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 sudo[67662]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 sudo[67662]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 sudo[67662]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 sudo[67662]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:57.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:57.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:57.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:55:57.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:57.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:55:57.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:55:57.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:57.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:55:57.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:57 smithi007 ceph-mon[29470]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:55:57.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 sudo[39888]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T08:55:57.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 sudo[39888]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:57.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 sudo[39888]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:57.749 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 sudo[39888]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 sudo[39891]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T08:55:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 sudo[39891]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T08:55:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 sudo[39891]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T08:55:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:57 smithi019 sudo[39891]: pam_unix(sudo:session): session closed for user root 2024-03-16T08:55:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: Detected new or changed devices on smithi007 2024-03-16T08:55:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: Adjusting osd_memory_target on smithi007 to 4289M 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/2598014230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1fa594a-67c7-4f8f-a745-74ef06dfd43c"}]: dispatch 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1fa594a-67c7-4f8f-a745-74ef06dfd43c"}]: dispatch 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: mgrmap e19: smithi007.uamwtp(active, since 2m), standbys: smithi019.iuqxhi 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1fa594a-67c7-4f8f-a745-74ef06dfd43c"}]': finished 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:55:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:55:58.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:55:58.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: Detected new or changed devices on smithi007 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: Adjusting osd_memory_target on smithi007 to 4289M 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/2598014230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1fa594a-67c7-4f8f-a745-74ef06dfd43c"}]: dispatch 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1fa594a-67c7-4f8f-a745-74ef06dfd43c"}]: dispatch 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: mgrmap e19: smithi007.uamwtp(active, since 2m), standbys: smithi019.iuqxhi 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1fa594a-67c7-4f8f-a745-74ef06dfd43c"}]': finished 2024-03-16T08:55:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T08:55:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:55:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T08:55:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T08:55:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:55:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T08:55:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:55:59.420 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:55:59 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/374879444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:55:59.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:55:59 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/374879444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:56:00.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:00 smithi007 ceph-mon[29470]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:56:00.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:00 smithi019 ceph-mon[36407]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:56:01.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:01 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:01.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:01 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:01.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:01 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:01.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:01 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:01.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:01 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:01 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:01 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:01 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:01 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:01 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:02.596 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:02 smithi019 ceph-mon[36407]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.4 KiB/s rd, 191 KiB/s wr, 13 op/s 2024-03-16T08:56:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:02 smithi007 ceph-mon[29470]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.4 KiB/s rd, 191 KiB/s wr, 13 op/s 2024-03-16T08:56:04.862 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:04 smithi019 ceph-mon[36407]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 128 KiB/s wr, 9 op/s 2024-03-16T08:56:04.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:04 smithi007 ceph-mon[29470]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 128 KiB/s wr, 9 op/s 2024-03-16T08:56:05.841 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:05 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T08:56:05.841 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:05 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:05.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:05 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T08:56:05.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:05 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:06.817 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:06 smithi019 ceph-mon[36407]: Deploying daemon osd.3 on smithi019 2024-03-16T08:56:06.818 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:06 smithi019 ceph-mon[36407]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 97 KiB/s wr, 6 op/s 2024-03-16T08:56:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:06 smithi007 ceph-mon[29470]: Deploying daemon osd.3 on smithi019 2024-03-16T08:56:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:06 smithi007 ceph-mon[29470]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 97 KiB/s wr, 6 op/s 2024-03-16T08:56:08.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:08 smithi019 ceph-mon[36407]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-16T08:56:08.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:08 smithi007 ceph-mon[29470]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-16T08:56:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:10 smithi019 ceph-mon[36407]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-16T08:56:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:56:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:10 smithi007 ceph-mon[29470]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-16T08:56:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:56:11.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:11 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:56:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:11 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:56:12.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:12 smithi019 ceph-mon[36407]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-16T08:56:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:12 smithi007 ceph-mon[29470]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-16T08:56:13.408 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-03-16T08:56:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.406+0000 7f26d37fe700 1 -- 172.21.15.19:0/3469754440 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f26dc10f980 con 0x7f26c8069210 2024-03-16T08:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.408+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f26c8069210 msgr2=0x7f26c806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.408+0000 7f26e18d1700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f26c8069210 0x7f26c806b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26d4015040 tx=0x7f26d4004d10 comp rx=0 tx=0).stop 2024-03-16T08:56:13.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.408+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fcf80 msgr2=0x7f26dc10e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:56:13.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.408+0000 7f26e18d1700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fcf80 0x7f26dc10e140 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f26c4002a50 tx=0x7f26c40046b0 comp rx=0 tx=0).stop 2024-03-16T08:56:13.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.409+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 shutdown_connections 2024-03-16T08:56:13.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.409+0000 7f26e18d1700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f26c8069210 0x7f26c806b6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:13.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.409+0000 7f26e18d1700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0fbcd0 0x7f26dc10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:13.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.409+0000 7f26e18d1700 1 --2- 172.21.15.19:0/3469754440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26dc0fcf80 0x7f26dc10e140 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:13.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.409+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 >> 172.21.15.19:0/3469754440 conn(0x7f26dc0f7430 msgr2=0x7f26dc065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:56:13.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.409+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 shutdown_connections 2024-03-16T08:56:13.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:13.409+0000 7f26e18d1700 1 -- 172.21.15.19:0/3469754440 wait complete. 2024-03-16T08:56:14.114 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.3.service 2024-03-16T08:56:14.117 INFO:tasks.cephadm:Deploying osd.4 on smithi019 with /dev/vg_nvme/lv_3... 2024-03-16T08:56:14.117 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T08:56:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:14 smithi019 ceph-mon[36407]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:56:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:14 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:14 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:14 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:14 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:14 smithi019 ceph-mon[36407]: from='osd.3 [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T08:56:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:14 smithi019 ceph-mon[36407]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T08:56:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:14 smithi007 ceph-mon[29470]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:56:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:14 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:14 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:14 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:14 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:14 smithi007 ceph-mon[29470]: from='osd.3 [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T08:56:14.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:14 smithi007 ceph-mon[29470]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T08:56:15.297 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi019/config 2024-03-16T08:56:15.752 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:15 smithi019 ceph-mon[36407]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T08:56:15.752 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:15 smithi019 ceph-mon[36407]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T08:56:15.752 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:15 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:56:15.752 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:15 smithi019 ceph-mon[36407]: from='osd.3 [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:56:15.752 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:15 smithi019 ceph-mon[36407]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:56:15.753 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:15 smithi019 ceph-mon[36407]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:56:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:15 smithi007 ceph-mon[29470]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T08:56:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:15 smithi007 ceph-mon[29470]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T08:56:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:15 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:56:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:15 smithi007 ceph-mon[29470]: from='osd.3 [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:56:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:15 smithi007 ceph-mon[29470]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:56:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:15 smithi007 ceph-mon[29470]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-16T08:56:16.044 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 08:56:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[41983]: 2024-03-16T08:56:15.749+0000 7f081dc39700 -1 osd.3 0 waiting for initial osdmap 2024-03-16T08:56:16.045 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 08:56:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[41983]: 2024-03-16T08:56:15.756+0000 7f0819235700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:56:16.857 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:16 smithi019 ceph-mon[36407]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-16T08:56:16.857 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:16 smithi019 ceph-mon[36407]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T08:56:16.857 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:16 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:56:16.857 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:16 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:56:16.857 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:16 smithi019 ceph-mon[36407]: osd.3 [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] boot 2024-03-16T08:56:16.857 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:16 smithi019 ceph-mon[36407]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T08:56:16.857 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:16 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:56:17.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:16 smithi007 ceph-mon[29470]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-16T08:56:17.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:16 smithi007 ceph-mon[29470]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T08:56:17.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:16 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:56:17.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:16 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:56:17.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:16 smithi007 ceph-mon[29470]: osd.3 [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] boot 2024-03-16T08:56:17.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:16 smithi007 ceph-mon[29470]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T08:56:17.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:16 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T08:56:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T08:56:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T08:56:17.883 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-03-16T08:56:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-03-16T08:56:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123188 s, 851 MB/s 2024-03-16T08:56:17.883 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-03-16T08:56:18.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:17 smithi019 ceph-mon[36407]: purged_snaps scrub starts 2024-03-16T08:56:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:17 smithi019 ceph-mon[36407]: purged_snaps scrub ok 2024-03-16T08:56:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:17 smithi019 ceph-mon[36407]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:17 smithi019 ceph-mon[36407]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T08:56:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:17 smithi007 ceph-mon[29470]: purged_snaps scrub starts 2024-03-16T08:56:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:17 smithi007 ceph-mon[29470]: purged_snaps scrub ok 2024-03-16T08:56:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:17 smithi007 ceph-mon[29470]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:17 smithi007 ceph-mon[29470]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T08:56:18.685 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-03-16T08:56:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi019/config 2024-03-16T08:56:19.274 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: Detected new or changed devices on smithi019 2024-03-16T08:56:19.274 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.274 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.274 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:56:19.274 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: Adjusting osd_memory_target on smithi019 to 15941M 2024-03-16T08:56:19.274 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.274 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:19.275 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:19.275 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.275 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:56:19.275 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:19.275 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:19.275 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.275 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:19 smithi019 ceph-mon[36407]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T08:56:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: Detected new or changed devices on smithi019 2024-03-16T08:56:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:56:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: Adjusting osd_memory_target on smithi019 to 15941M 2024-03-16T08:56:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:19.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:56:19.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:19.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:19.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:19.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:19 smithi007 ceph-mon[29470]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T08:56:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.500+0000 7f1c29849700 1 -- 172.21.15.19:0/2633579515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24102360 msgr2=0x7f1c24104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:56:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.500+0000 7f1c29849700 1 --2- 172.21.15.19:0/2633579515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24102360 0x7f1c24104740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c004940 tx=0x7f1c0c009e30 comp rx=0 tx=0).stop 2024-03-16T08:56:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.500+0000 7f1c29849700 1 -- 172.21.15.19:0/2633579515 shutdown_connections 2024-03-16T08:56:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.500+0000 7f1c29849700 1 --2- 172.21.15.19:0/2633579515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24102360 0x7f1c24104740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.500+0000 7f1c29849700 1 --2- 172.21.15.19:0/2633579515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c24066280 0x7f1c24101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.500+0000 7f1c29849700 1 -- 172.21.15.19:0/2633579515 >> 172.21.15.19:0/2633579515 conn(0x7f1c240f7430 msgr2=0x7f1c240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:56:20.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.500+0000 7f1c29849700 1 -- 172.21.15.19:0/2633579515 shutdown_connections 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.500+0000 7f1c29849700 1 -- 172.21.15.19:0/2633579515 wait complete. 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c29849700 1 Processor -- start 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c29849700 1 -- start start 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c29849700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24066280 0x7f1c2410fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c29849700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c24102360 0x7f1c24110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c29849700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24110770 con 0x7f1c24102360 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c29849700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24110fc0 con 0x7f1c24066280 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c22ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24066280 0x7f1c2410fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:56:20.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c22ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24066280 0x7f1c2410fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49796/0 (socket says 172.21.15.19:49796) 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c22ffd700 1 -- 172.21.15.19:0/1864293242 learned_addr learned my addr 172.21.15.19:0/1864293242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c227fc700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c24102360 0x7f1c24110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c22ffd700 1 -- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c24102360 msgr2=0x7f1c24110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c22ffd700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c24102360 0x7f1c24110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c22ffd700 1 -- 172.21.15.19:0/1864293242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c005040 con 0x7f1c24066280 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.501+0000 7f1c22ffd700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24066280 0x7f1c2410fc30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1c14001ce0 tx=0x7f1c14009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.502+0000 7f1c1b7fe700 1 -- 172.21.15.19:0/1864293242 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c14007b30 con 0x7f1c24066280 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.502+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c241112a0 con 0x7f1c24066280 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.502+0000 7f1c1b7fe700 1 -- 172.21.15.19:0/1864293242 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c1400f040 con 0x7f1c24066280 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.502+0000 7f1c1b7fe700 1 -- 172.21.15.19:0/1864293242 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c140096d0 con 0x7f1c24066280 2024-03-16T08:56:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.502+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c241117e0 con 0x7f1c24066280 2024-03-16T08:56:20.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.503+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2405f5f0 con 0x7f1c24066280 2024-03-16T08:56:20.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.503+0000 7f1c1b7fe700 1 -- 172.21.15.19:0/1864293242 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f1c14007c90 con 0x7f1c24066280 2024-03-16T08:56:20.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.503+0000 7f1c1b7fe700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1c100692a0 0x7f1c1006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:56:20.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.504+0000 7f1c1b7fe700 1 -- 172.21.15.19:0/1864293242 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f1c14086580 con 0x7f1c24066280 2024-03-16T08:56:20.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.504+0000 7f1c227fc700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1c100692a0 0x7f1c1006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:56:20.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.504+0000 7f1c227fc700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1c100692a0 0x7f1c1006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00b970 tx=0x7f1c0c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:56:20.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.508+0000 7f1c1b7fe700 1 -- 172.21.15.19:0/1864293242 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f1c1404cd70 con 0x7f1c24066280 2024-03-16T08:56:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:20.711+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c24111b30 con 0x7f1c100692a0 2024-03-16T08:56:20.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:20 smithi019 ceph-mon[36407]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:20.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:20 smithi019 ceph-mon[36407]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T08:56:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:20 smithi007 ceph-mon[29470]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:20 smithi007 ceph-mon[29470]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T08:56:21.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:21 smithi019 ceph-mon[36407]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T08:56:21.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:21 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:56:21.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:21 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:56:21.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:21 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:21 smithi007 ceph-mon[29470]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T08:56:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:21 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:56:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:21 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:56:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:21 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:22.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:22 smithi019 ceph-mon[36407]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:56:22.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:22 smithi019 ceph-mon[36407]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T08:56:22.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:22 smithi007 ceph-mon[29470]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:56:22.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:22 smithi007 ceph-mon[29470]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T08:56:23.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:23 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/3358687737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55356509-5ce4-4cb1-9195-3911332ec8fb"}]: dispatch 2024-03-16T08:56:23.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:23 smithi019 ceph-mon[36407]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55356509-5ce4-4cb1-9195-3911332ec8fb"}]: dispatch 2024-03-16T08:56:23.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:23 smithi019 ceph-mon[36407]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55356509-5ce4-4cb1-9195-3911332ec8fb"}]': finished 2024-03-16T08:56:23.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:23 smithi019 ceph-mon[36407]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T08:56:23.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:23 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:23.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:23 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/3358687737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55356509-5ce4-4cb1-9195-3911332ec8fb"}]: dispatch 2024-03-16T08:56:23.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:23 smithi007 ceph-mon[29470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55356509-5ce4-4cb1-9195-3911332ec8fb"}]: dispatch 2024-03-16T08:56:23.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:23 smithi007 ceph-mon[29470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55356509-5ce4-4cb1-9195-3911332ec8fb"}]': finished 2024-03-16T08:56:23.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:23 smithi007 ceph-mon[29470]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T08:56:23.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:23 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:24.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:24 smithi019 ceph-mon[36407]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-16T08:56:24.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:24 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/1590794931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:56:24.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:24 smithi007 ceph-mon[29470]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-03-16T08:56:24.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:24 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/1590794931' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:56:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:26 smithi019 ceph-mon[36407]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T08:56:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:56:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:26 smithi007 ceph-mon[29470]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T08:56:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:56:28.685 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:28 smithi019 ceph-mon[36407]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-16T08:56:28.997 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:28 smithi007 ceph-mon[29470]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-16T08:56:30.910 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:30 smithi019 ceph-mon[36407]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-16T08:56:30.910 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:30 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T08:56:30.911 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:30 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:30.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:30 smithi007 ceph-mon[29470]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-16T08:56:30.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:30 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T08:56:30.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:30 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:31.837 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:31 smithi019 ceph-mon[36407]: Deploying daemon osd.4 on smithi019 2024-03-16T08:56:31.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:31 smithi007 ceph-mon[29470]: Deploying daemon osd.4 on smithi019 2024-03-16T08:56:32.792 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:32 smithi019 ceph-mon[36407]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:32.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:32 smithi007 ceph-mon[29470]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:34.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:34 smithi019 ceph-mon[36407]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:34 smithi007 ceph-mon[29470]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:36.851 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:36 smithi019 ceph-mon[36407]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:36.851 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:36.851 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:36.851 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:56:36.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:36 smithi007 ceph-mon[29470]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:36.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:36.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:36.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:56:38.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:38 smithi019 ceph-mon[36407]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:38.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:38 smithi007 ceph-mon[29470]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:39.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.473+0000 7f1c1b7fe700 1 -- 172.21.15.19:0/1864293242 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1c24111b30 con 0x7f1c100692a0 2024-03-16T08:56:39.477 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 4 on host 'smithi019' 2024-03-16T08:56:39.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1c100692a0 msgr2=0x7f1c1006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:56:39.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1c100692a0 0x7f1c1006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00b970 tx=0x7f1c0c009e30 comp rx=0 tx=0).stop 2024-03-16T08:56:39.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24066280 msgr2=0x7f1c2410fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:56:39.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24066280 0x7f1c2410fc30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1c14001ce0 tx=0x7f1c14009e60 comp rx=0 tx=0).stop 2024-03-16T08:56:39.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 shutdown_connections 2024-03-16T08:56:39.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1c100692a0 0x7f1c1006b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:39.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c24066280 0x7f1c2410fc30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:39.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 --2- 172.21.15.19:0/1864293242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c24102360 0x7f1c24110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:39.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 >> 172.21.15.19:0/1864293242 conn(0x7f1c240f7430 msgr2=0x7f1c240f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:56:39.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 shutdown_connections 2024-03-16T08:56:39.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:39.476+0000 7f1c29849700 1 -- 172.21.15.19:0/1864293242 wait complete. 2024-03-16T08:56:40.100 DEBUG:teuthology.orchestra.run.smithi019:osd.4> sudo journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.4.service 2024-03-16T08:56:40.102 INFO:tasks.cephadm:Deploying osd.5 on smithi019 with /dev/vg_nvme/lv_2... 2024-03-16T08:56:40.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T08:56:40.405 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi019/config 2024-03-16T08:56:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:40 smithi019 ceph-mon[36407]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:40 smithi019 ceph-mon[36407]: from='osd.4 [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:56:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:40 smithi019 ceph-mon[36407]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:56:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:40 smithi007 ceph-mon[29470]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:40 smithi007 ceph-mon[29470]: from='osd.4 [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:56:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:40 smithi007 ceph-mon[29470]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T08:56:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:41.608 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:41 smithi019 ceph-mon[36407]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T08:56:41.608 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:41 smithi019 ceph-mon[36407]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T08:56:41.608 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:41.608 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:41 smithi019 ceph-mon[36407]: from='osd.4 [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:56:41.608 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:41 smithi019 ceph-mon[36407]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:56:41.609 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:56:41.609 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 08:56:41 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[48824]: 2024-03-16T08:56:41.440+0000 7f75de211700 -1 osd.4 0 waiting for initial osdmap 2024-03-16T08:56:41.609 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 08:56:41 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[48824]: 2024-03-16T08:56:41.447+0000 7f75db010700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:56:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:41 smithi007 ceph-mon[29470]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T08:56:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:41 smithi007 ceph-mon[29470]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T08:56:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:41 smithi007 ceph-mon[29470]: from='osd.4 [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:56:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:41 smithi007 ceph-mon[29470]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:56:41.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:56:42.573 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:42 smithi019 ceph-mon[36407]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:42.573 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:42 smithi019 ceph-mon[36407]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-16T08:56:42.573 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:42 smithi019 ceph-mon[36407]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T08:56:42.573 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:42.573 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:42 smithi007 ceph-mon[29470]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-16T08:56:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:42 smithi007 ceph-mon[29470]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-16T08:56:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:42 smithi007 ceph-mon[29470]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T08:56:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T08:56:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T08:56:43.339 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-03-16T08:56:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-03-16T08:56:43.339 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123215 s, 851 MB/s 2024-03-16T08:56:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-03-16T08:56:43.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:43 smithi019 ceph-mon[36407]: purged_snaps scrub starts 2024-03-16T08:56:43.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:43 smithi019 ceph-mon[36407]: purged_snaps scrub ok 2024-03-16T08:56:43.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:43 smithi019 ceph-mon[36407]: osd.4 [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] boot 2024-03-16T08:56:43.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:43 smithi019 ceph-mon[36407]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T08:56:43.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:43 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:43.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:43 smithi007 ceph-mon[29470]: purged_snaps scrub starts 2024-03-16T08:56:43.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:43 smithi007 ceph-mon[29470]: purged_snaps scrub ok 2024-03-16T08:56:43.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:43 smithi007 ceph-mon[29470]: osd.4 [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] boot 2024-03-16T08:56:43.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:43 smithi007 ceph-mon[29470]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T08:56:43.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:43 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T08:56:43.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-03-16T08:56:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi019/config 2024-03-16T08:56:44.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:44 smithi019 ceph-mon[36407]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:44.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:44 smithi019 ceph-mon[36407]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T08:56:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:44 smithi007 ceph-mon[29470]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:44 smithi007 ceph-mon[29470]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T08:56:45.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: Detected new or changed devices on smithi019 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: Adjusting osd_memory_target on smithi019 to 7970M 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:45.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:45.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:45 smithi019 ceph-mon[36407]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: Detected new or changed devices on smithi019 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: Adjusting osd_memory_target on smithi019 to 7970M 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:45.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:56:45.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:45.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:56:45.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:56:45.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:45 smithi007 ceph-mon[29470]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:46.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.194+0000 7f0914cd8700 1 -- 172.21.15.19:0/3255939859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 msgr2=0x7f09100fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.194+0000 7f0914cd8700 1 --2- 172.21.15.19:0/3255939859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 0x7f09100fe450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0904004940 tx=0x7f0904009e30 comp rx=0 tx=0).stop 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 -- 172.21.15.19:0/3255939859 shutdown_connections 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 --2- 172.21.15.19:0/3255939859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 0x7f09100fe450 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 --2- 172.21.15.19:0/3255939859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09100fce70 0x7f09100fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 -- 172.21.15.19:0/3255939859 >> 172.21.15.19:0/3255939859 conn(0x7f09100f8550 msgr2=0x7f09100fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 -- 172.21.15.19:0/3255939859 shutdown_connections 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 -- 172.21.15.19:0/3255939859 wait complete. 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 Processor -- start 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 -- start start 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.195+0000 7f0914cd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09100fce70 0x7f091010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f0914cd8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 0x7f091010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f0914cd8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091010e460 con 0x7f09100fce70 2024-03-16T08:56:46.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f0914cd8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091010e5d0 con 0x7f09100fdfe0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f090dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 0x7f091010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f090dd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 0x7f091010de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33348/0 (socket says 172.21.15.19:33348) 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f090dd9b700 1 -- 172.21.15.19:0/4167256641 learned_addr learned my addr 172.21.15.19:0/4167256641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f090e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09100fce70 0x7f091010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f090dd9b700 1 -- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09100fce70 msgr2=0x7f091010d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f090dd9b700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09100fce70 0x7f091010d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f090dd9b700 1 -- 172.21.15.19:0/4167256641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904005040 con 0x7f09100fdfe0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f090dd9b700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 0x7f091010de60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f090400b3b0 tx=0x7f0904009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f08feffd700 1 -- 172.21.15.19:0/4167256641 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090400e040 con 0x7f09100fdfe0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0910112fd0 con 0x7f09100fdfe0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f08feffd700 1 -- 172.21.15.19:0/4167256641 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f090400bbf0 con 0x7f09100fdfe0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f08feffd700 1 -- 172.21.15.19:0/4167256641 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090401c630 con 0x7f09100fdfe0 2024-03-16T08:56:46.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.196+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0910113560 con 0x7f09100fdfe0 2024-03-16T08:56:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.197+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091004c710 con 0x7f09100fdfe0 2024-03-16T08:56:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.198+0000 7f08feffd700 1 -- 172.21.15.19:0/4167256641 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f090400bd60 con 0x7f09100fdfe0 2024-03-16T08:56:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.198+0000 7f08feffd700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f08f80692a0 0x7f08f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:56:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.198+0000 7f08feffd700 1 -- 172.21.15.19:0/4167256641 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f0904087860 con 0x7f09100fdfe0 2024-03-16T08:56:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.198+0000 7f090e59c700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f08f80692a0 0x7f08f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:56:46.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.199+0000 7f090e59c700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f08f80692a0 0x7f08f806b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0900004f40 tx=0x7f090000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:56:46.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.203+0000 7f08feffd700 1 -- 172.21.15.19:0/4167256641 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f0904057bb0 con 0x7f09100fdfe0 2024-03-16T08:56:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:56:46.401+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0910102370 con 0x7f08f80692a0 2024-03-16T08:56:47.097 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:46 smithi019 ceph-mon[36407]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:56:47.097 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:46 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:56:47.098 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:46 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:56:47.098 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:46 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:46 smithi007 ceph-mon[29470]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:56:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:46 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T08:56:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:46 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T08:56:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:46 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:48.004 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:47 smithi019 ceph-mon[36407]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:48.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:47 smithi007 ceph-mon[29470]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:49.130 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:48 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/2920095819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "576cd741-4e7d-421a-a5dd-67ceb0af2440"}]: dispatch 2024-03-16T08:56:49.130 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:48 smithi007 ceph-mon[29470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "576cd741-4e7d-421a-a5dd-67ceb0af2440"}]: dispatch 2024-03-16T08:56:49.131 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:48 smithi007 ceph-mon[29470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "576cd741-4e7d-421a-a5dd-67ceb0af2440"}]': finished 2024-03-16T08:56:49.131 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:48 smithi007 ceph-mon[29470]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T08:56:49.131 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:48 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:56:49.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:48 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/2920095819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "576cd741-4e7d-421a-a5dd-67ceb0af2440"}]: dispatch 2024-03-16T08:56:49.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:48 smithi019 ceph-mon[36407]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "576cd741-4e7d-421a-a5dd-67ceb0af2440"}]: dispatch 2024-03-16T08:56:49.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:48 smithi019 ceph-mon[36407]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "576cd741-4e7d-421a-a5dd-67ceb0af2440"}]': finished 2024-03-16T08:56:49.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:48 smithi019 ceph-mon[36407]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T08:56:49.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:48 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:56:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:49 smithi019 ceph-mon[36407]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:49 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/271147604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:56:50.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:49 smithi007 ceph-mon[29470]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:50.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:49 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/271147604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T08:56:52.957 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:52 smithi019 ceph-mon[36407]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:52.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:52 smithi007 ceph-mon[29470]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:54.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:54 smithi019 ceph-mon[36407]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:54.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:54 smithi007 ceph-mon[29470]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:56.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:56 smithi019 ceph-mon[36407]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:56.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T08:56:56.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:56.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:56:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:56 smithi007 ceph-mon[29470]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T08:56:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:56:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:56:57.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:57 smithi019 ceph-mon[36407]: Deploying daemon osd.5 on smithi019 2024-03-16T08:56:57.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:57 smithi007 ceph-mon[29470]: Deploying daemon osd.5 on smithi019 2024-03-16T08:56:58.792 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:56:58 smithi019 ceph-mon[36407]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:56:58.899 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:56:58 smithi007 ceph-mon[29470]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:00 smithi019 ceph-mon[36407]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:00.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:00 smithi007 ceph-mon[29470]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:02.448 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:02.448 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:02.449 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:02 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:57:02.449 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:02 smithi019 ceph-mon[36407]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:02.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:02.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:02.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:02 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:57:02.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:02 smithi007 ceph-mon[29470]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:04.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.117+0000 7f08feffd700 1 -- 172.21.15.19:0/4167256641 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0910102370 con 0x7f08f80692a0 2024-03-16T08:57:04.119 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 5 on host 'smithi019' 2024-03-16T08:57:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f08f80692a0 msgr2=0x7f08f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f08f80692a0 0x7f08f806b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0900004f40 tx=0x7f090000b040 comp rx=0 tx=0).stop 2024-03-16T08:57:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 msgr2=0x7f091010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 0x7f091010de60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f090400b3b0 tx=0x7f0904009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 shutdown_connections 2024-03-16T08:57:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f08f80692a0 0x7f08f806b750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09100fce70 0x7f091010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 --2- 172.21.15.19:0/4167256641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09100fdfe0 0x7f091010de60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 >> 172.21.15.19:0/4167256641 conn(0x7f09100f8550 msgr2=0x7f09100f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 shutdown_connections 2024-03-16T08:57:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:04.119+0000 7f0914cd8700 1 -- 172.21.15.19:0/4167256641 wait complete. 2024-03-16T08:57:04.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:04 smithi019 ceph-mon[36407]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:04.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:04 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:04.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:04 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:04.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:04 smithi007 ceph-mon[29470]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:04.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:04 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:04.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:04 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:04.794 DEBUG:teuthology.orchestra.run.smithi019:osd.5> sudo journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.5.service 2024-03-16T08:57:04.796 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-16T08:57:04.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-16T08:57:05.068 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:05.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 08:57:04 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[55734]: 2024-03-16T08:57:04.918+0000 7f36c73fe540 -1 osd.5 0 log_to_monitors true 2024-03-16T08:57:05.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:05 smithi007 ceph-mon[29470]: from='osd.5 [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:57:05.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:05 smithi007 ceph-mon[29470]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:57:05.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:05 smithi019 ceph-mon[36407]: from='osd.5 [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:57:05.634 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:05 smithi019 ceph-mon[36407]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T08:57:06.476 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:06 smithi007 ceph-mon[29470]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:06.477 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:06 smithi007 ceph-mon[29470]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T08:57:06.477 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:06 smithi007 ceph-mon[29470]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T08:57:06.477 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:06 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:57:06.477 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:06 smithi007 ceph-mon[29470]: from='osd.5 [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:57:06.477 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:06 smithi007 ceph-mon[29470]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:57:06.477 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:06 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:06.477 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:06 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:06.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-mon[36407]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-mon[36407]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T08:57:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-mon[36407]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T08:57:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:57:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-mon[36407]: from='osd.5 [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:57:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-mon[36407]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T08:57:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:06.713 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[55734]: 2024-03-16T08:57:06.416+0000 7f36bf3d3700 -1 osd.5 0 waiting for initial osdmap 2024-03-16T08:57:06.714 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 08:57:06 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[55734]: 2024-03-16T08:57:06.424+0000 7f36ba1ce700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T08:57:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.823+0000 7f80d8578700 1 -- 172.21.15.7:0/519715555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 msgr2=0x7f80d00fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.823+0000 7f80d8578700 1 --2- 172.21.15.7:0/519715555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 0x7f80d00fd250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009ce0 tx=0x7f80c8004d10 comp rx=0 tx=0).stop 2024-03-16T08:57:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.823+0000 7f80d8578700 1 -- 172.21.15.7:0/519715555 shutdown_connections 2024-03-16T08:57:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.823+0000 7f80d8578700 1 --2- 172.21.15.7:0/519715555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80d00fe040 0x7f80d00fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.823+0000 7f80d8578700 1 --2- 172.21.15.7:0/519715555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 0x7f80d00fd250 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.823+0000 7f80d8578700 1 -- 172.21.15.7:0/519715555 >> 172.21.15.7:0/519715555 conn(0x7f80d00f8570 msgr2=0x7f80d00fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.823+0000 7f80d8578700 1 -- 172.21.15.7:0/519715555 shutdown_connections 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.823+0000 7f80d8578700 1 -- 172.21.15.7:0/519715555 wait complete. 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d8578700 1 Processor -- start 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d8578700 1 -- start start 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d8578700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 0x7f80d010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d8578700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80d00fe040 0x7f80d010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d8578700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d010e570 con 0x7f80d00fce40 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d8578700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d010edc0 con 0x7f80d00fe040 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d6314700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 0x7f80d010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d6314700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 0x7f80d010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46786/0 (socket says 172.21.15.7:46786) 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d6314700 1 -- 172.21.15.7:0/3452120567 learned_addr learned my addr 172.21.15.7:0/3452120567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d5b13700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80d00fe040 0x7f80d010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d6314700 1 -- 172.21.15.7:0/3452120567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80d00fe040 msgr2=0x7f80d010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d6314700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80d00fe040 0x7f80d010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d6314700 1 -- 172.21.15.7:0/3452120567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c8009940 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d5b13700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80d00fe040 0x7f80d010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d6314700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 0x7f80d010da30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009380 tx=0x7f80c8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80ceffd700 1 -- 172.21.15.7:0/3452120567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80c800e040 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80ceffd700 1 -- 172.21.15.7:0/3452120567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80c801cd00 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d010f040 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.824+0000 7f80ceffd700 1 -- 172.21.15.7:0/3452120567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80c8012bc0 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.825+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d010f580 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.825+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d005f5f0 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.826+0000 7f80ceffd700 1 -- 172.21.15.7:0/3452120567 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f80c8012410 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.826+0000 7f80ceffd700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f80b80692a0 0x7f80b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.826+0000 7f80d5b13700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f80b80692a0 0x7f80b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.826+0000 7f80ceffd700 1 -- 172.21.15.7:0/3452120567 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f80c8089130 con 0x7f80d00fce40 2024-03-16T08:57:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.827+0000 7f80d5b13700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f80b80692a0 0x7f80b806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f80c4004600 tx=0x7f80c400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:06.831+0000 7f80ceffd700 1 -- 172.21.15.7:0/3452120567 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f80c80592c0 con 0x7f80d00fce40 2024-03-16T08:57:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.024+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f80d010f8a0 con 0x7f80d00fce40 2024-03-16T08:57:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.024+0000 7f80ceffd700 1 -- 172.21.15.7:0/3452120567 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7f80c8058e50 con 0x7f80d00fce40 2024-03-16T08:57:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:57:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f80b80692a0 msgr2=0x7f80b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f80b80692a0 0x7f80b806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f80c4004600 tx=0x7f80c400b040 comp rx=0 tx=0).stop 2024-03-16T08:57:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 msgr2=0x7f80d010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 0x7f80d010da30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009380 tx=0x7f80c8004d10 comp rx=0 tx=0).stop 2024-03-16T08:57:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 shutdown_connections 2024-03-16T08:57:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f80b80692a0 0x7f80b806b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80d00fce40 0x7f80d010da30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 --2- 172.21.15.7:0/3452120567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80d00fe040 0x7f80d010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 >> 172.21.15.7:0/3452120567 conn(0x7f80d00f8570 msgr2=0x7f80d00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 shutdown_connections 2024-03-16T08:57:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:07.026+0000 7f80d8578700 1 -- 172.21.15.7:0/3452120567 wait complete. 2024-03-16T08:57:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:07 smithi019 ceph-mon[36407]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-16T08:57:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:07 smithi019 ceph-mon[36407]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T08:57:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:57:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:07 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:57:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3452120567' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T08:57:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:07 smithi007 ceph-mon[29470]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-03-16T08:57:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:07 smithi007 ceph-mon[29470]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T08:57:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:07 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:57:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:07 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:57:07.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:07 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3452120567' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T08:57:07.830 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1710579402,"num_in_osds":6,"osd_in_since":1710579408,"num_remapped_pgs":0} 2024-03-16T08:57:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:08 smithi019 ceph-mon[36407]: purged_snaps scrub starts 2024-03-16T08:57:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:08 smithi019 ceph-mon[36407]: purged_snaps scrub ok 2024-03-16T08:57:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:08 smithi019 ceph-mon[36407]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:08 smithi019 ceph-mon[36407]: osd.5 [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] boot 2024-03-16T08:57:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:08 smithi019 ceph-mon[36407]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T08:57:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:08 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:57:08.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:08 smithi007 ceph-mon[29470]: purged_snaps scrub starts 2024-03-16T08:57:08.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:08 smithi007 ceph-mon[29470]: purged_snaps scrub ok 2024-03-16T08:57:08.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:08 smithi007 ceph-mon[29470]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-16T08:57:08.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:08 smithi007 ceph-mon[29470]: osd.5 [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] boot 2024-03-16T08:57:08.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:08 smithi007 ceph-mon[29470]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T08:57:08.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:08 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T08:57:08.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-16T08:57:09.103 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:09.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T08:57:09.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:09.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:09.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:57:09.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:57:09.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:57:09.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:09.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:57:09.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:57:09.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:09 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:57:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:09 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:10.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:10 smithi007 ceph-mon[29470]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:10.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:10 smithi007 ceph-mon[29470]: Detected new or changed devices on smithi019 2024-03-16T08:57:10.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:10 smithi007 ceph-mon[29470]: Adjusting osd_memory_target on smithi019 to 5313M 2024-03-16T08:57:10.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:57:10.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:57:10.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:57:10.801 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:10.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 -- 172.21.15.7:0/1468602451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 msgr2=0x7feb5c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:10.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 --2- 172.21.15.7:0/1468602451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 0x7feb5c0fe450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb50004940 tx=0x7feb50009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:10.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 -- 172.21.15.7:0/1468602451 shutdown_connections 2024-03-16T08:57:10.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 --2- 172.21.15.7:0/1468602451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 0x7feb5c0fe450 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:10.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 --2- 172.21.15.7:0/1468602451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5c0fce70 0x7feb5c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:10.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 -- 172.21.15.7:0/1468602451 >> 172.21.15.7:0/1468602451 conn(0x7feb5c0f8550 msgr2=0x7feb5c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:10.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 -- 172.21.15.7:0/1468602451 shutdown_connections 2024-03-16T08:57:10.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 -- 172.21.15.7:0/1468602451 wait complete. 2024-03-16T08:57:10.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 Processor -- start 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.824+0000 7feb641ea700 1 -- start start 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb641ea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5c0fce70 0x7feb5c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb641ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 0x7feb5c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb641ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c10e440 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb641ea700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c10e5b0 con 0x7feb5c0fce70 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb61785700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 0x7feb5c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb61785700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 0x7feb5c10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46794/0 (socket says 172.21.15.7:46794) 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb61785700 1 -- 172.21.15.7:0/877601787 learned_addr learned my addr 172.21.15.7:0/877601787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb61f86700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5c0fce70 0x7feb5c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb61785700 1 -- 172.21.15.7:0/877601787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5c0fce70 msgr2=0x7feb5c10d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb61785700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5c0fce70 0x7feb5c10d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb61785700 1 -- 172.21.15.7:0/877601787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb50005040 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb61785700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 0x7feb5c10de40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feb5000b3b0 tx=0x7feb50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:10.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb5a7fc700 1 -- 172.21.15.7:0/877601787 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb5000e040 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb5a7fc700 1 -- 172.21.15.7:0/877601787 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb5000bbf0 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c112fb0 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb5a7fc700 1 -- 172.21.15.7:0/877601787 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb5001c630 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.825+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c113540 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.826+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5c04c710 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.827+0000 7feb5a7fc700 1 -- 172.21.15.7:0/877601787 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7feb5000bd60 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.827+0000 7feb5a7fc700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7feb440692a0 0x7feb4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:10.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.827+0000 7feb61f86700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7feb440692a0 0x7feb4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:10.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.827+0000 7feb5a7fc700 1 -- 172.21.15.7:0/877601787 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7feb50087b30 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.828+0000 7feb61f86700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7feb440692a0 0x7feb4406b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb4c004550 tx=0x7feb4c007ed0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:10.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:10.832+0000 7feb5a7fc700 1 -- 172.21.15.7:0/877601787 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7feb50057c30 con 0x7feb5c0fdfe0 2024-03-16T08:57:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:10 smithi019 ceph-mon[36407]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:10 smithi019 ceph-mon[36407]: Detected new or changed devices on smithi019 2024-03-16T08:57:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:10 smithi019 ceph-mon[36407]: Adjusting osd_memory_target on smithi019 to 5313M 2024-03-16T08:57:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:57:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:57:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:57:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:57:11.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.022+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7feb5c1058d0 con 0x7feb5c0fdfe0 2024-03-16T08:57:11.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.022+0000 7feb5a7fc700 1 -- 172.21.15.7:0/877601787 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7feb500577c0 con 0x7feb5c0fdfe0 2024-03-16T08:57:11.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7feb440692a0 msgr2=0x7feb4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7feb440692a0 0x7feb4406b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb4c004550 tx=0x7feb4c007ed0 comp rx=0 tx=0).stop 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 msgr2=0x7feb5c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 0x7feb5c10de40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feb5000b3b0 tx=0x7feb50009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 shutdown_connections 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7feb440692a0 0x7feb4406b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5c0fce70 0x7feb5c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 --2- 172.21.15.7:0/877601787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb5c0fdfe0 0x7feb5c10de40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 >> 172.21.15.7:0/877601787 conn(0x7feb5c0f8550 msgr2=0x7feb5c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.024+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 shutdown_connections 2024-03-16T08:57:11.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:11.025+0000 7feb641ea700 1 -- 172.21.15.7:0/877601787 wait complete. 2024-03-16T08:57:11.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:11 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/877601787' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T08:57:11.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:11 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:57:11.799 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1710579427,"num_in_osds":6,"osd_in_since":1710579408,"num_remapped_pgs":0} 2024-03-16T08:57:11.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T08:57:11.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:11 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/877601787' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T08:57:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:11 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:57:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:12.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:12 smithi007 ceph-mon[29470]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:12.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:12 smithi019 ceph-mon[36407]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 -- 172.21.15.7:0/2407550985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a4102e00 msgr2=0x7f14a41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 --2- 172.21.15.7:0/2407550985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a4102e00 0x7f14a41031d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1494004940 tx=0x7f1494009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 -- 172.21.15.7:0/2407550985 shutdown_connections 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 --2- 172.21.15.7:0/2407550985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a41037a0 0x7f14a40fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 --2- 172.21.15.7:0/2407550985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a4102e00 0x7f14a41031d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 -- 172.21.15.7:0/2407550985 >> 172.21.15.7:0/2407550985 conn(0x7f14a40f89b0 msgr2=0x7f14a40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 -- 172.21.15.7:0/2407550985 shutdown_connections 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 -- 172.21.15.7:0/2407550985 wait complete. 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 Processor -- start 2024-03-16T08:57:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 -- start start 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.906+0000 7f14aa1d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a4102e00 0x7f14a4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14aa1d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a41037a0 0x7f14a4113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14aa1d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a4114520 con 0x7f14a41037a0 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14aa1d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a410db70 con 0x7f14a4102e00 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a41037a0 0x7f14a4113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a41037a0 0x7f14a4113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39906/0 (socket says 172.21.15.7:39906) 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14a2ffd700 1 -- 172.21.15.7:0/356141679 learned_addr learned my addr 172.21.15.7:0/356141679 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14a37fe700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a4102e00 0x7f14a4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14a2ffd700 1 -- 172.21.15.7:0/356141679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a4102e00 msgr2=0x7f14a4113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14a2ffd700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a4102e00 0x7f14a4113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14a2ffd700 1 -- 172.21.15.7:0/356141679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494005040 con 0x7f14a41037a0 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14a2ffd700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a41037a0 0x7f14a4113ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1498002a50 tx=0x7f1498004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f148bfff700 1 -- 172.21.15.7:0/356141679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1498007cc0 con 0x7f14a41037a0 2024-03-16T08:57:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a410de50 con 0x7f14a41037a0 2024-03-16T08:57:13.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f148bfff700 1 -- 172.21.15.7:0/356141679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f149800f040 con 0x7f14a41037a0 2024-03-16T08:57:13.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f148bfff700 1 -- 172.21.15.7:0/356141679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f149800d9c0 con 0x7f14a41037a0 2024-03-16T08:57:13.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.907+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a410e410 con 0x7f14a41037a0 2024-03-16T08:57:13.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.908+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a4100110 con 0x7f14a41037a0 2024-03-16T08:57:13.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.909+0000 7f148bfff700 1 -- 172.21.15.7:0/356141679 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f1498009730 con 0x7f14a41037a0 2024-03-16T08:57:13.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.909+0000 7f148bfff700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f148c0692a0 0x7f148c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:13.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.909+0000 7f14a37fe700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f148c0692a0 0x7f148c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:13.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.909+0000 7f148bfff700 1 -- 172.21.15.7:0/356141679 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1498086bc0 con 0x7f14a41037a0 2024-03-16T08:57:13.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.910+0000 7f14a37fe700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f148c0692a0 0x7f148c06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1494004f90 tx=0x7f1494009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:13.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:13.913+0000 7f148bfff700 1 -- 172.21.15.7:0/356141679 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f1498056c40 con 0x7f14a41037a0 2024-03-16T08:57:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.105+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14a410efe0 con 0x7f14a41037a0 2024-03-16T08:57:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.105+0000 7f148bfff700 1 -- 172.21.15.7:0/356141679 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+10984 (secure 0 0 0) 0x7f14980567d0 con 0x7f14a41037a0 2024-03-16T08:57:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:57:14.108 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","created":"2024-03-16T08:51:36.700010+0000","modified":"2024-03-16T08:57:08.566020+0000","last_up_change":"2024-03-16T08:57:07.414518+0000","last_in_change":"2024-03-16T08:56:48.805178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:55:53.635803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c8f96044-56c4-4825-8138-629e5352a8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6803","nonce":1074142884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6805","nonce":1074142884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6809","nonce":1074142884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6807","nonce":1074142884}]},"public_addr":"172.21.15.7:6803/1074142884","cluster_addr":"172.21.15.7:6805/1074142884","heartbeat_back_addr":"172.21.15.7:6809/1074142884","heartbeat_front_addr":"172.21.15.7:6807/1074142884","state":["exists","up"]},{"osd":1,"uuid":"b3fcdd0c-0016-4781-803b-bbca53b4517b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6811","nonce":2075460664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6813","nonce":2075460664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6817","nonce":2075460664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6815","nonce":2075460664}]},"public_addr":"172.21.15.7:6811/2075460664","cluster_addr":"172.21.15.7:6813/2075460664","heartbeat_back_addr":"172.21.15.7:6817/2075460664","heartbeat_front_addr":"172.21.15.7:6815/2075460664","state":["exists","up"]},{"osd":2,"uuid":"4f400784-c979-4f41-bcd2-a52737e2e65e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6819","nonce":206561017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6821","nonce":206561017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6825","nonce":206561017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6823","nonce":206561017}]},"public_addr":"172.21.15.7:6819/206561017","cluster_addr":"172.21.15.7:6821/206561017","heartbeat_back_addr":"172.21.15.7:6825/206561017","heartbeat_front_addr":"172.21.15.7:6823/206561017","state":["exists","up"]},{"osd":3,"uuid":"c1fa594a-67c7-4f8f-a745-74ef06dfd43c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6801","nonce":157322165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6803","nonce":157322165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6807","nonce":157322165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6805","nonce":157322165}]},"public_addr":"172.21.15.19:6801/157322165","cluster_addr":"172.21.15.19:6803/157322165","heartbeat_back_addr":"172.21.15.19:6807/157322165","heartbeat_front_addr":"172.21.15.19:6805/157322165","state":["exists","up"]},{"osd":4,"uuid":"55356509-5ce4-4cb1-9195-3911332ec8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6809","nonce":2504473950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6811","nonce":2504473950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6815","nonce":2504473950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6813","nonce":2504473950}]},"public_addr":"172.21.15.19:6809/2504473950","cluster_addr":"172.21.15.19:6811/2504473950","heartbeat_back_addr":"172.21.15.19:6815/2504473950","heartbeat_front_addr":"172.21.15.19:6813/2504473950","state":["exists","up"]},{"osd":5,"uuid":"576cd741-4e7d-421a-a5dd-67ceb0af2440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6817","nonce":1303105831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6819","nonce":1303105831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6823","nonce":1303105831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6821","nonce":1303105831}]},"public_addr":"172.21.15.19:6817/1303105831","cluster_addr":"172.21.15.19:6819/1303105831","heartbeat_back_addr":"172.21.15.19:6823/1303105831","heartbeat_front_addr":"172.21.15.19:6821/1303105831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:54:54.847162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:23.867331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:50.950660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:14.805620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:40.511669+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:57:05.957371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1698981906":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6801/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/1002495884":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6800/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/1939052835":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/3257972232":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3238187551":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/4215733143":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6800/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3022930672":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3147176153":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/1599723782":"2024-03-17T08:52:36.181013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:57:14.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f148c0692a0 msgr2=0x7f148c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f148c0692a0 0x7f148c06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1494004f90 tx=0x7f1494009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a41037a0 msgr2=0x7f14a4113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a41037a0 0x7f14a4113ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1498002a50 tx=0x7f1498004600 comp rx=0 tx=0).stop 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 shutdown_connections 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f148c0692a0 0x7f148c06b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14a4102e00 0x7f14a4113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 --2- 172.21.15.7:0/356141679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a41037a0 0x7f14a4113ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.107+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 >> 172.21.15.7:0/356141679 conn(0x7f14a40f89b0 msgr2=0x7f14a41073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.108+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 shutdown_connections 2024-03-16T08:57:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:14.108+0000 7f14aa1d7700 1 -- 172.21.15.7:0/356141679 wait complete. 2024-03-16T08:57:14.704 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:55:53.635803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-16T08:57:14.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-16T08:57:14.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:14 smithi019 ceph-mon[36407]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:14.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:14 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/356141679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:57:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:14.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:14 smithi007 ceph-mon[29470]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:14.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:14 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/356141679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:57:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.635+0000 7f1399f70700 1 -- 172.21.15.7:0/3011428407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 msgr2=0x7f13940fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.635+0000 7f1399f70700 1 --2- 172.21.15.7:0/3011428407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 0x7f13940fb990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f137c004940 tx=0x7f137c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.635+0000 7f1399f70700 1 -- 172.21.15.7:0/3011428407 shutdown_connections 2024-03-16T08:57:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.635+0000 7f1399f70700 1 --2- 172.21.15.7:0/3011428407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 0x7f13940fb990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.635+0000 7f1399f70700 1 --2- 172.21.15.7:0/3011428407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1394102fd0 0x7f13941033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.635+0000 7f1399f70700 1 -- 172.21.15.7:0/3011428407 >> 172.21.15.7:0/3011428407 conn(0x7f1394071530 msgr2=0x7f1394071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.635+0000 7f1399f70700 1 -- 172.21.15.7:0/3011428407 shutdown_connections 2024-03-16T08:57:16.638 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:16 smithi007 ceph-mon[29470]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.635+0000 7f1399f70700 1 -- 172.21.15.7:0/3011428407 wait complete. 2024-03-16T08:57:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f1399f70700 1 Processor -- start 2024-03-16T08:57:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f1399f70700 1 -- start start 2024-03-16T08:57:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f1399f70700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 0x7f13940fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f1399f70700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1394102fd0 0x7f13940fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f1399f70700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940fe670 con 0x7f13940fb540 2024-03-16T08:57:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f1399f70700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940fd1a0 con 0x7f1394102fd0 2024-03-16T08:57:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f13937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 0x7f13940fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f13937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 0x7f13940fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39936/0 (socket says 172.21.15.7:39936) 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f13937fe700 1 -- 172.21.15.7:0/488123057 learned_addr learned my addr 172.21.15.7:0/488123057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f1392ffd700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1394102fd0 0x7f13940fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f13937fe700 1 -- 172.21.15.7:0/488123057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1394102fd0 msgr2=0x7f13940fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f13937fe700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1394102fd0 0x7f13940fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.636+0000 7f13937fe700 1 -- 172.21.15.7:0/488123057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c005040 con 0x7f13940fb540 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.637+0000 7f1392ffd700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1394102fd0 0x7f13940fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.637+0000 7f13937fe700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 0x7f13940fc720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f138400eb40 tx=0x7f138400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.637+0000 7f138bfff700 1 -- 172.21.15.7:0/488123057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1384007cc0 con 0x7f13940fb540 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.637+0000 7f138bfff700 1 -- 172.21.15.7:0/488123057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f138400f040 con 0x7f13940fb540 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.637+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13940fd420 con 0x7f13940fb540 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.637+0000 7f138bfff700 1 -- 172.21.15.7:0/488123057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f138400ca10 con 0x7f13940fb540 2024-03-16T08:57:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.637+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139411a020 con 0x7f13940fb540 2024-03-16T08:57:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.638+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13941053b0 con 0x7f13940fb540 2024-03-16T08:57:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.638+0000 7f138bfff700 1 -- 172.21.15.7:0/488123057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f138400cc80 con 0x7f13940fb540 2024-03-16T08:57:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.639+0000 7f138bfff700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f13800692a0 0x7f138006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.639+0000 7f138bfff700 1 -- 172.21.15.7:0/488123057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1384086a10 con 0x7f13940fb540 2024-03-16T08:57:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.639+0000 7f1392ffd700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f13800692a0 0x7f138006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.639+0000 7f1392ffd700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f13800692a0 0x7f138006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f13940fde50 tx=0x7f137c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.643+0000 7f138bfff700 1 -- 172.21.15.7:0/488123057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f138404dd70 con 0x7f13940fb540 2024-03-16T08:57:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.833+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13940fdc00 con 0x7f13940fb540 2024-03-16T08:57:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.834+0000 7f138bfff700 1 -- 172.21.15.7:0/488123057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13840530b0 con 0x7f13940fb540 2024-03-16T08:57:16.836 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f13800692a0 msgr2=0x7f138006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f13800692a0 0x7f138006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f13940fde50 tx=0x7f137c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 msgr2=0x7f13940fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 0x7f13940fc720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f138400eb40 tx=0x7f138400b040 comp rx=0 tx=0).stop 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 shutdown_connections 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f13800692a0 0x7f138006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13940fb540 0x7f13940fc720 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 --2- 172.21.15.7:0/488123057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1394102fd0 0x7f13940fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 >> 172.21.15.7:0/488123057 conn(0x7f1394071530 msgr2=0x7f13940fa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:16.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 shutdown_connections 2024-03-16T08:57:16.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:16.836+0000 7f1399f70700 1 -- 172.21.15.7:0/488123057 wait complete. 2024-03-16T08:57:16.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:16 smithi019 ceph-mon[36407]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:17.573 INFO:tasks.cephadm:Setting up client nodes... 2024-03-16T08:57:17.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-16T08:57:17.838 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:17.959 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:17 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/488123057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:57:17.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:17 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/488123057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:57:18.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:18 smithi019 ceph-mon[36407]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:18.968 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:18 smithi007 ceph-mon[29470]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.605+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/1887662013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fce80 msgr2=0x7f2cd80fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.605+0000 7f2cdd6bd700 1 --2- 172.21.15.7:0/1887662013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fce80 0x7f2cd80fd290 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8004940 tx=0x7f2cc8009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.605+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/1887662013 shutdown_connections 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.605+0000 7f2cdd6bd700 1 --2- 172.21.15.7:0/1887662013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cd80fdff0 0x7f2cd80fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.605+0000 7f2cdd6bd700 1 --2- 172.21.15.7:0/1887662013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fce80 0x7f2cd80fd290 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.605+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/1887662013 >> 172.21.15.7:0/1887662013 conn(0x7f2cd80f8580 msgr2=0x7f2cd80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.605+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/1887662013 shutdown_connections 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.605+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/1887662013 wait complete. 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cdd6bd700 1 Processor -- start 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cdd6bd700 1 -- start start 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cdd6bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cd80fce80 0x7f2cd810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cdd6bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fdff0 0x7f2cd810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cdd6bd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd810e4f0 con 0x7f2cd80fdff0 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cdd6bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd8112e90 con 0x7f2cd80fce80 2024-03-16T08:57:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cd67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fdff0 0x7f2cd810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cd67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fdff0 0x7f2cd810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39956/0 (socket says 172.21.15.7:39956) 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cd67fc700 1 -- 172.21.15.7:0/2183946550 learned_addr learned my addr 172.21.15.7:0/2183946550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.606+0000 7f2cd6ffd700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cd80fce80 0x7f2cd810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cd67fc700 1 -- 172.21.15.7:0/2183946550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cd80fce80 msgr2=0x7f2cd810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cd67fc700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cd80fce80 0x7f2cd810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cd67fc700 1 -- 172.21.15.7:0/2183946550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc8005040 con 0x7f2cd80fdff0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cd67fc700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fdff0 0x7f2cd810de60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00b790 tx=0x7f2ccc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cbf7fe700 1 -- 172.21.15.7:0/2183946550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ccc00f770 con 0x7f2cd80fdff0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cbf7fe700 1 -- 172.21.15.7:0/2183946550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ccc00fda0 con 0x7f2cd80fdff0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd81130f0 con 0x7f2cd80fdff0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cbf7fe700 1 -- 172.21.15.7:0/2183946550 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ccc00d310 con 0x7f2cd80fdff0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.607+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd8113660 con 0x7f2cd80fdff0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.608+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd804c710 con 0x7f2cd80fdff0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.608+0000 7f2cbf7fe700 1 -- 172.21.15.7:0/2183946550 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f2ccc010420 con 0x7f2cd80fdff0 2024-03-16T08:57:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.609+0000 7f2cbf7fe700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2cc00692a0 0x7f2cc006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.609+0000 7f2cbf7fe700 1 -- 172.21.15.7:0/2183946550 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2ccc086d60 con 0x7f2cd80fdff0 2024-03-16T08:57:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.609+0000 7f2cd6ffd700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2cc00692a0 0x7f2cc006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.609+0000 7f2cd6ffd700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2cc00692a0 0x7f2cc006b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800b3b0 tx=0x7f2cc8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.613+0000 7f2cbf7fe700 1 -- 172.21.15.7:0/2183946550 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f2ccc056f00 con 0x7f2cd80fdff0 2024-03-16T08:57:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.881+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2cd8113b10 con 0x7f2cd80fdff0 2024-03-16T08:57:19.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.885+0000 7f2cbf7fe700 1 -- 172.21.15.7:0/2183946550 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2ccc056a90 con 0x7f2cd80fdff0 2024-03-16T08:57:19.887 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-03-16T08:57:19.887 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvXvVlbpKgNBAAFRxxjcwp39qZQJStR3jN5w== 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2cc00692a0 msgr2=0x7f2cc006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2cc00692a0 0x7f2cc006b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800b3b0 tx=0x7f2cc8009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fdff0 msgr2=0x7f2cd810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fdff0 0x7f2cd810de60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00b790 tx=0x7f2ccc009250 comp rx=0 tx=0).stop 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 shutdown_connections 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2cc00692a0 0x7f2cc006b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cd80fce80 0x7f2cd810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 --2- 172.21.15.7:0/2183946550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd80fdff0 0x7f2cd810de60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 >> 172.21.15.7:0/2183946550 conn(0x7f2cd80f8580 msgr2=0x7f2cd80f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:19.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 shutdown_connections 2024-03-16T08:57:19.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:19.887+0000 7f2cdd6bd700 1 -- 172.21.15.7:0/2183946550 wait complete. 2024-03-16T08:57:20.576 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:57:20.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-16T08:57:20.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-16T08:57:20.626 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-16T08:57:20.900 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi019/config 2024-03-16T08:57:20.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:20 smithi019 ceph-mon[36407]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:20.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:20 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/2183946550' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:57:20.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:20 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/2183946550' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T08:57:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:20 smithi007 ceph-mon[29470]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:20 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/2183946550' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:57:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:20 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/2183946550' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.393+0000 7f798a091700 1 -- 172.21.15.19:0/1184913912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 msgr2=0x7f79840faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.393+0000 7f798a091700 1 --2- 172.21.15.19:0/1184913912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 0x7f79840faee0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7978004940 tx=0x7f7978009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.393+0000 7f798a091700 1 -- 172.21.15.19:0/1184913912 shutdown_connections 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.393+0000 7f798a091700 1 --2- 172.21.15.19:0/1184913912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 0x7f79840faee0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.393+0000 7f798a091700 1 --2- 172.21.15.19:0/1184913912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79840fe450 0x7f79840fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.393+0000 7f798a091700 1 -- 172.21.15.19:0/1184913912 >> 172.21.15.19:0/1184913912 conn(0x7f79840f6c10 msgr2=0x7f79840f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.393+0000 7f798a091700 1 -- 172.21.15.19:0/1184913912 shutdown_connections 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.393+0000 7f798a091700 1 -- 172.21.15.19:0/1184913912 wait complete. 2024-03-16T08:57:22.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f798a091700 1 Processor -- start 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f798a091700 1 -- start start 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f798a091700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79840fe450 0x7f79840fda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f798a091700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 0x7f79840fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f798a091700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840fc650 con 0x7f79840fe450 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f798a091700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79840fc7c0 con 0x7f79840feda0 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f7982ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 0x7f79840fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f7982ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 0x7f79840fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36286/0 (socket says 172.21.15.19:36286) 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f7982ffd700 1 -- 172.21.15.19:0/1442384755 learned_addr learned my addr 172.21.15.19:0/1442384755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.394+0000 7f79837fe700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79840fe450 0x7f79840fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f7982ffd700 1 -- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79840fe450 msgr2=0x7f79840fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f7982ffd700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79840fe450 0x7f79840fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:22.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f7982ffd700 1 -- 172.21.15.19:0/1442384755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978005040 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f7982ffd700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 0x7f79840fc110 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f797800b3b0 tx=0x7f7978009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f796bfff700 1 -- 172.21.15.19:0/1442384755 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f797800e040 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79840fca40 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f796bfff700 1 -- 172.21.15.19:0/1442384755 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f797800bbf0 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f796bfff700 1 -- 172.21.15.19:0/1442384755 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f797801c630 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.395+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7984115a30 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.396+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798404c710 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.396+0000 7f796bfff700 1 -- 172.21.15.19:0/1442384755 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f797800bd60 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.397+0000 7f796bfff700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f796c0692a0 0x7f796c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.397+0000 7f796bfff700 1 -- 172.21.15.19:0/1442384755 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7978087b00 con 0x7f79840feda0 2024-03-16T08:57:22.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.397+0000 7f79837fe700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f796c0692a0 0x7f796c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.397+0000 7f79837fe700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f796c0692a0 0x7f796c06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7974009bf0 tx=0x7f79740091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.401+0000 7f796bfff700 1 -- 172.21.15.19:0/1442384755 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f7978057c00 con 0x7f79840feda0 2024-03-16T08:57:22.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.671+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f7984105980 con 0x7f79840feda0 2024-03-16T08:57:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.675+0000 7f796bfff700 1 -- 172.21.15.19:0/1442384755 <== mon.1 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f7978057790 con 0x7f79840feda0 2024-03-16T08:57:22.677 INFO:teuthology.orchestra.run.smithi019.stdout:[client.1] 2024-03-16T08:57:22.677 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDyXvVlgbocKBAAXoH/zgoEh6QVX48hFw4Z0w== 2024-03-16T08:57:22.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f796c0692a0 msgr2=0x7f796c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:22.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f796c0692a0 0x7f796c06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7974009bf0 tx=0x7f79740091a0 comp rx=0 tx=0).stop 2024-03-16T08:57:22.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 msgr2=0x7f79840fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:22.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 0x7f79840fc110 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f797800b3b0 tx=0x7f7978009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 shutdown_connections 2024-03-16T08:57:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f796c0692a0 0x7f796c06b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79840fe450 0x7f79840fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 --2- 172.21.15.19:0/1442384755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79840feda0 0x7f79840fc110 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.677+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 >> 172.21.15.19:0/1442384755 conn(0x7f79840f6c10 msgr2=0x7f79840f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.678+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 shutdown_connections 2024-03-16T08:57:22.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:57:22.678+0000 7f798a091700 1 -- 172.21.15.19:0/1442384755 wait complete. 2024-03-16T08:57:22.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:22 smithi019 ceph-mon[36407]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:22.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:22 smithi007 ceph-mon[29470]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:23.279 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:57:23.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-16T08:57:23.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-16T08:57:23.321 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-16T08:57:23.321 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-16T08:57:23.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-16T08:57:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:23.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:23 smithi007 ceph-mon[29470]: from='client.? 172.21.15.19:0/1442384755' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:57:23.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:23 smithi007 ceph-mon[29470]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:57:23.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:23 smithi007 ceph-mon[29470]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T08:57:23.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:23 smithi019 ceph-mon[36407]: from='client.? 172.21.15.19:0/1442384755' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:57:23.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:23 smithi019 ceph-mon[36407]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T08:57:23.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:23 smithi019 ceph-mon[36407]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T08:57:24.960 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:24 smithi007 ceph-mon[29470]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:24.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:24 smithi019 ceph-mon[36407]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.288+0000 7f934c6f9700 1 -- 172.21.15.7:0/2259459606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 msgr2=0x7f93440fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.288+0000 7f934c6f9700 1 --2- 172.21.15.7:0/2259459606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 0x7f93440fe1e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9334004940 tx=0x7f9334009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.288+0000 7f934c6f9700 1 -- 172.21.15.7:0/2259459606 shutdown_connections 2024-03-16T08:57:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.288+0000 7f934c6f9700 1 --2- 172.21.15.7:0/2259459606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440fe7b0 0x7f9344106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.288+0000 7f934c6f9700 1 --2- 172.21.15.7:0/2259459606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 0x7f93440fe1e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.288+0000 7f934c6f9700 1 -- 172.21.15.7:0/2259459606 >> 172.21.15.7:0/2259459606 conn(0x7f93440f7880 msgr2=0x7f93440f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.288+0000 7f934c6f9700 1 -- 172.21.15.7:0/2259459606 shutdown_connections 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.288+0000 7f934c6f9700 1 -- 172.21.15.7:0/2259459606 wait complete. 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934c6f9700 1 Processor -- start 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934c6f9700 1 -- start start 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934c6f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 0x7f93440fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934c6f9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440fe7b0 0x7f93440fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934c6f9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440fc710 con 0x7f93440fde10 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934c6f9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440fc850 con 0x7f93440fe7b0 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934a495700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 0x7f93440fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934a495700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 0x7f93440fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53612/0 (socket says 172.21.15.7:53612) 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934a495700 1 -- 172.21.15.7:0/620742135 learned_addr learned my addr 172.21.15.7:0/620742135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f9349c94700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440fe7b0 0x7f93440fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934a495700 1 -- 172.21.15.7:0/620742135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440fe7b0 msgr2=0x7f93440fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934a495700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440fe7b0 0x7f93440fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934a495700 1 -- 172.21.15.7:0/620742135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334005040 con 0x7f93440fde10 2024-03-16T08:57:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.289+0000 7f934a495700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 0x7f93440fbc40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9334004890 tx=0x7f9334009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.290+0000 7f933affd700 1 -- 172.21.15.7:0/620742135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933400e040 con 0x7f93440fde10 2024-03-16T08:57:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.290+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93440fcaa0 con 0x7f93440fde10 2024-03-16T08:57:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.290+0000 7f933affd700 1 -- 172.21.15.7:0/620742135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9334009af0 con 0x7f93440fde10 2024-03-16T08:57:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.290+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934411a570 con 0x7f93440fde10 2024-03-16T08:57:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.290+0000 7f933affd700 1 -- 172.21.15.7:0/620742135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933401c3f0 con 0x7f93440fde10 2024-03-16T08:57:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.291+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9344066770 con 0x7f93440fde10 2024-03-16T08:57:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.291+0000 7f933affd700 1 -- 172.21.15.7:0/620742135 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f933400baf0 con 0x7f93440fde10 2024-03-16T08:57:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.292+0000 7f933affd700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f93300692a0 0x7f933006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.292+0000 7f933affd700 1 -- 172.21.15.7:0/620742135 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9334088080 con 0x7f93440fde10 2024-03-16T08:57:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.292+0000 7f9349c94700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f93300692a0 0x7f933006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.292+0000 7f9349c94700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f93300692a0 0x7f933006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93440fd580 tx=0x7f9340009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.296+0000 7f933affd700 1 -- 172.21.15.7:0/620742135 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f93340546f0 con 0x7f93440fde10 2024-03-16T08:57:25.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.540+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f934404bed0 con 0x7f93440fde10 2024-03-16T08:57:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.543+0000 7f933affd700 1 -- 172.21.15.7:0/620742135 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167159 (secure 0 0 0) 0x7f9334057d10 con 0x7f93440fde10 2024-03-16T08:57:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:57:25.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f93300692a0 msgr2=0x7f933006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:25.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f93300692a0 0x7f933006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93440fd580 tx=0x7f9340009040 comp rx=0 tx=0).stop 2024-03-16T08:57:25.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 msgr2=0x7f93440fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:25.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 0x7f93440fbc40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9334004890 tx=0x7f9334009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:25.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 shutdown_connections 2024-03-16T08:57:25.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f93300692a0 0x7f933006b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:25.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93440fde10 0x7f93440fbc40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:25.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 --2- 172.21.15.7:0/620742135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93440fe7b0 0x7f93440fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:25.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 >> 172.21.15.7:0/620742135 conn(0x7f93440f7880 msgr2=0x7f93440f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:25.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 shutdown_connections 2024-03-16T08:57:25.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:25.545+0000 7f934c6f9700 1 -- 172.21.15.7:0/620742135 wait complete. 2024-03-16T08:57:25.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:25 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/620742135' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T08:57:25.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:25 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/620742135' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T08:57:26.437 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"active_gid":14208,"active_name":"smithi007.uamwtp","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2519997150},{"type":"v1","addr":"172.21.15.7:6801","nonce":2519997150}]},"active_addr":"172.21.15.7:6801/2519997150","active_change":"2024-03-16T08:53:41.313094+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14230,"name":"smithi019.iuqxhi","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/","prometheus":"http://172.21.15.7:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1417367188}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2638653648}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3185447390}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":422599095}]}]} 2024-03-16T08:57:26.441 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-16T08:57:26.441 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-16T08:57:26.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T08:57:26.708 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:26.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:26 smithi019 ceph-mon[36407]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:57:26.979 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:26 smithi007 ceph-mon[29470]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:26.979 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:57:28.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.318+0000 7f043d40b700 1 -- 172.21.15.7:0/3948101229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0438101d20 msgr2=0x7f04381020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:28.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.318+0000 7f043d40b700 1 --2- 172.21.15.7:0/3948101229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0438101d20 0x7f04381020f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0428009ce0 tx=0x7f0428004d10 comp rx=0 tx=0).stop 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.318+0000 7f043d40b700 1 -- 172.21.15.7:0/3948101229 shutdown_connections 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.318+0000 7f043d40b700 1 --2- 172.21.15.7:0/3948101229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04381026c0 0x7f04380fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.318+0000 7f043d40b700 1 --2- 172.21.15.7:0/3948101229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0438101d20 0x7f04381020f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.318+0000 7f043d40b700 1 -- 172.21.15.7:0/3948101229 >> 172.21.15.7:0/3948101229 conn(0x7f04380f7870 msgr2=0x7f04380f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.318+0000 7f043d40b700 1 -- 172.21.15.7:0/3948101229 shutdown_connections 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.318+0000 7f043d40b700 1 -- 172.21.15.7:0/3948101229 wait complete. 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f043d40b700 1 Processor -- start 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f043d40b700 1 -- start start 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f043d40b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0438101d20 0x7f043810ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f043d40b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381026c0 0x7f043810f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f043d40b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043810f6f0 con 0x7f04381026c0 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f043d40b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043810f830 con 0x7f0438101d20 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f04367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381026c0 0x7f043810f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f04367fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381026c0 0x7f043810f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53618/0 (socket says 172.21.15.7:53618) 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f04367fc700 1 -- 172.21.15.7:0/1997880706 learned_addr learned my addr 172.21.15.7:0/1997880706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f0436ffd700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0438101d20 0x7f043810ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f04367fc700 1 -- 172.21.15.7:0/1997880706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0438101d20 msgr2=0x7f043810ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f04367fc700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0438101d20 0x7f043810ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f04367fc700 1 -- 172.21.15.7:0/1997880706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0428009940 con 0x7f04381026c0 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f04367fc700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381026c0 0x7f043810f1b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f042000b790 tx=0x7f0420009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.319+0000 7f042f7fe700 1 -- 172.21.15.7:0/1997880706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f042000f6f0 con 0x7f04381026c0 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.320+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0438117ce0 con 0x7f04381026c0 2024-03-16T08:57:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.320+0000 7f042f7fe700 1 -- 172.21.15.7:0/1997880706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f042000fd20 con 0x7f04381026c0 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.320+0000 7f042f7fe700 1 -- 172.21.15.7:0/1997880706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04200103a0 con 0x7f04381026c0 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.320+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0438118160 con 0x7f04381026c0 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.321+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04380fd470 con 0x7f04381026c0 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.321+0000 7f042f7fe700 1 -- 172.21.15.7:0/1997880706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f0420011400 con 0x7f04381026c0 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.322+0000 7f042f7fe700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f04240692a0 0x7f042406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.322+0000 7f042f7fe700 1 -- 172.21.15.7:0/1997880706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0420086c20 con 0x7f04381026c0 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.322+0000 7f0436ffd700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f04240692a0 0x7f042406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.322+0000 7f0436ffd700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f04240692a0 0x7f042406b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f042800b740 tx=0x7f0428004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:28.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.326+0000 7f042f7fe700 1 -- 172.21.15.7:0/1997880706 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f0420053290 con 0x7f04381026c0 2024-03-16T08:57:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.516+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f043804c710 con 0x7f04381026c0 2024-03-16T08:57:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.517+0000 7f042f7fe700 1 -- 172.21.15.7:0/1997880706 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+10984 (secure 0 0 0) 0x7f04200568b0 con 0x7f04381026c0 2024-03-16T08:57:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:57:28.519 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","created":"2024-03-16T08:51:36.700010+0000","modified":"2024-03-16T08:57:08.566020+0000","last_up_change":"2024-03-16T08:57:07.414518+0000","last_in_change":"2024-03-16T08:56:48.805178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:55:53.635803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c8f96044-56c4-4825-8138-629e5352a8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6803","nonce":1074142884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6805","nonce":1074142884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6809","nonce":1074142884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6807","nonce":1074142884}]},"public_addr":"172.21.15.7:6803/1074142884","cluster_addr":"172.21.15.7:6805/1074142884","heartbeat_back_addr":"172.21.15.7:6809/1074142884","heartbeat_front_addr":"172.21.15.7:6807/1074142884","state":["exists","up"]},{"osd":1,"uuid":"b3fcdd0c-0016-4781-803b-bbca53b4517b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6811","nonce":2075460664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6813","nonce":2075460664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6817","nonce":2075460664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6815","nonce":2075460664}]},"public_addr":"172.21.15.7:6811/2075460664","cluster_addr":"172.21.15.7:6813/2075460664","heartbeat_back_addr":"172.21.15.7:6817/2075460664","heartbeat_front_addr":"172.21.15.7:6815/2075460664","state":["exists","up"]},{"osd":2,"uuid":"4f400784-c979-4f41-bcd2-a52737e2e65e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6819","nonce":206561017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6821","nonce":206561017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6825","nonce":206561017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6823","nonce":206561017}]},"public_addr":"172.21.15.7:6819/206561017","cluster_addr":"172.21.15.7:6821/206561017","heartbeat_back_addr":"172.21.15.7:6825/206561017","heartbeat_front_addr":"172.21.15.7:6823/206561017","state":["exists","up"]},{"osd":3,"uuid":"c1fa594a-67c7-4f8f-a745-74ef06dfd43c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6801","nonce":157322165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6803","nonce":157322165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6807","nonce":157322165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6805","nonce":157322165}]},"public_addr":"172.21.15.19:6801/157322165","cluster_addr":"172.21.15.19:6803/157322165","heartbeat_back_addr":"172.21.15.19:6807/157322165","heartbeat_front_addr":"172.21.15.19:6805/157322165","state":["exists","up"]},{"osd":4,"uuid":"55356509-5ce4-4cb1-9195-3911332ec8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6809","nonce":2504473950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6811","nonce":2504473950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6815","nonce":2504473950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6813","nonce":2504473950}]},"public_addr":"172.21.15.19:6809/2504473950","cluster_addr":"172.21.15.19:6811/2504473950","heartbeat_back_addr":"172.21.15.19:6815/2504473950","heartbeat_front_addr":"172.21.15.19:6813/2504473950","state":["exists","up"]},{"osd":5,"uuid":"576cd741-4e7d-421a-a5dd-67ceb0af2440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6817","nonce":1303105831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6819","nonce":1303105831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6823","nonce":1303105831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6821","nonce":1303105831}]},"public_addr":"172.21.15.19:6817/1303105831","cluster_addr":"172.21.15.19:6819/1303105831","heartbeat_back_addr":"172.21.15.19:6823/1303105831","heartbeat_front_addr":"172.21.15.19:6821/1303105831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:54:54.847162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:23.867331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:50.950660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:14.805620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:40.511669+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:57:05.957371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1698981906":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6801/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/1002495884":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6800/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/1939052835":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/3257972232":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3238187551":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/4215733143":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6800/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3022930672":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3147176153":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/1599723782":"2024-03-17T08:52:36.181013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.518+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f04240692a0 msgr2=0x7f042406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.518+0000 7f043d40b700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f04240692a0 0x7f042406b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f042800b740 tx=0x7f0428004d10 comp rx=0 tx=0).stop 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.518+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381026c0 msgr2=0x7f043810f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.518+0000 7f043d40b700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381026c0 0x7f043810f1b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f042000b790 tx=0x7f0420009220 comp rx=0 tx=0).stop 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.519+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 shutdown_connections 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.519+0000 7f043d40b700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f04240692a0 0x7f042406b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.519+0000 7f043d40b700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0438101d20 0x7f043810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.519+0000 7f043d40b700 1 --2- 172.21.15.7:0/1997880706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04381026c0 0x7f043810f1b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.519+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 >> 172.21.15.7:0/1997880706 conn(0x7f04380f7870 msgr2=0x7f04380f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.519+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 shutdown_connections 2024-03-16T08:57:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:28.519+0000 7f043d40b700 1 -- 172.21.15.7:0/1997880706 wait complete. 2024-03-16T08:57:28.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:28 smithi007 ceph-mon[29470]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:28.814 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:28 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1997880706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:57:28.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:28 smithi019 ceph-mon[36407]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:28.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:28 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1997880706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:57:29.224 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-16T08:57:29.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T08:57:29.492 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:30.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:30 smithi007 ceph-mon[29470]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:30.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:30 smithi019 ceph-mon[36407]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.187+0000 7fa080010700 1 -- 172.21.15.7:0/3446020375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 msgr2=0x7fa0780fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.187+0000 7fa080010700 1 --2- 172.21.15.7:0/3446020375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 0x7fa0780fce30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa068004940 tx=0x7fa068009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.187+0000 7fa080010700 1 -- 172.21.15.7:0/3446020375 shutdown_connections 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.187+0000 7fa080010700 1 --2- 172.21.15.7:0/3446020375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 0x7fa0780fce30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.187+0000 7fa080010700 1 --2- 172.21.15.7:0/3446020375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa078102e20 0x7fa0781031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.187+0000 7fa080010700 1 -- 172.21.15.7:0/3446020375 >> 172.21.15.7:0/3446020375 conn(0x7fa0780f89d0 msgr2=0x7fa0780fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.187+0000 7fa080010700 1 -- 172.21.15.7:0/3446020375 shutdown_connections 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.187+0000 7fa080010700 1 -- 172.21.15.7:0/3446020375 wait complete. 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa080010700 1 Processor -- start 2024-03-16T08:57:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa080010700 1 -- start start 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa080010700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa078102e20 0x7fa078113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa080010700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 0x7fa0781140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa080010700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781147c0 con 0x7fa0781037c0 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa080010700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07810db20 con 0x7fa078102e20 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa07d5ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 0x7fa0781140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa07d5ab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 0x7fa0781140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53624/0 (socket says 172.21.15.7:53624) 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa07d5ab700 1 -- 172.21.15.7:0/1450110584 learned_addr learned my addr 172.21.15.7:0/1450110584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa07ddac700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa078102e20 0x7fa078113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa07d5ab700 1 -- 172.21.15.7:0/1450110584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa078102e20 msgr2=0x7fa078113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa07d5ab700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa078102e20 0x7fa078113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.188+0000 7fa07d5ab700 1 -- 172.21.15.7:0/1450110584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068005040 con 0x7fa0781037c0 2024-03-16T08:57:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.189+0000 7fa07d5ab700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 0x7fa0781140e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa068009100 tx=0x7fa068009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.189+0000 7fa06e7fc700 1 -- 172.21.15.7:0/1450110584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa06800e040 con 0x7fa0781037c0 2024-03-16T08:57:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.189+0000 7fa06e7fc700 1 -- 172.21.15.7:0/1450110584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa068009af0 con 0x7fa0781037c0 2024-03-16T08:57:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.189+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07810dda0 con 0x7fa0781037c0 2024-03-16T08:57:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.189+0000 7fa06e7fc700 1 -- 172.21.15.7:0/1450110584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa06801c3f0 con 0x7fa0781037c0 2024-03-16T08:57:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.189+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07810e360 con 0x7fa0781037c0 2024-03-16T08:57:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.190+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0780fe570 con 0x7fa0781037c0 2024-03-16T08:57:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.190+0000 7fa06e7fc700 1 -- 172.21.15.7:0/1450110584 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fa06800baf0 con 0x7fa0781037c0 2024-03-16T08:57:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.191+0000 7fa06e7fc700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa0640692a0 0x7fa06406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.191+0000 7fa06e7fc700 1 -- 172.21.15.7:0/1450110584 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa0680880d0 con 0x7fa0781037c0 2024-03-16T08:57:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.191+0000 7fa07ddac700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa0640692a0 0x7fa06406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.191+0000 7fa07ddac700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa0640692a0 0x7fa06406b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa074009b60 tx=0x7fa0740091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:31.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.195+0000 7fa06e7fc700 1 -- 172.21.15.7:0/1450110584 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fa0680581d0 con 0x7fa0781037c0 2024-03-16T08:57:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.389+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa07805f5f0 con 0x7fa0781037c0 2024-03-16T08:57:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.390+0000 7fa06e7fc700 1 -- 172.21.15.7:0/1450110584 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+10984 (secure 0 0 0) 0x7fa068057d60 con 0x7fa0781037c0 2024-03-16T08:57:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:57:31.393 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","created":"2024-03-16T08:51:36.700010+0000","modified":"2024-03-16T08:57:08.566020+0000","last_up_change":"2024-03-16T08:57:07.414518+0000","last_in_change":"2024-03-16T08:56:48.805178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:55:53.635803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c8f96044-56c4-4825-8138-629e5352a8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6803","nonce":1074142884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6805","nonce":1074142884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6809","nonce":1074142884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6807","nonce":1074142884}]},"public_addr":"172.21.15.7:6803/1074142884","cluster_addr":"172.21.15.7:6805/1074142884","heartbeat_back_addr":"172.21.15.7:6809/1074142884","heartbeat_front_addr":"172.21.15.7:6807/1074142884","state":["exists","up"]},{"osd":1,"uuid":"b3fcdd0c-0016-4781-803b-bbca53b4517b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6811","nonce":2075460664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6813","nonce":2075460664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6817","nonce":2075460664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6815","nonce":2075460664}]},"public_addr":"172.21.15.7:6811/2075460664","cluster_addr":"172.21.15.7:6813/2075460664","heartbeat_back_addr":"172.21.15.7:6817/2075460664","heartbeat_front_addr":"172.21.15.7:6815/2075460664","state":["exists","up"]},{"osd":2,"uuid":"4f400784-c979-4f41-bcd2-a52737e2e65e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6819","nonce":206561017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6821","nonce":206561017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6825","nonce":206561017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6823","nonce":206561017}]},"public_addr":"172.21.15.7:6819/206561017","cluster_addr":"172.21.15.7:6821/206561017","heartbeat_back_addr":"172.21.15.7:6825/206561017","heartbeat_front_addr":"172.21.15.7:6823/206561017","state":["exists","up"]},{"osd":3,"uuid":"c1fa594a-67c7-4f8f-a745-74ef06dfd43c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6801","nonce":157322165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6803","nonce":157322165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6807","nonce":157322165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6805","nonce":157322165}]},"public_addr":"172.21.15.19:6801/157322165","cluster_addr":"172.21.15.19:6803/157322165","heartbeat_back_addr":"172.21.15.19:6807/157322165","heartbeat_front_addr":"172.21.15.19:6805/157322165","state":["exists","up"]},{"osd":4,"uuid":"55356509-5ce4-4cb1-9195-3911332ec8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6809","nonce":2504473950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6811","nonce":2504473950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6815","nonce":2504473950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6813","nonce":2504473950}]},"public_addr":"172.21.15.19:6809/2504473950","cluster_addr":"172.21.15.19:6811/2504473950","heartbeat_back_addr":"172.21.15.19:6815/2504473950","heartbeat_front_addr":"172.21.15.19:6813/2504473950","state":["exists","up"]},{"osd":5,"uuid":"576cd741-4e7d-421a-a5dd-67ceb0af2440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6817","nonce":1303105831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6819","nonce":1303105831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6823","nonce":1303105831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6821","nonce":1303105831}]},"public_addr":"172.21.15.19:6817/1303105831","cluster_addr":"172.21.15.19:6819/1303105831","heartbeat_back_addr":"172.21.15.19:6823/1303105831","heartbeat_front_addr":"172.21.15.19:6821/1303105831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:54:54.847162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:23.867331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:50.950660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:14.805620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:40.511669+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:57:05.957371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1698981906":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6801/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/1002495884":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6800/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/1939052835":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/3257972232":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3238187551":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/4215733143":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6800/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3022930672":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3147176153":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/1599723782":"2024-03-17T08:52:36.181013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:57:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa0640692a0 msgr2=0x7fa06406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa0640692a0 0x7fa06406b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa074009b60 tx=0x7fa0740091a0 comp rx=0 tx=0).stop 2024-03-16T08:57:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 msgr2=0x7fa0781140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 0x7fa0781140e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa068009100 tx=0x7fa068009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 shutdown_connections 2024-03-16T08:57:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa0640692a0 0x7fa06406b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa078102e20 0x7fa078113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 --2- 172.21.15.7:0/1450110584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0781037c0 0x7fa0781140e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 >> 172.21.15.7:0/1450110584 conn(0x7fa0780f89d0 msgr2=0x7fa0780f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 shutdown_connections 2024-03-16T08:57:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:31.392+0000 7fa080010700 1 -- 172.21.15.7:0/1450110584 wait complete. 2024-03-16T08:57:31.940 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:31 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1450110584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:57:31.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:31 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1450110584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:57:31.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-16T08:57:31.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-16T08:57:31.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-16T08:57:31.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-16T08:57:31.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-16T08:57:31.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-16T08:57:32.279 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:32.280 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:32 smithi019 ceph-mon[36407]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:32.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:32 smithi007 ceph-mon[29470]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.821+0000 7f550b3d8700 1 -- 172.21.15.7:0/3591481386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 msgr2=0x7f5504106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.821+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3591481386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 0x7f5504106f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54f4004940 tx=0x7f54f4009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.821+0000 7f550b3d8700 1 -- 172.21.15.7:0/3591481386 shutdown_connections 2024-03-16T08:57:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.821+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3591481386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 0x7f5504106f40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.821+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3591481386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55040fded0 0x7f55040fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.821+0000 7f550b3d8700 1 -- 172.21.15.7:0/3591481386 >> 172.21.15.7:0/3591481386 conn(0x7f5504072bc0 msgr2=0x7f5504072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.821+0000 7f550b3d8700 1 -- 172.21.15.7:0/3591481386 shutdown_connections 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.821+0000 7f550b3d8700 1 -- 172.21.15.7:0/3591481386 wait complete. 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f550b3d8700 1 Processor -- start 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f550b3d8700 1 -- start start 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f550b3d8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55040fded0 0x7f55040fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f550b3d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 0x7f55040fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f550b3d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040fc860 con 0x7f55040fe870 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f550b3d8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040fc9a0 con 0x7f55040fded0 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f5508973700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 0x7f55040fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f5508973700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 0x7f55040fc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33746/0 (socket says 172.21.15.7:33746) 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f5508973700 1 -- 172.21.15.7:0/3203862358 learned_addr learned my addr 172.21.15.7:0/3203862358 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f5509174700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55040fded0 0x7f55040fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f5508973700 1 -- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55040fded0 msgr2=0x7f55040fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f5508973700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55040fded0 0x7f55040fbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.822+0000 7f5508973700 1 -- 172.21.15.7:0/3203862358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f4005040 con 0x7f55040fe870 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.823+0000 7f5508973700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 0x7f55040fc2d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009380 tx=0x7f54f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.823+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54f400e040 con 0x7f55040fe870 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.823+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54f4009af0 con 0x7f55040fe870 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.823+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5504119dc0 con 0x7f55040fe870 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.823+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54f401c630 con 0x7f55040fe870 2024-03-16T08:57:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.823+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550411a1c0 con 0x7f55040fe870 2024-03-16T08:57:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.824+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5504102460 con 0x7f55040fe870 2024-03-16T08:57:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.824+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f54f4013420 con 0x7f55040fe870 2024-03-16T08:57:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.825+0000 7f54f9ffb700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f54f00692a0 0x7f54f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.825+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f54f4088020 con 0x7f55040fe870 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.825+0000 7f5509174700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f54f00692a0 0x7f54f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.825+0000 7f54f9ffb700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] conn(0x7f54f006ef70 0x7f54f0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.825+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 --> [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f0071a30 con 0x7f54f006ef70 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.825+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54f408a690 con 0x7f55040fe870 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.825+0000 7f5509174700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f54f00692a0 0x7f54f006b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5500004c40 tx=0x7f5500009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.825+0000 7f5509975700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] conn(0x7f54f006ef70 0x7f54f0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.826+0000 7f5509975700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] conn(0x7f54f006ef70 0x7f54f0071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.826+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 <== osd.4 v2:172.21.15.19:6808/2504473950 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f54f0071a30 con 0x7f54f006ef70 2024-03-16T08:57:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.841+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 --> [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f55040fcb30 con 0x7f54f006ef70 2024-03-16T08:57:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.842+0000 7f54f9ffb700 1 -- 172.21.15.7:0/3203862358 <== osd.4 v2:172.21.15.19:6808/2504473950 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f54fc00a8d0 con 0x7f54f006ef70 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.842+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] conn(0x7f54f006ef70 msgr2=0x7f54f0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.842+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] conn(0x7f54f006ef70 0x7f54f0071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.842+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f54f00692a0 msgr2=0x7f54f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.842+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f54f00692a0 0x7f54f006b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5500004c40 tx=0x7f5500009040 comp rx=0 tx=0).stop 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.842+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 msgr2=0x7f55040fc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.842+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 0x7f55040fc2d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009380 tx=0x7f54f4009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.843+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 shutdown_connections 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.843+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:6808/2504473950,v1:172.21.15.19:6809/2504473950] conn(0x7f54f006ef70 0x7f54f0071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.843+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f54f00692a0 0x7f54f006b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.843+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55040fded0 0x7f55040fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.843+0000 7f550b3d8700 1 --2- 172.21.15.7:0/3203862358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55040fe870 0x7f55040fc2d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.843+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 >> 172.21.15.7:0/3203862358 conn(0x7f5504072bc0 msgr2=0x7f5504103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.843+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 shutdown_connections 2024-03-16T08:57:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:34.843+0000 7f550b3d8700 1 -- 172.21.15.7:0/3203862358 wait complete. 2024-03-16T08:57:34.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:34 smithi019 ceph-mon[36407]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:34 smithi007 ceph-mon[29470]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.456+0000 7f5872a60700 1 -- 172.21.15.7:0/1352112160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 msgr2=0x7f586c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:35.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.456+0000 7f5872a60700 1 --2- 172.21.15.7:0/1352112160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 0x7f586c1031d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f585c004940 tx=0x7f585c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:35.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.456+0000 7f5872a60700 1 -- 172.21.15.7:0/1352112160 shutdown_connections 2024-03-16T08:57:35.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.456+0000 7f5872a60700 1 --2- 172.21.15.7:0/1352112160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586c1037a0 0x7f586c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:35.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.456+0000 7f5872a60700 1 --2- 172.21.15.7:0/1352112160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 0x7f586c1031d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:35.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.456+0000 7f5872a60700 1 -- 172.21.15.7:0/1352112160 >> 172.21.15.7:0/1352112160 conn(0x7f586c0f89b0 msgr2=0x7f586c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:35.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.456+0000 7f5872a60700 1 -- 172.21.15.7:0/1352112160 shutdown_connections 2024-03-16T08:57:35.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.456+0000 7f5872a60700 1 -- 172.21.15.7:0/1352112160 wait complete. 2024-03-16T08:57:35.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f5872a60700 1 Processor -- start 2024-03-16T08:57:35.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f5872a60700 1 -- start start 2024-03-16T08:57:35.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f5872a60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 0x7f586c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:35.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f5872a60700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586c1037a0 0x7f586c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:35.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f5872a60700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c114570 con 0x7f586c102e00 2024-03-16T08:57:35.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f5872a60700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c10dae0 con 0x7f586c1037a0 2024-03-16T08:57:35.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f586bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 0x7f586c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:35.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f586bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 0x7f586c113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33768/0 (socket says 172.21.15.7:33768) 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.457+0000 7f586bfff700 1 -- 172.21.15.7:0/2075634183 learned_addr learned my addr 172.21.15.7:0/2075634183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f586b7fe700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586c1037a0 0x7f586c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f586bfff700 1 -- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586c1037a0 msgr2=0x7f586c113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f586bfff700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586c1037a0 0x7f586c113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f586bfff700 1 -- 172.21.15.7:0/2075634183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c005040 con 0x7f586c102e00 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f586bfff700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 0x7f586c113950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f585c009150 tx=0x7f585c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c00e040 con 0x7f586c102e00 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f585c013420 con 0x7f586c102e00 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c10dd60 con 0x7f586c102e00 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c01c630 con 0x7f586c102e00 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.458+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c10e320 con 0x7f586c102e00 2024-03-16T08:57:35.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.459+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f586c10e840 con 0x7f586c102e00 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f585c013590 con 0x7f586c102e00 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f5868ff9700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f58540692a0 0x7f585406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f585c088b90 con 0x7f586c102e00 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f586b7fe700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f58540692a0 0x7f585406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f5868ff9700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] conn(0x7f585406ef70 0x7f5854071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 --> [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5854071a30 con 0x7f585406ef70 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f585c088f70 con 0x7f586c102e00 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f5870ffd700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] conn(0x7f585406ef70 0x7f5854071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.460+0000 7f586b7fe700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f58540692a0 0x7f585406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5860006fd0 tx=0x7f5860008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.461+0000 7f5870ffd700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] conn(0x7f585406ef70 0x7f5854071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:35.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.462+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 <== osd.3 v2:172.21.15.19:6800/157322165 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5854071a30 con 0x7f585406ef70 2024-03-16T08:57:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.477+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 --> [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f586c0fa290 con 0x7f585406ef70 2024-03-16T08:57:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.477+0000 7f5868ff9700 1 -- 172.21.15.7:0/2075634183 <== osd.3 v2:172.21.15.19:6800/157322165 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f586c0fa290 con 0x7f585406ef70 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] conn(0x7f585406ef70 msgr2=0x7f5854071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] conn(0x7f585406ef70 0x7f5854071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f58540692a0 msgr2=0x7f585406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f58540692a0 0x7f585406b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5860006fd0 tx=0x7f5860008040 comp rx=0 tx=0).stop 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 msgr2=0x7f586c113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 0x7f586c113950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f585c009150 tx=0x7f585c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 shutdown_connections 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:6800/157322165,v1:172.21.15.19:6801/157322165] conn(0x7f585406ef70 0x7f5854071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f58540692a0 0x7f585406b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586c102e00 0x7f586c113950 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 --2- 172.21.15.7:0/2075634183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586c1037a0 0x7f586c113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 >> 172.21.15.7:0/2075634183 conn(0x7f586c0f89b0 msgr2=0x7f586c0f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 shutdown_connections 2024-03-16T08:57:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:35.478+0000 7f5872a60700 1 -- 172.21.15.7:0/2075634183 wait complete. 2024-03-16T08:57:35.496 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:111669149713 2024-03-16T08:57:36.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-16T08:57:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:36.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:36 smithi019 ceph-mon[36407]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:36.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:36 smithi007 ceph-mon[29470]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:37.905 INFO:teuthology.orchestra.run.smithi007.stdout:146028888077 2024-03-16T08:57:37.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-16T08:57:38.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:38 smithi019 ceph-mon[36407]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:38.976 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:38 smithi007 ceph-mon[29470]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:40.870 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:40 smithi007 ceph-mon[29470]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:40.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:40 smithi019 ceph-mon[36407]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.360+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2394342928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 msgr2=0x7fc1ac0fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.360+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2394342928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 0x7fc1ac0fb910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc19c004940 tx=0x7fc19c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.360+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2394342928 shutdown_connections 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.360+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2394342928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 0x7fc1ac0fb910 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.360+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2394342928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ac102e70 0x7fc1ac103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.360+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2394342928 >> 172.21.15.7:0/2394342928 conn(0x7fc1ac0714c0 msgr2=0x7fc1ac0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.360+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2394342928 shutdown_connections 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.360+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2394342928 wait complete. 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b45c4700 1 Processor -- start 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b45c4700 1 -- start start 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b45c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 0x7fc1ac113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b45c4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ac102e70 0x7fc1ac1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b45c4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac114700 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b45c4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac10daf0 con 0x7fc1ac102e70 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b2360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 0x7fc1ac113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b2360700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 0x7fc1ac113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33776/0 (socket says 172.21.15.7:33776) 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b2360700 1 -- 172.21.15.7:0/2770303896 learned_addr learned my addr 172.21.15.7:0/2770303896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b1b5f700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ac102e70 0x7fc1ac1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b2360700 1 -- 172.21.15.7:0/2770303896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ac102e70 msgr2=0x7fc1ac1140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b2360700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ac102e70 0x7fc1ac1140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.361+0000 7fc1b2360700 1 -- 172.21.15.7:0/2770303896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c005040 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.362+0000 7fc1b1b5f700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ac102e70 0x7fc1ac1140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:57:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.362+0000 7fc1b2360700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 0x7fc1ac113b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8002a50 tx=0x7fc1a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.362+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a8007cc0 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.362+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1a800f040 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.362+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac10ddd0 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.362+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1a8009640 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.362+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac10e390 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.363+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1ac10e970 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.363+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fc1a80097c0 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.364+0000 7fc1a2ffd700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc1980692a0 0x7fc19806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.364+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc1a8086c80 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.364+0000 7fc1b1b5f700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc1980692a0 0x7fc19806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.364+0000 7fc1a2ffd700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] conn(0x7fc19806ef70 0x7fc198071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.364+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 --> [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc198071a30 con 0x7fc19806ef70 2024-03-16T08:57:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.364+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a808c2e0 con 0x7fc1ac0fb4a0 2024-03-16T08:57:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.364+0000 7fc1b2b61700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] conn(0x7fc19806ef70 0x7fc198071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.364+0000 7fc1b1b5f700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc1980692a0 0x7fc19806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b3b0 tx=0x7fc19c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.365+0000 7fc1b2b61700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] conn(0x7fc19806ef70 0x7fc198071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.365+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 <== osd.0 v2:172.21.15.7:6802/1074142884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc198071a30 con 0x7fc19806ef70 2024-03-16T08:57:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.381+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 --> [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1ac05e6c0 con 0x7fc19806ef70 2024-03-16T08:57:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1a2ffd700 1 -- 172.21.15.7:0/2770303896 <== osd.0 v2:172.21.15.7:6802/1074142884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc1ac05e6c0 con 0x7fc19806ef70 2024-03-16T08:57:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] conn(0x7fc19806ef70 msgr2=0x7fc198071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] conn(0x7fc19806ef70 0x7fc198071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc1980692a0 msgr2=0x7fc19806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc1980692a0 0x7fc19806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b3b0 tx=0x7fc19c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 msgr2=0x7fc1ac113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 0x7fc1ac113b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8002a50 tx=0x7fc1a80046b0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 shutdown_connections 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc1980692a0 0x7fc19806b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1ac0fb4a0 0x7fc1ac113b70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.7:6802/1074142884,v1:172.21.15.7:6803/1074142884] conn(0x7fc19806ef70 0x7fc198071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 --2- 172.21.15.7:0/2770303896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ac102e70 0x7fc1ac1140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 >> 172.21.15.7:0/2770303896 conn(0x7fc1ac0714c0 msgr2=0x7fc1ac0fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 shutdown_connections 2024-03-16T08:57:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.382+0000 7fc1b45c4700 1 -- 172.21.15.7:0/2770303896 wait complete. 2024-03-16T08:57:41.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:57:41.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:57:41.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.963+0000 7f5c4e584700 1 -- 172.21.15.7:0/1610840165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 msgr2=0x7f5c48102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.963+0000 7f5c4e584700 1 --2- 172.21.15.7:0/1610840165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 0x7f5c48102140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c30004940 tx=0x7f5c30009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:41.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.963+0000 7f5c4e584700 1 -- 172.21.15.7:0/1610840165 shutdown_connections 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.963+0000 7f5c4e584700 1 --2- 172.21.15.7:0/1610840165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c48102710 0x7f5c480fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.963+0000 7f5c4e584700 1 --2- 172.21.15.7:0/1610840165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 0x7f5c48102140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.963+0000 7f5c4e584700 1 -- 172.21.15.7:0/1610840165 >> 172.21.15.7:0/1610840165 conn(0x7f5c48072bc0 msgr2=0x7f5c48072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.963+0000 7f5c4e584700 1 -- 172.21.15.7:0/1610840165 shutdown_connections 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.963+0000 7f5c4e584700 1 -- 172.21.15.7:0/1610840165 wait complete. 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c4e584700 1 Processor -- start 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c4e584700 1 -- start start 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c4e584700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 0x7f5c4810eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c4e584700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c48102710 0x7f5c4810f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c4e584700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4810f650 con 0x7f5c48101d70 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c4e584700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4810f790 con 0x7f5c48102710 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 0x7f5c4810eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c47fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 0x7f5c4810eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33780/0 (socket says 172.21.15.7:33780) 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c47fff700 1 -- 172.21.15.7:0/3318819278 learned_addr learned my addr 172.21.15.7:0/3318819278 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.964+0000 7f5c477fe700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c48102710 0x7f5c4810f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c47fff700 1 -- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c48102710 msgr2=0x7f5c4810f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c47fff700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c48102710 0x7f5c4810f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c47fff700 1 -- 172.21.15.7:0/3318819278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c30005040 con 0x7f5c48101d70 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c47fff700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 0x7f5c4810eac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009b30 tx=0x7f5c30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c30021070 con 0x7f5c48101d70 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c30013c00 con 0x7f5c48101d70 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c48117b30 con 0x7f5c48101d70 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c3000e070 con 0x7f5c48101d70 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.965+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c48117f90 con 0x7f5c48101d70 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.966+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5c4805e6c0 con 0x7f5c48101d70 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.966+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f5c3000b790 con 0x7f5c48101d70 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.967+0000 7f5c44ff9700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5c340692a0 0x7f5c3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.967+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5c30088270 con 0x7f5c48101d70 2024-03-16T08:57:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.967+0000 7f5c477fe700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5c340692a0 0x7f5c3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.967+0000 7f5c44ff9700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] conn(0x7f5c3406ef70 0x7f5c34071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.967+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 --> [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c34071a30 con 0x7f5c3406ef70 2024-03-16T08:57:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.967+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c3008a8c0 con 0x7f5c48101d70 2024-03-16T08:57:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.967+0000 7f5c4cb21700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] conn(0x7f5c3406ef70 0x7f5c34071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.967+0000 7f5c477fe700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5c340692a0 0x7f5c3406b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c48066000 tx=0x7f5c3800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.968+0000 7f5c4cb21700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] conn(0x7f5c3406ef70 0x7f5c34071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.968+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 <== osd.5 v2:172.21.15.19:6816/1303105831 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5c34071a30 con 0x7f5c3406ef70 2024-03-16T08:57:41.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.987+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 --> [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5c480fd4c0 con 0x7f5c3406ef70 2024-03-16T08:57:41.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c44ff9700 1 -- 172.21.15.7:0/3318819278 <== osd.5 v2:172.21.15.19:6816/1303105831 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5c480fd4c0 con 0x7f5c3406ef70 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] conn(0x7f5c3406ef70 msgr2=0x7f5c34071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] conn(0x7f5c3406ef70 0x7f5c34071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5c340692a0 msgr2=0x7f5c3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5c340692a0 0x7f5c3406b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c48066000 tx=0x7f5c3800b040 comp rx=0 tx=0).stop 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 msgr2=0x7f5c4810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 0x7f5c4810eac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009b30 tx=0x7f5c30009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 shutdown_connections 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:6816/1303105831,v1:172.21.15.19:6817/1303105831] conn(0x7f5c3406ef70 0x7f5c34071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5c340692a0 0x7f5c3406b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48101d70 0x7f5c4810eac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 --2- 172.21.15.7:0/3318819278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c48102710 0x7f5c4810f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:41.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.988+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 >> 172.21.15.7:0/3318819278 conn(0x7f5c48072bc0 msgr2=0x7f5c480faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.989+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 shutdown_connections 2024-03-16T08:57:41.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:41.989+0000 7f5c4e584700 1 -- 172.21.15.7:0/3318819278 wait complete. 2024-03-16T08:57:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:42.947 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:42 smithi007 ceph-mon[29470]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:42.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:42 smithi019 ceph-mon[36407]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:43.752 INFO:teuthology.orchestra.run.smithi007.stdout:167503724554 2024-03-16T08:57:43.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-16T08:57:43.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:43 smithi019 ceph-mon[36407]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:43 smithi007 ceph-mon[29470]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:44.698 INFO:teuthology.orchestra.run.smithi007.stdout:38654705700 2024-03-16T08:57:44.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-16T08:57:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.918+0000 7f2c41a2b700 1 -- 172.21.15.7:0/477893145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 msgr2=0x7f2c3c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.918+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/477893145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 0x7f2c3c109800 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c004940 tx=0x7f2c2c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.918+0000 7f2c41a2b700 1 -- 172.21.15.7:0/477893145 shutdown_connections 2024-03-16T08:57:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.918+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/477893145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c109dd0 0x7f2c3c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.918+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/477893145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 0x7f2c3c109800 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.918+0000 7f2c41a2b700 1 -- 172.21.15.7:0/477893145 >> 172.21.15.7:0/477893145 conn(0x7f2c3c072c00 msgr2=0x7f2c3c075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.919+0000 7f2c41a2b700 1 -- 172.21.15.7:0/477893145 shutdown_connections 2024-03-16T08:57:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.919+0000 7f2c41a2b700 1 -- 172.21.15.7:0/477893145 wait complete. 2024-03-16T08:57:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.919+0000 7f2c41a2b700 1 Processor -- start 2024-03-16T08:57:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.919+0000 7f2c41a2b700 1 -- start start 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.919+0000 7f2c41a2b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 0x7f2c3c119ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c41a2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c109dd0 0x7f2c3c11a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c41a2b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c11aa50 con 0x7f2c3c109430 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c41a2b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c114030 con 0x7f2c3c109dd0 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c3affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 0x7f2c3c119ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c3affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 0x7f2c3c119ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46002/0 (socket says 172.21.15.7:46002) 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c3affd700 1 -- 172.21.15.7:0/229986863 learned_addr learned my addr 172.21.15.7:0/229986863 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c3a7fc700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c109dd0 0x7f2c3c11a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c3affd700 1 -- 172.21.15.7:0/229986863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c109dd0 msgr2=0x7f2c3c11a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c3affd700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c109dd0 0x7f2c3c11a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:44.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c3affd700 1 -- 172.21.15.7:0/229986863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c005040 con 0x7f2c3c109430 2024-03-16T08:57:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c3affd700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 0x7f2c3c119ec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009b30 tx=0x7f2c2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c2c021070 con 0x7f2c3c109430 2024-03-16T08:57:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.920+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c1142b0 con 0x7f2c3c109430 2024-03-16T08:57:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.921+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c114790 con 0x7f2c3c109430 2024-03-16T08:57:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.921+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c2c013c00 con 0x7f2c3c109430 2024-03-16T08:57:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.921+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c2c00e070 con 0x7f2c3c109430 2024-03-16T08:57:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.921+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c3c114c90 con 0x7f2c3c109430 2024-03-16T08:57:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.922+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f2c2c026ae0 con 0x7f2c3c109430 2024-03-16T08:57:44.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.923+0000 7f2c237fe700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2c240692a0 0x7f2c2406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:44.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.923+0000 7f2c3a7fc700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2c240692a0 0x7f2c2406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:44.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.923+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2c2c089160 con 0x7f2c3c109430 2024-03-16T08:57:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.923+0000 7f2c237fe700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] conn(0x7f2c2406ef70 0x7f2c24071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.923+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 --> [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c24071a30 con 0x7f2c2406ef70 2024-03-16T08:57:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.923+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2c089540 con 0x7f2c3c109430 2024-03-16T08:57:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.924+0000 7f2c3a7fc700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2c240692a0 0x7f2c2406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c115760 tx=0x7f2c3000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.924+0000 7f2c3b7fe700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] conn(0x7f2c2406ef70 0x7f2c24071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.924+0000 7f2c3b7fe700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] conn(0x7f2c2406ef70 0x7f2c24071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.925+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 <== osd.2 v2:172.21.15.7:6818/206561017 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2c24071a30 con 0x7f2c2406ef70 2024-03-16T08:57:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.943+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 --> [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c3c104b80 con 0x7f2c2406ef70 2024-03-16T08:57:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.943+0000 7f2c237fe700 1 -- 172.21.15.7:0/229986863 <== osd.2 v2:172.21.15.7:6818/206561017 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2c3c104b80 con 0x7f2c2406ef70 2024-03-16T08:57:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] conn(0x7f2c2406ef70 msgr2=0x7f2c24071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] conn(0x7f2c2406ef70 0x7f2c24071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2c240692a0 msgr2=0x7f2c2406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2c240692a0 0x7f2c2406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c115760 tx=0x7f2c3000b040 comp rx=0 tx=0).stop 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 msgr2=0x7f2c3c119ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 0x7f2c3c119ec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009b30 tx=0x7f2c2c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 shutdown_connections 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6818/206561017,v1:172.21.15.7:6819/206561017] conn(0x7f2c2406ef70 0x7f2c24071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2c240692a0 0x7f2c2406b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c3c109430 0x7f2c3c119ec0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 --2- 172.21.15.7:0/229986863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c109dd0 0x7f2c3c11a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 >> 172.21.15.7:0/229986863 conn(0x7f2c3c072c00 msgr2=0x7f2c3c074bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 shutdown_connections 2024-03-16T08:57:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:44.944+0000 7f2c41a2b700 1 -- 172.21.15.7:0/229986863 wait complete. 2024-03-16T08:57:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.905+0000 7f3594b16700 1 -- 172.21.15.7:0/3031438355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3590102e70 msgr2=0x7f3590103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.905+0000 7f3594b16700 1 --2- 172.21.15.7:0/3031438355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3590102e70 0x7f3590103240 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3580004940 tx=0x7f3580009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.905+0000 7f3594b16700 1 -- 172.21.15.7:0/3031438355 shutdown_connections 2024-03-16T08:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.905+0000 7f3594b16700 1 --2- 172.21.15.7:0/3031438355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35900fb4a0 0x7f35900fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.905+0000 7f3594b16700 1 --2- 172.21.15.7:0/3031438355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3590102e70 0x7f3590103240 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.905+0000 7f3594b16700 1 -- 172.21.15.7:0/3031438355 >> 172.21.15.7:0/3031438355 conn(0x7f35900714c0 msgr2=0x7f35900718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.905+0000 7f3594b16700 1 -- 172.21.15.7:0/3031438355 shutdown_connections 2024-03-16T08:57:45.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.905+0000 7f3594b16700 1 -- 172.21.15.7:0/3031438355 wait complete. 2024-03-16T08:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f3594b16700 1 Processor -- start 2024-03-16T08:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f3594b16700 1 -- start start 2024-03-16T08:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f3594b16700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35900fb4a0 0x7f3590113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f3594b16700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3590102e70 0x7f35901140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f3594b16700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3590114700 con 0x7f35900fb4a0 2024-03-16T08:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f3594b16700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359010daf0 con 0x7f3590102e70 2024-03-16T08:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f358e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35900fb4a0 0x7f3590113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:45.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f358e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35900fb4a0 0x7f3590113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46022/0 (socket says 172.21.15.7:46022) 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f358e59c700 1 -- 172.21.15.7:0/3505762704 learned_addr learned my addr 172.21.15.7:0/3505762704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f358dd9b700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3590102e70 0x7f35901140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f358e59c700 1 -- 172.21.15.7:0/3505762704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3590102e70 msgr2=0x7f35901140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f358e59c700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3590102e70 0x7f35901140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.906+0000 7f358e59c700 1 -- 172.21.15.7:0/3505762704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3580005040 con 0x7f35900fb4a0 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.907+0000 7f358e59c700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35900fb4a0 0x7f3590113b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f358000b8b0 tx=0x7f3580009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.907+0000 7f357effd700 1 -- 172.21.15.7:0/3505762704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358000e040 con 0x7f35900fb4a0 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.907+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359010dd70 con 0x7f35900fb4a0 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.907+0000 7f357effd700 1 -- 172.21.15.7:0/3505762704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3580004290 con 0x7f35900fb4a0 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.907+0000 7f357effd700 1 -- 172.21.15.7:0/3505762704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358001c3f0 con 0x7f35900fb4a0 2024-03-16T08:57:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.907+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359010e330 con 0x7f35900fb4a0 2024-03-16T08:57:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.908+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3590100110 con 0x7f35900fb4a0 2024-03-16T08:57:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.908+0000 7f357effd700 1 -- 172.21.15.7:0/3505762704 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f35800137b0 con 0x7f35900fb4a0 2024-03-16T08:57:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.909+0000 7f357effd700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f35780692a0 0x7f357806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.909+0000 7f357effd700 1 -- 172.21.15.7:0/3505762704 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f35800881d0 con 0x7f35900fb4a0 2024-03-16T08:57:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.909+0000 7f358dd9b700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f35780692a0 0x7f357806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.909+0000 7f358dd9b700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f35780692a0 0x7f357806b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3584009fd0 tx=0x7f3584009f20 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:45.913+0000 7f357effd700 1 -- 172.21.15.7:0/3505762704 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f35800582d0 con 0x7f35900fb4a0 2024-03-16T08:57:46.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.107+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f359010f1c0 con 0x7f35900fb4a0 2024-03-16T08:57:46.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.107+0000 7f357effd700 1 -- 172.21.15.7:0/3505762704 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3580057e60 con 0x7f35900fb4a0 2024-03-16T08:57:46.110 INFO:teuthology.orchestra.run.smithi007.stdout:111669149715 2024-03-16T08:57:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f35780692a0 msgr2=0x7f357806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f35780692a0 0x7f357806b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3584009fd0 tx=0x7f3584009f20 comp rx=0 tx=0).stop 2024-03-16T08:57:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35900fb4a0 msgr2=0x7f3590113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35900fb4a0 0x7f3590113b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f358000b8b0 tx=0x7f3580009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 shutdown_connections 2024-03-16T08:57:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f35780692a0 0x7f357806b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35900fb4a0 0x7f3590113b70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 --2- 172.21.15.7:0/3505762704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3590102e70 0x7f35901140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 >> 172.21.15.7:0/3505762704 conn(0x7f35900714c0 msgr2=0x7f35900fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 shutdown_connections 2024-03-16T08:57:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:46.109+0000 7f3594b16700 1 -- 172.21.15.7:0/3505762704 wait complete. 2024-03-16T08:57:46.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:46 smithi007 ceph-mon[29470]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:46.688 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:46 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3505762704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T08:57:46.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:46 smithi019 ceph-mon[36407]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:46.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:46 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3505762704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T08:57:46.740 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:47.545 INFO:teuthology.orchestra.run.smithi007.stdout:81604378650 2024-03-16T08:57:47.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-16T08:57:48.002 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149713 got 111669149715 for osd.3 2024-03-16T08:57:48.003 DEBUG:teuthology.parallel:result is None 2024-03-16T08:57:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.502+0000 7fcd30bdb700 1 -- 172.21.15.7:0/1335834584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 msgr2=0x7fcd2c0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.502+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/1335834584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 0x7fcd2c0fbd30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c004940 tx=0x7fcd1c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.502+0000 7fcd30bdb700 1 -- 172.21.15.7:0/1335834584 shutdown_connections 2024-03-16T08:57:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.502+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/1335834584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 0x7fcd2c0fbd30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.502+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/1335834584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c101d20 0x7fcd2c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.502+0000 7fcd30bdb700 1 -- 172.21.15.7:0/1335834584 >> 172.21.15.7:0/1335834584 conn(0x7fcd2c0f7870 msgr2=0x7fcd2c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.502+0000 7fcd30bdb700 1 -- 172.21.15.7:0/1335834584 shutdown_connections 2024-03-16T08:57:48.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.502+0000 7fcd30bdb700 1 -- 172.21.15.7:0/1335834584 wait complete. 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd30bdb700 1 Processor -- start 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd30bdb700 1 -- start start 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd30bdb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c101d20 0x7fcd2c10eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd30bdb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 0x7fcd2c10f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd30bdb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c10f5c0 con 0x7fcd2c1026c0 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd30bdb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c10f700 con 0x7fcd2c101d20 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd29d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 0x7fcd2c10f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd29d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 0x7fcd2c10f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46040/0 (socket says 172.21.15.7:46040) 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd29d9b700 1 -- 172.21.15.7:0/4146639820 learned_addr learned my addr 172.21.15.7:0/4146639820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd2a59c700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c101d20 0x7fcd2c10eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd29d9b700 1 -- 172.21.15.7:0/4146639820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c101d20 msgr2=0x7fcd2c10eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd29d9b700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c101d20 0x7fcd2c10eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.503+0000 7fcd29d9b700 1 -- 172.21.15.7:0/4146639820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c005040 con 0x7fcd2c1026c0 2024-03-16T08:57:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.504+0000 7fcd29d9b700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 0x7fcd2c10f080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00b3b0 tx=0x7fcd1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.504+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd1c00e040 con 0x7fcd2c1026c0 2024-03-16T08:57:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.504+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd1c009b10 con 0x7fcd2c1026c0 2024-03-16T08:57:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.504+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c10f980 con 0x7fcd2c1026c0 2024-03-16T08:57:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.504+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd1c013420 con 0x7fcd2c1026c0 2024-03-16T08:57:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.504+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c117f50 con 0x7fcd2c1026c0 2024-03-16T08:57:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.505+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd2c0f92c0 con 0x7fcd2c1026c0 2024-03-16T08:57:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.505+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fcd1c01c7c0 con 0x7fcd2c1026c0 2024-03-16T08:57:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd22ffd700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcd1c0877d0 con 0x7fcd2c1026c0 2024-03-16T08:57:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd2a59c700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd22ffd700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] conn(0x7fcd1806ef70 0x7fcd18071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 --> [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd18071a30 con 0x7fcd1806ef70 2024-03-16T08:57:48.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd1c089de0 con 0x7fcd2c1026c0 2024-03-16T08:57:48.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd2ad9d700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] conn(0x7fcd1806ef70 0x7fcd18071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:48.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd2a59c700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcd180692a0 0x7fcd1806b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009bf0 tx=0x7fcd140091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:48.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.506+0000 7fcd2ad9d700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] conn(0x7fcd1806ef70 0x7fcd18071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:48.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.507+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 <== osd.1 v2:172.21.15.7:6810/2075460664 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fcd18071a30 con 0x7fcd1806ef70 2024-03-16T08:57:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.522+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 --> [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcd2c0fd470 con 0x7fcd1806ef70 2024-03-16T08:57:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.522+0000 7fcd22ffd700 1 -- 172.21.15.7:0/4146639820 <== osd.1 v2:172.21.15.7:6810/2075460664 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcd2c0fd470 con 0x7fcd1806ef70 2024-03-16T08:57:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] conn(0x7fcd1806ef70 msgr2=0x7fcd18071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] conn(0x7fcd1806ef70 0x7fcd18071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcd180692a0 msgr2=0x7fcd1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcd180692a0 0x7fcd1806b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009bf0 tx=0x7fcd140091a0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 msgr2=0x7fcd2c10f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:48.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 0x7fcd2c10f080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00b3b0 tx=0x7fcd1c009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 shutdown_connections 2024-03-16T08:57:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6810/2075460664,v1:172.21.15.7:6811/2075460664] conn(0x7fcd1806ef70 0x7fcd18071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c101d20 0x7fcd2c10eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 --2- 172.21.15.7:0/4146639820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd2c1026c0 0x7fcd2c10f080 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 >> 172.21.15.7:0/4146639820 conn(0x7fcd2c0f7870 msgr2=0x7fcd2c0f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 shutdown_connections 2024-03-16T08:57:48.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:48.523+0000 7fcd30bdb700 1 -- 172.21.15.7:0/4146639820 wait complete. 2024-03-16T08:57:48.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:48 smithi019 ceph-mon[36407]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:48.769 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:48.868 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:48 smithi007 ceph-mon[29470]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:49.707 INFO:teuthology.orchestra.run.smithi007.stdout:60129542175 2024-03-16T08:57:49.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-16T08:57:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.906+0000 7f9bf46ce700 1 -- 172.21.15.7:0/4165699447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fe7a0 msgr2=0x7f9bec106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.906+0000 7f9bf46ce700 1 --2- 172.21.15.7:0/4165699447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fe7a0 0x7f9bec106e70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc004940 tx=0x7f9bdc009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.906+0000 7f9bf46ce700 1 -- 172.21.15.7:0/4165699447 shutdown_connections 2024-03-16T08:57:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.906+0000 7f9bf46ce700 1 --2- 172.21.15.7:0/4165699447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fe7a0 0x7f9bec106e70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.906+0000 7f9bf46ce700 1 --2- 172.21.15.7:0/4165699447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bec0fde00 0x7f9bec0fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.906+0000 7f9bf46ce700 1 -- 172.21.15.7:0/4165699447 >> 172.21.15.7:0/4165699447 conn(0x7f9bec0f7880 msgr2=0x7f9bec0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.906+0000 7f9bf46ce700 1 -- 172.21.15.7:0/4165699447 shutdown_connections 2024-03-16T08:57:49.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.906+0000 7f9bf46ce700 1 -- 172.21.15.7:0/4165699447 wait complete. 2024-03-16T08:57:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf46ce700 1 Processor -- start 2024-03-16T08:57:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf46ce700 1 -- start start 2024-03-16T08:57:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf46ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fde00 0x7f9bec113d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf46ce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bec0fe7a0 0x7f9bec10bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf46ce700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec10c130 con 0x7f9bec0fde00 2024-03-16T08:57:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf46ce700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec10c2a0 con 0x7f9bec0fe7a0 2024-03-16T08:57:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf246a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fde00 0x7f9bec113d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf246a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fde00 0x7f9bec113d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46046/0 (socket says 172.21.15.7:46046) 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf246a700 1 -- 172.21.15.7:0/3186923519 learned_addr learned my addr 172.21.15.7:0/3186923519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf1c69700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bec0fe7a0 0x7f9bec10bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf246a700 1 -- 172.21.15.7:0/3186923519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bec0fe7a0 msgr2=0x7f9bec10bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf246a700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bec0fe7a0 0x7f9bec10bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.907+0000 7f9bf246a700 1 -- 172.21.15.7:0/3186923519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc005040 con 0x7f9bec0fde00 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.908+0000 7f9bf246a700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fde00 0x7f9bec113d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9be800c8c0 tx=0x7f9be80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.908+0000 7f9be2ffd700 1 -- 172.21.15.7:0/3186923519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be800a760 con 0x7f9bec0fde00 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.908+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec10c520 con 0x7f9bec0fde00 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.908+0000 7f9be2ffd700 1 -- 172.21.15.7:0/3186923519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9be800ad90 con 0x7f9bec0fde00 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.908+0000 7f9be2ffd700 1 -- 172.21.15.7:0/3186923519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be80113a0 con 0x7f9bec0fde00 2024-03-16T08:57:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.908+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec10ca60 con 0x7f9bec0fde00 2024-03-16T08:57:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.909+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec0666f0 con 0x7f9bec0fde00 2024-03-16T08:57:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.909+0000 7f9be2ffd700 1 -- 172.21.15.7:0/3186923519 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f9be8012400 con 0x7f9bec0fde00 2024-03-16T08:57:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.910+0000 7f9be2ffd700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9bd80692b0 0x7f9bd806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.910+0000 7f9be2ffd700 1 -- 172.21.15.7:0/3186923519 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9be8086d20 con 0x7f9bec0fde00 2024-03-16T08:57:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.910+0000 7f9bf1c69700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9bd80692b0 0x7f9bd806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.910+0000 7f9bf1c69700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9bd80692b0 0x7f9bd806b760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9bec10d530 tx=0x7f9bdc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:49.915+0000 7f9be2ffd700 1 -- 172.21.15.7:0/3186923519 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f9be8056e20 con 0x7f9bec0fde00 2024-03-16T08:57:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.113+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9bec10ca60 con 0x7f9bec0fde00 2024-03-16T08:57:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.113+0000 7f9be2ffd700 1 -- 172.21.15.7:0/3186923519 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9bec10ca60 con 0x7f9bec0fde00 2024-03-16T08:57:50.116 INFO:teuthology.orchestra.run.smithi007.stdout:146028888080 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9bd80692b0 msgr2=0x7f9bd806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9bd80692b0 0x7f9bd806b760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9bec10d530 tx=0x7f9bdc009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fde00 msgr2=0x7f9bec113d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fde00 0x7f9bec113d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9be800c8c0 tx=0x7f9be80044c0 comp rx=0 tx=0).stop 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 shutdown_connections 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9bd80692b0 0x7f9bd806b760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bec0fde00 0x7f9bec113d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 --2- 172.21.15.7:0/3186923519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bec0fe7a0 0x7f9bec10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.116+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 >> 172.21.15.7:0/3186923519 conn(0x7f9bec0f7880 msgr2=0x7f9bec0f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.117+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 shutdown_connections 2024-03-16T08:57:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:50.117+0000 7f9bf46ce700 1 -- 172.21.15.7:0/3186923519 wait complete. 2024-03-16T08:57:50.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:50 smithi007 ceph-mon[29470]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:50.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:50 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3186923519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T08:57:50.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:50 smithi019 ceph-mon[36407]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:50.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:50 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3186923519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T08:57:51.170 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888077 got 146028888080 for osd.4 2024-03-16T08:57:51.170 DEBUG:teuthology.parallel:result is None 2024-03-16T08:57:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:52.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:52 smithi007 ceph-mon[29470]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:52.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:52 smithi019 ceph-mon[36407]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:57:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.910+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2821185222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a40fb4a0 msgr2=0x7f09a40fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.910+0000 7f09ab2eb700 1 --2- 172.21.15.7:0/2821185222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a40fb4a0 0x7f09a40fb910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0994004940 tx=0x7f0994009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.910+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2821185222 shutdown_connections 2024-03-16T08:57:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.910+0000 7f09ab2eb700 1 --2- 172.21.15.7:0/2821185222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a40fb4a0 0x7f09a40fb910 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.910+0000 7f09ab2eb700 1 --2- 172.21.15.7:0/2821185222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a41014f0 0x7f09a41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.910+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2821185222 >> 172.21.15.7:0/2821185222 conn(0x7f09a40714c0 msgr2=0x7f09a40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.911+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2821185222 shutdown_connections 2024-03-16T08:57:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.911+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2821185222 wait complete. 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.911+0000 7f09ab2eb700 1 Processor -- start 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.911+0000 7f09ab2eb700 1 -- start start 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.911+0000 7f09ab2eb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a40fb4a0 0x7f09a410eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.911+0000 7f09ab2eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a41014f0 0x7f09a410f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.911+0000 7f09ab2eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a410f750 con 0x7f09a41014f0 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.911+0000 7f09ab2eb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a410f890 con 0x7f09a40fb4a0 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09a8886700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a41014f0 0x7f09a410f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09a8886700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a41014f0 0x7f09a410f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47850/0 (socket says 172.21.15.7:47850) 2024-03-16T08:57:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09a8886700 1 -- 172.21.15.7:0/2903267096 learned_addr learned my addr 172.21.15.7:0/2903267096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09a9087700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a40fb4a0 0x7f09a410eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09a8886700 1 -- 172.21.15.7:0/2903267096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a40fb4a0 msgr2=0x7f09a410eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09a8886700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a40fb4a0 0x7f09a410eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09a8886700 1 -- 172.21.15.7:0/2903267096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994005040 con 0x7f09a41014f0 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09a8886700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a41014f0 0x7f09a410f070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f099400b8b0 tx=0x7f0994009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f0999ffb700 1 -- 172.21.15.7:0/2903267096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099400e040 con 0x7f09a41014f0 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a410c920 con 0x7f09a41014f0 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f0999ffb700 1 -- 172.21.15.7:0/2903267096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0994004290 con 0x7f09a41014f0 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.912+0000 7f0999ffb700 1 -- 172.21.15.7:0/2903267096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099401c3f0 con 0x7f09a41014f0 2024-03-16T08:57:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.913+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a410ceb0 con 0x7f09a41014f0 2024-03-16T08:57:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.914+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a41038d0 con 0x7f09a41014f0 2024-03-16T08:57:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.914+0000 7f0999ffb700 1 -- 172.21.15.7:0/2903267096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f0994013420 con 0x7f09a41014f0 2024-03-16T08:57:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.914+0000 7f0999ffb700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f09900692a0 0x7f099006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.914+0000 7f0999ffb700 1 -- 172.21.15.7:0/2903267096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0994087f70 con 0x7f09a41014f0 2024-03-16T08:57:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.914+0000 7f09a9087700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f09900692a0 0x7f099006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:53.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.915+0000 7f09a9087700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f09900692a0 0x7f099006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09a0002170 tx=0x7f09a000d630 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:53.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:53.919+0000 7f0999ffb700 1 -- 172.21.15.7:0/2903267096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f09940b9d70 con 0x7f09a41014f0 2024-03-16T08:57:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.111+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f09a40fcbd0 con 0x7f09a41014f0 2024-03-16T08:57:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.111+0000 7f0999ffb700 1 -- 172.21.15.7:0/2903267096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0994054730 con 0x7f09a41014f0 2024-03-16T08:57:54.114 INFO:teuthology.orchestra.run.smithi007.stdout:38654705703 2024-03-16T08:57:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f09900692a0 msgr2=0x7f099006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f09900692a0 0x7f099006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09a0002170 tx=0x7f09a000d630 comp rx=0 tx=0).stop 2024-03-16T08:57:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a41014f0 msgr2=0x7f09a410f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a41014f0 0x7f09a410f070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f099400b8b0 tx=0x7f0994009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 shutdown_connections 2024-03-16T08:57:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f09900692a0 0x7f099006b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a40fb4a0 0x7f09a410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 --2- 172.21.15.7:0/2903267096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09a41014f0 0x7f09a410f070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 >> 172.21.15.7:0/2903267096 conn(0x7f09a40714c0 msgr2=0x7f09a40fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 shutdown_connections 2024-03-16T08:57:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:54.113+0000 7f09ab2eb700 1 -- 172.21.15.7:0/2903267096 wait complete. 2024-03-16T08:57:54.638 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:54 smithi007 ceph-mon[29470]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:54.638 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:54 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/2903267096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T08:57:54.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:54 smithi019 ceph-mon[36407]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:54.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:54 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/2903267096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T08:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 -- 172.21.15.7:0/3229623997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c500fb4c0 msgr2=0x7f7c500fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:55.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 --2- 172.21.15.7:0/3229623997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c500fb4c0 0x7f7c500fb930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7c38004a50 tx=0x7f7c3800af40 comp rx=0 tx=0).stop 2024-03-16T08:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 -- 172.21.15.7:0/3229623997 shutdown_connections 2024-03-16T08:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 --2- 172.21.15.7:0/3229623997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c500fb4c0 0x7f7c500fb930 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 --2- 172.21.15.7:0/3229623997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c50101520 0x7f7c501018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 -- 172.21.15.7:0/3229623997 >> 172.21.15.7:0/3229623997 conn(0x7f7c50072310 msgr2=0x7f7c50072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 -- 172.21.15.7:0/3229623997 shutdown_connections 2024-03-16T08:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 -- 172.21.15.7:0/3229623997 wait complete. 2024-03-16T08:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.282+0000 7f7c570d4700 1 Processor -- start 2024-03-16T08:57:55.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c570d4700 1 -- start start 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c570d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c500fb4c0 0x7f7c500fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c570d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50101520 0x7f7c500fda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c570d4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500ff440 con 0x7f7c50101520 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c570d4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500fdf70 con 0x7f7c500fb4c0 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50101520 0x7f7c500fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50101520 0x7f7c500fda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47868/0 (socket says 172.21.15.7:47868) 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c4ffff700 1 -- 172.21.15.7:0/1047792057 learned_addr learned my addr 172.21.15.7:0/1047792057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c54e70700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c500fb4c0 0x7f7c500fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c4ffff700 1 -- 172.21.15.7:0/1047792057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c500fb4c0 msgr2=0x7f7c500fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c4ffff700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c500fb4c0 0x7f7c500fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c4ffff700 1 -- 172.21.15.7:0/1047792057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38005040 con 0x7f7c50101520 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.283+0000 7f7c4ffff700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50101520 0x7f7c500fda30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c38008b90 tx=0x7f7c3800af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:55.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.284+0000 7f7c4d7fa700 1 -- 172.21.15.7:0/1047792057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3800f070 con 0x7f7c50101520 2024-03-16T08:57:55.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.284+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c500fe1f0 con 0x7f7c50101520 2024-03-16T08:57:55.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.284+0000 7f7c4d7fa700 1 -- 172.21.15.7:0/1047792057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c3800a4c0 con 0x7f7c50101520 2024-03-16T08:57:55.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.284+0000 7f7c4d7fa700 1 -- 172.21.15.7:0/1047792057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c38014660 con 0x7f7c50101520 2024-03-16T08:57:55.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.284+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5011a050 con 0x7f7c50101520 2024-03-16T08:57:55.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.285+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c50105020 con 0x7f7c50101520 2024-03-16T08:57:55.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.285+0000 7f7c4d7fa700 1 -- 172.21.15.7:0/1047792057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f7c3801c420 con 0x7f7c50101520 2024-03-16T08:57:55.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.285+0000 7f7c4d7fa700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f7c3c0692a0 0x7f7c3c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:55.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.286+0000 7f7c4d7fa700 1 -- 172.21.15.7:0/1047792057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7c38088710 con 0x7f7c50101520 2024-03-16T08:57:55.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.286+0000 7f7c54e70700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f7c3c0692a0 0x7f7c3c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:55.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.286+0000 7f7c54e70700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f7c3c0692a0 0x7f7c3c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c48001450 tx=0x7f7c4800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:55.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.290+0000 7f7c4d7fa700 1 -- 172.21.15.7:0/1047792057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f7c38054d80 con 0x7f7c50101520 2024-03-16T08:57:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.484+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7c500fe9d0 con 0x7f7c50101520 2024-03-16T08:57:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.484+0000 7f7c4d7fa700 1 -- 172.21.15.7:0/1047792057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c380583a0 con 0x7f7c50101520 2024-03-16T08:57:55.486 INFO:teuthology.orchestra.run.smithi007.stdout:81604378651 2024-03-16T08:57:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f7c3c0692a0 msgr2=0x7f7c3c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f7c3c0692a0 0x7f7c3c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c48001450 tx=0x7f7c4800b040 comp rx=0 tx=0).stop 2024-03-16T08:57:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50101520 msgr2=0x7f7c500fda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50101520 0x7f7c500fda30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c38008b90 tx=0x7f7c3800af40 comp rx=0 tx=0).stop 2024-03-16T08:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 shutdown_connections 2024-03-16T08:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f7c3c0692a0 0x7f7c3c06b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c500fb4c0 0x7f7c500fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 --2- 172.21.15.7:0/1047792057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50101520 0x7f7c500fda30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 >> 172.21.15.7:0/1047792057 conn(0x7f7c50072310 msgr2=0x7f7c501021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 shutdown_connections 2024-03-16T08:57:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.486+0000 7f7c570d4700 1 -- 172.21.15.7:0/1047792057 wait complete. 2024-03-16T08:57:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.490+0000 7fa7fc385700 1 -- 172.21.15.7:0/1300838965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 msgr2=0x7fa7f40fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.490+0000 7fa7fc385700 1 --2- 172.21.15.7:0/1300838965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 0x7fa7f40fbd10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8004940 tx=0x7fa7e8009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 -- 172.21.15.7:0/1300838965 shutdown_connections 2024-03-16T08:57:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 --2- 172.21.15.7:0/1300838965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 0x7fa7f40fbd10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 --2- 172.21.15.7:0/1300838965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7f4101d00 0x7fa7f41020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 -- 172.21.15.7:0/1300838965 >> 172.21.15.7:0/1300838965 conn(0x7fa7f40f7830 msgr2=0x7fa7f40f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 -- 172.21.15.7:0/1300838965 shutdown_connections 2024-03-16T08:57:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 -- 172.21.15.7:0/1300838965 wait complete. 2024-03-16T08:57:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 Processor -- start 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 -- start start 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7f4101d00 0x7fa7f4115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 0x7fa7f41160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4116730 con 0x7fa7f41026a0 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7fc385700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f410fd10 con 0x7fa7f4101d00 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.491+0000 7fa7f9920700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 0x7fa7f41160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f9920700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 0x7fa7f41160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47884/0 (socket says 172.21.15.7:47884) 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f9920700 1 -- 172.21.15.7:0/12589818 learned_addr learned my addr 172.21.15.7:0/12589818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7fa121700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7f4101d00 0x7fa7f4115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f9920700 1 -- 172.21.15.7:0/12589818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7f4101d00 msgr2=0x7fa7f4115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f9920700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7f4101d00 0x7fa7f4115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f9920700 1 -- 172.21.15.7:0/12589818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e8005040 con 0x7fa7f41026a0 2024-03-16T08:57:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f9920700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 0x7fa7f41160e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009100 tx=0x7fa7e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f2ffd700 1 -- 172.21.15.7:0/12589818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7e800e040 con 0x7fa7f41026a0 2024-03-16T08:57:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f2ffd700 1 -- 172.21.15.7:0/12589818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7e8009af0 con 0x7fa7f41026a0 2024-03-16T08:57:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f410ff90 con 0x7fa7f41026a0 2024-03-16T08:57:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7f2ffd700 1 -- 172.21.15.7:0/12589818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7e801c3f0 con 0x7fa7f41026a0 2024-03-16T08:57:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.492+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f4110550 con 0x7fa7f41026a0 2024-03-16T08:57:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.494+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f40fd450 con 0x7fa7f41026a0 2024-03-16T08:57:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.494+0000 7fa7f2ffd700 1 -- 172.21.15.7:0/12589818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fa7e800baf0 con 0x7fa7f41026a0 2024-03-16T08:57:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.494+0000 7fa7f2ffd700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa7dc0692a0 0x7fa7dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.494+0000 7fa7f2ffd700 1 -- 172.21.15.7:0/12589818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa7e8087fc0 con 0x7fa7f41026a0 2024-03-16T08:57:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.494+0000 7fa7fa121700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa7dc0692a0 0x7fa7dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.495+0000 7fa7fa121700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa7dc0692a0 0x7fa7dc06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40041d0 tx=0x7fa7e40043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:55.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.499+0000 7fa7f2ffd700 1 -- 172.21.15.7:0/12589818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fa7e8058160 con 0x7fa7f41026a0 2024-03-16T08:57:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.693+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa7f404c710 con 0x7fa7f41026a0 2024-03-16T08:57:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.694+0000 7fa7f2ffd700 1 -- 172.21.15.7:0/12589818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa7e8057cf0 con 0x7fa7f41026a0 2024-03-16T08:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:167503724557 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa7dc0692a0 msgr2=0x7fa7dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa7dc0692a0 0x7fa7dc06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40041d0 tx=0x7fa7e40043c0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 msgr2=0x7fa7f41160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 0x7fa7f41160e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009100 tx=0x7fa7e8009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 shutdown_connections 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa7dc0692a0 0x7fa7dc06b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7f4101d00 0x7fa7f4115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 --2- 172.21.15.7:0/12589818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f41026a0 0x7fa7f41160e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 >> 172.21.15.7:0/12589818 conn(0x7fa7f40f7830 msgr2=0x7fa7f40f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 shutdown_connections 2024-03-16T08:57:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:55.696+0000 7fa7fc385700 1 -- 172.21.15.7:0/12589818 wait complete. 2024-03-16T08:57:55.842 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:55 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1047792057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T08:57:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:55 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1047792057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T08:57:56.600 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:56 smithi007 ceph-mon[29470]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:56.600 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:56 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/12589818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T08:57:56.600 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:57:56.962 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378650 got 81604378651 for osd.2 2024-03-16T08:57:56.962 DEBUG:teuthology.parallel:result is None 2024-03-16T08:57:56.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:56 smithi019 ceph-mon[36407]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:56.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:56 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/12589818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T08:57:56.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:57:57.484 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705703 for osd.0 2024-03-16T08:57:57.485 DEBUG:teuthology.parallel:result is None 2024-03-16T08:57:57.902 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724557 for osd.5 2024-03-16T08:57:57.902 DEBUG:teuthology.parallel:result is None 2024-03-16T08:57:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 -- 172.21.15.7:0/1335740082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 msgr2=0x7f3f180fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 --2- 172.21.15.7:0/1335740082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 0x7f3f180fe250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004940 tx=0x7f3f08009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 -- 172.21.15.7:0/1335740082 shutdown_connections 2024-03-16T08:57:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 --2- 172.21.15.7:0/1335740082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f180fe820 0x7f3f18106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 --2- 172.21.15.7:0/1335740082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 0x7f3f180fe250 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 -- 172.21.15.7:0/1335740082 >> 172.21.15.7:0/1335740082 conn(0x7f3f180f7870 msgr2=0x7f3f180f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 -- 172.21.15.7:0/1335740082 shutdown_connections 2024-03-16T08:57:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 -- 172.21.15.7:0/1335740082 wait complete. 2024-03-16T08:57:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 Processor -- start 2024-03-16T08:57:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.346+0000 7f3f1d35d700 1 -- start start 2024-03-16T08:57:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f1d35d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 0x7f3f18113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f1d35d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f180fe820 0x7f3f18113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f1d35d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f181144a0 con 0x7f3f180fde80 2024-03-16T08:57:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f1d35d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1810daa0 con 0x7f3f180fe820 2024-03-16T08:57:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 0x7f3f18113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 0x7f3f18113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47904/0 (socket says 172.21.15.7:47904) 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f16ffd700 1 -- 172.21.15.7:0/410686832 learned_addr learned my addr 172.21.15.7:0/410686832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f167fc700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f180fe820 0x7f3f18113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f16ffd700 1 -- 172.21.15.7:0/410686832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f180fe820 msgr2=0x7f3f18113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f16ffd700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f180fe820 0x7f3f18113e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f16ffd700 1 -- 172.21.15.7:0/410686832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08005040 con 0x7f3f180fde80 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.347+0000 7f3f16ffd700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 0x7f3f18113910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009380 tx=0x7f3f08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.348+0000 7f3eff7fe700 1 -- 172.21.15.7:0/410686832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0800e040 con 0x7f3f180fde80 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.348+0000 7f3eff7fe700 1 -- 172.21.15.7:0/410686832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f08009af0 con 0x7f3f180fde80 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.348+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1810dd80 con 0x7f3f180fde80 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.348+0000 7f3eff7fe700 1 -- 172.21.15.7:0/410686832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0801c3f0 con 0x7f3f180fde80 2024-03-16T08:57:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.348+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1810e2c0 con 0x7f3f180fde80 2024-03-16T08:57:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.349+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f18066b10 con 0x7f3f180fde80 2024-03-16T08:57:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.349+0000 7f3eff7fe700 1 -- 172.21.15.7:0/410686832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f3f08013420 con 0x7f3f180fde80 2024-03-16T08:57:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.350+0000 7f3eff7fe700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3f000692a0 0x7f3f0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:57:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.350+0000 7f3eff7fe700 1 -- 172.21.15.7:0/410686832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3f08088100 con 0x7f3f180fde80 2024-03-16T08:57:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.350+0000 7f3f167fc700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3f000692a0 0x7f3f0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:57:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.350+0000 7f3f167fc700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3f000692a0 0x7f3f0006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f1810ec60 tx=0x7f3f0c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:57:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.354+0000 7f3eff7fe700 1 -- 172.21.15.7:0/410686832 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f3f08058200 con 0x7f3f180fde80 2024-03-16T08:57:58.549 INFO:teuthology.orchestra.run.smithi007.stdout:60129542177 2024-03-16T08:57:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.546+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3f1810efc0 con 0x7f3f180fde80 2024-03-16T08:57:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.547+0000 7f3eff7fe700 1 -- 172.21.15.7:0/410686832 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f08057d90 con 0x7f3f180fde80 2024-03-16T08:57:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3f000692a0 msgr2=0x7f3f0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3f000692a0 0x7f3f0006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f1810ec60 tx=0x7f3f0c00b040 comp rx=0 tx=0).stop 2024-03-16T08:57:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 msgr2=0x7f3f18113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:57:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 0x7f3f18113910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009380 tx=0x7f3f08009e30 comp rx=0 tx=0).stop 2024-03-16T08:57:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 shutdown_connections 2024-03-16T08:57:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3f000692a0 0x7f3f0006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f180fde80 0x7f3f18113910 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 --2- 172.21.15.7:0/410686832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f180fe820 0x7f3f18113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:57:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 >> 172.21.15.7:0/410686832 conn(0x7f3f180f7870 msgr2=0x7f3f180f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:57:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 shutdown_connections 2024-03-16T08:57:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:57:58.550+0000 7f3f1d35d700 1 -- 172.21.15.7:0/410686832 wait complete. 2024-03-16T08:57:58.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:58 smithi007 ceph-mon[29470]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:58.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:57:58 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/410686832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T08:57:58.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:58 smithi019 ceph-mon[36407]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:57:58.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:57:58 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/410686832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T08:57:59.323 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542177 for osd.1 2024-03-16T08:57:59.323 DEBUG:teuthology.parallel:result is None 2024-03-16T08:57:59.323 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-16T08:57:59.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T08:57:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:00.893 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:00 smithi007 ceph-mon[29470]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:00.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:00 smithi019 ceph-mon[36407]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.259+0000 7f9302911700 1 -- 172.21.15.7:0/1065990134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070c10 msgr2=0x7f92fc109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.259+0000 7f9302911700 1 --2- 172.21.15.7:0/1065990134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070c10 0x7f92fc109170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92ec004970 tx=0x7f92ec009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 -- 172.21.15.7:0/1065990134 shutdown_connections 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 --2- 172.21.15.7:0/1065990134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070c10 0x7f92fc109170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 --2- 172.21.15.7:0/1065990134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92fc070300 0x7f92fc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 -- 172.21.15.7:0/1065990134 >> 172.21.15.7:0/1065990134 conn(0x7f92fc0f89b0 msgr2=0x7f92fc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 -- 172.21.15.7:0/1065990134 shutdown_connections 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 -- 172.21.15.7:0/1065990134 wait complete. 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 Processor -- start 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 -- start start 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070300 0x7f92fc115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92fc070c10 0x7f92fc1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:01.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc116720 con 0x7f92fc070300 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.260+0000 7f9302911700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc10fd00 con 0x7f92fc070c10 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070300 0x7f92fc115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070300 0x7f92fc115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47918/0 (socket says 172.21.15.7:47918) 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92fbfff700 1 -- 172.21.15.7:0/218492346 learned_addr learned my addr 172.21.15.7:0/218492346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92fb7fe700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92fc070c10 0x7f92fc1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92fbfff700 1 -- 172.21.15.7:0/218492346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92fc070c10 msgr2=0x7f92fc1160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92fbfff700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92fc070c10 0x7f92fc1160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92fbfff700 1 -- 172.21.15.7:0/218492346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec005040 con 0x7f92fc070300 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92fbfff700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070300 0x7f92fc115b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f92e4002a50 tx=0x7f92e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92f8ff9700 1 -- 172.21.15.7:0/218492346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e4007cc0 con 0x7f92fc070300 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92f8ff9700 1 -- 172.21.15.7:0/218492346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92e400f040 con 0x7f92fc070300 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc10ffe0 con 0x7f92fc070300 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f92f8ff9700 1 -- 172.21.15.7:0/218492346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e4009640 con 0x7f92fc070300 2024-03-16T08:58:01.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.261+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc1104f0 con 0x7f92fc070300 2024-03-16T08:58:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.262+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92fc071a40 con 0x7f92fc070300 2024-03-16T08:58:01.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.263+0000 7f92f8ff9700 1 -- 172.21.15.7:0/218492346 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f92e40097a0 con 0x7f92fc070300 2024-03-16T08:58:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.263+0000 7f92f8ff9700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f92e80692a0 0x7f92e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.263+0000 7f92f8ff9700 1 -- 172.21.15.7:0/218492346 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f92e4086b70 con 0x7f92fc070300 2024-03-16T08:58:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.263+0000 7f92fb7fe700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f92e80692a0 0x7f92e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:01.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.263+0000 7f92fb7fe700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f92e80692a0 0x7f92e806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92ec004f50 tx=0x7f92ec009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:01.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.267+0000 7f92f8ff9700 1 -- 172.21.15.7:0/218492346 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f92e4056c70 con 0x7f92fc070300 2024-03-16T08:58:01.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.455+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92fc1111e0 con 0x7f92e80692a0 2024-03-16T08:58:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.456+0000 7f92f8ff9700 1 -- 172.21.15.7:0/218492346 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28190 (secure 0 0 0) 0x7f92fc1111e0 con 0x7f92e80692a0 2024-03-16T08:58:01.459 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f92e80692a0 msgr2=0x7f92e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f92e80692a0 0x7f92e806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92ec004f50 tx=0x7f92ec009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070300 msgr2=0x7f92fc115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070300 0x7f92fc115b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f92e4002a50 tx=0x7f92e40046b0 comp rx=0 tx=0).stop 2024-03-16T08:58:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 shutdown_connections 2024-03-16T08:58:01.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f92e80692a0 0x7f92e806b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92fc070300 0x7f92fc115b90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 --2- 172.21.15.7:0/218492346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92fc070c10 0x7f92fc1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 >> 172.21.15.7:0/218492346 conn(0x7f92fc0f89b0 msgr2=0x7f92fc0f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 shutdown_connections 2024-03-16T08:58:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:01.459+0000 7f9302911700 1 -- 172.21.15.7:0/218492346 wait complete. 2024-03-16T08:58:01.462 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-16T08:58:02.061 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-03-16T08:58:01.360835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787600,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683312,"statfs":{"total":575970213888,"available":574139711488,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002229"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-16T08:57:07.418387+0000","last_change":"2024-03-16T08:56:42.442751+0000","last_active":"2024-03-16T08:57:07.418387+0000","last_peered":"2024-03-16T08:57:07.418387+0000","last_clean":"2024-03-16T08:57:07.418387+0000","last_became_active":"2024-03-16T08:56:42.442495+0000","last_became_peered":"2024-03-16T08:56:42.442495+0000","last_unstale":"2024-03-16T08:57:07.418387+0000","last_undegraded":"2024-03-16T08:57:07.418387+0000","last_fullsized":"2024-03-16T08:57:07.418387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:55:53.946307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:55:53.946307+0000","last_clean_scrub_stamp":"2024-03-16T08:55:53.946307+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:23.165056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69799999999999995}]},{"osd":1,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.56399999999999995}]},{"osd":2,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.36499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 08:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.374}]},{"osd":1,"last update":"Sat Mar 16 08:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.438}]},{"osd":2,"last update":"Sat Mar 16 08:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.51200000000000001}]},{"osd":4,"last update":"Sat Mar 16 08:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.312}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447512,"statfs":{"total":95995035648,"available":95690252288,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 08:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.252,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.33800000000000002}]},{"osd":1,"last update":"Sat Mar 16 08:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Mar 16 08:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.625}]},{"osd":4,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 16 08:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32400000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.253},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33800000000000002},"min":{"1min":0.25800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.28599999999999998}]},{"osd":2,"last update":"Sat Mar 16 08:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.373}]},{"osd":3,"last update":"Sat Mar 16 08:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.501}]},{"osd":4,"last update":"Sat Mar 16 08:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 08:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32000000000000001},"min":{"1min":0.24199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.37,"5min":0.34399999999999997,"15min":0.34000000000000002},"min":{"1min":0.26900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.28999999999999998}]},{"osd":2,"last update":"Sat Mar 16 08:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34599999999999997,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.314},{"interface":"front","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.40400000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.33100000000000002}]},{"osd":3,"last update":"Sat Mar 16 08:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.498},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60399999999999998}]},{"osd":4,"last update":"Sat Mar 16 08:57:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:58:02.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T08:58:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:02.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:02 smithi007 ceph-mon[29470]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:02.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:02 smithi007 ceph-mon[29470]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:58:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:02 smithi019 ceph-mon[36407]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:02 smithi019 ceph-mon[36407]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 -- 172.21.15.7:0/71711355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fe7f0 msgr2=0x7f2784106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 --2- 172.21.15.7:0/71711355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fe7f0 0x7f2784106ec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2778004940 tx=0x7f2778009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 -- 172.21.15.7:0/71711355 shutdown_connections 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 --2- 172.21.15.7:0/71711355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fe7f0 0x7f2784106ec0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 --2- 172.21.15.7:0/71711355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840fde50 0x7f27840fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 -- 172.21.15.7:0/71711355 >> 172.21.15.7:0/71711355 conn(0x7f27840f7870 msgr2=0x7f27840f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 -- 172.21.15.7:0/71711355 shutdown_connections 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 -- 172.21.15.7:0/71711355 wait complete. 2024-03-16T08:58:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.020+0000 7f2789fae700 1 Processor -- start 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f2789fae700 1 -- start start 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f2789fae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fde50 0x7f2784113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f2789fae700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840fe7f0 0x7f2784113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f2789fae700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2784114500 con 0x7f27840fde50 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f2789fae700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278410db20 con 0x7f27840fe7f0 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f27837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fde50 0x7f2784113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f27837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fde50 0x7f2784113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52006/0 (socket says 172.21.15.7:52006) 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f27837fe700 1 -- 172.21.15.7:0/1978568818 learned_addr learned my addr 172.21.15.7:0/1978568818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f2782ffd700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840fe7f0 0x7f2784113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f27837fe700 1 -- 172.21.15.7:0/1978568818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840fe7f0 msgr2=0x7f2784113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f27837fe700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840fe7f0 0x7f2784113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f27837fe700 1 -- 172.21.15.7:0/1978568818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778005040 con 0x7f27840fde50 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f27837fe700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fde50 0x7f2784113970 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f277400c8c0 tx=0x7f27740044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.021+0000 7f276bfff700 1 -- 172.21.15.7:0/1978568818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277400a980 con 0x7f27840fde50 2024-03-16T08:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.022+0000 7f276bfff700 1 -- 172.21.15.7:0/1978568818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f277400ce60 con 0x7f27840fde50 2024-03-16T08:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.022+0000 7f276bfff700 1 -- 172.21.15.7:0/1978568818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27740115e0 con 0x7f27840fde50 2024-03-16T08:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.022+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278410de00 con 0x7f27840fde50 2024-03-16T08:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.022+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278410e340 con 0x7f27840fde50 2024-03-16T08:58:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.023+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2784066ab0 con 0x7f27840fde50 2024-03-16T08:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.023+0000 7f276bfff700 1 -- 172.21.15.7:0/1978568818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f2774012400 con 0x7f27840fde50 2024-03-16T08:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.023+0000 7f276bfff700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f276c0692a0 0x7f276c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.024+0000 7f276bfff700 1 -- 172.21.15.7:0/1978568818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2774086e50 con 0x7f27840fde50 2024-03-16T08:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.024+0000 7f2782ffd700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f276c0692a0 0x7f276c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.024+0000 7f2782ffd700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f276c0692a0 0x7f276c06b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f278410f210 tx=0x7f2778009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:04.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.028+0000 7f276bfff700 1 -- 172.21.15.7:0/1978568818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f2774053560 con 0x7f27840fde50 2024-03-16T08:58:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.215+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f278410efc0 con 0x7f276c0692a0 2024-03-16T08:58:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.217+0000 7f276bfff700 1 -- 172.21.15.7:0/1978568818 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28176 (secure 0 0 0) 0x7f278410efc0 con 0x7f276c0692a0 2024-03-16T08:58:04.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f276c0692a0 msgr2=0x7f276c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f276c0692a0 0x7f276c06b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f278410f210 tx=0x7f2778009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fde50 msgr2=0x7f2784113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fde50 0x7f2784113970 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f277400c8c0 tx=0x7f27740044c0 comp rx=0 tx=0).stop 2024-03-16T08:58:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 shutdown_connections 2024-03-16T08:58:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f276c0692a0 0x7f276c06b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27840fde50 0x7f2784113970 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 --2- 172.21.15.7:0/1978568818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27840fe7f0 0x7f2784113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 >> 172.21.15.7:0/1978568818 conn(0x7f27840f7870 msgr2=0x7f27840f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 shutdown_connections 2024-03-16T08:58:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:04.219+0000 7f2789fae700 1 -- 172.21.15.7:0/1978568818 wait complete. 2024-03-16T08:58:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-16T08:58:04.901 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:04 smithi007 ceph-mon[29470]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:04.926 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-03-16T08:58:03.361123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787600,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683312,"statfs":{"total":575970213888,"available":574139711488,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002014"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-16T08:57:07.418387+0000","last_change":"2024-03-16T08:56:42.442751+0000","last_active":"2024-03-16T08:57:07.418387+0000","last_peered":"2024-03-16T08:57:07.418387+0000","last_clean":"2024-03-16T08:57:07.418387+0000","last_became_active":"2024-03-16T08:56:42.442495+0000","last_became_peered":"2024-03-16T08:56:42.442495+0000","last_unstale":"2024-03-16T08:57:07.418387+0000","last_undegraded":"2024-03-16T08:57:07.418387+0000","last_fullsized":"2024-03-16T08:57:07.418387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T08:55:53.946307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T08:55:53.946307+0000","last_clean_scrub_stamp":"2024-03-16T08:55:53.946307+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:40:23.165056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.443}]},{"osd":1,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.47199999999999998}]},{"osd":2,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 08:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.501}]},{"osd":1,"last update":"Sat Mar 16 08:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.53900000000000003}]},{"osd":2,"last update":"Sat Mar 16 08:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52300000000000002}]},{"osd":4,"last update":"Sat Mar 16 08:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447512,"statfs":{"total":95995035648,"available":95690252288,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 08:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.252,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.33800000000000002}]},{"osd":1,"last update":"Sat Mar 16 08:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.499,"5min":0.499,"15min":0.499},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.376},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Mar 16 08:57:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.625}]},{"osd":4,"last update":"Sat Mar 16 08:57:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 16 08:57:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32400000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33800000000000002},"min":{"1min":0.25800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.34499999999999997}]},{"osd":2,"last update":"Sat Mar 16 08:57:58 2024","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.38900000000000001,"15min":0.40200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.40699999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.318},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.41799999999999998,"15min":0.434},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.42699999999999999,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.27100000000000002}]},{"osd":3,"last update":"Sat Mar 16 08:57:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.64900000000000002}]},{"osd":4,"last update":"Sat Mar 16 08:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.499},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 08:57:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32000000000000001},"min":{"1min":0.24199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.37,"5min":0.34399999999999997,"15min":0.34000000000000002},"min":{"1min":0.26900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.28999999999999998}]},{"osd":2,"last update":"Sat Mar 16 08:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34599999999999997,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.314},{"interface":"front","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.40400000000000003},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.33100000000000002}]},{"osd":3,"last update":"Sat Mar 16 08:57:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.498},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60399999999999998}]},{"osd":4,"last update":"Sat Mar 16 08:57:43 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T08:58:04.927 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-16T08:58:04.927 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-16T08:58:04.927 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-16T08:58:04.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-16T08:58:04.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:04 smithi019 ceph-mon[36407]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:05.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:05 smithi019 ceph-mon[36407]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:58:05.997 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:05 smithi007 ceph-mon[29470]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T08:58:06.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:06 smithi007 ceph-mon[29470]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:06.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:06 smithi019 ceph-mon[36407]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.992+0000 7f3db327c700 1 -- 172.21.15.7:0/3277278007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 msgr2=0x7f3dac0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.992+0000 7f3db327c700 1 --2- 172.21.15.7:0/3277278007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 0x7f3dac0fce10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3da0004940 tx=0x7f3da0009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.992+0000 7f3db327c700 1 -- 172.21.15.7:0/3277278007 shutdown_connections 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.992+0000 7f3db327c700 1 --2- 172.21.15.7:0/3277278007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 0x7f3dac0fce10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.992+0000 7f3db327c700 1 --2- 172.21.15.7:0/3277278007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dac102e00 0x7f3dac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.992+0000 7f3db327c700 1 -- 172.21.15.7:0/3277278007 >> 172.21.15.7:0/3277278007 conn(0x7f3dac0f89b0 msgr2=0x7f3dac0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.992+0000 7f3db327c700 1 -- 172.21.15.7:0/3277278007 shutdown_connections 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.992+0000 7f3db327c700 1 -- 172.21.15.7:0/3277278007 wait complete. 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db327c700 1 Processor -- start 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db327c700 1 -- start start 2024-03-16T08:58:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db327c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dac102e00 0x7f3dac113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db327c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 0x7f3dac113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db327c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac1145b0 con 0x7f3dac1037a0 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db327c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac10db40 con 0x7f3dac102e00 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db0817700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 0x7f3dac113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db0817700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 0x7f3dac113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52014/0 (socket says 172.21.15.7:52014) 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db0817700 1 -- 172.21.15.7:0/3355031865 learned_addr learned my addr 172.21.15.7:0/3355031865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db1018700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dac102e00 0x7f3dac113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db0817700 1 -- 172.21.15.7:0/3355031865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dac102e00 msgr2=0x7f3dac113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.993+0000 7f3db0817700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dac102e00 0x7f3dac113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.994+0000 7f3db0817700 1 -- 172.21.15.7:0/3355031865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da0005040 con 0x7f3dac1037a0 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.994+0000 7f3db0817700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 0x7f3dac113ed0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009100 tx=0x7f3da0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.994+0000 7f3da9ffb700 1 -- 172.21.15.7:0/3355031865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da000e040 con 0x7f3dac1037a0 2024-03-16T08:58:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.994+0000 7f3da9ffb700 1 -- 172.21.15.7:0/3355031865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3da0013420 con 0x7f3dac1037a0 2024-03-16T08:58:06.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.994+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac10ddc0 con 0x7f3dac1037a0 2024-03-16T08:58:06.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.994+0000 7f3da9ffb700 1 -- 172.21.15.7:0/3355031865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da001c610 con 0x7f3dac1037a0 2024-03-16T08:58:06.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.994+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac10e380 con 0x7f3dac1037a0 2024-03-16T08:58:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.995+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac100110 con 0x7f3dac1037a0 2024-03-16T08:58:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.995+0000 7f3da9ffb700 1 -- 172.21.15.7:0/3355031865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f3da000baf0 con 0x7f3dac1037a0 2024-03-16T08:58:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.996+0000 7f3da9ffb700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3d940692a0 0x7f3d9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.996+0000 7f3da9ffb700 1 -- 172.21.15.7:0/3355031865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3da00880d0 con 0x7f3dac1037a0 2024-03-16T08:58:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.996+0000 7f3db1018700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3d940692a0 0x7f3d9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:06.996+0000 7f3db1018700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3d940692a0 0x7f3d9406b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009f50 tx=0x7f3d9c009590 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.000+0000 7f3da9ffb700 1 -- 172.21.15.7:0/3355031865 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f3da0054740 con 0x7f3dac1037a0 2024-03-16T08:58:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.236+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3dac10efe0 con 0x7f3dac1037a0 2024-03-16T08:58:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.236+0000 7f3da9ffb700 1 -- 172.21.15.7:0/3355031865 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3da0057d60 con 0x7f3dac1037a0 2024-03-16T08:58:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:07.239 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-16T08:58:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3d940692a0 msgr2=0x7f3d9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3d940692a0 0x7f3d9406b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009f50 tx=0x7f3d9c009590 comp rx=0 tx=0).stop 2024-03-16T08:58:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 msgr2=0x7f3dac113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 0x7f3dac113ed0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009100 tx=0x7f3da0009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 shutdown_connections 2024-03-16T08:58:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3d940692a0 0x7f3d9406b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3dac102e00 0x7f3dac113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 --2- 172.21.15.7:0/3355031865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1037a0 0x7f3dac113ed0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:07.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 >> 172.21.15.7:0/3355031865 conn(0x7f3dac0f89b0 msgr2=0x7f3dac0fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:07.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 shutdown_connections 2024-03-16T08:58:07.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:07.239+0000 7f3db327c700 1 -- 172.21.15.7:0/3355031865 wait complete. 2024-03-16T08:58:07.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3355031865' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T08:58:07.965 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:07 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3355031865' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T08:58:07.995 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-16T08:58:07.995 INFO:tasks.cephadm:Setup complete, yielding 2024-03-16T08:58:07.996 INFO:teuthology.run_tasks:Running task print... 2024-03-16T08:58:08.005 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-16T08:58:08.006 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:58:08.062 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:58:08.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-16T08:58:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:08.718 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:08 smithi007 ceph-mon[29470]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:08.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:08 smithi019 ceph-mon[36407]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.038+0000 7fda142ed700 1 -- 172.21.15.7:0/2784667760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 msgr2=0x7fda0c0fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.038+0000 7fda142ed700 1 --2- 172.21.15.7:0/2784667760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 0x7fda0c0fe4e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc004940 tx=0x7fd9fc009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.038+0000 7fda142ed700 1 -- 172.21.15.7:0/2784667760 shutdown_connections 2024-03-16T08:58:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.038+0000 7fda142ed700 1 --2- 172.21.15.7:0/2784667760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 0x7fda0c0fe4e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.038+0000 7fda142ed700 1 --2- 172.21.15.7:0/2784667760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda0c0fcf00 0x7fda0c0fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.038+0000 7fda142ed700 1 -- 172.21.15.7:0/2784667760 >> 172.21.15.7:0/2784667760 conn(0x7fda0c0f85a0 msgr2=0x7fda0c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.038+0000 7fda142ed700 1 -- 172.21.15.7:0/2784667760 shutdown_connections 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.038+0000 7fda142ed700 1 -- 172.21.15.7:0/2784667760 wait complete. 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda142ed700 1 Processor -- start 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda142ed700 1 -- start start 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda142ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda0c0fcf00 0x7fda0c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda142ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 0x7fda0c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda142ed700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0708c0 con 0x7fda0c0fe070 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda142ed700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c070a30 con 0x7fda0c0fcf00 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda11888700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 0x7fda0c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda11888700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 0x7fda0c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52032/0 (socket says 172.21.15.7:52032) 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda11888700 1 -- 172.21.15.7:0/1319467438 learned_addr learned my addr 172.21.15.7:0/1319467438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda12089700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda0c0fcf00 0x7fda0c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda11888700 1 -- 172.21.15.7:0/1319467438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda0c0fcf00 msgr2=0x7fda0c071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda11888700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda0c0fcf00 0x7fda0c071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda11888700 1 -- 172.21.15.7:0/1319467438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc005040 con 0x7fda0c0fe070 2024-03-16T08:58:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.039+0000 7fda11888700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 0x7fda0c070310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc013870 tx=0x7fd9fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.040+0000 7fda02ffd700 1 -- 172.21.15.7:0/1319467438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9fc021070 con 0x7fda0c0fe070 2024-03-16T08:58:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.040+0000 7fda02ffd700 1 -- 172.21.15.7:0/1319467438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9fc009430 con 0x7fda0c0fe070 2024-03-16T08:58:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.040+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c070cb0 con 0x7fda0c0fe070 2024-03-16T08:58:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.040+0000 7fda02ffd700 1 -- 172.21.15.7:0/1319467438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9fc00e070 con 0x7fda0c0fe070 2024-03-16T08:58:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.040+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c115a00 con 0x7fda0c0fe070 2024-03-16T08:58:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.041+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0c05f5f0 con 0x7fda0c0fe070 2024-03-16T08:58:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.041+0000 7fda02ffd700 1 -- 172.21.15.7:0/1319467438 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fd9fc01c750 con 0x7fda0c0fe070 2024-03-16T08:58:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.042+0000 7fda02ffd700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd9f8069230 0x7fd9f806b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.042+0000 7fda02ffd700 1 -- 172.21.15.7:0/1319467438 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd9fc088110 con 0x7fda0c0fe070 2024-03-16T08:58:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.042+0000 7fda12089700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd9f8069230 0x7fd9f806b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:10.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.042+0000 7fda12089700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd9f8069230 0x7fd9f806b6e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fda0c071bd0 tx=0x7fda0800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.046+0000 7fda02ffd700 1 -- 172.21.15.7:0/1319467438 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fd9fc054780 con 0x7fda0c0fe070 2024-03-16T08:58:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.297+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fda0c04bed0 con 0x7fda0c0fe070 2024-03-16T08:58:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.301+0000 7fda02ffd700 1 -- 172.21.15.7:0/1319467438 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd9fc057da0 con 0x7fda0c0fe070 2024-03-16T08:58:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd9f8069230 msgr2=0x7fd9f806b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd9f8069230 0x7fd9f806b6e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fda0c071bd0 tx=0x7fda0800b040 comp rx=0 tx=0).stop 2024-03-16T08:58:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 msgr2=0x7fda0c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 0x7fda0c070310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc013870 tx=0x7fd9fc009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 shutdown_connections 2024-03-16T08:58:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd9f8069230 0x7fd9f806b6e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda0c0fcf00 0x7fda0c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 --2- 172.21.15.7:0/1319467438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c0fe070 0x7fda0c070310 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 >> 172.21.15.7:0/1319467438 conn(0x7fda0c0f85a0 msgr2=0x7fda0c0f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 shutdown_connections 2024-03-16T08:58:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:10.303+0000 7fda142ed700 1 -- 172.21.15.7:0/1319467438 wait complete. 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1319467438' entity='client.admin' 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:10.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:10 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1319467438' entity='client.admin' 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:10.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:10 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:11.060 INFO:teuthology.run_tasks:Running task print... 2024-03-16T08:58:11.070 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-16T08:58:11.070 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:58:11.150 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:58:11.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-16T08:58:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:11.891 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:11 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:58:11.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:11 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:58:12.805 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:12 smithi007 ceph-mon[29470]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:12.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:12 smithi019 ceph-mon[36407]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 -- 172.21.15.7:0/2613851510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fbcd0 msgr2=0x7f5e980fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 --2- 172.21.15.7:0/2613851510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fbcd0 0x7f5e980fc0e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c004940 tx=0x7f5e8c009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 -- 172.21.15.7:0/2613851510 shutdown_connections 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 --2- 172.21.15.7:0/2613851510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e980fcf80 0x7f5e980fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 --2- 172.21.15.7:0/2613851510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fbcd0 0x7f5e980fc0e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 -- 172.21.15.7:0/2613851510 >> 172.21.15.7:0/2613851510 conn(0x7f5e980f7450 msgr2=0x7f5e980f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 -- 172.21.15.7:0/2613851510 shutdown_connections 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 -- 172.21.15.7:0/2613851510 wait complete. 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 Processor -- start 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 -- start start 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e980fbcd0 0x7f5e980ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fcf80 0x7f5e980fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.048+0000 7f5e9cde8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e98100170 con 0x7f5e980fcf80 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e9cde8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980fe800 con 0x7f5e980fbcd0 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e95d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fcf80 0x7f5e980fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e95d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fcf80 0x7f5e980fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35272/0 (socket says 172.21.15.7:35272) 2024-03-16T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e95d9b700 1 -- 172.21.15.7:0/1873278920 learned_addr learned my addr 172.21.15.7:0/1873278920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e9659c700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e980fbcd0 0x7f5e980ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e95d9b700 1 -- 172.21.15.7:0/1873278920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e980fbcd0 msgr2=0x7f5e980ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e95d9b700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e980fbcd0 0x7f5e980ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e95d9b700 1 -- 172.21.15.7:0/1873278920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c005040 con 0x7f5e980fcf80 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e95d9b700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fcf80 0x7f5e980fe2c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800c8c0 tx=0x7f5e880044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e86ffd700 1 -- 172.21.15.7:0/1873278920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e8800a760 con 0x7f5e980fcf80 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e86ffd700 1 -- 172.21.15.7:0/1873278920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e8800ad90 con 0x7f5e980fcf80 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e980feae0 con 0x7f5e980fcf80 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e86ffd700 1 -- 172.21.15.7:0/1873278920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e880113a0 con 0x7f5e980fcf80 2024-03-16T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.049+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e980ff0a0 con 0x7f5e980fcf80 2024-03-16T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.050+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e98101b10 con 0x7f5e980fcf80 2024-03-16T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.051+0000 7f5e86ffd700 1 -- 172.21.15.7:0/1873278920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f5e88012400 con 0x7f5e980fcf80 2024-03-16T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.051+0000 7f5e86ffd700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5e800692a0 0x7f5e8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.051+0000 7f5e86ffd700 1 -- 172.21.15.7:0/1873278920 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5e88086de0 con 0x7f5e980fcf80 2024-03-16T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.051+0000 7f5e9659c700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5e800692a0 0x7f5e8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.051+0000 7f5e9659c700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5e800692a0 0x7f5e8006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00b970 tx=0x7f5e8c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.055+0000 7f5e86ffd700 1 -- 172.21.15.7:0/1873278920 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f5e88056f80 con 0x7f5e980fcf80 2024-03-16T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.266+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e9802b310 con 0x7f5e800692a0 2024-03-16T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.267+0000 7f5e86ffd700 1 -- 172.21.15.7:0/1873278920 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5e9802b310 con 0x7f5e800692a0 2024-03-16T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-03-16T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-03-16T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-03-16T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5e800692a0 msgr2=0x7f5e8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5e800692a0 0x7f5e8006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00b970 tx=0x7f5e8c009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fcf80 msgr2=0x7f5e980fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fcf80 0x7f5e980fe2c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800c8c0 tx=0x7f5e880044c0 comp rx=0 tx=0).stop 2024-03-16T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 shutdown_connections 2024-03-16T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f5e800692a0 0x7f5e8006b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e980fbcd0 0x7f5e980ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 --2- 172.21.15.7:0/1873278920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e980fcf80 0x7f5e980fe2c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 >> 172.21.15.7:0/1873278920 conn(0x7f5e980f7450 msgr2=0x7f5e98066810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 shutdown_connections 2024-03-16T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:13.269+0000 7f5e9cde8700 1 -- 172.21.15.7:0/1873278920 wait complete. 2024-03-16T08:58:13.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-16T08:58:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:14.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:14 smithi019 ceph-mon[36407]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:14.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:14 smithi019 ceph-mon[36407]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:14.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:14 smithi007 ceph-mon[29470]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:14.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:14 smithi007 ceph-mon[29470]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.760+0000 7f57e5679700 1 -- 172.21.15.7:0/28345059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 msgr2=0x7f57e00fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.760+0000 7f57e5679700 1 --2- 172.21.15.7:0/28345059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 0x7f57e00fc210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f57d4004940 tx=0x7f57d4009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.760+0000 7f57e5679700 1 -- 172.21.15.7:0/28345059 shutdown_connections 2024-03-16T08:58:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.760+0000 7f57e5679700 1 --2- 172.21.15.7:0/28345059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e00fcf70 0x7f57e00fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.760+0000 7f57e5679700 1 --2- 172.21.15.7:0/28345059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 0x7f57e00fc210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.760+0000 7f57e5679700 1 -- 172.21.15.7:0/28345059 >> 172.21.15.7:0/28345059 conn(0x7f57e00f74e0 msgr2=0x7f57e00f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.760+0000 7f57e5679700 1 -- 172.21.15.7:0/28345059 shutdown_connections 2024-03-16T08:58:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.760+0000 7f57e5679700 1 -- 172.21.15.7:0/28345059 wait complete. 2024-03-16T08:58:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57e5679700 1 Processor -- start 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57e5679700 1 -- start start 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57e5679700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 0x7f57e0110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57e5679700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e00fcf70 0x7f57e0112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57e5679700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0066be0 con 0x7f57e00fbe00 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57e5679700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0066d20 con 0x7f57e00fcf70 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57deffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 0x7f57e0110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57deffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 0x7f57e0110750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35298/0 (socket says 172.21.15.7:35298) 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57deffd700 1 -- 172.21.15.7:0/2852164815 learned_addr learned my addr 172.21.15.7:0/2852164815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57de7fc700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e00fcf70 0x7f57e0112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57deffd700 1 -- 172.21.15.7:0/2852164815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e00fcf70 msgr2=0x7f57e0112ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57deffd700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e00fcf70 0x7f57e0112ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57deffd700 1 -- 172.21.15.7:0/2852164815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4005040 con 0x7f57e00fbe00 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.761+0000 7f57deffd700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 0x7f57e0110750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57d400b3b0 tx=0x7f57d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.762+0000 7f57c77fe700 1 -- 172.21.15.7:0/2852164815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d400e040 con 0x7f57e00fbe00 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.762+0000 7f57c77fe700 1 -- 172.21.15.7:0/2852164815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57d400bbf0 con 0x7f57e00fbe00 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.762+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e01131e0 con 0x7f57e00fbe00 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.762+0000 7f57c77fe700 1 -- 172.21.15.7:0/2852164815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d401c630 con 0x7f57e00fbe00 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.762+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e01137a0 con 0x7f57e00fbe00 2024-03-16T08:58:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.763+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e005f5f0 con 0x7f57e00fbe00 2024-03-16T08:58:15.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.764+0000 7f57c77fe700 1 -- 172.21.15.7:0/2852164815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f57d400bd60 con 0x7f57e00fbe00 2024-03-16T08:58:15.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.764+0000 7f57c77fe700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f57c80692a0 0x7f57c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:15.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.764+0000 7f57c77fe700 1 -- 172.21.15.7:0/2852164815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f57d4087a50 con 0x7f57e00fbe00 2024-03-16T08:58:15.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.764+0000 7f57de7fc700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f57c80692a0 0x7f57c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:15.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.764+0000 7f57de7fc700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f57c80692a0 0x7f57c806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57d0004f40 tx=0x7f57d000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:15.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.768+0000 7f57c77fe700 1 -- 172.21.15.7:0/2852164815 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f57d4054160 con 0x7f57e00fbe00 2024-03-16T08:58:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.968+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f57e0101290 con 0x7f57c80692a0 2024-03-16T08:58:15.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.977+0000 7f57c77fe700 1 -- 172.21.15.7:0/2852164815 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7f57e0101290 con 0x7f57c80692a0 2024-03-16T08:58:15.979 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T08:58:15.979 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (3m) 2m ago 5m 26.6M - 0.25.0 c8568f914cd2 d5db4ce74c0d 2024-03-16T08:58:15.979 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (5m) 2m ago 5m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T08:58:15.979 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (4m) 69s ago 4m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (3m) 2m ago 4m 80.2M - 9.4.7 954c08fa6188 4f8b149e57d8 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:9283 running (6m) 2m ago 6m 472M - 17.2.7-648-g1df27247 0bdcfccf14ce 376995a75ba3 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283 running (4m) 69s ago 4m 432M - 17.2.7-648-g1df27247 0bdcfccf14ce ab3071a32d6e 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (6m) 2m ago 6m 42.4M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (4m) 69s ago 4m 41.7M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (5m) 2m ago 5m 14.3M - 1.5.0 0da6a335fe13 9ca7275ea387 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (4m) 69s ago 4m 16.8M - 1.5.0 0da6a335fe13 9edea61eb01f 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (3m) 2m ago 3m 41.3M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (2m) 2m ago 2m 38.3M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (2m) 2m ago 2m 38.0M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (2m) 69s ago 2m 42.0M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (100s) 69s ago 100s 37.0M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (75s) 69s ago 74s 13.4M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T08:58:15.980 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (3m) 2m ago 4m 37.7M - 2.43.0 a07b618ecd1d b457178ffb05 2024-03-16T08:58:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f57c80692a0 msgr2=0x7f57c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f57c80692a0 0x7f57c806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57d0004f40 tx=0x7f57d000b040 comp rx=0 tx=0).stop 2024-03-16T08:58:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 msgr2=0x7f57e0110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 0x7f57e0110750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f57d400b3b0 tx=0x7f57d4009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 shutdown_connections 2024-03-16T08:58:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f57c80692a0 0x7f57c806b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e00fbe00 0x7f57e0110750 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 --2- 172.21.15.7:0/2852164815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e00fcf70 0x7f57e0112ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 >> 172.21.15.7:0/2852164815 conn(0x7f57e00f74e0 msgr2=0x7f57e00f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 shutdown_connections 2024-03-16T08:58:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:15.979+0000 7f57e5679700 1 -- 172.21.15.7:0/2852164815 wait complete. 2024-03-16T08:58:16.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-16T08:58:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:16.932 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:16 smithi007 ceph-mon[29470]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:16.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:16 smithi019 ceph-mon[36407]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:17.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:17 smithi007 ceph-mon[29470]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:17.966 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:17 smithi019 ceph-mon[36407]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 -- 172.21.15.7:0/791531329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 msgr2=0x7f1ad40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 --2- 172.21.15.7:0/791531329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 0x7f1ad40fd3f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4004940 tx=0x7f1ac4009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 -- 172.21.15.7:0/791531329 shutdown_connections 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 --2- 172.21.15.7:0/791531329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 0x7f1ad40fd3f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 --2- 172.21.15.7:0/791531329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad40fbcd0 0x7f1ad40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 -- 172.21.15.7:0/791531329 >> 172.21.15.7:0/791531329 conn(0x7f1ad40f7430 msgr2=0x7f1ad40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 -- 172.21.15.7:0/791531329 shutdown_connections 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 -- 172.21.15.7:0/791531329 wait complete. 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 Processor -- start 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.373+0000 7f1adc63b700 1 -- start start 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1adc63b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad40fbcd0 0x7f1ad4066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1adc63b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 0x7f1ad4112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1adc63b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4112c70 con 0x7f1ad40fcf80 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1adc63b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4112db0 con 0x7f1ad40fbcd0 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1ad9bd6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 0x7f1ad4112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1ad9bd6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 0x7f1ad4112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35310/0 (socket says 172.21.15.7:35310) 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1ad9bd6700 1 -- 172.21.15.7:0/1885944015 learned_addr learned my addr 172.21.15.7:0/1885944015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1ada3d7700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad40fbcd0 0x7f1ad4066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1ad9bd6700 1 -- 172.21.15.7:0/1885944015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad40fbcd0 msgr2=0x7f1ad4066cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1ad9bd6700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad40fbcd0 0x7f1ad4066cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1ad9bd6700 1 -- 172.21.15.7:0/1885944015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac4005040 con 0x7f1ad40fcf80 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1ad9bd6700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 0x7f1ad4112730 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400b3b0 tx=0x7f1ac4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1acaffd700 1 -- 172.21.15.7:0/1885944015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac400e040 con 0x7f1ad40fcf80 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1acaffd700 1 -- 172.21.15.7:0/1885944015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ac400bbf0 con 0x7f1ad40fcf80 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad4112fb0 con 0x7f1ad40fcf80 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1acaffd700 1 -- 172.21.15.7:0/1885944015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac401c630 con 0x7f1ad40fcf80 2024-03-16T08:58:18.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.374+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad4113540 con 0x7f1ad40fcf80 2024-03-16T08:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.375+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad404c710 con 0x7f1ad40fcf80 2024-03-16T08:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.376+0000 7f1acaffd700 1 -- 172.21.15.7:0/1885944015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f1ac4013420 con 0x7f1ad40fcf80 2024-03-16T08:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.376+0000 7f1acaffd700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1ac00692a0 0x7f1ac006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.376+0000 7f1acaffd700 1 -- 172.21.15.7:0/1885944015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1ac4087b20 con 0x7f1ad40fcf80 2024-03-16T08:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.376+0000 7f1ada3d7700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1ac00692a0 0x7f1ac006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.377+0000 7f1ada3d7700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1ac00692a0 0x7f1ac006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0004f40 tx=0x7f1ad000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.381+0000 7f1acaffd700 1 -- 172.21.15.7:0/1885944015 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f1ac4057c20 con 0x7f1ad40fcf80 2024-03-16T08:58:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.581+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ad4113730 con 0x7f1ac00692a0 2024-03-16T08:58:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.585+0000 7f1acaffd700 1 -- 172.21.15.7:0/1885944015 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1152 (secure 0 0 0) 0x7f1ad4113730 con 0x7f1ac00692a0 2024-03-16T08:58:18.587 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-16T08:58:18.588 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-03-16T08:58:18.588 INFO:teuthology.orchestra.run.smithi007.stdout:crash 2/2 2m ago 5m * 2024-03-16T08:58:18.588 INFO:teuthology.orchestra.run.smithi007.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-03-16T08:58:18.588 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 2/2 2m ago 6m count:2 2024-03-16T08:58:18.588 INFO:teuthology.orchestra.run.smithi007.stdout:mon 2/2 2m ago 5m smithi007:172.21.15.7=smithi007;smithi019:172.21.15.19=smithi019;count:2 2024-03-16T08:58:18.588 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-03-16T08:58:18.588 INFO:teuthology.orchestra.run.smithi007.stdout:osd 6 2m ago - 2024-03-16T08:58:18.588 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-03-16T08:58:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.587+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1ac00692a0 msgr2=0x7f1ac006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.587+0000 7f1adc63b700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1ac00692a0 0x7f1ac006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0004f40 tx=0x7f1ad000b040 comp rx=0 tx=0).stop 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.587+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 msgr2=0x7f1ad4112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.587+0000 7f1adc63b700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 0x7f1ad4112730 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400b3b0 tx=0x7f1ac4009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.588+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 shutdown_connections 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.588+0000 7f1adc63b700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1ac00692a0 0x7f1ac006b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.588+0000 7f1adc63b700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad40fbcd0 0x7f1ad4066cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.588+0000 7f1adc63b700 1 --2- 172.21.15.7:0/1885944015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad40fcf80 0x7f1ad4112730 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.588+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 >> 172.21.15.7:0/1885944015 conn(0x7f1ad40f7430 msgr2=0x7f1ad40f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.588+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 shutdown_connections 2024-03-16T08:58:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:18.588+0000 7f1adc63b700 1 -- 172.21.15.7:0/1885944015 wait complete. 2024-03-16T08:58:18.944 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:18 smithi007 ceph-mon[29470]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:18.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:18 smithi019 ceph-mon[36407]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:19.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-16T08:58:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:19.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:19 smithi019 ceph-mon[36407]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:19.981 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:19 smithi007 ceph-mon[29470]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:20.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:20 smithi007 ceph-mon[29470]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:20.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:20 smithi019 ceph-mon[36407]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 -- 172.21.15.7:0/694035757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fce50 msgr2=0x7f9b8c0fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 --2- 172.21.15.7:0/694035757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fce50 0x7f9b8c0fd260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9b80004940 tx=0x7f9b80009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 -- 172.21.15.7:0/694035757 shutdown_connections 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 --2- 172.21.15.7:0/694035757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c0fe050 0x7f9b8c0fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 --2- 172.21.15.7:0/694035757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fce50 0x7f9b8c0fd260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 -- 172.21.15.7:0/694035757 >> 172.21.15.7:0/694035757 conn(0x7f9b8c0f8580 msgr2=0x7f9b8c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 -- 172.21.15.7:0/694035757 shutdown_connections 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 -- 172.21.15.7:0/694035757 wait complete. 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 Processor -- start 2024-03-16T08:58:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.332+0000 7f9b92e39700 1 -- start start 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b92e39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c0fce50 0x7f9b8c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b92e39700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fe050 0x7f9b8c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b92e39700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c10e4b0 con 0x7f9b8c0fe050 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b92e39700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c10e620 con 0x7f9b8c0fce50 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fe050 0x7f9b8c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fe050 0x7f9b8c10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35328/0 (socket says 172.21.15.7:35328) 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b8bfff700 1 -- 172.21.15.7:0/2730843595 learned_addr learned my addr 172.21.15.7:0/2730843595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b90bd5700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c0fce50 0x7f9b8c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b8bfff700 1 -- 172.21.15.7:0/2730843595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c0fce50 msgr2=0x7f9b8c10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b8bfff700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c0fce50 0x7f9b8c10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b8bfff700 1 -- 172.21.15.7:0/2730843595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b80005040 con 0x7f9b8c0fe050 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b8bfff700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fe050 0x7f9b8c10deb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800c8c0 tx=0x7f9b780044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b897fa700 1 -- 172.21.15.7:0/2730843595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7800a980 con 0x7f9b8c0fe050 2024-03-16T08:58:21.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b897fa700 1 -- 172.21.15.7:0/2730843595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b7800ce60 con 0x7f9b8c0fe050 2024-03-16T08:58:21.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c113080 con 0x7f9b8c0fe050 2024-03-16T08:58:21.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b897fa700 1 -- 172.21.15.7:0/2730843595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b780115e0 con 0x7f9b8c0fe050 2024-03-16T08:58:21.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.333+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c113640 con 0x7f9b8c0fe050 2024-03-16T08:58:21.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.334+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c04c710 con 0x7f9b8c0fe050 2024-03-16T08:58:21.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.335+0000 7f9b897fa700 1 -- 172.21.15.7:0/2730843595 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f9b78012400 con 0x7f9b8c0fe050 2024-03-16T08:58:21.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.335+0000 7f9b897fa700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9b7c0692a0 0x7f9b7c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:21.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.335+0000 7f9b897fa700 1 -- 172.21.15.7:0/2730843595 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9b78086e50 con 0x7f9b8c0fe050 2024-03-16T08:58:21.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.335+0000 7f9b90bd5700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9b7c0692a0 0x7f9b7c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:21.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.336+0000 7f9b90bd5700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9b7c0692a0 0x7f9b7c06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000b970 tx=0x7f9b80009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:21.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.340+0000 7f9b897fa700 1 -- 172.21.15.7:0/2730843595 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f9b78056ff0 con 0x7f9b8c0fe050 2024-03-16T08:58:21.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.537+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b8c1023c0 con 0x7f9b7c0692a0 2024-03-16T08:58:21.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.539+0000 7f9b897fa700 1 -- 172.21.15.7:0/2730843595 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f9b8c1023c0 con 0x7f9b7c0692a0 2024-03-16T08:58:21.541 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-03-16T08:58:21.541 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-03-16T08:58:21.541 INFO:teuthology.orchestra.run.smithi007.stdout:smithi019 172.21.15.19 2024-03-16T08:58:21.541 INFO:teuthology.orchestra.run.smithi007.stdout:2 hosts in cluster 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9b7c0692a0 msgr2=0x7f9b7c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9b7c0692a0 0x7f9b7c06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000b970 tx=0x7f9b80009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fe050 msgr2=0x7f9b8c10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fe050 0x7f9b8c10deb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800c8c0 tx=0x7f9b780044c0 comp rx=0 tx=0).stop 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 shutdown_connections 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f9b7c0692a0 0x7f9b7c06b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c0fce50 0x7f9b8c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 --2- 172.21.15.7:0/2730843595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0fe050 0x7f9b8c10deb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 >> 172.21.15.7:0/2730843595 conn(0x7f9b8c0f8580 msgr2=0x7f9b8c0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 shutdown_connections 2024-03-16T08:58:21.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:21.541+0000 7f9b92e39700 1 -- 172.21.15.7:0/2730843595 wait complete. 2024-03-16T08:58:22.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-16T08:58:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:22.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:22 smithi007 ceph-mon[29470]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:22.844 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:22 smithi007 ceph-mon[29470]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:22.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:22 smithi019 ceph-mon[36407]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:22.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:22 smithi019 ceph-mon[36407]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 -- 172.21.15.7:0/1735145667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 msgr2=0x7f6abc0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 --2- 172.21.15.7:0/1735145667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 0x7f6abc0fd280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6aac004940 tx=0x7f6aac009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 -- 172.21.15.7:0/1735145667 shutdown_connections 2024-03-16T08:58:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 --2- 172.21.15.7:0/1735145667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6abc0fdfe0 0x7f6abc0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 --2- 172.21.15.7:0/1735145667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 0x7f6abc0fd280 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 -- 172.21.15.7:0/1735145667 >> 172.21.15.7:0/1735145667 conn(0x7f6abc0f8550 msgr2=0x7f6abc0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 -- 172.21.15.7:0/1735145667 shutdown_connections 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 -- 172.21.15.7:0/1735145667 wait complete. 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.995+0000 7f6ac3712700 1 Processor -- start 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac3712700 1 -- start start 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac3712700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 0x7f6abc10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac3712700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6abc0fdfe0 0x7f6abc10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac3712700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc10e490 con 0x7f6abc0fce70 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac3712700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc10e600 con 0x7f6abc0fdfe0 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac14ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 0x7f6abc10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac14ae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 0x7f6abc10d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57938/0 (socket says 172.21.15.7:57938) 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac14ae700 1 -- 172.21.15.7:0/505128751 learned_addr learned my addr 172.21.15.7:0/505128751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac0cad700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6abc0fdfe0 0x7f6abc10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac14ae700 1 -- 172.21.15.7:0/505128751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6abc0fdfe0 msgr2=0x7f6abc10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac14ae700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6abc0fdfe0 0x7f6abc10de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac14ae700 1 -- 172.21.15.7:0/505128751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac005040 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac14ae700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 0x7f6abc10d950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00b3b0 tx=0x7f6aac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ab1ffb700 1 -- 172.21.15.7:0/505128751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aac00e040 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc113000 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ab1ffb700 1 -- 172.21.15.7:0/505128751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6aac00bcc0 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.996+0000 7f6ab1ffb700 1 -- 172.21.15.7:0/505128751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aac01c630 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.997+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc1135c0 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.998+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc04c710 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.998+0000 7f6ab1ffb700 1 -- 172.21.15.7:0/505128751 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6aac013980 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.998+0000 7f6ab1ffb700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6aa80692a0 0x7f6aa806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.999+0000 7f6ab1ffb700 1 -- 172.21.15.7:0/505128751 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6aac087c70 con 0x7f6abc0fce70 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.999+0000 7f6ac0cad700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6aa80692a0 0x7f6aa806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:23.999+0000 7f6ac0cad700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6aa80692a0 0x7f6aa806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8004d60 tx=0x7f6ab800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.003+0000 7f6ab1ffb700 1 -- 172.21.15.7:0/505128751 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f6aac057cf0 con 0x7f6abc0fce70 2024-03-16T08:58:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.202+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6abc102340 con 0x7f6aa80692a0 2024-03-16T08:58:24.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.204+0000 7f6ab1ffb700 1 -- 172.21.15.7:0/505128751 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f6abc102340 con 0x7f6aa80692a0 2024-03-16T08:58:24.206 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-16T08:58:24.206 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-16T08:58:24.206 INFO:teuthology.orchestra.run.smithi007.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 74s ago Has a FileSystem, LVM detected 2024-03-16T08:58:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6aa80692a0 msgr2=0x7f6aa806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6aa80692a0 0x7f6aa806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8004d60 tx=0x7f6ab800b040 comp rx=0 tx=0).stop 2024-03-16T08:58:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 msgr2=0x7f6abc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 0x7f6abc10d950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00b3b0 tx=0x7f6aac009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 shutdown_connections 2024-03-16T08:58:24.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6aa80692a0 0x7f6aa806b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc0fce70 0x7f6abc10d950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 --2- 172.21.15.7:0/505128751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6abc0fdfe0 0x7f6abc10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 >> 172.21.15.7:0/505128751 conn(0x7f6abc0f8550 msgr2=0x7f6abc0f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 shutdown_connections 2024-03-16T08:58:24.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:24.206+0000 7f6ac3712700 1 -- 172.21.15.7:0/505128751 wait complete. 2024-03-16T08:58:24.755 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:58:24.765 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:58:24.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-16T08:58:24.790 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:24 smithi007 ceph-mon[29470]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:24.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:24 smithi019 ceph-mon[36407]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:25.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:25 smithi007 ceph-mon[29470]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:25.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:25 smithi019 ceph-mon[36407]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 -- 172.21.15.7:0/2587647144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fbcd0 msgr2=0x7fd1fc0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 --2- 172.21.15.7:0/2587647144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fbcd0 0x7fd1fc0fc0e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8004940 tx=0x7fd1f8009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 -- 172.21.15.7:0/2587647144 shutdown_connections 2024-03-16T08:58:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 --2- 172.21.15.7:0/2587647144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1fc0fcf80 0x7fd1fc0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 --2- 172.21.15.7:0/2587647144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fbcd0 0x7fd1fc0fc0e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 -- 172.21.15.7:0/2587647144 >> 172.21.15.7:0/2587647144 conn(0x7fd1fc0f7450 msgr2=0x7fd1fc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 -- 172.21.15.7:0/2587647144 shutdown_connections 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 -- 172.21.15.7:0/2587647144 wait complete. 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 Processor -- start 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.603+0000 7fd203eed700 1 -- start start 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd203eed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1fc0fbcd0 0x7fd1fc066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd203eed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fcf80 0x7fd1fc1126f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd203eed700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc112c30 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd203eed700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc112d70 con 0x7fd1fc0fbcd0 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd201488700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fcf80 0x7fd1fc1126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd201488700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fcf80 0x7fd1fc1126f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57952/0 (socket says 172.21.15.7:57952) 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd201488700 1 -- 172.21.15.7:0/935327082 learned_addr learned my addr 172.21.15.7:0/935327082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd201488700 1 -- 172.21.15.7:0/935327082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1fc0fbcd0 msgr2=0x7fd1fc066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd201c89700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1fc0fbcd0 0x7fd1fc066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd201488700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1fc0fbcd0 0x7fd1fc066d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd201488700 1 -- 172.21.15.7:0/935327082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f8005040 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd201488700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fcf80 0x7fd1fc1126f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c830 tx=0x7fd1ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd1f27fc700 1 -- 172.21.15.7:0/935327082 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1ec00a760 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd1f27fc700 1 -- 172.21.15.7:0/935327082 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1ec00ad90 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc113050 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd1f27fc700 1 -- 172.21.15.7:0/935327082 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1ec0113a0 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.604+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc1135e0 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.605+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1fc04c710 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.606+0000 7fd1f27fc700 1 -- 172.21.15.7:0/935327082 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fd1ec012400 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.606+0000 7fd1f27fc700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1e80692a0 0x7fd1e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.606+0000 7fd1f27fc700 1 -- 172.21.15.7:0/935327082 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd1ec086eb0 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.606+0000 7fd201c89700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1e80692a0 0x7fd1e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.607+0000 7fd201c89700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1e80692a0 0x7fd1e806b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8000c00 tx=0x7fd1f8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.611+0000 7fd1f27fc700 1 -- 172.21.15.7:0/935327082 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fd1ec056fb0 con 0x7fd1fc0fcf80 2024-03-16T08:58:26.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:26.830+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1fc1012e0 con 0x7fd1e80692a0 2024-03-16T08:58:26.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:26 smithi019 ceph-mon[36407]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:58:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:26 smithi007 ceph-mon[29470]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:58:27.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:27 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-16T08:58:27.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:27 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-16T08:58:28.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.690+0000 7fd1f27fc700 1 -- 172.21.15.7:0/935327082 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd1fc1012e0 con 0x7fd1e80692a0 2024-03-16T08:58:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1e80692a0 msgr2=0x7fd1e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1e80692a0 0x7fd1e806b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8000c00 tx=0x7fd1f8009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fcf80 msgr2=0x7fd1fc1126f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fcf80 0x7fd1fc1126f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c830 tx=0x7fd1ec004470 comp rx=0 tx=0).stop 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 shutdown_connections 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd1e80692a0 0x7fd1e806b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1fc0fbcd0 0x7fd1fc066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 --2- 172.21.15.7:0/935327082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1fc0fcf80 0x7fd1fc1126f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.693+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 >> 172.21.15.7:0/935327082 conn(0x7fd1fc0f7450 msgr2=0x7fd1fc0f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.694+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 shutdown_connections 2024-03-16T08:58:28.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:28.694+0000 7fd203eed700 1 -- 172.21.15.7:0/935327082 wait complete. 2024-03-16T08:58:28.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:28 smithi019 ceph-mon[36407]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:28.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:28 smithi019 ceph-mon[36407]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:28.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:28 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-16T08:58:28.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:28 smithi019 ceph-mon[36407]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T08:58:28.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:28 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-16T08:58:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:28 smithi007 ceph-mon[29470]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:58:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:28 smithi007 ceph-mon[29470]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:28 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-16T08:58:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:28 smithi007 ceph-mon[29470]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T08:58:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:28 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-16T08:58:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:28 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[29466]: 2024-03-16T08:58:28.666+0000 7f9efe034700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T08:58:29.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T08:58:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-16T08:58:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T08:58:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-16T08:58:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T08:58:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T08:58:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T08:58:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-16T08:58:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: fsmap cephfs:0 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: Saving service mds.cephfs spec with placement count:4 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.bsxctp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.bsxctp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:58:29.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:29 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-16T08:58:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T08:58:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-16T08:58:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T08:58:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T08:58:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T08:58:29.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: fsmap cephfs:0 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: Saving service mds.cephfs spec with placement count:4 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.bsxctp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.bsxctp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:58:29.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:29 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:30.816 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:30.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:30 smithi007 ceph-mon[29470]: Deploying daemon mds.cephfs.smithi007.bsxctp on smithi007 2024-03-16T08:58:30.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:30 smithi007 ceph-mon[29470]: pgmap v175: 65 pgs: 9 creating+peering, 6 active+clean, 50 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:30.939 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:30 smithi007 ceph-mon[29470]: osdmap e44: 6 total, 6 up, 6 in 2024-03-16T08:58:30.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:30 smithi019 ceph-mon[36407]: Deploying daemon mds.cephfs.smithi007.bsxctp on smithi007 2024-03-16T08:58:30.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:30 smithi019 ceph-mon[36407]: pgmap v175: 65 pgs: 9 creating+peering, 6 active+clean, 50 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:30.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:30 smithi019 ceph-mon[36407]: osdmap e44: 6 total, 6 up, 6 in 2024-03-16T08:58:31.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:31 smithi019 ceph-mon[36407]: osdmap e45: 6 total, 6 up, 6 in 2024-03-16T08:58:31.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:31 smithi007 ceph-mon[29470]: osdmap e45: 6 total, 6 up, 6 in 2024-03-16T08:58:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.592+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/2399518483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fee10 msgr2=0x7fa6c00faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.592+0000 7fa6c7ad8700 1 --2- 172.21.15.7:0/2399518483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fee10 0x7fa6c00faf50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4004940 tx=0x7fa6b4009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.592+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/2399518483 shutdown_connections 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.592+0000 7fa6c7ad8700 1 --2- 172.21.15.7:0/2399518483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fee10 0x7fa6c00faf50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.592+0000 7fa6c7ad8700 1 --2- 172.21.15.7:0/2399518483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c00fe4c0 0x7fa6c00fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.592+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/2399518483 >> 172.21.15.7:0/2399518483 conn(0x7fa6c00f6ba0 msgr2=0x7fa6c00f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.592+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/2399518483 shutdown_connections 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.592+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/2399518483 wait complete. 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c7ad8700 1 Processor -- start 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c7ad8700 1 -- start start 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c7ad8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fe4c0 0x7fa6c010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c7ad8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c00fee10 0x7fa6c010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c7ad8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c010e570 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c7ad8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c010edc0 con 0x7fa6c00fee10 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c5874700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fe4c0 0x7fa6c010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c5874700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fe4c0 0x7fa6c010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41064/0 (socket says 172.21.15.7:41064) 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c5874700 1 -- 172.21.15.7:0/415186755 learned_addr learned my addr 172.21.15.7:0/415186755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c5073700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c00fee10 0x7fa6c010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c5874700 1 -- 172.21.15.7:0/415186755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c00fee10 msgr2=0x7fa6c010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c5874700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c00fee10 0x7fa6c010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c5874700 1 -- 172.21.15.7:0/415186755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4005040 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6c5874700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fe4c0 0x7fa6c010da30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800cac0 tx=0x7fa6b8007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.593+0000 7fa6be7fc700 1 -- 172.21.15.7:0/415186755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b800aa50 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.594+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c010f0a0 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.594+0000 7fa6be7fc700 1 -- 172.21.15.7:0/415186755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6b800abb0 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.594+0000 7fa6be7fc700 1 -- 172.21.15.7:0/415186755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b80115e0 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.594+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c010f5e0 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.594+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c005f5f0 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.595+0000 7fa6be7fc700 1 -- 172.21.15.7:0/415186755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fa6b800ad20 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.595+0000 7fa6be7fc700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa6a80692a0 0x7fa6a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.595+0000 7fa6be7fc700 1 -- 172.21.15.7:0/415186755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa6b8087920 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.595+0000 7fa6c5073700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa6a80692a0 0x7fa6a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:32.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.596+0000 7fa6c5073700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa6a80692a0 0x7fa6a806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b3b0 tx=0x7fa6b4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:32.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.600+0000 7fa6be7fc700 1 -- 172.21.15.7:0/415186755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fa6b8057660 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.845+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa6c004bed0 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.845+0000 7fa6be7fc700 1 -- 172.21.15.7:0/415186755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fa6b80571f0 con 0x7fa6c00fe4c0 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:e2 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 2 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T08:58:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T08:58:28.666644+0000 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-03-16T08:58:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 1 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:in 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T08:58:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T08:58:32.851 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-03-16T08:58:32.851 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:32.851 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa6a80692a0 msgr2=0x7fa6a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa6a80692a0 0x7fa6a806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b3b0 tx=0x7fa6b4009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fe4c0 msgr2=0x7fa6c010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fe4c0 0x7fa6c010da30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800cac0 tx=0x7fa6b8007ee0 comp rx=0 tx=0).stop 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 shutdown_connections 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fa6a80692a0 0x7fa6a806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c00fe4c0 0x7fa6c010da30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 --2- 172.21.15.7:0/415186755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6c00fee10 0x7fa6c010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 >> 172.21.15.7:0/415186755 conn(0x7fa6c00f6ba0 msgr2=0x7fa6c00f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 shutdown_connections 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:32.848+0000 7fa6c7ad8700 1 -- 172.21.15.7:0/415186755 wait complete. 2024-03-16T08:58:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-03-16T08:58:32.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:32 smithi019 ceph-mon[36407]: pgmap v178: 65 pgs: 9 creating+peering, 18 active+clean, 38 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:32.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:32 smithi007 ceph-mon[29470]: pgmap v178: 65 pgs: 9 creating+peering, 18 active+clean, 38 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:33.723 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:58:33.733 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:58:33.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-03-16T08:58:33.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:33 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/415186755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T08:58:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:33 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:33 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:33 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:33 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.bncsil", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:58:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:33 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.bncsil", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:58:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:33 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:33 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/415186755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T08:58:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:33 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:33 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:33 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:33 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.bncsil", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:58:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:33 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.bncsil", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:58:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:33 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: Deploying daemon mds.cephfs.smithi019.bncsil on smithi019 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: pgmap v179: 65 pgs: 9 creating+peering, 48 active+clean, 8 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6826/2352390069,v1:172.21.15.7:6827/2352390069] up:boot 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: daemon mds.cephfs.smithi007.bsxctp assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: Cluster is now healthy 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: fsmap cephfs:0 1 up:standby 2024-03-16T08:58:34.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T08:58:34.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: fsmap cephfs:1 {0=cephfs.smithi007.bsxctp=up:creating} 2024-03-16T08:58:34.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:34 smithi019 ceph-mon[36407]: daemon mds.cephfs.smithi007.bsxctp is now active in filesystem cephfs as rank 0 2024-03-16T08:58:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: Deploying daemon mds.cephfs.smithi019.bncsil on smithi019 2024-03-16T08:58:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: pgmap v179: 65 pgs: 9 creating+peering, 48 active+clean, 8 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:58:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6826/2352390069,v1:172.21.15.7:6827/2352390069] up:boot 2024-03-16T08:58:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: daemon mds.cephfs.smithi007.bsxctp assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-16T08:58:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T08:58:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T08:58:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: Cluster is now healthy 2024-03-16T08:58:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: fsmap cephfs:0 1 up:standby 2024-03-16T08:58:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T08:58:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: fsmap cephfs:1 {0=cephfs.smithi007.bsxctp=up:creating} 2024-03-16T08:58:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:34 smithi007 ceph-mon[29470]: daemon mds.cephfs.smithi007.bsxctp is now active in filesystem cephfs as rank 0 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 -- 172.21.15.7:0/3351883688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 msgr2=0x7f6d880fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 --2- 172.21.15.7:0/3351883688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 0x7f6d880fd250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009d80 tx=0x7f6d80004cb0 comp rx=0 tx=0).stop 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 -- 172.21.15.7:0/3351883688 shutdown_connections 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 --2- 172.21.15.7:0/3351883688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d880fe040 0x7f6d880fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 --2- 172.21.15.7:0/3351883688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 0x7f6d880fd250 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 -- 172.21.15.7:0/3351883688 >> 172.21.15.7:0/3351883688 conn(0x7f6d880f8570 msgr2=0x7f6d880fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 -- 172.21.15.7:0/3351883688 shutdown_connections 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 -- 172.21.15.7:0/3351883688 wait complete. 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 Processor -- start 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.611+0000 7f6d8d037700 1 -- start start 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d8d037700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 0x7f6d88071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d8d037700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d880fe040 0x7f6d88070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d8d037700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880708c0 con 0x7f6d880fce40 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d8d037700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88070a30 con 0x7f6d880fe040 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d86d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 0x7f6d88071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d86d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 0x7f6d88071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41094/0 (socket says 172.21.15.7:41094) 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d86d9d700 1 -- 172.21.15.7:0/3264831389 learned_addr learned my addr 172.21.15.7:0/3264831389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d8659c700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d880fe040 0x7f6d88070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d86d9d700 1 -- 172.21.15.7:0/3264831389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d880fe040 msgr2=0x7f6d88070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d86d9d700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d880fe040 0x7f6d88070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d86d9d700 1 -- 172.21.15.7:0/3264831389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80009940 con 0x7f6d880fce40 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d86d9d700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 0x7f6d88071e40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000bb50 tx=0x7f6d80004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d7f7fe700 1 -- 172.21.15.7:0/3264831389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d8000e070 con 0x7f6d880fce40 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d7f7fe700 1 -- 172.21.15.7:0/3264831389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d80004dd0 con 0x7f6d880fce40 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d88070cb0 con 0x7f6d880fce40 2024-03-16T08:58:35.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d7f7fe700 1 -- 172.21.15.7:0/3264831389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d80013cd0 con 0x7f6d880fce40 2024-03-16T08:58:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.612+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d88115a90 con 0x7f6d880fce40 2024-03-16T08:58:35.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.613+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8805f5f0 con 0x7f6d880fce40 2024-03-16T08:58:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.614+0000 7f6d7f7fe700 1 -- 172.21.15.7:0/3264831389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6d80013520 con 0x7f6d880fce40 2024-03-16T08:58:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.614+0000 7f6d7f7fe700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6d740692a0 0x7f6d7406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.614+0000 7f6d8659c700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6d740692a0 0x7f6d7406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.614+0000 7f6d7f7fe700 1 -- 172.21.15.7:0/3264831389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6d80088ed0 con 0x7f6d880fce40 2024-03-16T08:58:35.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.615+0000 7f6d8659c700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6d740692a0 0x7f6d7406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d70004fd0 tx=0x7f6d7000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:35.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.619+0000 7f6d7f7fe700 1 -- 172.21.15.7:0/3264831389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f6d80055200 con 0x7f6d880fce40 2024-03-16T08:58:35.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:35.867+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f6d88115ce0 con 0x7f6d880fce40 2024-03-16T08:58:35.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:35 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6826/2352390069,v1:172.21.15.7:6827/2352390069] up:active 2024-03-16T08:58:35.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:35 smithi019 ceph-mon[36407]: fsmap cephfs:1 {0=cephfs.smithi007.bsxctp=up:active} 2024-03-16T08:58:35.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:35 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6826/2352390069,v1:172.21.15.7:6827/2352390069] up:active 2024-03-16T08:58:35.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:35 smithi007 ceph-mon[29470]: fsmap cephfs:1 {0=cephfs.smithi007.bsxctp=up:active} 2024-03-16T08:58:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.585+0000 7f6d7f7fe700 1 -- 172.21.15.7:0/3264831389 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f6d80058820 con 0x7f6d880fce40 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6d740692a0 msgr2=0x7f6d7406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6d740692a0 0x7f6d7406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6d70004fd0 tx=0x7f6d7000b040 comp rx=0 tx=0).stop 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 msgr2=0x7f6d88071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 0x7f6d88071e40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000bb50 tx=0x7f6d80004cb0 comp rx=0 tx=0).stop 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 shutdown_connections 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6d740692a0 0x7f6d7406b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d880fce40 0x7f6d88071e40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 --2- 172.21.15.7:0/3264831389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d880fe040 0x7f6d88070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 >> 172.21.15.7:0/3264831389 conn(0x7f6d880f8570 msgr2=0x7f6d880f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 shutdown_connections 2024-03-16T08:58:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:36.587+0000 7f6d8d037700 1 -- 172.21.15.7:0/3264831389 wait complete. 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 2 op/s 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.znurst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.znurst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3264831389' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: daemon mds.cephfs.smithi019.bncsil assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] up:boot 2024-03-16T08:58:36.833 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3264831389' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-16T08:58:36.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:creating} 2024-03-16T08:58:36.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T08:58:36.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:36 smithi007 ceph-mon[29470]: daemon mds.cephfs.smithi019.bncsil is now active in filesystem cephfs as rank 1 2024-03-16T08:58:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: pgmap v180: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 2 op/s 2024-03-16T08:58:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.znurst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:58:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.znurst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:58:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3264831389' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-16T08:58:36.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: daemon mds.cephfs.smithi019.bncsil assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T08:58:36.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-16T08:58:36.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] up:boot 2024-03-16T08:58:36.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3264831389' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-16T08:58:36.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:creating} 2024-03-16T08:58:36.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T08:58:36.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:36 smithi019 ceph-mon[36407]: daemon mds.cephfs.smithi019.bncsil is now active in filesystem cephfs as rank 1 2024-03-16T08:58:37.491 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:58:37.501 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:58:37.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-03-16T08:58:37.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:37 smithi019 ceph-mon[36407]: Deploying daemon mds.cephfs.smithi007.znurst on smithi007 2024-03-16T08:58:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:37 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] up:active 2024-03-16T08:58:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:37 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 2024-03-16T08:58:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:37 smithi007 ceph-mon[29470]: Deploying daemon mds.cephfs.smithi007.znurst on smithi007 2024-03-16T08:58:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:37 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] up:active 2024-03-16T08:58:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:37 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 2024-03-16T08:58:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:38.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:38 smithi019 ceph-mon[36407]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 6 op/s 2024-03-16T08:58:38.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:38 smithi007 ceph-mon[29470]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 6 op/s 2024-03-16T08:58:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.161+0000 7f45b7245700 1 -- 172.21.15.7:0/840549484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 msgr2=0x7f45b00fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.161+0000 7f45b7245700 1 --2- 172.21.15.7:0/840549484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 0x7f45b00fe450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f459c004940 tx=0x7f459c009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.161+0000 7f45b7245700 1 -- 172.21.15.7:0/840549484 shutdown_connections 2024-03-16T08:58:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.161+0000 7f45b7245700 1 --2- 172.21.15.7:0/840549484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 0x7f45b00fe450 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.161+0000 7f45b7245700 1 --2- 172.21.15.7:0/840549484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b00fce70 0x7f45b00fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.161+0000 7f45b7245700 1 -- 172.21.15.7:0/840549484 >> 172.21.15.7:0/840549484 conn(0x7f45b00f8550 msgr2=0x7f45b00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.161+0000 7f45b7245700 1 -- 172.21.15.7:0/840549484 shutdown_connections 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.161+0000 7f45b7245700 1 -- 172.21.15.7:0/840549484 wait complete. 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45b7245700 1 Processor -- start 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45b7245700 1 -- start start 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45b7245700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b00fce70 0x7f45b010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45b7245700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 0x7f45b01100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45b7245700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01106d0 con 0x7f45b00fdfe0 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45b7245700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0110840 con 0x7f45b00fce70 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 0x7f45b01100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 0x7f45b01100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41146/0 (socket says 172.21.15.7:41146) 2024-03-16T08:58:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45affff700 1 -- 172.21.15.7:0/836341104 learned_addr learned my addr 172.21.15.7:0/836341104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.162+0000 7f45b4fe1700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b00fce70 0x7f45b010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45affff700 1 -- 172.21.15.7:0/836341104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b00fce70 msgr2=0x7f45b010fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45affff700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b00fce70 0x7f45b010fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45affff700 1 -- 172.21.15.7:0/836341104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c005040 con 0x7f45b00fdfe0 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45affff700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 0x7f45b01100d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f459c00b3b0 tx=0x7f459c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45ad7fa700 1 -- 172.21.15.7:0/836341104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f459c00e040 con 0x7f45b00fdfe0 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45ad7fa700 1 -- 172.21.15.7:0/836341104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f459c00bbf0 con 0x7f45b00fdfe0 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b01151f0 con 0x7f45b00fdfe0 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45ad7fa700 1 -- 172.21.15.7:0/836341104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f459c01c630 con 0x7f45b00fdfe0 2024-03-16T08:58:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.163+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b01157b0 con 0x7f45b00fdfe0 2024-03-16T08:58:40.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.164+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b004c710 con 0x7f45b00fdfe0 2024-03-16T08:58:40.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.165+0000 7f45ad7fa700 1 -- 172.21.15.7:0/836341104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f459c00bd60 con 0x7f45b00fdfe0 2024-03-16T08:58:40.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.165+0000 7f45ad7fa700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f45a00692a0 0x7f45a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:40.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.165+0000 7f45b4fe1700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f45a00692a0 0x7f45a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:40.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.165+0000 7f45ad7fa700 1 -- 172.21.15.7:0/836341104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f459c087eb0 con 0x7f45b00fdfe0 2024-03-16T08:58:40.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.166+0000 7f45b4fe1700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f45a00692a0 0x7f45a006b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45a4004f60 tx=0x7f45a400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:40.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.170+0000 7f45ad7fa700 1 -- 172.21.15.7:0/836341104 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f459c057c70 con 0x7f45b00fdfe0 2024-03-16T08:58:40.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.406+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f45b0070f60 con 0x7f45b00fdfe0 2024-03-16T08:58:40.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.709+0000 7f45ad7fa700 1 -- 172.21.15.7:0/836341104 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f459c057800 con 0x7f45b00fdfe0 2024-03-16T08:58:40.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.711+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f45a00692a0 msgr2=0x7f45a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.711+0000 7f45b7245700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f45a00692a0 0x7f45a006b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45a4004f60 tx=0x7f45a400b040 comp rx=0 tx=0).stop 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.711+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 msgr2=0x7f45b01100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.711+0000 7f45b7245700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 0x7f45b01100d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f459c00b3b0 tx=0x7f459c009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.711+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 shutdown_connections 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.712+0000 7f45b7245700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f45a00692a0 0x7f45a006b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.712+0000 7f45b7245700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45b00fce70 0x7f45b010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.712+0000 7f45b7245700 1 --2- 172.21.15.7:0/836341104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00fdfe0 0x7f45b01100d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.712+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 >> 172.21.15.7:0/836341104 conn(0x7f45b00f8550 msgr2=0x7f45b00f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:40.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.712+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 shutdown_connections 2024-03-16T08:58:40.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:40.712+0000 7f45b7245700 1 -- 172.21.15.7:0/836341104 wait complete. 2024-03-16T08:58:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:40 smithi007 ceph-mon[29470]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 7 op/s 2024-03-16T08:58:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.gmhzgo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:58:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.gmhzgo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:58:40.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:40 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:40.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:40 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/836341104' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-16T08:58:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:40 smithi019 ceph-mon[36407]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 7 op/s 2024-03-16T08:58:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.gmhzgo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T08:58:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.gmhzgo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T08:58:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:40 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:40 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/836341104' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-16T08:58:41.403 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:58:41.436 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:58:41.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-16T08:58:41.707 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:41.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: Deploying daemon mds.cephfs.smithi019.gmhzgo on smithi019 2024-03-16T08:58:41.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-16T08:58:41.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: Cluster is now healthy 2024-03-16T08:58:41.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:boot 2024-03-16T08:58:41.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/836341104' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-16T08:58:41.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 1 up:standby 2024-03-16T08:58:41.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T08:58:41.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:58:41.955 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.043 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: Deploying daemon mds.cephfs.smithi019.gmhzgo on smithi019 2024-03-16T08:58:42.043 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-16T08:58:42.043 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: Cluster is now healthy 2024-03-16T08:58:42.043 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:boot 2024-03-16T08:58:42.043 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/836341104' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-16T08:58:42.043 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 1 up:standby 2024-03-16T08:58:42.044 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T08:58:42.044 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:58:42.044 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.980 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:42 smithi019 ceph-mon[36407]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-16T08:58:42.980 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.980 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.981 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.981 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.981 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.981 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:42 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:42.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:42 smithi007 ceph-mon[29470]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-16T08:58:42.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:42.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:42 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.419+0000 7f242cf1f700 1 -- 172.21.15.7:0/417032327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 msgr2=0x7f24280fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.419+0000 7f242cf1f700 1 --2- 172.21.15.7:0/417032327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 0x7f24280fd3f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f241c004940 tx=0x7f241c009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.420+0000 7f242cf1f700 1 -- 172.21.15.7:0/417032327 shutdown_connections 2024-03-16T08:58:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.420+0000 7f242cf1f700 1 --2- 172.21.15.7:0/417032327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 0x7f24280fd3f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.420+0000 7f242cf1f700 1 --2- 172.21.15.7:0/417032327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24280fbcd0 0x7f24280fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.420+0000 7f242cf1f700 1 -- 172.21.15.7:0/417032327 >> 172.21.15.7:0/417032327 conn(0x7f24280f7450 msgr2=0x7f24280f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.420+0000 7f242cf1f700 1 -- 172.21.15.7:0/417032327 shutdown_connections 2024-03-16T08:58:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.420+0000 7f242cf1f700 1 -- 172.21.15.7:0/417032327 wait complete. 2024-03-16T08:58:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f242cf1f700 1 Processor -- start 2024-03-16T08:58:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f242cf1f700 1 -- start start 2024-03-16T08:58:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f242cf1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24280fbcd0 0x7f2428066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f242cf1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 0x7f24281126c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f242cf1f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428112c00 con 0x7f24280fcf80 2024-03-16T08:58:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f242cf1f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428112d40 con 0x7f24280fbcd0 2024-03-16T08:58:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 0x7f24281126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f2425d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 0x7f24281126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54014/0 (socket says 172.21.15.7:54014) 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f2425d9b700 1 -- 172.21.15.7:0/1507155378 learned_addr learned my addr 172.21.15.7:0/1507155378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f242659c700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24280fbcd0 0x7f2428066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f2425d9b700 1 -- 172.21.15.7:0/1507155378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24280fbcd0 msgr2=0x7f2428066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f2425d9b700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24280fbcd0 0x7f2428066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f2425d9b700 1 -- 172.21.15.7:0/1507155378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c005040 con 0x7f24280fcf80 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.421+0000 7f2425d9b700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 0x7f24281126c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f241c013870 tx=0x7f241c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.422+0000 7f2416ffd700 1 -- 172.21.15.7:0/1507155378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241c021070 con 0x7f24280fcf80 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.422+0000 7f2416ffd700 1 -- 172.21.15.7:0/1507155378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f241c009430 con 0x7f24280fcf80 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.422+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2428112f40 con 0x7f24280fcf80 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.422+0000 7f2416ffd700 1 -- 172.21.15.7:0/1507155378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241c00e070 con 0x7f24280fcf80 2024-03-16T08:58:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.422+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2428113450 con 0x7f24280fcf80 2024-03-16T08:58:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.423+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242804c710 con 0x7f24280fcf80 2024-03-16T08:58:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.423+0000 7f2416ffd700 1 -- 172.21.15.7:0/1507155378 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f241c01c750 con 0x7f24280fcf80 2024-03-16T08:58:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.424+0000 7f2416ffd700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2410069240 0x7f241006b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.424+0000 7f2416ffd700 1 -- 172.21.15.7:0/1507155378 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f241c0883d0 con 0x7f24280fcf80 2024-03-16T08:58:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.424+0000 7f242659c700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2410069240 0x7f241006b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.424+0000 7f242659c700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2410069240 0x7f241006b6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2418009b80 tx=0x7f241800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.428+0000 7f2416ffd700 1 -- 172.21.15.7:0/1507155378 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f241c058190 con 0x7f24280fcf80 2024-03-16T08:58:43.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.674+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f2428113740 con 0x7f24280fcf80 2024-03-16T08:58:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.724+0000 7f2416ffd700 1 -- 172.21.15.7:0/1507155378 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v10) v1 ==== 133+0+0 (secure 0 0 0) 0x7f241c057d20 con 0x7f24280fcf80 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.726+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2410069240 msgr2=0x7f241006b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.726+0000 7f242cf1f700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2410069240 0x7f241006b6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2418009b80 tx=0x7f241800b040 comp rx=0 tx=0).stop 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.726+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 msgr2=0x7f24281126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.726+0000 7f242cf1f700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 0x7f24281126c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f241c013870 tx=0x7f241c009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.727+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 shutdown_connections 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.727+0000 7f242cf1f700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f2410069240 0x7f241006b6f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.727+0000 7f242cf1f700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24280fbcd0 0x7f2428066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.727+0000 7f242cf1f700 1 --2- 172.21.15.7:0/1507155378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24280fcf80 0x7f24281126c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.727+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 >> 172.21.15.7:0/1507155378 conn(0x7f24280f7450 msgr2=0x7f24280f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.727+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 shutdown_connections 2024-03-16T08:58:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:43.727+0000 7f242cf1f700 1 -- 172.21.15.7:0/1507155378 wait complete. 2024-03-16T08:58:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:inline data disabled 2024-03-16T08:58:43.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:43 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] up:boot 2024-03-16T08:58:43.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:43 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:43.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:43 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T08:58:43.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:43 smithi019 ceph-mon[36407]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-03-16T08:58:43.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:43 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1507155378' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-16T08:58:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:43 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] up:boot 2024-03-16T08:58:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:43 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:43 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T08:58:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:43 smithi007 ceph-mon[29470]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-03-16T08:58:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:43 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1507155378' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-16T08:58:44.411 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T08:58:44.421 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:58:44.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T08:58:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:44.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:44 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1507155378' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-16T08:58:44.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:44 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:44.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:44 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:44.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:44 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:45.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:44 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1507155378' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-16T08:58:45.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:44 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:45.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:44 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:45.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:44 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:45.928 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:standby 2024-03-16T08:58:45.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] up:active 2024-03-16T08:58:45.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: Dropping low affinity active daemon mds.cephfs.smithi007.bsxctp in favor of higher affinity standby. 2024-03-16T08:58:45.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: Replacing daemon mds.cephfs.smithi007.bsxctp as rank 0 with standby daemon mds.cephfs.smithi007.znurst 2024-03-16T08:58:45.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T08:58:45.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:45.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: osdmap e46: 6 total, 6 up, 6 in 2024-03-16T08:58:45.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: fsmap cephfs:2/2 {0=cephfs.smithi007.znurst=up:replay,1=cephfs.smithi019.bncsil=up:active} 1 up:standby 2024-03-16T08:58:45.929 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:45 smithi007 ceph-mon[29470]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 6 op/s 2024-03-16T08:58:46.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:standby 2024-03-16T08:58:46.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] up:active 2024-03-16T08:58:46.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: Dropping low affinity active daemon mds.cephfs.smithi007.bsxctp in favor of higher affinity standby. 2024-03-16T08:58:46.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: Replacing daemon mds.cephfs.smithi007.bsxctp as rank 0 with standby daemon mds.cephfs.smithi007.znurst 2024-03-16T08:58:46.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T08:58:46.115 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:46.115 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: osdmap e46: 6 total, 6 up, 6 in 2024-03-16T08:58:46.115 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: fsmap cephfs:2/2 {0=cephfs.smithi007.znurst=up:replay,1=cephfs.smithi019.bncsil=up:active} 1 up:standby 2024-03-16T08:58:46.115 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:45 smithi019 ceph-mon[36407]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 6 op/s 2024-03-16T08:58:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:46 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:resolve 2024-03-16T08:58:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:46 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] up:boot 2024-03-16T08:58:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:46 smithi007 ceph-mon[29470]: fsmap cephfs:2/2 {0=cephfs.smithi007.znurst=up:resolve,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:46 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T08:58:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:46 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:46 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:resolve 2024-03-16T08:58:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:46 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] up:boot 2024-03-16T08:58:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:46 smithi019 ceph-mon[36407]: fsmap cephfs:2/2 {0=cephfs.smithi007.znurst=up:resolve,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:46 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T08:58:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:46 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.557+0000 7f0e817ec700 1 -- 172.21.15.7:0/2152193331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c071e20 msgr2=0x7f0e7c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.557+0000 7f0e817ec700 1 --2- 172.21.15.7:0/2152193331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c071e20 0x7f0e7c070330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009ce0 tx=0x7f0e74004d10 comp rx=0 tx=0).stop 2024-03-16T08:58:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.557+0000 7f0e817ec700 1 -- 172.21.15.7:0/2152193331 shutdown_connections 2024-03-16T08:58:47.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.557+0000 7f0e817ec700 1 --2- 172.21.15.7:0/2152193331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e7c070870 0x7f0e7c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:47.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.557+0000 7f0e817ec700 1 --2- 172.21.15.7:0/2152193331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c071e20 0x7f0e7c070330 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:47.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.557+0000 7f0e817ec700 1 -- 172.21.15.7:0/2152193331 >> 172.21.15.7:0/2152193331 conn(0x7f0e7c0f85a0 msgr2=0x7f0e7c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:47.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.557+0000 7f0e817ec700 1 -- 172.21.15.7:0/2152193331 shutdown_connections 2024-03-16T08:58:47.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.557+0000 7f0e817ec700 1 -- 172.21.15.7:0/2152193331 wait complete. 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e817ec700 1 Processor -- start 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e817ec700 1 -- start start 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e817ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c070870 0x7f0e7c10fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e817ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e7c071e20 0x7f0e7c1101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e817ec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1107a0 con 0x7f0e7c070870 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e817ec700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c110ff0 con 0x7f0e7c071e20 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c070870 0x7f0e7c10fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c070870 0x7f0e7c10fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54068/0 (socket says 172.21.15.7:54068) 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e7affd700 1 -- 172.21.15.7:0/3833357416 learned_addr learned my addr 172.21.15.7:0/3833357416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e7a7fc700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e7c071e20 0x7f0e7c1101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e7affd700 1 -- 172.21.15.7:0/3833357416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e7c071e20 msgr2=0x7f0e7c1101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e7affd700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e7c071e20 0x7f0e7c1101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e7affd700 1 -- 172.21.15.7:0/3833357416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e74009940 con 0x7f0e7c070870 2024-03-16T08:58:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.558+0000 7f0e7affd700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c070870 0x7f0e7c10fc60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e740098e0 tx=0x7f0e74004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.559+0000 7f0e737fe700 1 -- 172.21.15.7:0/3833357416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e7400e040 con 0x7f0e7c070870 2024-03-16T08:58:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.559+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c111270 con 0x7f0e7c070870 2024-03-16T08:58:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.559+0000 7f0e737fe700 1 -- 172.21.15.7:0/3833357416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e740049f0 con 0x7f0e7c070870 2024-03-16T08:58:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.559+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c1117b0 con 0x7f0e7c070870 2024-03-16T08:58:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.559+0000 7f0e737fe700 1 -- 172.21.15.7:0/3833357416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e74012c00 con 0x7f0e7c070870 2024-03-16T08:58:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.560+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c05f5f0 con 0x7f0e7c070870 2024-03-16T08:58:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.560+0000 7f0e737fe700 1 -- 172.21.15.7:0/3833357416 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f0e74012410 con 0x7f0e7c070870 2024-03-16T08:58:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.560+0000 7f0e737fe700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0e680692a0 0x7f0e6806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.561+0000 7f0e737fe700 1 -- 172.21.15.7:0/3833357416 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0e740894f0 con 0x7f0e7c070870 2024-03-16T08:58:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.561+0000 7f0e7a7fc700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0e680692a0 0x7f0e6806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.561+0000 7f0e7a7fc700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0e680692a0 0x7f0e6806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009b80 tx=0x7f0e6400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:47.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.565+0000 7f0e737fe700 1 -- 172.21.15.7:0/3833357416 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f0e740591e0 con 0x7f0e7c070870 2024-03-16T08:58:47.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.831+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0e7c111ae0 con 0x7f0e7c070870 2024-03-16T08:58:47.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:47 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:reconnect 2024-03-16T08:58:47.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:47 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] up:standby 2024-03-16T08:58:47.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:47 smithi007 ceph-mon[29470]: fsmap cephfs:2/2 {0=cephfs.smithi007.znurst=up:reconnect,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:47.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:47 smithi007 ceph-mon[29470]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1.8 KiB/s wr, 4 op/s 2024-03-16T08:58:47.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.832+0000 7f0e737fe700 1 -- 172.21.15.7:0/3833357416 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1810 (secure 0 0 0) 0x7f0e74017730 con 0x7f0e7c070870 2024-03-16T08:58:47.835 INFO:teuthology.orchestra.run.smithi007.stdout:e15 2024-03-16T08:58:47.835 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T08:58:47.835 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T08:58:47.835 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T08:58:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:47.835 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T08:58:47.835 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 15 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T08:58:47.739264+0000 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T08:58:47.836 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:rejoin seq 5 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:47.837 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T08:58:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:47.838 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T08:58:47.838 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T08:58:47.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0e680692a0 msgr2=0x7f0e6806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0e680692a0 0x7f0e6806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009b80 tx=0x7f0e6400b040 comp rx=0 tx=0).stop 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c070870 msgr2=0x7f0e7c10fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c070870 0x7f0e7c10fc60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e740098e0 tx=0x7f0e74004d10 comp rx=0 tx=0).stop 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 shutdown_connections 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0e680692a0 0x7f0e6806b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e7c070870 0x7f0e7c10fc60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 --2- 172.21.15.7:0/3833357416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e7c071e20 0x7f0e7c1101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 >> 172.21.15.7:0/3833357416 conn(0x7f0e7c0f85a0 msgr2=0x7f0e7c0f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 shutdown_connections 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:47.835+0000 7f0e817ec700 1 -- 172.21.15.7:0/3833357416 wait complete. 2024-03-16T08:58:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2024-03-16T08:58:48.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:47 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:reconnect 2024-03-16T08:58:48.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:47 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] up:standby 2024-03-16T08:58:48.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:47 smithi019 ceph-mon[36407]: fsmap cephfs:2/2 {0=cephfs.smithi007.znurst=up:reconnect,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:48.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:47 smithi019 ceph-mon[36407]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1.8 KiB/s wr, 4 op/s 2024-03-16T08:58:48.766 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:48 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:rejoin 2024-03-16T08:58:48.766 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:48 smithi007 ceph-mon[29470]: fsmap cephfs:2/2 {0=cephfs.smithi007.znurst=up:rejoin,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:48.766 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:48 smithi007 ceph-mon[29470]: daemon mds.cephfs.smithi007.znurst is now active in filesystem cephfs as rank 0 2024-03-16T08:58:48.766 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:48 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3833357416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T08:58:48.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-16T08:58:49.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:48 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:rejoin 2024-03-16T08:58:49.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:48 smithi019 ceph-mon[36407]: fsmap cephfs:2/2 {0=cephfs.smithi007.znurst=up:rejoin,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:49.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:48 smithi019 ceph-mon[36407]: daemon mds.cephfs.smithi007.znurst is now active in filesystem cephfs as rank 0 2024-03-16T08:58:49.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:48 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3833357416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T08:58:49.569 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:49 smithi007 ceph-mon[29470]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T08:58:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:49 smithi007 ceph-mon[29470]: Cluster is now healthy 2024-03-16T08:58:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:49 smithi007 ceph-mon[29470]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:active 2024-03-16T08:58:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:49 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:49 smithi007 ceph-mon[29470]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 409 B/s wr, 3 op/s 2024-03-16T08:58:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:49 smithi019 ceph-mon[36407]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T08:58:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:49 smithi019 ceph-mon[36407]: Cluster is now healthy 2024-03-16T08:58:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:49 smithi019 ceph-mon[36407]: mds.? [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] up:active 2024-03-16T08:58:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:49 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T08:58:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:49 smithi019 ceph-mon[36407]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 409 B/s wr, 3 op/s 2024-03-16T08:58:51.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 -- 172.21.15.7:0/1943783359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c066280 msgr2=0x7f6c6c0ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:51.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 --2- 172.21.15.7:0/1943783359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c066280 0x7f6c6c0ffa20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6c68004940 tx=0x7f6c68009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:51.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 -- 172.21.15.7:0/1943783359 shutdown_connections 2024-03-16T08:58:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 --2- 172.21.15.7:0/1943783359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6c0fff60 0x7f6c6c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 --2- 172.21.15.7:0/1943783359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c066280 0x7f6c6c0ffa20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 -- 172.21.15.7:0/1943783359 >> 172.21.15.7:0/1943783359 conn(0x7f6c6c0f7430 msgr2=0x7f6c6c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 -- 172.21.15.7:0/1943783359 shutdown_connections 2024-03-16T08:58:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 -- 172.21.15.7:0/1943783359 wait complete. 2024-03-16T08:58:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 Processor -- start 2024-03-16T08:58:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.212+0000 7f6c73fb1700 1 -- start start 2024-03-16T08:58:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c73fb1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6c066280 0x7f6c6c114100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c73fb1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c0fff60 0x7f6c6c114640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c73fb1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c114c40 con 0x7f6c6c0fff60 2024-03-16T08:58:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c73fb1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c115440 con 0x7f6c6c066280 2024-03-16T08:58:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c7154c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c0fff60 0x7f6c6c114640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c71d4d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6c066280 0x7f6c6c114100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c7154c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c0fff60 0x7f6c6c114640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54090/0 (socket says 172.21.15.7:54090) 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c7154c700 1 -- 172.21.15.7:0/3695619669 learned_addr learned my addr 172.21.15.7:0/3695619669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c7154c700 1 -- 172.21.15.7:0/3695619669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6c066280 msgr2=0x7f6c6c114100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c7154c700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6c066280 0x7f6c6c114100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c7154c700 1 -- 172.21.15.7:0/3695619669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c68005040 con 0x7f6c6c0fff60 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.213+0000 7f6c7154c700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c0fff60 0x7f6c6c114640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c002a50 tx=0x7f6c5c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.214+0000 7f6c627fc700 1 -- 172.21.15.7:0/3695619669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5c007cc0 con 0x7f6c6c0fff60 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.214+0000 7f6c627fc700 1 -- 172.21.15.7:0/3695619669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c5c00f040 con 0x7f6c6c0fff60 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.214+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c115720 con 0x7f6c6c0fff60 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.214+0000 7f6c627fc700 1 -- 172.21.15.7:0/3695619669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5c00da30 con 0x7f6c6c0fff60 2024-03-16T08:58:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.214+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c115c60 con 0x7f6c6c0fff60 2024-03-16T08:58:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.215+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c05f5f0 con 0x7f6c6c0fff60 2024-03-16T08:58:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.215+0000 7f6c627fc700 1 -- 172.21.15.7:0/3695619669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6c5c00db90 con 0x7f6c6c0fff60 2024-03-16T08:58:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.215+0000 7f6c627fc700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6c580692a0 0x7f6c5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.216+0000 7f6c627fc700 1 -- 172.21.15.7:0/3695619669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6c5c086ee0 con 0x7f6c6c0fff60 2024-03-16T08:58:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.216+0000 7f6c71d4d700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6c580692a0 0x7f6c5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.216+0000 7f6c71d4d700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6c580692a0 0x7f6c5806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b970 tx=0x7f6c68009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:51.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.221+0000 7f6c627fc700 1 -- 172.21.15.7:0/3695619669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f6c5c056c50 con 0x7f6c6c0fff60 2024-03-16T08:58:51.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.469+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c6c04bed0 con 0x7f6c6c0fff60 2024-03-16T08:58:51.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.470+0000 7f6c627fc700 1 -- 172.21.15.7:0/3695619669 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4694 (secure 0 0 0) 0x7f6c5c0567e0 con 0x7f6c6c0fff60 2024-03-16T08:58:51.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6c580692a0 msgr2=0x7f6c5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:51.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6c580692a0 0x7f6c5806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b970 tx=0x7f6c68009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:51.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c0fff60 msgr2=0x7f6c6c114640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c0fff60 0x7f6c6c114640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c002a50 tx=0x7f6c5c0046b0 comp rx=0 tx=0).stop 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 shutdown_connections 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f6c580692a0 0x7f6c5806b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c6c066280 0x7f6c6c114100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 --2- 172.21.15.7:0/3695619669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6c0fff60 0x7f6c6c114640 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 >> 172.21.15.7:0/3695619669 conn(0x7f6c6c0f7430 msgr2=0x7f6c6c0fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 shutdown_connections 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:51.472+0000 7f6c73fb1700 1 -- 172.21.15.7:0/3695619669 wait complete. 2024-03-16T08:58:51.475 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-03-16T08:58:51.484 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T08:58:51.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:51 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:51.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:51 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:51.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:51 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:51.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:51 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:51.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:51 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:51.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:51 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:51.619 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:51 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:51.619 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:51 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:51.619 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:51 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:51.619 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:51 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:51.620 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:51 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:51.620 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:51 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:58:52.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:52 smithi007 ceph-mon[29470]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-16T08:58:52.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:52 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3695619669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:58:52.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-16T08:58:52.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:52 smithi019 ceph-mon[36407]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-16T08:58:52.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:52 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3695619669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:58:53.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:53.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:53.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:53.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:54.591 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:54 smithi007 ceph-mon[29470]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-16T08:58:54.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:54 smithi019 ceph-mon[36407]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 -- 172.21.15.7:0/3792757939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 msgr2=0x7f0f800fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 --2- 172.21.15.7:0/3792757939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 0x7f0f800fc0e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c004940 tx=0x7f0f7c009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 -- 172.21.15.7:0/3792757939 shutdown_connections 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 --2- 172.21.15.7:0/3792757939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800fcf80 0x7f0f800fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 --2- 172.21.15.7:0/3792757939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 0x7f0f800fc0e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 -- 172.21.15.7:0/3792757939 >> 172.21.15.7:0/3792757939 conn(0x7f0f800f7430 msgr2=0x7f0f800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 -- 172.21.15.7:0/3792757939 shutdown_connections 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 -- 172.21.15.7:0/3792757939 wait complete. 2024-03-16T08:58:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 Processor -- start 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.395+0000 7f0f87a88700 1 -- start start 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f87a88700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 0x7f0f80066c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f87a88700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800fcf80 0x7f0f80112760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f87a88700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80112ca0 con 0x7f0f800fbcd0 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f87a88700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80112e10 con 0x7f0f800fcf80 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f85824700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 0x7f0f80066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f85824700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 0x7f0f80066c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57330/0 (socket says 172.21.15.7:57330) 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f85824700 1 -- 172.21.15.7:0/2516389876 learned_addr learned my addr 172.21.15.7:0/2516389876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f85023700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800fcf80 0x7f0f80112760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f85824700 1 -- 172.21.15.7:0/2516389876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800fcf80 msgr2=0x7f0f80112760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f85824700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800fcf80 0x7f0f80112760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f85824700 1 -- 172.21.15.7:0/2516389876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c005040 con 0x7f0f800fbcd0 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f85824700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 0x7f0f80066c70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009150 tx=0x7f0f7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f767fc700 1 -- 172.21.15.7:0/2516389876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7c00e040 con 0x7f0f800fbcd0 2024-03-16T08:58:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f767fc700 1 -- 172.21.15.7:0/2516389876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f7c004290 con 0x7f0f800fbcd0 2024-03-16T08:58:56.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f80113090 con 0x7f0f800fbcd0 2024-03-16T08:58:56.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f767fc700 1 -- 172.21.15.7:0/2516389876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7c01c630 con 0x7f0f800fbcd0 2024-03-16T08:58:56.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.396+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f80113650 con 0x7f0f800fbcd0 2024-03-16T08:58:56.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.397+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8004c710 con 0x7f0f800fbcd0 2024-03-16T08:58:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.398+0000 7f0f767fc700 1 -- 172.21.15.7:0/2516389876 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f0f7c00bbf0 con 0x7f0f800fbcd0 2024-03-16T08:58:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.398+0000 7f0f767fc700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0f6c0692a0 0x7f0f6c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.398+0000 7f0f767fc700 1 -- 172.21.15.7:0/2516389876 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0f7c087ea0 con 0x7f0f800fbcd0 2024-03-16T08:58:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.398+0000 7f0f85023700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0f6c0692a0 0x7f0f6c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.399+0000 7f0f85023700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0f6c0692a0 0x7f0f6c06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009bf0 tx=0x7f0f700091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.403+0000 7f0f767fc700 1 -- 172.21.15.7:0/2516389876 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f0f7c057c10 con 0x7f0f800fbcd0 2024-03-16T08:58:56.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:56 smithi019 ceph-mon[36407]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-16T08:58:56.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:58:56.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.678+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f0f80104840 con 0x7f0f800fbcd0 2024-03-16T08:58:56.681 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:56 smithi007 ceph-mon[29470]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-16T08:58:56.681 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:58:56.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.679+0000 7f0f767fc700 1 -- 172.21.15.7:0/2516389876 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v16) v1 ==== 78+0+99 (secure 0 0 0) 0x7f0f7c0577a0 con 0x7f0f800fbcd0 2024-03-16T08:58:56.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0f6c0692a0 msgr2=0x7f0f6c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0f6c0692a0 0x7f0f6c06b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009bf0 tx=0x7f0f700091a0 comp rx=0 tx=0).stop 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 msgr2=0x7f0f80066c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 0x7f0f80066c70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009150 tx=0x7f0f7c009e30 comp rx=0 tx=0).stop 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 shutdown_connections 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f0f6c0692a0 0x7f0f6c06b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800fbcd0 0x7f0f80066c70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 --2- 172.21.15.7:0/2516389876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800fcf80 0x7f0f80112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 >> 172.21.15.7:0/2516389876 conn(0x7f0f800f7430 msgr2=0x7f0f800f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 shutdown_connections 2024-03-16T08:58:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:56.681+0000 7f0f87a88700 1 -- 172.21.15.7:0/2516389876 wait complete. 2024-03-16T08:58:56.693 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T08:58:57.467 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-16T08:58:57.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T08:58:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.595+0000 7fd43815d640 1 -- 172.21.15.7:0/3260529548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 msgr2=0x7fd430101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.595+0000 7fd43815d640 1 --2- 172.21.15.7:0/3260529548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 0x7fd430101730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4200048d0 tx=0x7fd420009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.595+0000 7fd43815d640 1 -- 172.21.15.7:0/3260529548 shutdown_connections 2024-03-16T08:58:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.595+0000 7fd43815d640 1 --2- 172.21.15.7:0/3260529548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 0x7fd430101730 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.595+0000 7fd43815d640 1 --2- 172.21.15.7:0/3260529548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd430064fd0 0x7fd4300fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.595+0000 7fd43815d640 1 -- 172.21.15.7:0/3260529548 >> 172.21.15.7:0/3260529548 conn(0x7fd4300f6840 msgr2=0x7fd4300f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.595+0000 7fd43815d640 1 -- 172.21.15.7:0/3260529548 shutdown_connections 2024-03-16T08:58:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.595+0000 7fd43815d640 1 -- 172.21.15.7:0/3260529548 wait complete. 2024-03-16T08:58:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd43815d640 1 Processor -- start 2024-03-16T08:58:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd43815d640 1 -- start start 2024-03-16T08:58:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd43815d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd430064fd0 0x7fd43010cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd43815d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 0x7fd43010d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd43815d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43010d8c0 con 0x7fd4300ff350 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd43815d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd430112230 con 0x7fd430064fd0 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd4356f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 0x7fd43010d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd4356f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 0x7fd43010d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57350/0 (socket says 172.21.15.7:57350) 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd4356f8640 1 -- 172.21.15.7:0/711477469 learned_addr learned my addr 172.21.15.7:0/711477469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd435ef9640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd430064fd0 0x7fd43010cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd4356f8640 1 -- 172.21.15.7:0/711477469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd430064fd0 msgr2=0x7fd43010cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd4356f8640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd430064fd0 0x7fd43010cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd4356f8640 1 -- 172.21.15.7:0/711477469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420005040 con 0x7fd4300ff350 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.596+0000 7fd435ef9640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd430064fd0 0x7fd43010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.597+0000 7fd4356f8640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 0x7fd43010d240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd420003080 tx=0x7fd420009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.597+0000 7fd4267fc640 1 -- 172.21.15.7:0/711477469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd420030e90 con 0x7fd4300ff350 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.597+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd430112370 con 0x7fd4300ff350 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.597+0000 7fd4267fc640 1 -- 172.21.15.7:0/711477469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4200092e0 con 0x7fd4300ff350 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.597+0000 7fd4267fc640 1 -- 172.21.15.7:0/711477469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd420043640 con 0x7fd4300ff350 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.597+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd430112860 con 0x7fd4300ff350 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.598+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43004bcc0 con 0x7fd4300ff350 2024-03-16T08:58:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.599+0000 7fd4267fc640 1 -- 172.21.15.7:0/711477469 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fd42003b9f0 con 0x7fd4300ff350 2024-03-16T08:58:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.599+0000 7fd4267fc640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd41c069210 0x7fd41c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.599+0000 7fd4267fc640 1 -- 172.21.15.7:0/711477469 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd4200aa000 con 0x7fd4300ff350 2024-03-16T08:58:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.599+0000 7fd435ef9640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd41c069210 0x7fd41c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.599+0000 7fd435ef9640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd41c069210 0x7fd41c06b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4300fde40 tx=0x7fd42c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.603+0000 7fd4267fc640 1 -- 172.21.15.7:0/711477469 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fd4200762e0 con 0x7fd4300ff350 2024-03-16T08:58:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:57 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/2516389876' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-16T08:58:57.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.779+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd430104d00 con 0x7fd4300ff350 2024-03-16T08:58:57.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.780+0000 7fd4267fc640 1 -- 172.21.15.7:0/711477469 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14297 (secure 0 0 0) 0x7fd420079900 con 0x7fd4300ff350 2024-03-16T08:58:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:57.783 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","created":"2024-03-16T08:51:36.700010+0000","modified":"2024-03-16T08:58:44.727587+0000","last_up_change":"2024-03-16T08:57:07.414518+0000","last_in_change":"2024-03-16T08:56:48.805178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:55:53.635803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-16T08:58:27.077630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-16T08:58:27.912287+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c8f96044-56c4-4825-8138-629e5352a8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6803","nonce":1074142884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6805","nonce":1074142884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6809","nonce":1074142884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":1074142884},{"type":"v1","addr":"172.21.15.7:6807","nonce":1074142884}]},"public_addr":"172.21.15.7:6803/1074142884","cluster_addr":"172.21.15.7:6805/1074142884","heartbeat_back_addr":"172.21.15.7:6809/1074142884","heartbeat_front_addr":"172.21.15.7:6807/1074142884","state":["exists","up"]},{"osd":1,"uuid":"b3fcdd0c-0016-4781-803b-bbca53b4517b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6811","nonce":2075460664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6813","nonce":2075460664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6817","nonce":2075460664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":2075460664},{"type":"v1","addr":"172.21.15.7:6815","nonce":2075460664}]},"public_addr":"172.21.15.7:6811/2075460664","cluster_addr":"172.21.15.7:6813/2075460664","heartbeat_back_addr":"172.21.15.7:6817/2075460664","heartbeat_front_addr":"172.21.15.7:6815/2075460664","state":["exists","up"]},{"osd":2,"uuid":"4f400784-c979-4f41-bcd2-a52737e2e65e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6819","nonce":206561017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6821","nonce":206561017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6825","nonce":206561017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":206561017},{"type":"v1","addr":"172.21.15.7:6823","nonce":206561017}]},"public_addr":"172.21.15.7:6819/206561017","cluster_addr":"172.21.15.7:6821/206561017","heartbeat_back_addr":"172.21.15.7:6825/206561017","heartbeat_front_addr":"172.21.15.7:6823/206561017","state":["exists","up"]},{"osd":3,"uuid":"c1fa594a-67c7-4f8f-a745-74ef06dfd43c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6801","nonce":157322165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6803","nonce":157322165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6807","nonce":157322165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":157322165},{"type":"v1","addr":"172.21.15.19:6805","nonce":157322165}]},"public_addr":"172.21.15.19:6801/157322165","cluster_addr":"172.21.15.19:6803/157322165","heartbeat_back_addr":"172.21.15.19:6807/157322165","heartbeat_front_addr":"172.21.15.19:6805/157322165","state":["exists","up"]},{"osd":4,"uuid":"55356509-5ce4-4cb1-9195-3911332ec8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6809","nonce":2504473950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6811","nonce":2504473950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6815","nonce":2504473950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":2504473950},{"type":"v1","addr":"172.21.15.19:6813","nonce":2504473950}]},"public_addr":"172.21.15.19:6809/2504473950","cluster_addr":"172.21.15.19:6811/2504473950","heartbeat_back_addr":"172.21.15.19:6815/2504473950","heartbeat_front_addr":"172.21.15.19:6813/2504473950","state":["exists","up"]},{"osd":5,"uuid":"576cd741-4e7d-421a-a5dd-67ceb0af2440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6817","nonce":1303105831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6819","nonce":1303105831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6823","nonce":1303105831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":1303105831},{"type":"v1","addr":"172.21.15.19:6821","nonce":1303105831}]},"public_addr":"172.21.15.19:6817/1303105831","cluster_addr":"172.21.15.19:6819/1303105831","heartbeat_back_addr":"172.21.15.19:6823/1303105831","heartbeat_front_addr":"172.21.15.19:6821/1303105831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:54:54.847162+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:23.867331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:50.950660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:14.805620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:40.511669+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:57:05.957371+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6827/2352390069":"2024-03-17T08:58:44.727553+0000","172.21.15.7:6826/2352390069":"2024-03-17T08:58:44.727553+0000","172.21.15.7:0/3147176153":"2024-03-17T08:52:05.202127+0000","172.21.15.7:6801/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:6800/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/3022930672":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/4215733143":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/3238187551":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/3257972232":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/1939052835":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/1002495884":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6800/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/1599723782":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6801/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/1698981906":"2024-03-17T08:53:41.312979+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd41c069210 msgr2=0x7fd41c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd41c069210 0x7fd41c06b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd4300fde40 tx=0x7fd42c00b040 comp rx=0 tx=0).stop 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 msgr2=0x7fd43010d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 0x7fd43010d240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd420003080 tx=0x7fd420009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 shutdown_connections 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fd41c069210 0x7fd41c06b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4300ff350 0x7fd43010d240 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 --2- 172.21.15.7:0/711477469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd430064fd0 0x7fd43010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.782+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 >> 172.21.15.7:0/711477469 conn(0x7fd4300f6840 msgr2=0x7fd4300f6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.783+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 shutdown_connections 2024-03-16T08:58:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.783+0000 7fd43815d640 1 -- 172.21.15.7:0/711477469 wait complete. 2024-03-16T08:58:57.796 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:55:53.635803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-16T08:58:27.077630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-16T08:58:27.912287+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T08:58:57.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T08:58:57.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.895+0000 7f839275d640 1 -- 172.21.15.7:0/3661848928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 msgr2=0x7f838c0fa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:57.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.895+0000 7f839275d640 1 --2- 172.21.15.7:0/3661848928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 0x7f838c0fa2a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f837c0048d0 tx=0x7f837c009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.895+0000 7f839275d640 1 -- 172.21.15.7:0/3661848928 shutdown_connections 2024-03-16T08:58:57.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.895+0000 7f839275d640 1 --2- 172.21.15.7:0/3661848928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 0x7f838c0fa2a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.895+0000 7f839275d640 1 --2- 172.21.15.7:0/3661848928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f838c0fd7f0 0x7f838c0fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.895+0000 7f839275d640 1 -- 172.21.15.7:0/3661848928 >> 172.21.15.7:0/3661848928 conn(0x7f838c0f5ff0 msgr2=0x7f838c0f8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:57.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.895+0000 7f839275d640 1 -- 172.21.15.7:0/3661848928 shutdown_connections 2024-03-16T08:58:57.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.895+0000 7f839275d640 1 -- 172.21.15.7:0/3661848928 wait complete. 2024-03-16T08:58:57.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f839275d640 1 Processor -- start 2024-03-16T08:58:57.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f839275d640 1 -- start start 2024-03-16T08:58:57.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f839275d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f838c0fd7f0 0x7f838c0fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:57.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f839275d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 0x7f838c0fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:57.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f839275d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0fd450 con 0x7f838c0fe140 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f839275d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0fba10 con 0x7f838c0fd7f0 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f838b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 0x7f838c0fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f838b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 0x7f838c0fb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57362/0 (socket says 172.21.15.7:57362) 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.896+0000 7f838b7fe640 1 -- 172.21.15.7:0/7973963 learned_addr learned my addr 172.21.15.7:0/7973963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f838bfff640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f838c0fd7f0 0x7f838c0fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f838b7fe640 1 -- 172.21.15.7:0/7973963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f838c0fd7f0 msgr2=0x7f838c0fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f838b7fe640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f838c0fd7f0 0x7f838c0fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f838b7fe640 1 -- 172.21.15.7:0/7973963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c005040 con 0x7f838c0fe140 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f838b7fe640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 0x7f838c0fb4d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f837c02bda0 tx=0x7f837c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f8388ff9640 1 -- 172.21.15.7:0/7973963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f837c0094c0 con 0x7f838c0fe140 2024-03-16T08:58:57.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c0fbc90 con 0x7f838c0fe140 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f8388ff9640 1 -- 172.21.15.7:0/7973963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f837c009620 con 0x7f838c0fe140 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f8388ff9640 1 -- 172.21.15.7:0/7973963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f837c03b640 con 0x7f838c0fe140 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.897+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c0fc1b0 con 0x7f838c0fe140 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.898+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c105540 con 0x7f838c0fe140 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.899+0000 7f8388ff9640 1 -- 172.21.15.7:0/7973963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f837c02f070 con 0x7f838c0fe140 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.899+0000 7f8388ff9640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8374069210 0x7f837406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.899+0000 7f8388ff9640 1 -- 172.21.15.7:0/7973963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f837c0a9720 con 0x7f838c0fe140 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.899+0000 7f838bfff640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8374069210 0x7f837406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:57.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.899+0000 7f838bfff640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8374069210 0x7f837406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8380004ed0 tx=0x7f838000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:57.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:57.903+0000 7f8388ff9640 1 -- 172.21.15.7:0/7973963 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f837c075980 con 0x7f838c0fe140 2024-03-16T08:58:57.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:57 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/2516389876' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-16T08:58:58.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.074+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f838c114ec0 con 0x7f838c0fe140 2024-03-16T08:58:58.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.074+0000 7f8388ff9640 1 -- 172.21.15.7:0/7973963 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f837c078fa0 con 0x7f838c0fe140 2024-03-16T08:58:58.077 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-16T08:58:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8374069210 msgr2=0x7f837406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8374069210 0x7f837406b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8380004ed0 tx=0x7f838000b040 comp rx=0 tx=0).stop 2024-03-16T08:58:58.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 msgr2=0x7f838c0fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 0x7f838c0fb4d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f837c02bda0 tx=0x7f837c009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 shutdown_connections 2024-03-16T08:58:58.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8374069210 0x7f837406b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f838c0fe140 0x7f838c0fb4d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 --2- 172.21.15.7:0/7973963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f838c0fd7f0 0x7f838c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 >> 172.21.15.7:0/7973963 conn(0x7f838c0f5ff0 msgr2=0x7f838c0f76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:58.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 shutdown_connections 2024-03-16T08:58:58.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.076+0000 7f839275d640 1 -- 172.21.15.7:0/7973963 wait complete. 2024-03-16T08:58:58.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-16T08:58:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.189+0000 7f16104ae640 1 -- 172.21.15.7:0/1065589102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fc390 msgr2=0x7f16080fc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.189+0000 7f16104ae640 1 --2- 172.21.15.7:0/1065589102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fc390 0x7f16080fc800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0048d0 tx=0x7f15fc009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.189+0000 7f16104ae640 1 -- 172.21.15.7:0/1065589102 shutdown_connections 2024-03-16T08:58:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.189+0000 7f16104ae640 1 --2- 172.21.15.7:0/1065589102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fc390 0x7f16080fc800 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.189+0000 7f16104ae640 1 --2- 172.21.15.7:0/1065589102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16080fb0c0 0x7f16080fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.189+0000 7f16104ae640 1 -- 172.21.15.7:0/1065589102 >> 172.21.15.7:0/1065589102 conn(0x7f16080f6860 msgr2=0x7f16080f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.189+0000 7f16104ae640 1 -- 172.21.15.7:0/1065589102 shutdown_connections 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.189+0000 7f16104ae640 1 -- 172.21.15.7:0/1065589102 wait complete. 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f16104ae640 1 Processor -- start 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f16104ae640 1 -- start start 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f16104ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fb0c0 0x7f16080fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f16104ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16080fc390 0x7f16080fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f16104ae640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16080ff600 con 0x7f16080fb0c0 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f16104ae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16080fdc10 con 0x7f16080fc390 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f160e24a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fb0c0 0x7f16080fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f160e24a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fb0c0 0x7f16080fefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57380/0 (socket says 172.21.15.7:57380) 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f160e24a640 1 -- 172.21.15.7:0/4270530666 learned_addr learned my addr 172.21.15.7:0/4270530666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f160da49640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16080fc390 0x7f16080fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f160e24a640 1 -- 172.21.15.7:0/4270530666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16080fc390 msgr2=0x7f16080fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f160e24a640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16080fc390 0x7f16080fd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f160e24a640 1 -- 172.21.15.7:0/4270530666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc005040 con 0x7f16080fb0c0 2024-03-16T08:58:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f160e24a640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fb0c0 0x7f16080fefb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f160400c830 tx=0x7f1604004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f15faffd640 1 -- 172.21.15.7:0/4270530666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160400a850 con 0x7f16080fb0c0 2024-03-16T08:58:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f15faffd640 1 -- 172.21.15.7:0/4270530666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f160400ae80 con 0x7f16080fb0c0 2024-03-16T08:58:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16080fdea0 con 0x7f16080fb0c0 2024-03-16T08:58:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f15faffd640 1 -- 172.21.15.7:0/4270530666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1604012ea0 con 0x7f16080fb0c0 2024-03-16T08:58:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.190+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16080fe3c0 con 0x7f16080fb0c0 2024-03-16T08:58:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.192+0000 7f15faffd640 1 -- 172.21.15.7:0/4270530666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f1604012400 con 0x7f16080fb0c0 2024-03-16T08:58:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.192+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1608104420 con 0x7f16080fb0c0 2024-03-16T08:58:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.192+0000 7f15faffd640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f15f4069210 0x7f15f406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.192+0000 7f15faffd640 1 -- 172.21.15.7:0/4270530666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1604087440 con 0x7f16080fb0c0 2024-03-16T08:58:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.192+0000 7f160da49640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f15f4069210 0x7f15f406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.193+0000 7f160da49640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f15f4069210 0x7f15f406b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16080fed60 tx=0x7f15fc009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.197+0000 7f15faffd640 1 -- 172.21.15.7:0/4270530666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f1604053720 con 0x7f16080fb0c0 2024-03-16T08:58:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.399+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f160804bcc0 con 0x7f16080fb0c0 2024-03-16T08:58:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.399+0000 7f15faffd640 1 -- 172.21.15.7:0/4270530666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f16040126b0 con 0x7f16080fb0c0 2024-03-16T08:58:58.401 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f15f4069210 msgr2=0x7f15f406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f15f4069210 0x7f15f406b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16080fed60 tx=0x7f15fc009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fb0c0 msgr2=0x7f16080fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fb0c0 0x7f16080fefb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f160400c830 tx=0x7f1604004430 comp rx=0 tx=0).stop 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 shutdown_connections 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f15f4069210 0x7f15f406b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16080fc390 0x7f16080fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 --2- 172.21.15.7:0/4270530666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16080fb0c0 0x7f16080fefb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.401+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 >> 172.21.15.7:0/4270530666 conn(0x7f16080f6860 msgr2=0x7f16080f8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.402+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 shutdown_connections 2024-03-16T08:58:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.402+0000 7f16104ae640 1 -- 172.21.15.7:0/4270530666 wait complete. 2024-03-16T08:58:58.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.518+0000 7fcf986ae640 1 -- 172.21.15.7:0/523257058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 msgr2=0x7fcf901016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.518+0000 7fcf986ae640 1 --2- 172.21.15.7:0/523257058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 0x7fcf901016c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcf800048d0 tx=0x7fcf80009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.518+0000 7fcf986ae640 1 -- 172.21.15.7:0/523257058 shutdown_connections 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.518+0000 7fcf986ae640 1 --2- 172.21.15.7:0/523257058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 0x7fcf901016c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.518+0000 7fcf986ae640 1 --2- 172.21.15.7:0/523257058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90065030 0x7fcf900feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.518+0000 7fcf986ae640 1 -- 172.21.15.7:0/523257058 >> 172.21.15.7:0/523257058 conn(0x7fcf900f6870 msgr2=0x7fcf900f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.518+0000 7fcf986ae640 1 -- 172.21.15.7:0/523257058 shutdown_connections 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.518+0000 7fcf986ae640 1 -- 172.21.15.7:0/523257058 wait complete. 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf986ae640 1 Processor -- start 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf986ae640 1 -- start start 2024-03-16T08:58:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf986ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90065030 0x7fcf900fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf986ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 0x7fcf900faca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf986ae640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900fcbd0 con 0x7fcf900ff2e0 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf986ae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900fb1e0 con 0x7fcf90065030 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf95c49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 0x7fcf900faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf95c49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 0x7fcf900faca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57394/0 (socket says 172.21.15.7:57394) 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf95c49640 1 -- 172.21.15.7:0/1361717725 learned_addr learned my addr 172.21.15.7:0/1361717725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf9644a640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90065030 0x7fcf900fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf95c49640 1 -- 172.21.15.7:0/1361717725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90065030 msgr2=0x7fcf900fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf95c49640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90065030 0x7fcf900fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf95c49640 1 -- 172.21.15.7:0/1361717725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf80005040 con 0x7fcf900ff2e0 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.519+0000 7fcf95c49640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 0x7fcf900faca0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf80007da0 tx=0x7fcf80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.520+0000 7fcf86ffd640 1 -- 172.21.15.7:0/1361717725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf800094c0 con 0x7fcf900ff2e0 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.520+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf900fb410 con 0x7fcf900ff2e0 2024-03-16T08:58:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.520+0000 7fcf86ffd640 1 -- 172.21.15.7:0/1361717725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf80009620 con 0x7fcf900ff2e0 2024-03-16T08:58:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.520+0000 7fcf86ffd640 1 -- 172.21.15.7:0/1361717725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf8003b620 con 0x7fcf900ff2e0 2024-03-16T08:58:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.520+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf900fb930 con 0x7fcf900ff2e0 2024-03-16T08:58:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.521+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9004bcc0 con 0x7fcf900ff2e0 2024-03-16T08:58:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.522+0000 7fcf86ffd640 1 -- 172.21.15.7:0/1361717725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fcf8002f070 con 0x7fcf900ff2e0 2024-03-16T08:58:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.522+0000 7fcf86ffd640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcf7c069210 0x7fcf7c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.522+0000 7fcf86ffd640 1 -- 172.21.15.7:0/1361717725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcf800a9750 con 0x7fcf900ff2e0 2024-03-16T08:58:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.522+0000 7fcf9644a640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcf7c069210 0x7fcf7c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.523+0000 7fcf9644a640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcf7c069210 0x7fcf7c06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf90064d90 tx=0x7fcf8c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.530+0000 7fcf86ffd640 1 -- 172.21.15.7:0/1361717725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fcf800759b0 con 0x7fcf900ff2e0 2024-03-16T08:58:58.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.704+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fcf900fbac0 con 0x7fcf900ff2e0 2024-03-16T08:58:58.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.705+0000 7fcf86ffd640 1 -- 172.21.15.7:0/1361717725 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcf80078fd0 con 0x7fcf900ff2e0 2024-03-16T08:58:58.707 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.707+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcf7c069210 msgr2=0x7fcf7c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.707+0000 7fcf986ae640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcf7c069210 0x7fcf7c06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf90064d90 tx=0x7fcf8c00b040 comp rx=0 tx=0).stop 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.707+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 msgr2=0x7fcf900faca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.707+0000 7fcf986ae640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 0x7fcf900faca0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf80007da0 tx=0x7fcf80009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.708+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 shutdown_connections 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.708+0000 7fcf986ae640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcf7c069210 0x7fcf7c06b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.708+0000 7fcf986ae640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf900ff2e0 0x7fcf900faca0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.708+0000 7fcf986ae640 1 --2- 172.21.15.7:0/1361717725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90065030 0x7fcf900fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.708+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 >> 172.21.15.7:0/1361717725 conn(0x7fcf900f6870 msgr2=0x7fcf900f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:58.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.708+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 shutdown_connections 2024-03-16T08:58:58.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.708+0000 7fcf986ae640 1 -- 172.21.15.7:0/1361717725 wait complete. 2024-03-16T08:58:58.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T08:58:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.823+0000 7f1f6a86e640 1 -- 172.21.15.7:0/1795502003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fa680 msgr2=0x7f1f640faa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.823+0000 7f1f6a86e640 1 --2- 172.21.15.7:0/1795502003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fa680 0x7f1f640faa90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f540048d0 tx=0x7f1f54009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.823+0000 7f1f6a86e640 1 -- 172.21.15.7:0/1795502003 shutdown_connections 2024-03-16T08:58:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.823+0000 7f1f6a86e640 1 --2- 172.21.15.7:0/1795502003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f640fafd0 0x7f1f64101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.823+0000 7f1f6a86e640 1 --2- 172.21.15.7:0/1795502003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fa680 0x7f1f640faa90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.823+0000 7f1f6a86e640 1 -- 172.21.15.7:0/1795502003 >> 172.21.15.7:0/1795502003 conn(0x7f1f640f6080 msgr2=0x7f1f640f84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.823+0000 7f1f6a86e640 1 -- 172.21.15.7:0/1795502003 shutdown_connections 2024-03-16T08:58:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.823+0000 7f1f6a86e640 1 -- 172.21.15.7:0/1795502003 wait complete. 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f6a86e640 1 Processor -- start 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f6a86e640 1 -- start start 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f6a86e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f640fa680 0x7f1f640fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f6a86e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fafd0 0x7f1f640fb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f6a86e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640fd5b0 con 0x7f1f640fafd0 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f6a86e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640fbbc0 con 0x7f1f640fa680 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fafd0 0x7f1f640fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f637fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fafd0 0x7f1f640fb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57406/0 (socket says 172.21.15.7:57406) 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f637fe640 1 -- 172.21.15.7:0/2029394778 learned_addr learned my addr 172.21.15.7:0/2029394778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f63fff640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f640fa680 0x7f1f640fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f637fe640 1 -- 172.21.15.7:0/2029394778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f640fa680 msgr2=0x7f1f640fcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f637fe640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f640fa680 0x7f1f640fcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f637fe640 1 -- 172.21.15.7:0/2029394778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f54005040 con 0x7f1f640fafd0 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.824+0000 7f1f63fff640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f640fa680 0x7f1f640fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T08:58:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.825+0000 7f1f637fe640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fafd0 0x7f1f640fb680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c002840 tx=0x7f1f4c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:58.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.825+0000 7f1f60ff9640 1 -- 172.21.15.7:0/2029394778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f4c007c30 con 0x7f1f640fafd0 2024-03-16T08:58:58.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.825+0000 7f1f60ff9640 1 -- 172.21.15.7:0/2029394778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f4c00f040 con 0x7f1f640fafd0 2024-03-16T08:58:58.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.825+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f640fbe50 con 0x7f1f640fafd0 2024-03-16T08:58:58.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.825+0000 7f1f60ff9640 1 -- 172.21.15.7:0/2029394778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f4c0094e0 con 0x7f1f640fafd0 2024-03-16T08:58:58.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.825+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640fc370 con 0x7f1f640fafd0 2024-03-16T08:58:58.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.826+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6404bcc0 con 0x7f1f640fafd0 2024-03-16T08:58:58.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.826+0000 7f1f60ff9640 1 -- 172.21.15.7:0/2029394778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f1f4c0097a0 con 0x7f1f640fafd0 2024-03-16T08:58:58.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.827+0000 7f1f60ff9640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1f50069210 0x7f1f5006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:58:58.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.827+0000 7f1f63fff640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1f50069210 0x7f1f5006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:58:58.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.827+0000 7f1f60ff9640 1 -- 172.21.15.7:0/2029394778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1f4c0871a0 con 0x7f1f640fafd0 2024-03-16T08:58:58.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.827+0000 7f1f63fff640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1f50069210 0x7f1f5006b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1f540317e0 tx=0x7f1f54009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:58:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:58.831+0000 7f1f60ff9640 1 -- 172.21.15.7:0/2029394778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f1f4c053400 con 0x7f1f640fafd0 2024-03-16T08:58:58.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-16T08:58:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/711477469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:58:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/7973963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:58:58.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:58 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/4270530666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T08:58:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-16T08:58:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:58:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:58:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:58:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/711477469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T08:58:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/7973963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T08:58:58.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:58 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/4270530666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T08:58:59.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.059+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f64104d60 con 0x7f1f640fafd0 2024-03-16T08:58:59.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.060+0000 7f1f60ff9640 1 -- 172.21.15.7:0/2029394778 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4694 (secure 0 0 0) 0x7f1f4c056a20 con 0x7f1f640fafd0 2024-03-16T08:58:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T08:58:59.062 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2680162193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2680162193},{"type":"v1","addr":"172.21.15.7:6827","nonce":2680162193}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T08:58:48.744076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14482,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14482":{"gid":14482,"name":"cephfs.smithi007.znurst","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.7:6829/3212461733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3212461733},{"type":"v1","addr":"172.21.15.7:6829","nonce":3212461733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi019.bncsil","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6825/4120977378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":4120977378},{"type":"v1","addr":"172.21.15.19:6825","nonce":4120977378}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-16T08:58:59.062 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-03-16T08:58:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.061+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1f50069210 msgr2=0x7f1f5006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.061+0000 7f1f6a86e640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1f50069210 0x7f1f5006b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1f540317e0 tx=0x7f1f54009da0 comp rx=0 tx=0).stop 2024-03-16T08:58:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.061+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fafd0 msgr2=0x7f1f640fb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:58:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.062+0000 7f1f6a86e640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fafd0 0x7f1f640fb680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c002840 tx=0x7f1f4c004570 comp rx=0 tx=0).stop 2024-03-16T08:58:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.062+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 shutdown_connections 2024-03-16T08:58:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.062+0000 7f1f6a86e640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f1f50069210 0x7f1f5006b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.062+0000 7f1f6a86e640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640fafd0 0x7f1f640fb680 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:59.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.062+0000 7f1f6a86e640 1 --2- 172.21.15.7:0/2029394778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f640fa680 0x7f1f640fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:58:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.062+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 >> 172.21.15.7:0/2029394778 conn(0x7f1f640f6080 msgr2=0x7f1f640ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:58:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.062+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 shutdown_connections 2024-03-16T08:58:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:58:59.062+0000 7f1f6a86e640 1 -- 172.21.15.7:0/2029394778 wait complete. 2024-03-16T08:58:59.074 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 18} 2024-03-16T08:58:59.074 INFO:teuthology.run_tasks:Running task kclient... 2024-03-16T08:58:59.099 INFO:tasks.kclient:Mounting kernel clients... 2024-03-16T08:58:59.099 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-16T08:58:59.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:58:59.100 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-16T08:58:59.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:58:59.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-03-16T08:58:59.184 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-03-16T08:58:59.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:58:59.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:58:59.185 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-16T08:58:59.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:58:59.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-03-16T08:58:59.280 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-03-16T08:58:59.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:58:59.281 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-16T08:58:59.281 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T08:58:59.281 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T08:58:59.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-16T08:58:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:58:59.919 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:59 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1361717725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T08:58:59.920 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:58:59 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/2029394778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:58:59.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:59 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1361717725' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T08:58:59.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:58:59 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/2029394778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T08:59:00.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:00 smithi019 ceph-mon[36407]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-16T08:59:00.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:00 smithi007 ceph-mon[29470]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-16T08:59:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 -- 172.21.15.7:0/2430312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 msgr2=0x7f8ffc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 --2- 172.21.15.7:0/2430312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 0x7f8ffc0fce10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0004940 tx=0x7f8ff0009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 -- 172.21.15.7:0/2430312983 shutdown_connections 2024-03-16T08:59:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 --2- 172.21.15.7:0/2430312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 0x7f8ffc0fce10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 --2- 172.21.15.7:0/2430312983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc102e00 0x7f8ffc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 -- 172.21.15.7:0/2430312983 >> 172.21.15.7:0/2430312983 conn(0x7f8ffc0f89b0 msgr2=0x7f8ffc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 -- 172.21.15.7:0/2430312983 shutdown_connections 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 -- 172.21.15.7:0/2430312983 wait complete. 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 Processor -- start 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.223+0000 7f9002eb1700 1 -- start start 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f9002eb1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc102e00 0x7f8ffc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f9002eb1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 0x7f8ffc070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f9002eb1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc072210 con 0x7f8ffc1037a0 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f9002eb1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc070d40 con 0x7f8ffc102e00 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ffbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 0x7f8ffc070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ffbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 0x7f8ffc070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57420/0 (socket says 172.21.15.7:57420) 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ffbfff700 1 -- 172.21.15.7:0/617769512 learned_addr learned my addr 172.21.15.7:0/617769512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f9000c4d700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc102e00 0x7f8ffc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ffbfff700 1 -- 172.21.15.7:0/617769512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc102e00 msgr2=0x7f8ffc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ffbfff700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc102e00 0x7f8ffc0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ffbfff700 1 -- 172.21.15.7:0/617769512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0005040 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ffbfff700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 0x7f8ffc070800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009100 tx=0x7f8ff0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ff97fa700 1 -- 172.21.15.7:0/617769512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff000e040 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f8ff97fa700 1 -- 172.21.15.7:0/617769512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ff0009af0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.224+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc070fc0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.225+0000 7f8ff97fa700 1 -- 172.21.15.7:0/617769512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ff001c3f0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.225+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc11a0d0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.225+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc1052d0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.226+0000 7f8ff97fa700 1 -- 172.21.15.7:0/617769512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f8ff000baf0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.226+0000 7f8ff97fa700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8fe40692a0 0x7f8fe406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.227+0000 7f9000c4d700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8fe40692a0 0x7f8fe406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.227+0000 7f8ff97fa700 1 -- 172.21.15.7:0/617769512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8ff0088350 con 0x7f8ffc1037a0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.227+0000 7f9000c4d700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8fe40692a0 0x7f8fe406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8fec004eb0 tx=0x7f8fec00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.231+0000 7f8ff97fa700 1 -- 172.21.15.7:0/617769512 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f8ff00546d0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.472+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8ffc109ee0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.473+0000 7f8ff97fa700 1 -- 172.21.15.7:0/617769512 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8ff0057cf0 con 0x7f8ffc1037a0 2024-03-16T08:59:01.475 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.474+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8fe40692a0 msgr2=0x7f8fe406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.474+0000 7f9002eb1700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8fe40692a0 0x7f8fe406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8fec004eb0 tx=0x7f8fec00b040 comp rx=0 tx=0).stop 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.474+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 msgr2=0x7f8ffc070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.474+0000 7f9002eb1700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 0x7f8ffc070800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009100 tx=0x7f8ff0009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.475+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 shutdown_connections 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.475+0000 7f9002eb1700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8fe40692a0 0x7f8fe406b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.475+0000 7f9002eb1700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc102e00 0x7f8ffc0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.475+0000 7f9002eb1700 1 --2- 172.21.15.7:0/617769512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ffc1037a0 0x7f8ffc070800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.475+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 >> 172.21.15.7:0/617769512 conn(0x7f8ffc0f89b0 msgr2=0x7f8ffc1021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.475+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 shutdown_connections 2024-03-16T08:59:01.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:01.475+0000 7f9002eb1700 1 -- 172.21.15.7:0/617769512 wait complete. 2024-03-16T08:59:01.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:01 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/617769512' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T08:59:01.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:01 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/617769512' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T08:59:02.109 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T08:59:02.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:02.110 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-03-16T08:59:02.126 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T08:59:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T08:59:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T08:59:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:59:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-03-16T08:59:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-16T08:59:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42241sec preferred_lft 42241sec 2024-03-16T08:59:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-16T08:59:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:59:02.128 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:59:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-03-16T08:59:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-03-16T08:59:02.128 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T08:59:02.128 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:02.128 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-16T08:59:02.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-03-16T08:59:02.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-03-16T08:59:02.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-03-16T08:59:02.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T08:59:02.129 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-16T08:59:02.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:02.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:02.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:02.322 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T08:59:02.348 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-03-16T08:59:02.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:02.350 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-03-16T08:59:02.420 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-03-16T08:59:02.420 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T08:59:02.420 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-16T08:59:02.420 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-16T08:59:02.420 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T08:59:02.420 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:02.421 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-16T08:59:02.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T08:59:02.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T08:59:02.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T08:59:02.421 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-16T08:59:02.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:02.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:02.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:02.725 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-16T08:59:02.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:02.740 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-03-16T08:59:02.794 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:02.794 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-16T08:59:02.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T08:59:02.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-16T08:59:02.795 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-16T08:59:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:02.893 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:02 smithi007 ceph-mon[29470]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-16T08:59:02.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:02.916 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-16T08:59:02.916 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:02.916 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-16T08:59:02.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-16T08:59:02.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T08:59:02.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-16T08:59:02.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-16T08:59:02.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-16T08:59:02.917 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-16T08:59:02.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:02 smithi019 ceph-mon[36407]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-16T08:59:02.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:03.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:03.058 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:03.058 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-16T08:59:03.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-03-16T08:59:03.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T08:59:03.058 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-16T08:59:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:03.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:03.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T08:59:03.139 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T08:59:03.153 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T08:59:03.154 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T08:59:03.154 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T08:59:03.208 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-16T08:59:03.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:03.208 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T08:59:03.293 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T08:59:03.294 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T08:59:03.294 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-03-16T08:59:03.418 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T08:59:03.418 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T08:59:03.418 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-03-16T08:59:03.418 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.19:3300" 2024-03-16T08:59:03.418 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T08:59:03.423 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T08:59:03.423 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T08:59:03.423 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-03-16T08:59:03.423 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.19:3300" 2024-03-16T08:59:03.423 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T08:59:03.423 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.19:6789" 2024-03-16T08:59:03.423 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.19:6789:/ 2024-03-16T08:59:03.423 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=4c46b77e-e372-11ee-95c9-87774f69a715" will pass to kernel 2024-03-16T08:59:03.424 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-16T08:59:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-03-16T08:59:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T08:59:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T08:59:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T08:59:03.426 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T08:59:03.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:03.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T08:59:03.454 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:59:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:59:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:03.554 INFO:teuthology.orchestra.run.smithi007.stdout:client24329 2024-03-16T08:59:03.554 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T08:59:03.554 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T08:59:03.571 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:59:03.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:59:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:03.670 INFO:teuthology.orchestra.run.smithi007.stdout:client24329 2024-03-16T08:59:03.671 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:59:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:59:03.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:03.731 INFO:teuthology.orchestra.run.smithi007.stdout:client24329 2024-03-16T08:59:03.731 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4c46b77e-e372-11ee-95c9-87774f69a715.client24329/status') 2024-03-16T08:59:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:03.794 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.24329 (3)172.21.15.7:0/34442062 2024-03-16T08:59:03.795 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-03-16T08:59:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-03-16T08:59:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-03-16T08:59:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:68 bytes copied, 2.1254e-05 s, 3.2 MB/s 2024-03-16T08:59:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:03.802 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-16T08:59:03.802 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T08:59:03.802 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-16T08:59:03.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-16T08:59:04.065 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:59:04.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:04 smithi007 ceph-mon[29470]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-16T08:59:04.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:04 smithi019 ceph-mon[36407]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-16T08:59:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.765+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1590504018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 msgr2=0x7f3e580fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.765+0000 7f3e5d7a4700 1 --2- 172.21.15.7:0/1590504018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 0x7f3e580fbd80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e40004940 tx=0x7f3e40009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.765+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1590504018 shutdown_connections 2024-03-16T08:59:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.765+0000 7f3e5d7a4700 1 --2- 172.21.15.7:0/1590504018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 0x7f3e580fbd80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.765+0000 7f3e5d7a4700 1 --2- 172.21.15.7:0/1590504018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58101d70 0x7f3e58102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.765+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1590504018 >> 172.21.15.7:0/1590504018 conn(0x7f3e58072bc0 msgr2=0x7f3e58072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.765+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1590504018 shutdown_connections 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.765+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1590504018 wait complete. 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e5d7a4700 1 Processor -- start 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e5d7a4700 1 -- start start 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e5d7a4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58101d70 0x7f3e580fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e5d7a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 0x7f3e580fdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e5d7a4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580fe4c0 con 0x7f3e58102710 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e5d7a4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580fe600 con 0x7f3e58101d70 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e567fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 0x7f3e580fdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e567fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 0x7f3e580fdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36172/0 (socket says 172.21.15.7:36172) 2024-03-16T08:59:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e567fc700 1 -- 172.21.15.7:0/1937836546 learned_addr learned my addr 172.21.15.7:0/1937836546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e56ffd700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58101d70 0x7f3e580fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e567fc700 1 -- 172.21.15.7:0/1937836546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58101d70 msgr2=0x7f3e580fd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e567fc700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58101d70 0x7f3e580fd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:05.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.766+0000 7f3e567fc700 1 -- 172.21.15.7:0/1937836546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40005040 con 0x7f3e58102710 2024-03-16T08:59:05.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.767+0000 7f3e567fc700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 0x7f3e580fdf30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009180 tx=0x7f3e40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.767+0000 7f3e4f7fe700 1 -- 172.21.15.7:0/1937836546 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4000e040 con 0x7f3e58102710 2024-03-16T08:59:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.767+0000 7f3e4f7fe700 1 -- 172.21.15.7:0/1937836546 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e40013420 con 0x7f3e58102710 2024-03-16T08:59:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.767+0000 7f3e4f7fe700 1 -- 172.21.15.7:0/1937836546 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e4001c610 con 0x7f3e58102710 2024-03-16T08:59:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.767+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e580fe850 con 0x7f3e58102710 2024-03-16T08:59:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.767+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5811a290 con 0x7f3e58102710 2024-03-16T08:59:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.768+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e580fd4c0 con 0x7f3e58102710 2024-03-16T08:59:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.768+0000 7f3e4f7fe700 1 -- 172.21.15.7:0/1937836546 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f3e4000baf0 con 0x7f3e58102710 2024-03-16T08:59:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.769+0000 7f3e4f7fe700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3e440692a0 0x7f3e4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.769+0000 7f3e4f7fe700 1 -- 172.21.15.7:0/1937836546 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3e40088330 con 0x7f3e58102710 2024-03-16T08:59:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.769+0000 7f3e56ffd700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3e440692a0 0x7f3e4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.769+0000 7f3e56ffd700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3e440692a0 0x7f3e4406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e50007fd0 tx=0x7f3e50009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:05.773+0000 7f3e4f7fe700 1 -- 172.21.15.7:0/1937836546 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f3e40054610 con 0x7f3e58102710 2024-03-16T08:59:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.008+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3e58108e50 con 0x7f3e58102710 2024-03-16T08:59:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.009+0000 7f3e4f7fe700 1 -- 172.21.15.7:0/1937836546 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7f3e40057c30 con 0x7f3e58102710 2024-03-16T08:59:06.011 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.010+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3e440692a0 msgr2=0x7f3e4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.010+0000 7f3e5d7a4700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3e440692a0 0x7f3e4406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e50007fd0 tx=0x7f3e50009040 comp rx=0 tx=0).stop 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.010+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 msgr2=0x7f3e580fdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.010+0000 7f3e5d7a4700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 0x7f3e580fdf30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009180 tx=0x7f3e40009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.010+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 shutdown_connections 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.011+0000 7f3e5d7a4700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f3e440692a0 0x7f3e4406b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.011+0000 7f3e5d7a4700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e58101d70 0x7f3e580fd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.011+0000 7f3e5d7a4700 1 --2- 172.21.15.7:0/1937836546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e58102710 0x7f3e580fdf30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.011+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 >> 172.21.15.7:0/1937836546 conn(0x7f3e58072bc0 msgr2=0x7f3e580659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.011+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 shutdown_connections 2024-03-16T08:59:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:06.011+0000 7f3e5d7a4700 1 -- 172.21.15.7:0/1937836546 wait complete. 2024-03-16T08:59:06.643 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T08:59:06.643 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T08:59:06.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-16T08:59:06.643 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-16T08:59:06.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-16T08:59:06.643 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T08:59:06.644 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T08:59:06.644 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T08:59:06.644 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-16T08:59:06.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:06.644 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-16T08:59:06.662 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T08:59:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T08:59:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T08:59:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:59:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42260sec preferred_lft 42260sec 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-16T08:59:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-16T08:59:06.664 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T08:59:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-16T08:59:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-16T08:59:06.664 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T08:59:06.664 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:06.664 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-16T08:59:06.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-03-16T08:59:06.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-03-16T08:59:06.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-03-16T08:59:06.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T08:59:06.664 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-16T08:59:06.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:06.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:06.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:06.883 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T08:59:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-16T08:59:06.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:06.913 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-16T08:59:06.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:06 smithi019 ceph-mon[36407]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:06.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:06 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1937836546' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T08:59:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:06 smithi007 ceph-mon[29470]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:06 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1937836546' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T08:59:07.027 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-16T08:59:07.028 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T08:59:07.028 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-16T08:59:07.028 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-16T08:59:07.028 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T08:59:07.028 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:07.028 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-16T08:59:07.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T08:59:07.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T08:59:07.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T08:59:07.029 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-16T08:59:07.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:07.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:07.305 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-16T08:59:07.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:07.320 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-03-16T08:59:07.374 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:07.374 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-16T08:59:07.374 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T08:59:07.375 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-16T08:59:07.375 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-16T08:59:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:07.499 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-16T08:59:07.499 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:07.499 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-16T08:59:07.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-16T08:59:07.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T08:59:07.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-16T08:59:07.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-16T08:59:07.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-16T08:59:07.499 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-16T08:59:07.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:07.636 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T08:59:07.636 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-16T08:59:07.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-16T08:59:07.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T08:59:07.636 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-16T08:59:07.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:07.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:07.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T08:59:07.720 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-16T08:59:07.734 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-16T08:59:07.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T08:59:07.734 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-16T08:59:07.788 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-16T08:59:07.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:07.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-16T08:59:07.886 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-16T08:59:07.886 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-16T08:59:07.886 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-16T08:59:08.000 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-16T08:59:08.000 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-16T08:59:08.000 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-16T08:59:08.001 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.19:3300" 2024-03-16T08:59:08.001 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T08:59:08.008 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-16T08:59:08.008 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-16T08:59:08.008 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-16T08:59:08.008 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.19:3300" 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.19:6789" 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.19:6789:/ 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=4c46b77e-e372-11ee-95c9-87774f69a715" will pass to kernel 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T08:59:08.009 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T08:59:08.011 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T08:59:08.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T08:59:08.012 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-16T08:59:08.034 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:07 smithi019 ceph-mon[36407]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T08:59:08.038 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:59:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:59:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:08.134 INFO:teuthology.orchestra.run.smithi019.stdout:client14540 2024-03-16T08:59:08.134 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T08:59:08.134 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T08:59:08.150 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:59:08.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:59:08.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:08.244 INFO:teuthology.orchestra.run.smithi019.stdout:client14540 2024-03-16T08:59:08.244 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T08:59:08.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:07 smithi007 ceph-mon[29470]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T08:59:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T08:59:08.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:08.299 INFO:teuthology.orchestra.run.smithi019.stdout:client14540 2024-03-16T08:59:08.299 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4c46b77e-e372-11ee-95c9-87774f69a715.client14540/status') 2024-03-16T08:59:08.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T08:59:08.359 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.14540 (3)172.21.15.19:0/480350363 2024-03-16T08:59:08.360 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-03-16T08:59:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-03-16T08:59:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-03-16T08:59:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:70 bytes copied, 2.115e-05 s, 3.3 MB/s 2024-03-16T08:59:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-16T08:59:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T08:59:08.367 INFO:teuthology.run_tasks:Running task print... 2024-03-16T08:59:08.377 INFO:teuthology.task.print:**** done client 2024-03-16T08:59:08.377 INFO:teuthology.run_tasks:Running task parallel... 2024-03-16T08:59:08.427 INFO:teuthology.task.parallel:starting parallel... 2024-03-16T08:59:08.427 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-16T08:59:08.427 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-16T08:59:08.428 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T08:59:08.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-16T08:59:08.428 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-16T08:59:08.429 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-16T08:59:08.433 INFO:tasks.workunit:Pulling workunits from ref 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:59:08.434 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-16T08:59:08.434 INFO:tasks.workunit:timeout=3h 2024-03-16T08:59:08.434 INFO:tasks.workunit:cleanup=True 2024-03-16T08:59:08.434 DEBUG:teuthology.orchestra.run.smithi007:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-16T08:59:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-16T08:59:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-16T08:59:08.454 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-03-16T08:59:08.454 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:59:08.454 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-16T08:59:08.454 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-16T08:59:08.454 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-16 08:58:33.691548275 +0000 2024-03-16T08:59:08.454 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-16 08:59:03.449527421 +0000 2024-03-16T08:59:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-16 08:58:33.691548275 +0000 2024-03-16T08:59:08.454 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-16T08:59:08.454 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-16T08:59:08.527 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-16T08:59:08.544 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-16T08:59:08.544 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-03-16T08:59:08.544 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-03-16T08:59:08.545 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T08:59:08.545 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-16T08:59:08.545 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-16T08:59:08.545 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-16 08:59:08.519456277 +0000 2024-03-16T08:59:08.545 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-16 08:59:08.519456277 +0000 2024-03-16T08:59:08.545 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-16 08:58:33.691548275 +0000 2024-03-16T08:59:08.545 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-16T08:59:08.545 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-16T08:59:08.618 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:59:08.619 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:59:08.634 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-16T08:59:08.674 INFO:tasks.workunit.client.1.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-16T08:59:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.368+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1602873119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 msgr2=0x7fc9a80fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.368+0000 7fc9ae7a7700 1 --2- 172.21.15.7:0/1602873119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 0x7fc9a80fc110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc998004940 tx=0x7fc998009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1602873119 shutdown_connections 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 --2- 172.21.15.7:0/1602873119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a80fcfb0 0x7fc9a80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 --2- 172.21.15.7:0/1602873119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 0x7fc9a80fc110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1602873119 >> 172.21.15.7:0/1602873119 conn(0x7fc9a80f7440 msgr2=0x7fc9a80f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1602873119 shutdown_connections 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1602873119 wait complete. 2024-03-16T08:59:10.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 Processor -- start 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 -- start start 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 0x7fc9a8066c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a80fcfb0 0x7fc9a8112720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a8112c60 con 0x7fc9a80fbd00 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.369+0000 7fc9ae7a7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a8112dd0 con 0x7fc9a80fcfb0 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 0x7fc9a8066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 0x7fc9a8066c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36188/0 (socket says 172.21.15.7:36188) 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a7fff700 1 -- 172.21.15.7:0/1496546249 learned_addr learned my addr 172.21.15.7:0/1496546249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:10.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a7fff700 1 -- 172.21.15.7:0/1496546249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a80fcfb0 msgr2=0x7fc9a8112720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a7fff700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a80fcfb0 0x7fc9a8112720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a7fff700 1 -- 172.21.15.7:0/1496546249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc998005040 con 0x7fc9a80fbd00 2024-03-16T08:59:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a7fff700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 0x7fc9a8066c90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc99800b3b0 tx=0x7fc998009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a4ff9700 1 -- 172.21.15.7:0/1496546249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc99800e040 con 0x7fc9a80fbd00 2024-03-16T08:59:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a8113050 con 0x7fc9a80fbd00 2024-03-16T08:59:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a4ff9700 1 -- 172.21.15.7:0/1496546249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc99800bcc0 con 0x7fc9a80fbd00 2024-03-16T08:59:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9a4ff9700 1 -- 172.21.15.7:0/1496546249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc99801c630 con 0x7fc9a80fbd00 2024-03-16T08:59:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.370+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a8113610 con 0x7fc9a80fbd00 2024-03-16T08:59:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.371+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a804c710 con 0x7fc9a80fbd00 2024-03-16T08:59:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.371+0000 7fc9a4ff9700 1 -- 172.21.15.7:0/1496546249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fc998013980 con 0x7fc9a80fbd00 2024-03-16T08:59:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.372+0000 7fc9a4ff9700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc9900692a0 0x7fc99006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.372+0000 7fc9a4ff9700 1 -- 172.21.15.7:0/1496546249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc998088010 con 0x7fc9a80fbd00 2024-03-16T08:59:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.372+0000 7fc9a77fe700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc9900692a0 0x7fc99006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.372+0000 7fc9a77fe700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc9900692a0 0x7fc99006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006fd0 tx=0x7fc99c006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.376+0000 7fc9a4ff9700 1 -- 172.21.15.7:0/1496546249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fc998057d00 con 0x7fc9a80fbd00 2024-03-16T08:59:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.570+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fc9a8104870 con 0x7fc9a80fbd00 2024-03-16T08:59:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.570+0000 7fc9a4ff9700 1 -- 172.21.15.7:0/1496546249 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fc998057890 con 0x7fc9a80fbd00 2024-03-16T08:59:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc9900692a0 msgr2=0x7fc99006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc9900692a0 0x7fc99006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006fd0 tx=0x7fc99c006db0 comp rx=0 tx=0).stop 2024-03-16T08:59:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 msgr2=0x7fc9a8066c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 0x7fc9a8066c90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc99800b3b0 tx=0x7fc998009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 shutdown_connections 2024-03-16T08:59:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc9900692a0 0x7fc99006b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9a80fbd00 0x7fc9a8066c90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 --2- 172.21.15.7:0/1496546249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9a80fcfb0 0x7fc9a8112720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 >> 172.21.15.7:0/1496546249 conn(0x7fc9a80f7440 msgr2=0x7fc9a80f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 shutdown_connections 2024-03-16T08:59:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:10.572+0000 7fc9ae7a7700 1 -- 172.21.15.7:0/1496546249 wait complete. 2024-03-16T08:59:10.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:10 smithi019 ceph-mon[36407]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-16T08:59:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:10 smithi007 ceph-mon[29470]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-16T08:59:11.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-16T08:59:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:59:11.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:11 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:59:11.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:11 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:59:12.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:12 smithi007 ceph-mon[29470]: pgmap v199: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-16T08:59:12.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:12 smithi019 ceph-mon[36407]: pgmap v199: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 -- 172.21.15.7:0/548531480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fed80 msgr2=0x7fcc7c0faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 --2- 172.21.15.7:0/548531480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fed80 0x7fcc7c0faec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc70004940 tx=0x7fcc70009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 -- 172.21.15.7:0/548531480 shutdown_connections 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 --2- 172.21.15.7:0/548531480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fed80 0x7fcc7c0faec0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 --2- 172.21.15.7:0/548531480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc7c0fe430 0x7fcc7c0fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 -- 172.21.15.7:0/548531480 >> 172.21.15.7:0/548531480 conn(0x7fcc7c0f6bd0 msgr2=0x7fcc7c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 -- 172.21.15.7:0/548531480 shutdown_connections 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 -- 172.21.15.7:0/548531480 wait complete. 2024-03-16T08:59:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.320+0000 7fcc812b6700 1 Processor -- start 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc812b6700 1 -- start start 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc812b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fe430 0x7fcc7c0fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc812b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc7c0fed80 0x7fcc7c0fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc812b6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c0fe040 con 0x7fcc7c0fe430 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc812b6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c0fc630 con 0x7fcc7c0fed80 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fe430 0x7fcc7c0fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fe430 0x7fcc7c0fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39160/0 (socket says 172.21.15.7:39160) 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc7affd700 1 -- 172.21.15.7:0/2364473803 learned_addr learned my addr 172.21.15.7:0/2364473803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc7a7fc700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc7c0fed80 0x7fcc7c0fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc7affd700 1 -- 172.21.15.7:0/2364473803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc7c0fed80 msgr2=0x7fcc7c0fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc7affd700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc7c0fed80 0x7fcc7c0fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc7affd700 1 -- 172.21.15.7:0/2364473803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc70005040 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc7affd700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fe430 0x7fcc7c0fda40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400c810 tx=0x7fcc74004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc637fe700 1 -- 172.21.15.7:0/2364473803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc74012810 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c0fc8e0 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc637fe700 1 -- 172.21.15.7:0/2364473803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc74012e40 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.321+0000 7fcc637fe700 1 -- 172.21.15.7:0/2364473803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7400a370 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.322+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c0fcea0 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.322+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c04c710 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.323+0000 7fcc637fe700 1 -- 172.21.15.7:0/2364473803 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fcc74011810 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.323+0000 7fcc637fe700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcc640692a0 0x7fcc6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.323+0000 7fcc637fe700 1 -- 172.21.15.7:0/2364473803 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcc74087670 con 0x7fcc7c0fe430 2024-03-16T08:59:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.323+0000 7fcc7a7fc700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcc640692a0 0x7fcc6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.324+0000 7fcc7a7fc700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcc640692a0 0x7fcc6406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000b3b0 tx=0x7fcc70009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.328+0000 7fcc637fe700 1 -- 172.21.15.7:0/2364473803 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fcc740570d0 con 0x7fcc7c0fe430 2024-03-16T08:59:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.519+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fcc7c105900 con 0x7fcc7c0fe430 2024-03-16T08:59:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.520+0000 7fcc637fe700 1 -- 172.21.15.7:0/2364473803 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fcc74056ef0 con 0x7fcc7c0fe430 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcc640692a0 msgr2=0x7fcc6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcc640692a0 0x7fcc6406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000b3b0 tx=0x7fcc70009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fe430 msgr2=0x7fcc7c0fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fe430 0x7fcc7c0fda40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400c810 tx=0x7fcc74004470 comp rx=0 tx=0).stop 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 shutdown_connections 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fcc640692a0 0x7fcc6406b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc7c0fe430 0x7fcc7c0fda40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 --2- 172.21.15.7:0/2364473803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc7c0fed80 0x7fcc7c0fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 >> 172.21.15.7:0/2364473803 conn(0x7fcc7c0f6bd0 msgr2=0x7fcc7c0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:13.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 shutdown_connections 2024-03-16T08:59:13.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:13.522+0000 7fcc812b6700 1 -- 172.21.15.7:0/2364473803 wait complete. 2024-03-16T08:59:14.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-16T08:59:14.454 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:59:14.730 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:14 smithi007 ceph-mon[29470]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-16T08:59:14.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:14 smithi019 ceph-mon[36407]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-16T08:59:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.195+0000 7f8e839c2700 1 -- 172.21.15.7:0/652686751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c066f10 msgr2=0x7f8e7c0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.195+0000 7f8e839c2700 1 --2- 172.21.15.7:0/652686751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c066f10 0x7f8e7c0ffa50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8e78004940 tx=0x7f8e78009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 -- 172.21.15.7:0/652686751 shutdown_connections 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 --2- 172.21.15.7:0/652686751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e7c0fff90 0x7f8e7c102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 --2- 172.21.15.7:0/652686751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c066f10 0x7f8e7c0ffa50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 -- 172.21.15.7:0/652686751 >> 172.21.15.7:0/652686751 conn(0x7f8e7c0f7480 msgr2=0x7f8e7c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 -- 172.21.15.7:0/652686751 shutdown_connections 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 -- 172.21.15.7:0/652686751 wait complete. 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 Processor -- start 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 -- start start 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e7c066f10 0x7f8e7c10fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c0fff90 0x7f8e7c1100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1106c0 con 0x7f8e7c0fff90 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e839c2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c110830 con 0x7f8e7c066f10 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e80f5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c0fff90 0x7f8e7c1100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e80f5d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c0fff90 0x7f8e7c1100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39174/0 (socket says 172.21.15.7:39174) 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.196+0000 7f8e80f5d700 1 -- 172.21.15.7:0/1206796024 learned_addr learned my addr 172.21.15.7:0/1206796024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e8175e700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e7c066f10 0x7f8e7c10fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e80f5d700 1 -- 172.21.15.7:0/1206796024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e7c066f10 msgr2=0x7f8e7c10fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e80f5d700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e7c066f10 0x7f8e7c10fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e80f5d700 1 -- 172.21.15.7:0/1206796024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78005040 con 0x7f8e7c0fff90 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e80f5d700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c0fff90 0x7f8e7c1100c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c002a50 tx=0x7f8e6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e71ffb700 1 -- 172.21.15.7:0/1206796024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e6c007cc0 con 0x7f8e7c0fff90 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e71ffb700 1 -- 172.21.15.7:0/1206796024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e6c00f040 con 0x7f8e7c0fff90 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c115290 con 0x7f8e7c0fff90 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e71ffb700 1 -- 172.21.15.7:0/1206796024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e6c00da30 con 0x7f8e7c0fff90 2024-03-16T08:59:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.197+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c115850 con 0x7f8e7c0fff90 2024-03-16T08:59:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.198+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c04c710 con 0x7f8e7c0fff90 2024-03-16T08:59:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.199+0000 7f8e71ffb700 1 -- 172.21.15.7:0/1206796024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f8e6c00db90 con 0x7f8e7c0fff90 2024-03-16T08:59:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.199+0000 7f8e71ffb700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8e680692a0 0x7f8e6806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.199+0000 7f8e71ffb700 1 -- 172.21.15.7:0/1206796024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8e6c086f90 con 0x7f8e7c0fff90 2024-03-16T08:59:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.199+0000 7f8e8175e700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8e680692a0 0x7f8e6806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.199+0000 7f8e8175e700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8e680692a0 0x7f8e6806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8e78004890 tx=0x7f8e78009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.203+0000 7f8e71ffb700 1 -- 172.21.15.7:0/1206796024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f8e6c056c80 con 0x7f8e7c0fff90 2024-03-16T08:59:16.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.393+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f8e7c1058e0 con 0x7f8e7c0fff90 2024-03-16T08:59:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.393+0000 7f8e71ffb700 1 -- 172.21.15.7:0/1206796024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f8e6c056810 con 0x7f8e7c0fff90 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8e680692a0 msgr2=0x7f8e6806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8e680692a0 0x7f8e6806b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8e78004890 tx=0x7f8e78009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c0fff90 msgr2=0x7f8e7c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c0fff90 0x7f8e7c1100c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c002a50 tx=0x7f8e6c0046b0 comp rx=0 tx=0).stop 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 shutdown_connections 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f8e680692a0 0x7f8e6806b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e7c066f10 0x7f8e7c10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 --2- 172.21.15.7:0/1206796024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e7c0fff90 0x7f8e7c1100c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 >> 172.21.15.7:0/1206796024 conn(0x7f8e7c0f7480 msgr2=0x7f8e7c0fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.395+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 shutdown_connections 2024-03-16T08:59:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:16.396+0000 7f8e839c2700 1 -- 172.21.15.7:0/1206796024 wait complete. 2024-03-16T08:59:16.918 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:16 smithi019 ceph-mon[36407]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-16T08:59:16.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:16 smithi007 ceph-mon[29470]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-16T08:59:17.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-16T08:59:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:59:18.943 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:18 smithi007 ceph-mon[29470]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-16T08:59:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:18 smithi019 ceph-mon[36407]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-16T08:59:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.182+0000 7fb256b43700 1 -- 172.21.15.7:0/3088179076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 msgr2=0x7fb2500fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.182+0000 7fb256b43700 1 --2- 172.21.15.7:0/3088179076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 0x7fb2500fe450 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb23c004940 tx=0x7fb23c009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.182+0000 7fb256b43700 1 -- 172.21.15.7:0/3088179076 shutdown_connections 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.182+0000 7fb256b43700 1 --2- 172.21.15.7:0/3088179076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 0x7fb2500fe450 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.182+0000 7fb256b43700 1 --2- 172.21.15.7:0/3088179076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2500fce70 0x7fb2500fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.182+0000 7fb256b43700 1 -- 172.21.15.7:0/3088179076 >> 172.21.15.7:0/3088179076 conn(0x7fb2500f8550 msgr2=0x7fb2500fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.182+0000 7fb256b43700 1 -- 172.21.15.7:0/3088179076 shutdown_connections 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb256b43700 1 -- 172.21.15.7:0/3088179076 wait complete. 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb256b43700 1 Processor -- start 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb256b43700 1 -- start start 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb256b43700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2500fce70 0x7fb25010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb256b43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 0x7fb25010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb256b43700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25010e510 con 0x7fb2500fdfe0 2024-03-16T08:59:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb256b43700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb250112eb0 con 0x7fb2500fce70 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb24ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 0x7fb25010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb24ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 0x7fb25010de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39194/0 (socket says 172.21.15.7:39194) 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.183+0000 7fb24ffff700 1 -- 172.21.15.7:0/3017818964 learned_addr learned my addr 172.21.15.7:0/3017818964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb2548df700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2500fce70 0x7fb25010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb24ffff700 1 -- 172.21.15.7:0/3017818964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2500fce70 msgr2=0x7fb25010d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb24ffff700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2500fce70 0x7fb25010d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb24ffff700 1 -- 172.21.15.7:0/3017818964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c005040 con 0x7fb2500fdfe0 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb24ffff700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 0x7fb25010de80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00b3b0 tx=0x7fb23c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb24d7fa700 1 -- 172.21.15.7:0/3017818964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb23c00e040 con 0x7fb2500fdfe0 2024-03-16T08:59:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2501130b0 con 0x7fb2500fdfe0 2024-03-16T08:59:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb24d7fa700 1 -- 172.21.15.7:0/3017818964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb23c00bbf0 con 0x7fb2500fdfe0 2024-03-16T08:59:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb24d7fa700 1 -- 172.21.15.7:0/3017818964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb23c01c630 con 0x7fb2500fdfe0 2024-03-16T08:59:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.184+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb250113620 con 0x7fb2500fdfe0 2024-03-16T08:59:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.185+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25004c710 con 0x7fb2500fdfe0 2024-03-16T08:59:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.186+0000 7fb24d7fa700 1 -- 172.21.15.7:0/3017818964 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fb23c00bd60 con 0x7fb2500fdfe0 2024-03-16T08:59:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.186+0000 7fb24d7fa700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb2400692a0 0x7fb24006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.186+0000 7fb24d7fa700 1 -- 172.21.15.7:0/3017818964 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb23c087ef0 con 0x7fb2500fdfe0 2024-03-16T08:59:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.186+0000 7fb2548df700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb2400692a0 0x7fb24006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.187+0000 7fb2548df700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb2400692a0 0x7fb24006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb244009bf0 tx=0x7fb2440091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.191+0000 7fb24d7fa700 1 -- 172.21.15.7:0/3017818964 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fb23c057c00 con 0x7fb2500fdfe0 2024-03-16T08:59:19.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:19.399+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fb250102280 con 0x7fb2400692a0 2024-03-16T08:59:20.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:20 smithi019 ceph-mon[36407]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-16T08:59:20.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:20 smithi019 ceph-mon[36407]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:20 smithi007 ceph-mon[29470]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-16T08:59:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:20 smithi007 ceph-mon[29470]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:22.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:22 smithi019 ceph-mon[36407]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-16T08:59:22.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:22 smithi007 ceph-mon[29470]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-16T08:59:24.910 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:24 smithi007 ceph-mon[29470]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T08:59:24.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:24 smithi019 ceph-mon[36407]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T08:59:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:26 smithi019 ceph-mon[36407]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:59:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:26 smithi007 ceph-mon[29470]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:59:28.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:28 smithi019 ceph-mon[36407]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:28 smithi007 ceph-mon[29470]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:30.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:30 smithi019 ceph-mon[36407]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:30.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:30 smithi007 ceph-mon[29470]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:32 smithi019 ceph-mon[36407]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:32.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:32 smithi007 ceph-mon[29470]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:34.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:34 smithi019 ceph-mon[36407]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:34 smithi007 ceph-mon[29470]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:36 smithi019 ceph-mon[36407]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:36.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:36 smithi007 ceph-mon[29470]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:38.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:38 smithi019 ceph-mon[36407]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:38.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:38 smithi007 ceph-mon[29470]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:40.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:40 smithi019 ceph-mon[36407]: pgmap v213: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:40 smithi007 ceph-mon[29470]: pgmap v213: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:41.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:41 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:59:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:41 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:59:42.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:42 smithi019 ceph-mon[36407]: pgmap v214: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:42.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:42 smithi007 ceph-mon[29470]: pgmap v214: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T08:59:44.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:44 smithi019 ceph-mon[36407]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T08:59:44.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:44 smithi007 ceph-mon[29470]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T08:59:46.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:46 smithi019 ceph-mon[36407]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T08:59:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:46 smithi007 ceph-mon[29470]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T08:59:48.914 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:48 smithi007 ceph-mon[29470]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T08:59:48.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.912+0000 7fb24d7fa700 1 -- 172.21.15.7:0/3017818964 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fb250102280 con 0x7fb2400692a0 2024-03-16T08:59:48.915 INFO:teuthology.orchestra.run.smithi007.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.914+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb2400692a0 msgr2=0x7fb24006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.914+0000 7fb256b43700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb2400692a0 0x7fb24006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb244009bf0 tx=0x7fb2440091a0 comp rx=0 tx=0).stop 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.914+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 msgr2=0x7fb25010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.914+0000 7fb256b43700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 0x7fb25010de80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00b3b0 tx=0x7fb23c009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.915+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 shutdown_connections 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.915+0000 7fb256b43700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fb2400692a0 0x7fb24006b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.915+0000 7fb256b43700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2500fce70 0x7fb25010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.915+0000 7fb256b43700 1 --2- 172.21.15.7:0/3017818964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2500fdfe0 0x7fb25010de80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.915+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 >> 172.21.15.7:0/3017818964 conn(0x7fb2500f8550 msgr2=0x7fb2500f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.915+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 shutdown_connections 2024-03-16T08:59:48.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:48.915+0000 7fb256b43700 1 -- 172.21.15.7:0/3017818964 wait complete. 2024-03-16T08:59:48.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:48 smithi019 ceph-mon[36407]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T08:59:49.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-16T08:59:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:49 smithi007 ceph-mon[29470]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:59:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:59:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:59:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:49 smithi007 ceph-mon[29470]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T08:59:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:59:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:59:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:49 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:59:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:49 smithi007 ceph-mon[29470]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:59:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T08:59:50.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:49 smithi019 ceph-mon[36407]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:59:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:59:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:59:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:49 smithi019 ceph-mon[36407]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T08:59:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T08:59:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T08:59:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:49 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:59:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:49 smithi019 ceph-mon[36407]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:59:52.667 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:52 smithi007 ceph-mon[29470]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-16T08:59:52.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:52 smithi019 ceph-mon[36407]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-16T08:59:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 -- 172.21.15.7:0/3408680289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00662e0 msgr2=0x7f49e00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 --2- 172.21.15.7:0/3408680289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00662e0 0x7f49e00ff9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f49d0004940 tx=0x7f49d0009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 -- 172.21.15.7:0/3408680289 shutdown_connections 2024-03-16T08:59:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 --2- 172.21.15.7:0/3408680289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e00fff00 0x7f49e0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 --2- 172.21.15.7:0/3408680289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00662e0 0x7f49e00ff9c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 -- 172.21.15.7:0/3408680289 >> 172.21.15.7:0/3408680289 conn(0x7f49e00f7430 msgr2=0x7f49e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:53.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 -- 172.21.15.7:0/3408680289 shutdown_connections 2024-03-16T08:59:53.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 -- 172.21.15.7:0/3408680289 wait complete. 2024-03-16T08:59:53.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.157+0000 7f49e6464700 1 Processor -- start 2024-03-16T08:59:53.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49e6464700 1 -- start start 2024-03-16T08:59:53.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49e6464700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e00662e0 0x7f49e00fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49e6464700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00fff00 0x7f49e00fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49e6464700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00fbe70 con 0x7f49e00fff00 2024-03-16T08:59:53.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49e6464700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00fbfe0 con 0x7f49e00662e0 2024-03-16T08:59:53.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00fff00 0x7f49e00fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49dffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e00662e0 0x7f49e00fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00fff00 0x7f49e00fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58576/0 (socket says 172.21.15.7:58576) 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49df7fe700 1 -- 172.21.15.7:0/3737623826 learned_addr learned my addr 172.21.15.7:0/3737623826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49df7fe700 1 -- 172.21.15.7:0/3737623826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e00662e0 msgr2=0x7f49e00fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49df7fe700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e00662e0 0x7f49e00fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49df7fe700 1 -- 172.21.15.7:0/3737623826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0005040 con 0x7f49e00fff00 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.158+0000 7f49df7fe700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00fff00 0x7f49e00fb8c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49d4002a50 tx=0x7f49d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.159+0000 7f49dcff9700 1 -- 172.21.15.7:0/3737623826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d4007cc0 con 0x7f49e00fff00 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.159+0000 7f49dcff9700 1 -- 172.21.15.7:0/3737623826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49d400f040 con 0x7f49e00fff00 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.159+0000 7f49dcff9700 1 -- 172.21.15.7:0/3737623826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d400da30 con 0x7f49e00fff00 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.159+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e00fc2c0 con 0x7f49e00fff00 2024-03-16T08:59:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.159+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e0115a60 con 0x7f49e00fff00 2024-03-16T08:59:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.160+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e005f5f0 con 0x7f49e00fff00 2024-03-16T08:59:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.160+0000 7f49dcff9700 1 -- 172.21.15.7:0/3737623826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f49d400db90 con 0x7f49e00fff00 2024-03-16T08:59:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.161+0000 7f49dcff9700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f49c80692a0 0x7f49c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.161+0000 7f49dcff9700 1 -- 172.21.15.7:0/3737623826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f49d4086eb0 con 0x7f49e00fff00 2024-03-16T08:59:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.161+0000 7f49dffff700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f49c80692a0 0x7f49c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.161+0000 7f49dffff700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f49c80692a0 0x7f49c806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49d000b970 tx=0x7f49d0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:53.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.165+0000 7f49dcff9700 1 -- 172.21.15.7:0/3737623826 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f49d4053190 con 0x7f49e00fff00 2024-03-16T08:59:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.387+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49e00f8f70 con 0x7f49c80692a0 2024-03-16T08:59:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.389+0000 7f49dcff9700 1 -- 172.21.15.7:0/3737623826 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f49e00f8f70 con 0x7f49c80692a0 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f49c80692a0 msgr2=0x7f49c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f49c80692a0 0x7f49c806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49d000b970 tx=0x7f49d0009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00fff00 msgr2=0x7f49e00fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00fff00 0x7f49e00fb8c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49d4002a50 tx=0x7f49d40046b0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 shutdown_connections 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f49c80692a0 0x7f49c806b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e00662e0 0x7f49e00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 --2- 172.21.15.7:0/3737623826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49e00fff00 0x7f49e00fb8c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.391+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 >> 172.21.15.7:0/3737623826 conn(0x7f49e00f7430 msgr2=0x7f49e00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.392+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 shutdown_connections 2024-03-16T08:59:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.392+0000 7f49e6464700 1 -- 172.21.15.7:0/3737623826 wait complete. 2024-03-16T08:59:53.404 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T08:59:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.498+0000 7fae53393700 1 -- 172.21.15.7:0/2180637970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 msgr2=0x7fae4c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.498+0000 7fae53393700 1 --2- 172.21.15.7:0/2180637970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c0fc0e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009ce0 tx=0x7fae3c004d10 comp rx=0 tx=0).stop 2024-03-16T08:59:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.498+0000 7fae53393700 1 -- 172.21.15.7:0/2180637970 shutdown_connections 2024-03-16T08:59:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.498+0000 7fae53393700 1 --2- 172.21.15.7:0/2180637970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fcf80 0x7fae4c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.498+0000 7fae53393700 1 --2- 172.21.15.7:0/2180637970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c0fc0e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.498+0000 7fae53393700 1 -- 172.21.15.7:0/2180637970 >> 172.21.15.7:0/2180637970 conn(0x7fae4c0f7430 msgr2=0x7fae4c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.498+0000 7fae53393700 1 -- 172.21.15.7:0/2180637970 shutdown_connections 2024-03-16T08:59:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.499+0000 7fae53393700 1 -- 172.21.15.7:0/2180637970 wait complete. 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.499+0000 7fae53393700 1 Processor -- start 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.499+0000 7fae53393700 1 -- start start 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.499+0000 7fae53393700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.499+0000 7fae53393700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fcf80 0x7fae4c112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.499+0000 7fae53393700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c066350 con 0x7fae4c0fcf80 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.499+0000 7fae53393700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c066490 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5092e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fcf80 0x7fae4c112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5112f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5112f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:51024/0 (socket says 172.21.15.7:51024) 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5112f700 1 -- 172.21.15.7:0/861919537 learned_addr learned my addr 172.21.15.7:0/861919537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5112f700 1 -- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fcf80 msgr2=0x7fae4c112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5112f700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fcf80 0x7fae4c112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5112f700 1 -- 172.21.15.7:0/861919537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c009940 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5092e700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fcf80 0x7fae4c112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae5112f700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c110720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00bfd0 tx=0x7fae3c004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae49ffb700 1 -- 172.21.15.7:0/861919537 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3c00e040 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c1131b0 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.500+0000 7fae49ffb700 1 -- 172.21.15.7:0/861919537 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae3c01cd00 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.501+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c113770 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.501+0000 7fae49ffb700 1 -- 172.21.15.7:0/861919537 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3c012c40 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.502+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c05f5f0 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.502+0000 7fae49ffb700 1 -- 172.21.15.7:0/861919537 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fae3c0124a0 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.503+0000 7fae49ffb700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae340692a0 0x7fae3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.503+0000 7fae49ffb700 1 -- 172.21.15.7:0/861919537 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fae3c0897c0 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.503+0000 7fae5092e700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae340692a0 0x7fae3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.504+0000 7fae5092e700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae340692a0 0x7fae3406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fae40009b80 tx=0x7fae4000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.507+0000 7fae49ffb700 1 -- 172.21.15.7:0/861919537 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fae3c055a00 con 0x7fae4c0fbcd0 2024-03-16T08:59:53.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.721+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae4c101290 con 0x7fae340692a0 2024-03-16T08:59:53.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.722+0000 7fae49ffb700 1 -- 172.21.15.7:0/861919537 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fae4c101290 con 0x7fae340692a0 2024-03-16T08:59:53.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae340692a0 msgr2=0x7fae3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae340692a0 0x7fae3406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fae40009b80 tx=0x7fae4000b040 comp rx=0 tx=0).stop 2024-03-16T08:59:53.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fbcd0 msgr2=0x7fae4c110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c110720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00bfd0 tx=0x7fae3c004d10 comp rx=0 tx=0).stop 2024-03-16T08:59:53.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 shutdown_connections 2024-03-16T08:59:53.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae340692a0 0x7fae3406b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c110720 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 --2- 172.21.15.7:0/861919537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fcf80 0x7fae4c112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 >> 172.21.15.7:0/861919537 conn(0x7fae4c0f7430 msgr2=0x7fae4c0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:53.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 shutdown_connections 2024-03-16T08:59:53.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.724+0000 7fae53393700 1 -- 172.21.15.7:0/861919537 wait complete. 2024-03-16T08:59:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.832+0000 7fed19b48700 1 -- 172.21.15.7:0/2208180697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 msgr2=0x7fed140ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.832+0000 7fed19b48700 1 --2- 172.21.15.7:0/2208180697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 0x7fed140ff9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fed04004940 tx=0x7fed04009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.833+0000 7fed19b48700 1 -- 172.21.15.7:0/2208180697 shutdown_connections 2024-03-16T08:59:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.833+0000 7fed19b48700 1 --2- 172.21.15.7:0/2208180697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed140fff00 0x7fed14102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.833+0000 7fed19b48700 1 --2- 172.21.15.7:0/2208180697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 0x7fed140ff9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.833+0000 7fed19b48700 1 -- 172.21.15.7:0/2208180697 >> 172.21.15.7:0/2208180697 conn(0x7fed140f7430 msgr2=0x7fed140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.833+0000 7fed19b48700 1 -- 172.21.15.7:0/2208180697 shutdown_connections 2024-03-16T08:59:53.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.833+0000 7fed19b48700 1 -- 172.21.15.7:0/2208180697 wait complete. 2024-03-16T08:59:53.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:59:53.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:53 smithi007 ceph-mon[29470]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (unknown) 2024-03-16T08:59:53.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:53 smithi007 ceph-mon[29470]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-16T08:59:53.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:59:53.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:53 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:59:53.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:53 smithi007 ceph-mon[29470]: Upgrade: Need to upgrade myself (mgr.smithi007.uamwtp) 2024-03-16T08:59:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.833+0000 7fed19b48700 1 Processor -- start 2024-03-16T08:59:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed19b48700 1 -- start start 2024-03-16T08:59:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed19b48700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 0x7fed1410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed19b48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed140fff00 0x7fed1410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed19b48700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1410e570 con 0x7fed140662e0 2024-03-16T08:59:53.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed19b48700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1410edc0 con 0x7fed140fff00 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 0x7fed1410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 0x7fed1410da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58618/0 (socket says 172.21.15.7:58618) 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed12ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed140fff00 0x7fed1410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed137fe700 1 -- 172.21.15.7:0/951471693 learned_addr learned my addr 172.21.15.7:0/951471693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed137fe700 1 -- 172.21.15.7:0/951471693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed140fff00 msgr2=0x7fed1410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed137fe700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed140fff00 0x7fed1410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.834+0000 7fed137fe700 1 -- 172.21.15.7:0/951471693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04005040 con 0x7fed140662e0 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.835+0000 7fed137fe700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 0x7fed1410da30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed04000c00 tx=0x7fed04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.835+0000 7fecfbfff700 1 -- 172.21.15.7:0/951471693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0400e040 con 0x7fed140662e0 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.835+0000 7fecfbfff700 1 -- 172.21.15.7:0/951471693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed0400bbf0 con 0x7fed140662e0 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.835+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1410f040 con 0x7fed140662e0 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.835+0000 7fecfbfff700 1 -- 172.21.15.7:0/951471693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0401c630 con 0x7fed140662e0 2024-03-16T08:59:53.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.835+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1410f580 con 0x7fed140662e0 2024-03-16T08:59:53.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.836+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1405f5f0 con 0x7fed140662e0 2024-03-16T08:59:53.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.837+0000 7fecfbfff700 1 -- 172.21.15.7:0/951471693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fed04013420 con 0x7fed140662e0 2024-03-16T08:59:53.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.837+0000 7fecfbfff700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fecfc0692a0 0x7fecfc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:53.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.837+0000 7fecfbfff700 1 -- 172.21.15.7:0/951471693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fed04088500 con 0x7fed140662e0 2024-03-16T08:59:53.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.837+0000 7fed12ffd700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fecfc0692a0 0x7fecfc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:53.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.838+0000 7fed12ffd700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fecfc0692a0 0x7fecfc06b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed08004f40 tx=0x7fed0800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:53.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:53.842+0000 7fecfbfff700 1 -- 172.21.15.7:0/951471693 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fed04058270 con 0x7fed140662e0 2024-03-16T08:59:54.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.046+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed1410f860 con 0x7fecfc0692a0 2024-03-16T08:59:54.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.055+0000 7fecfbfff700 1 -- 172.21.15.7:0/951471693 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fed1410f860 con 0x7fecfc0692a0 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (5m) 56s ago 6m 25.3M - 0.25.0 c8568f914cd2 d5db4ce74c0d 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (7m) 56s ago 7m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (6m) 61s ago 5m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (5m) 56s ago 6m 88.5M - 9.4.7 954c08fa6188 4f8b149e57d8 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (81s) 56s ago 80s 10.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (74s) 56s ago 74s 13.9M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (78s) 61s ago 78s 14.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (72s) 61s ago 71s 11.0M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:9283 running (8m) 56s ago 8m 480M - 17.2.7-648-g1df27247 0bdcfccf14ce 376995a75ba3 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283 running (5m) 61s ago 5m 432M - 17.2.7-648-g1df27247 0bdcfccf14ce ab3071a32d6e 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (8m) 56s ago 8m 43.9M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T08:59:54.058 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (5m) 61s ago 5m 38.5M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (6m) 56s ago 6m 14.4M - 1.5.0 0da6a335fe13 9ca7275ea387 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (5m) 61s ago 5m 16.8M - 1.5.0 0da6a335fe13 9edea61eb01f 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (5m) 56s ago 5m 47.8M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (4m) 56s ago 4m 45.5M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (4m) 56s ago 4m 44.1M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (3m) 61s ago 3m 47.0M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (3m) 61s ago 3m 42.5M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (2m) 61s ago 2m 44.4M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T08:59:54.059 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (5m) 56s ago 6m 57.4M - 2.43.0 a07b618ecd1d b457178ffb05 2024-03-16T08:59:54.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fecfc0692a0 msgr2=0x7fecfc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fecfc0692a0 0x7fecfc06b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed08004f40 tx=0x7fed0800b040 comp rx=0 tx=0).stop 2024-03-16T08:59:54.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 msgr2=0x7fed1410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 0x7fed1410da30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed04000c00 tx=0x7fed04009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:54.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 shutdown_connections 2024-03-16T08:59:54.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fecfc0692a0 0x7fecfc06b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed140662e0 0x7fed1410da30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 --2- 172.21.15.7:0/951471693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed140fff00 0x7fed1410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 >> 172.21.15.7:0/951471693 conn(0x7fed140f7430 msgr2=0x7fed140f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:54.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 shutdown_connections 2024-03-16T08:59:54.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.058+0000 7fed19b48700 1 -- 172.21.15.7:0/951471693 wait complete. 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.162+0000 7fae52104700 1 -- 172.21.15.7:0/2654138453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 msgr2=0x7fae4c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.162+0000 7fae52104700 1 --2- 172.21.15.7:0/2654138453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c0fc0e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fae3c004940 tx=0x7fae3c009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.162+0000 7fae52104700 1 -- 172.21.15.7:0/2654138453 shutdown_connections 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.162+0000 7fae52104700 1 --2- 172.21.15.7:0/2654138453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fcf80 0x7fae4c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.162+0000 7fae52104700 1 --2- 172.21.15.7:0/2654138453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c0fc0e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.162+0000 7fae52104700 1 -- 172.21.15.7:0/2654138453 >> 172.21.15.7:0/2654138453 conn(0x7fae4c0f7430 msgr2=0x7fae4c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.162+0000 7fae52104700 1 -- 172.21.15.7:0/2654138453 shutdown_connections 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.162+0000 7fae52104700 1 -- 172.21.15.7:0/2654138453 wait complete. 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae52104700 1 Processor -- start 2024-03-16T08:59:54.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae52104700 1 -- start start 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae52104700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c10dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae52104700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fcf80 0x7fae4c10e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae52104700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c10e620 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae52104700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c10edd0 con 0x7fae4c0fcf80 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae4b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c10dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae4b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c10dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58634/0 (socket says 172.21.15.7:58634) 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae4b7fe700 1 -- 172.21.15.7:0/753374942 learned_addr learned my addr 172.21.15.7:0/753374942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae42ffd700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fcf80 0x7fae4c10e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae4b7fe700 1 -- 172.21.15.7:0/753374942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fcf80 msgr2=0x7fae4c10e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae4b7fe700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fcf80 0x7fae4c10e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.163+0000 7fae4b7fe700 1 -- 172.21.15.7:0/753374942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c005040 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.164+0000 7fae4b7fe700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c10dae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009150 tx=0x7fae3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.164+0000 7fae48ff9700 1 -- 172.21.15.7:0/753374942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3c00e040 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.164+0000 7fae48ff9700 1 -- 172.21.15.7:0/753374942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae3c004290 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.164+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c10f050 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.164+0000 7fae48ff9700 1 -- 172.21.15.7:0/753374942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3c01c630 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.164+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c10f590 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.165+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c101b10 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.166+0000 7fae48ff9700 1 -- 172.21.15.7:0/753374942 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fae3c00bbf0 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.166+0000 7fae48ff9700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae380692a0 0x7fae3806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.167+0000 7fae48ff9700 1 -- 172.21.15.7:0/753374942 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fae3c087ea0 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.167+0000 7fae42ffd700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae380692a0 0x7fae3806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.167+0000 7fae42ffd700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae380692a0 0x7fae3806b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae34004f40 tx=0x7fae3400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:54.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.171+0000 7fae48ff9700 1 -- 172.21.15.7:0/753374942 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fae3c057c10 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:59:54.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:53 smithi019 ceph-mon[36407]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (unknown) 2024-03-16T08:59:54.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:53 smithi019 ceph-mon[36407]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-16T08:59:54.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T08:59:54.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:53 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T08:59:54.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:53 smithi019 ceph-mon[36407]: Upgrade: Need to upgrade myself (mgr.smithi007.uamwtp) 2024-03-16T08:59:54.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.456+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fae4c104950 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.457+0000 7fae48ff9700 1 -- 172.21.15.7:0/753374942 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fae4c104950 con 0x7fae4c0fbcd0 2024-03-16T08:59:54.459 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T08:59:54.459 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T08:59:54.459 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T08:59:54.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T08:59:54.459 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 14 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T08:59:54.460 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T08:59:54.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae380692a0 msgr2=0x7fae3806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae380692a0 0x7fae3806b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae34004f40 tx=0x7fae3400b040 comp rx=0 tx=0).stop 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 msgr2=0x7fae4c10dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c10dae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009150 tx=0x7fae3c009e30 comp rx=0 tx=0).stop 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 shutdown_connections 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fae380692a0 0x7fae3806b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4c0fbcd0 0x7fae4c10dae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 --2- 172.21.15.7:0/753374942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae4c0fcf80 0x7fae4c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.459+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 >> 172.21.15.7:0/753374942 conn(0x7fae4c0f7430 msgr2=0x7fae4c065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.460+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 shutdown_connections 2024-03-16T08:59:54.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.460+0000 7fae52104700 1 -- 172.21.15.7:0/753374942 wait complete. 2024-03-16T08:59:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.572+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3340594486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 msgr2=0x7fc8dc0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.572+0000 7fc8e1cb3700 1 --2- 172.21.15.7:0/3340594486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 0x7fc8dc0fd250 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc004a90 tx=0x7fc8cc009dd0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.572+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3340594486 shutdown_connections 2024-03-16T08:59:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.572+0000 7fc8e1cb3700 1 --2- 172.21.15.7:0/3340594486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8dc0fe040 0x7fc8dc0fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.572+0000 7fc8e1cb3700 1 --2- 172.21.15.7:0/3340594486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 0x7fc8dc0fd250 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.572+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3340594486 >> 172.21.15.7:0/3340594486 conn(0x7fc8dc0f8570 msgr2=0x7fc8dc0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.573+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3340594486 shutdown_connections 2024-03-16T08:59:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.573+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3340594486 wait complete. 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.573+0000 7fc8e1cb3700 1 Processor -- start 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.573+0000 7fc8e1cb3700 1 -- start start 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8e1cb3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 0x7fc8dc10d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8e1cb3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8dc0fe040 0x7fc8dc10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8e1cb3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc10e510 con 0x7fc8dc0fce40 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8e1cb3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc10ed60 con 0x7fc8dc0fe040 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 0x7fc8dc10d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 0x7fc8dc10d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58644/0 (socket says 172.21.15.7:58644) 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8db7fe700 1 -- 172.21.15.7:0/3543703320 learned_addr learned my addr 172.21.15.7:0/3543703320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8daffd700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8dc0fe040 0x7fc8dc10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8db7fe700 1 -- 172.21.15.7:0/3543703320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8dc0fe040 msgr2=0x7fc8dc10df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8db7fe700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8dc0fe040 0x7fc8dc10df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8db7fe700 1 -- 172.21.15.7:0/3543703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc004650 con 0x7fc8dc0fce40 2024-03-16T08:59:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.574+0000 7fc8db7fe700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 0x7fc8dc10d9d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009a00 tx=0x7fc8cc009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.575+0000 7fc8c3fff700 1 -- 172.21.15.7:0/3543703320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8cc019070 con 0x7fc8dc0fce40 2024-03-16T08:59:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.575+0000 7fc8c3fff700 1 -- 172.21.15.7:0/3543703320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8cc00bc60 con 0x7fc8dc0fce40 2024-03-16T08:59:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.575+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc10efe0 con 0x7fc8dc0fce40 2024-03-16T08:59:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.575+0000 7fc8c3fff700 1 -- 172.21.15.7:0/3543703320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8cc00e040 con 0x7fc8dc0fce40 2024-03-16T08:59:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.576+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc10f4c0 con 0x7fc8dc0fce40 2024-03-16T08:59:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.576+0000 7fc8c3fff700 1 -- 172.21.15.7:0/3543703320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fc8cc014430 con 0x7fc8dc0fce40 2024-03-16T08:59:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.577+0000 7fc8c3fff700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc8c40692a0 0x7fc8c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T08:59:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.577+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc05f5f0 con 0x7fc8dc0fce40 2024-03-16T08:59:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.577+0000 7fc8daffd700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc8c40692a0 0x7fc8c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T08:59:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.577+0000 7fc8c3fff700 1 -- 172.21.15.7:0/3543703320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc8cc089e30 con 0x7fc8dc0fce40 2024-03-16T08:59:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.577+0000 7fc8daffd700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc8c40692a0 0x7fc8c406b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009b80 tx=0x7fc8d000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T08:59:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.582+0000 7fc8c3fff700 1 -- 172.21.15.7:0/3543703320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fc8cc08d060 con 0x7fc8dc0fce40 2024-03-16T08:59:54.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.824+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8dc10f7c0 con 0x7fc8c40692a0 2024-03-16T08:59:54.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.825+0000 7fc8c3fff700 1 -- 172.21.15.7:0/3543703320 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fc8dc10f7c0 con 0x7fc8c40692a0 2024-03-16T08:59:54.827 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T08:59:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T08:59:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T08:59:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T08:59:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-03-16T08:59:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "0/2 daemons upgraded", 2024-03-16T08:59:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 image on host smithi019", 2024-03-16T08:59:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T08:59:54.828 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T08:59:54.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc8c40692a0 msgr2=0x7fc8c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc8c40692a0 0x7fc8c406b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009b80 tx=0x7fc8d000b040 comp rx=0 tx=0).stop 2024-03-16T08:59:54.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 msgr2=0x7fc8dc10d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T08:59:54.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 0x7fc8dc10d9d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009a00 tx=0x7fc8cc009dd0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 shutdown_connections 2024-03-16T08:59:54.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fc8c40692a0 0x7fc8c406b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8dc0fce40 0x7fc8dc10d9d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 --2- 172.21.15.7:0/3543703320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8dc0fe040 0x7fc8dc10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T08:59:54.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 >> 172.21.15.7:0/3543703320 conn(0x7fc8dc0f8570 msgr2=0x7fc8dc0f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T08:59:54.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 shutdown_connections 2024-03-16T08:59:54.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T08:59:54.828+0000 7fc8e1cb3700 1 -- 172.21.15.7:0/3543703320 wait complete. 2024-03-16T08:59:54.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:54 smithi007 ceph-mon[29470]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 on smithi019 2024-03-16T08:59:54.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:54 smithi007 ceph-mon[29470]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:54.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:54 smithi007 ceph-mon[29470]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-16T08:59:54.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:54 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/753374942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T08:59:55.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:54 smithi019 ceph-mon[36407]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 on smithi019 2024-03-16T08:59:55.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:54 smithi019 ceph-mon[36407]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:55.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:54 smithi019 ceph-mon[36407]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-16T08:59:55.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:54 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/753374942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T08:59:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:55 smithi007 ceph-mon[29470]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:55 smithi007 ceph-mon[29470]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:56.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:55 smithi019 ceph-mon[36407]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:55 smithi019 ceph-mon[36407]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:56 smithi007 ceph-mon[29470]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:56 smithi007 ceph-mon[29470]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T08:59:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:56 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:59:57.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:56 smithi019 ceph-mon[36407]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T08:59:57.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:56 smithi019 ceph-mon[36407]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T08:59:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:56 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T08:59:58.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 08:59:57 smithi019 ceph-mon[36407]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T08:59:58.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 08:59:57 smithi007 ceph-mon[29470]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T09:00:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:00 smithi019 ceph-mon[36407]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:00:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:00 smithi019 ceph-mon[36407]: overall HEALTH_OK 2024-03-16T09:00:00.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:00 smithi007 ceph-mon[29470]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:00:00.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:00 smithi007 ceph-mon[29470]: overall HEALTH_OK 2024-03-16T09:00:02.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:02 smithi019 ceph-mon[36407]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:00:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:02 smithi007 ceph-mon[29470]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:00:04.966 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:04 smithi019 ceph-mon[36407]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:04.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:04 smithi007 ceph-mon[29470]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:06.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:06 smithi019 ceph-mon[36407]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:06 smithi007 ceph-mon[29470]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:08.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:08 smithi019 ceph-mon[36407]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:08.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:08 smithi007 ceph-mon[29470]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:10.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:10 smithi019 ceph-mon[36407]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:10 smithi007 ceph-mon[29470]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:11.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:11 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:00:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:11 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:00:12.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:12 smithi019 ceph-mon[36407]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:12 smithi007 ceph-mon[29470]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:14.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:14 smithi019 ceph-mon[36407]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:14.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:14 smithi007 ceph-mon[29470]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:16.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:16 smithi019 ceph-mon[36407]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:16.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:16 smithi007 ceph-mon[29470]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:18.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:18 smithi019 ceph-mon[36407]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:18.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:18 smithi007 ceph-mon[29470]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:20.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:20 smithi019 ceph-mon[36407]: pgmap v233: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:20 smithi007 ceph-mon[29470]: pgmap v233: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:22.881 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:22 smithi019 ceph-mon[36407]: pgmap v234: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:22.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:22 smithi007 ceph-mon[29470]: pgmap v234: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:24.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 -- 172.21.15.7:0/2712982129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 msgr2=0x7f401c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:24.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 --2- 172.21.15.7:0/2712982129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 0x7f401c102320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4010004940 tx=0x7f4010009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:24.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 -- 172.21.15.7:0/2712982129 shutdown_connections 2024-03-16T09:00:24.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 --2- 172.21.15.7:0/2712982129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 0x7f401c102320 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:24.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 --2- 172.21.15.7:0/2712982129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401c0662e0 0x7f401c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:24.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 -- 172.21.15.7:0/2712982129 >> 172.21.15.7:0/2712982129 conn(0x7f401c0f7430 msgr2=0x7f401c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:24.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 -- 172.21.15.7:0/2712982129 shutdown_connections 2024-03-16T09:00:24.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 -- 172.21.15.7:0/2712982129 wait complete. 2024-03-16T09:00:24.942 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:24 smithi007 ceph-mon[29470]: pgmap v235: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:24.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.939+0000 7f4020b4c700 1 Processor -- start 2024-03-16T09:00:24.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4020b4c700 1 -- start start 2024-03-16T09:00:24.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4020b4c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401c0662e0 0x7f401c10da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:24.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4020b4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 0x7f401c10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:24.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4020b4c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c10e580 con 0x7f401c0fff00 2024-03-16T09:00:24.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4020b4c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c10ed80 con 0x7f401c0662e0 2024-03-16T09:00:24.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4019d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 0x7f401c10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:24.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4019d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 0x7f401c10df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48150/0 (socket says 172.21.15.7:48150) 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f401a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401c0662e0 0x7f401c10da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4019d9b700 1 -- 172.21.15.7:0/2199896740 learned_addr learned my addr 172.21.15.7:0/2199896740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4019d9b700 1 -- 172.21.15.7:0/2199896740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401c0662e0 msgr2=0x7f401c10da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4019d9b700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401c0662e0 0x7f401c10da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4019d9b700 1 -- 172.21.15.7:0/2199896740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4010005040 con 0x7f401c0fff00 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.940+0000 7f4019d9b700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 0x7f401c10df80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4010009150 tx=0x7f4010009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.941+0000 7f400affd700 1 -- 172.21.15.7:0/2199896740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f401000e040 con 0x7f401c0fff00 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.941+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c10f000 con 0x7f401c0fff00 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.941+0000 7f400affd700 1 -- 172.21.15.7:0/2199896740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4010004290 con 0x7f401c0fff00 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.941+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c10f540 con 0x7f401c0fff00 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.941+0000 7f400affd700 1 -- 172.21.15.7:0/2199896740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f401001c630 con 0x7f401c0fff00 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.942+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ffc0052e0 con 0x7f401c0fff00 2024-03-16T09:00:24.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.942+0000 7f400affd700 1 -- 172.21.15.7:0/2199896740 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f401000bbf0 con 0x7f401c0fff00 2024-03-16T09:00:24.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.943+0000 7f400affd700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f40040692a0 0x7f400406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:24.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.943+0000 7f400affd700 1 -- 172.21.15.7:0/2199896740 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4010087cf0 con 0x7f401c0fff00 2024-03-16T09:00:24.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.943+0000 7f401a59c700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f40040692a0 0x7f400406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:24.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.943+0000 7f401a59c700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f40040692a0 0x7f400406b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f400c009fd0 tx=0x7f400c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:24.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:24.947+0000 7f400affd700 1 -- 172.21.15.7:0/2199896740 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f4010057b00 con 0x7f401c0fff00 2024-03-16T09:00:24.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:24 smithi019 ceph-mon[36407]: pgmap v235: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:25.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.156+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ffc000bf0 con 0x7f40040692a0 2024-03-16T09:00:25.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.157+0000 7f400affd700 1 -- 172.21.15.7:0/2199896740 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f3ffc000bf0 con 0x7f40040692a0 2024-03-16T09:00:25.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f40040692a0 msgr2=0x7f400406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f40040692a0 0x7f400406b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f400c009fd0 tx=0x7f400c00b040 comp rx=0 tx=0).stop 2024-03-16T09:00:25.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 msgr2=0x7f401c10df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 0x7f401c10df80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4010009150 tx=0x7f4010009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 shutdown_connections 2024-03-16T09:00:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f40040692a0 0x7f400406b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401c0662e0 0x7f401c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 --2- 172.21.15.7:0/2199896740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f401c0fff00 0x7f401c10df80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 >> 172.21.15.7:0/2199896740 conn(0x7f401c0f7430 msgr2=0x7f401c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 shutdown_connections 2024-03-16T09:00:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.159+0000 7f4020b4c700 1 -- 172.21.15.7:0/2199896740 wait complete. 2024-03-16T09:00:25.172 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:00:25.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.267+0000 7fea5636a700 1 -- 172.21.15.7:0/2794672723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 msgr2=0x7fea500fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.267+0000 7fea5636a700 1 --2- 172.21.15.7:0/2794672723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 0x7fea500fc0f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea44004940 tx=0x7fea44009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 -- 172.21.15.7:0/2794672723 shutdown_connections 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 --2- 172.21.15.7:0/2794672723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea500fcf90 0x7fea500fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 --2- 172.21.15.7:0/2794672723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 0x7fea500fc0f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 -- 172.21.15.7:0/2794672723 >> 172.21.15.7:0/2794672723 conn(0x7fea500f7440 msgr2=0x7fea500f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 -- 172.21.15.7:0/2794672723 shutdown_connections 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 -- 172.21.15.7:0/2794672723 wait complete. 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 Processor -- start 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 -- start start 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.268+0000 7fea5636a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 0x7fea5010d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea5636a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea500fcf90 0x7fea5010df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea5636a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5010e510 con 0x7fea500fbce0 2024-03-16T09:00:25.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea5636a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5010ed60 con 0x7fea500fcf90 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 0x7fea5010d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 0x7fea5010d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48162/0 (socket says 172.21.15.7:48162) 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4ffff700 1 -- 172.21.15.7:0/3679952332 learned_addr learned my addr 172.21.15.7:0/3679952332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4f7fe700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea500fcf90 0x7fea5010df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4ffff700 1 -- 172.21.15.7:0/3679952332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea500fcf90 msgr2=0x7fea5010df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4ffff700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea500fcf90 0x7fea5010df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4ffff700 1 -- 172.21.15.7:0/3679952332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44005040 con 0x7fea500fbce0 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4ffff700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 0x7fea5010d9d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea44013870 tx=0x7fea44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4cff9700 1 -- 172.21.15.7:0/3679952332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea44021070 con 0x7fea500fbce0 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4cff9700 1 -- 172.21.15.7:0/3679952332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea44009430 con 0x7fea500fbce0 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5010efe0 con 0x7fea500fbce0 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea4cff9700 1 -- 172.21.15.7:0/3679952332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea4400e070 con 0x7fea500fbce0 2024-03-16T09:00:25.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.269+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5010f520 con 0x7fea500fbce0 2024-03-16T09:00:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.271+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea50101b20 con 0x7fea500fbce0 2024-03-16T09:00:25.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.271+0000 7fea4cff9700 1 -- 172.21.15.7:0/3679952332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fea44026410 con 0x7fea500fbce0 2024-03-16T09:00:25.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.271+0000 7fea4cff9700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fea38069230 0x7fea3806b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.271+0000 7fea4cff9700 1 -- 172.21.15.7:0/3679952332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fea440884a0 con 0x7fea500fbce0 2024-03-16T09:00:25.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.271+0000 7fea4f7fe700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fea38069230 0x7fea3806b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.272+0000 7fea4f7fe700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fea38069230 0x7fea3806b6e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea40009b80 tx=0x7fea4000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.276+0000 7fea4cff9700 1 -- 172.21.15.7:0/3679952332 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fea44058280 con 0x7fea500fbce0 2024-03-16T09:00:25.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.488+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea50104940 con 0x7fea38069230 2024-03-16T09:00:25.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.489+0000 7fea4cff9700 1 -- 172.21.15.7:0/3679952332 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fea50104940 con 0x7fea38069230 2024-03-16T09:00:25.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fea38069230 msgr2=0x7fea3806b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fea38069230 0x7fea3806b6e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea40009b80 tx=0x7fea4000b040 comp rx=0 tx=0).stop 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 msgr2=0x7fea5010d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 0x7fea5010d9d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea44013870 tx=0x7fea44009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 shutdown_connections 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fea38069230 0x7fea3806b6e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea500fbce0 0x7fea5010d9d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 --2- 172.21.15.7:0/3679952332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea500fcf90 0x7fea5010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 >> 172.21.15.7:0/3679952332 conn(0x7fea500f7440 msgr2=0x7fea50066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 shutdown_connections 2024-03-16T09:00:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.491+0000 7fea5636a700 1 -- 172.21.15.7:0/3679952332 wait complete. 2024-03-16T09:00:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.594+0000 7f77fe24e700 1 -- 172.21.15.7:0/3235025456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 msgr2=0x7f77f80fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.594+0000 7f77fe24e700 1 --2- 172.21.15.7:0/3235025456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 0x7f77f80fd310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f77e0004970 tx=0x7f77e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.594+0000 7f77fe24e700 1 -- 172.21.15.7:0/3235025456 shutdown_connections 2024-03-16T09:00:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.594+0000 7f77fe24e700 1 --2- 172.21.15.7:0/3235025456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77f80fe0d0 0x7f77f80fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.594+0000 7f77fe24e700 1 --2- 172.21.15.7:0/3235025456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 0x7f77f80fd310 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.594+0000 7f77fe24e700 1 -- 172.21.15.7:0/3235025456 >> 172.21.15.7:0/3235025456 conn(0x7f77f80f8560 msgr2=0x7f77f80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.594+0000 7f77fe24e700 1 -- 172.21.15.7:0/3235025456 shutdown_connections 2024-03-16T09:00:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.594+0000 7f77fe24e700 1 -- 172.21.15.7:0/3235025456 wait complete. 2024-03-16T09:00:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.595+0000 7f77fe24e700 1 Processor -- start 2024-03-16T09:00:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.595+0000 7f77fe24e700 1 -- start start 2024-03-16T09:00:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77fe24e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 0x7f77f810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77fe24e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77f80fe0d0 0x7f77f810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77fe24e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f810e540 con 0x7f77f80fe0d0 2024-03-16T09:00:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77fe24e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f810ed40 con 0x7f77f80fcee0 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 0x7f77f810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77f80fe0d0 0x7f77f810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 0x7f77f810da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:47578/0 (socket says 172.21.15.7:47578) 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f77fe700 1 -- 172.21.15.7:0/1408608515 learned_addr learned my addr 172.21.15.7:0/1408608515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f77fe700 1 -- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77f80fe0d0 msgr2=0x7f77f810df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f77fe700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77f80fe0d0 0x7f77f810df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f77fe700 1 -- 172.21.15.7:0/1408608515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e8005040 con 0x7f77f80fcee0 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f6ffd700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77f80fe0d0 0x7f77f810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.596+0000 7f77f77fe700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 0x7f77f810da00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b370 tx=0x7f77e0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.597+0000 7f77effff700 1 -- 172.21.15.7:0/1408608515 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e000e040 con 0x7f77f80fcee0 2024-03-16T09:00:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.597+0000 7f77effff700 1 -- 172.21.15.7:0/1408608515 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77e000be90 con 0x7f77f80fcee0 2024-03-16T09:00:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.597+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e0005040 con 0x7f77f80fcee0 2024-03-16T09:00:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.597+0000 7f77effff700 1 -- 172.21.15.7:0/1408608515 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e001c630 con 0x7f77f80fcee0 2024-03-16T09:00:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.597+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f810f370 con 0x7f77f80fcee0 2024-03-16T09:00:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.598+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f805f5f0 con 0x7f77f80fcee0 2024-03-16T09:00:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.599+0000 7f77effff700 1 -- 172.21.15.7:0/1408608515 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f77e00139f0 con 0x7f77f80fcee0 2024-03-16T09:00:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.599+0000 7f77effff700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f77e40692a0 0x7f77e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.599+0000 7f77f6ffd700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f77e40692a0 0x7f77e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.599+0000 7f77effff700 1 -- 172.21.15.7:0/1408608515 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f77e00886e0 con 0x7f77f80fcee0 2024-03-16T09:00:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.600+0000 7f77f6ffd700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f77e40692a0 0x7f77e406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009fd0 tx=0x7f77e800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:25.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.607+0000 7f77effff700 1 -- 172.21.15.7:0/1408608515 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f77e0058430 con 0x7f77f80fcee0 2024-03-16T09:00:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.809+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f77f810f790 con 0x7f77e40692a0 2024-03-16T09:00:25.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.819+0000 7f77effff700 1 -- 172.21.15.7:0/1408608515 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f77f810f790 con 0x7f77e40692a0 2024-03-16T09:00:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:00:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (5m) 88s ago 7m 25.3M - 0.25.0 c8568f914cd2 d5db4ce74c0d 2024-03-16T09:00:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (7m) 88s ago 7m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:00:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (6m) 93s ago 6m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:00:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (5m) 88s ago 7m 88.5M - 9.4.7 954c08fa6188 4f8b149e57d8 2024-03-16T09:00:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (113s) 88s ago 112s 10.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:00:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (106s) 88s ago 105s 13.9M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:00:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (110s) 93s ago 110s 14.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (103s) 93s ago 103s 11.0M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:9283 running (8m) 88s ago 8m 480M - 17.2.7-648-g1df27247 0bdcfccf14ce 376995a75ba3 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283 running (6m) 93s ago 6m 432M - 17.2.7-648-g1df27247 0bdcfccf14ce ab3071a32d6e 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (8m) 88s ago 8m 43.9M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (6m) 93s ago 6m 38.5M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (7m) 88s ago 7m 14.4M - 1.5.0 0da6a335fe13 9ca7275ea387 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (6m) 93s ago 6m 16.8M - 1.5.0 0da6a335fe13 9edea61eb01f 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (5m) 88s ago 5m 47.8M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (5m) 88s ago 5m 45.5M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (4m) 88s ago 4m 44.1M 4289M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (4m) 93s ago 4m 47.0M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (3m) 93s ago 3m 42.5M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:00:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (3m) 93s ago 3m 44.4M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:00:25.825 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (5m) 88s ago 6m 57.4M - 2.43.0 a07b618ecd1d b457178ffb05 2024-03-16T09:00:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f77e40692a0 msgr2=0x7f77e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f77e40692a0 0x7f77e406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009fd0 tx=0x7f77e800b040 comp rx=0 tx=0).stop 2024-03-16T09:00:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 msgr2=0x7f77f810da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 0x7f77f810da00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b370 tx=0x7f77e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 shutdown_connections 2024-03-16T09:00:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f77e40692a0 0x7f77e406b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77f80fcee0 0x7f77f810da00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 --2- 172.21.15.7:0/1408608515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77f80fe0d0 0x7f77f810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 >> 172.21.15.7:0/1408608515 conn(0x7f77f80f8560 msgr2=0x7f77f80f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 shutdown_connections 2024-03-16T09:00:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.821+0000 7f77fe24e700 1 -- 172.21.15.7:0/1408608515 wait complete. 2024-03-16T09:00:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 -- 172.21.15.7:0/2407091797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40fcee0 msgr2=0x7fabf40fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 --2- 172.21.15.7:0/2407091797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40fcee0 0x7fabf40fd310 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fabe8004940 tx=0x7fabe8009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 -- 172.21.15.7:0/2407091797 shutdown_connections 2024-03-16T09:00:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 --2- 172.21.15.7:0/2407091797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf40fe0d0 0x7fabf40fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 --2- 172.21.15.7:0/2407091797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40fcee0 0x7fabf40fd310 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 -- 172.21.15.7:0/2407091797 >> 172.21.15.7:0/2407091797 conn(0x7fabf40f8560 msgr2=0x7fabf40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 -- 172.21.15.7:0/2407091797 shutdown_connections 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 -- 172.21.15.7:0/2407091797 wait complete. 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 Processor -- start 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.925+0000 7fabfa682700 1 -- start start 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabfa682700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf40fcee0 0x7fabf4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabfa682700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40fe0d0 0x7fabf4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabfa682700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40708c0 con 0x7fabf40fe0d0 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabfa682700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4070a30 con 0x7fabf40fcee0 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabf3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf40fcee0 0x7fabf4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabf3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf40fcee0 0x7fabf4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:47606/0 (socket says 172.21.15.7:47606) 2024-03-16T09:00:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabf3fff700 1 -- 172.21.15.7:0/3250319520 learned_addr learned my addr 172.21.15.7:0/3250319520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabf37fe700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40fe0d0 0x7fabf4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabf3fff700 1 -- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40fe0d0 msgr2=0x7fabf4070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabf3fff700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40fe0d0 0x7fabf4070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.926+0000 7fabf3fff700 1 -- 172.21.15.7:0/3250319520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe8005040 con 0x7fabf40fcee0 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.927+0000 7fabf3fff700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf40fcee0 0x7fabf4071e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fabe8000c00 tx=0x7fabe8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.927+0000 7fabf0ff9700 1 -- 172.21.15.7:0/3250319520 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe800e030 con 0x7fabf40fcee0 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.927+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf4070cb0 con 0x7fabf40fcee0 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.927+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf4115a00 con 0x7fabf40fcee0 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.927+0000 7fabf0ff9700 1 -- 172.21.15.7:0/3250319520 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabe801c400 con 0x7fabf40fcee0 2024-03-16T09:00:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.927+0000 7fabf0ff9700 1 -- 172.21.15.7:0/3250319520 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe801cea0 con 0x7fabf40fcee0 2024-03-16T09:00:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.928+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe00052e0 con 0x7fabf40fcee0 2024-03-16T09:00:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.929+0000 7fabf0ff9700 1 -- 172.21.15.7:0/3250319520 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fabe801c940 con 0x7fabf40fcee0 2024-03-16T09:00:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.929+0000 7fabf0ff9700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fabdc0692a0 0x7fabdc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.930+0000 7fabf0ff9700 1 -- 172.21.15.7:0/3250319520 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fabe80885f0 con 0x7fabf40fcee0 2024-03-16T09:00:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.930+0000 7fabf37fe700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fabdc0692a0 0x7fabdc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.930+0000 7fabf37fe700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fabdc0692a0 0x7fabdc06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009fd0 tx=0x7fabe400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:25.938+0000 7fabf0ff9700 1 -- 172.21.15.7:0/3250319520 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7fabe8054970 con 0x7fabf40fcee0 2024-03-16T09:00:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.215+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fabe0005da0 con 0x7fabf40fcee0 2024-03-16T09:00:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.215+0000 7fabf0ff9700 1 -- 172.21.15.7:0/3250319520 <== mon.1 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fabe8057f90 con 0x7fabf40fcee0 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:00:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:00:26.219 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:00:26.219 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 14 2024-03-16T09:00:26.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:00:26.219 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.217+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fabdc0692a0 msgr2=0x7fabdc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.217+0000 7fabfa682700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fabdc0692a0 0x7fabdc06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009fd0 tx=0x7fabe400b040 comp rx=0 tx=0).stop 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.217+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf40fcee0 msgr2=0x7fabf4071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.217+0000 7fabfa682700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf40fcee0 0x7fabf4071e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fabe8000c00 tx=0x7fabe8009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.218+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 shutdown_connections 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.218+0000 7fabfa682700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7fabdc0692a0 0x7fabdc06b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.218+0000 7fabfa682700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabf40fcee0 0x7fabf4071e40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.218+0000 7fabfa682700 1 --2- 172.21.15.7:0/3250319520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf40fe0d0 0x7fabf4070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.218+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 >> 172.21.15.7:0/3250319520 conn(0x7fabf40f8560 msgr2=0x7fabf40f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.218+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 shutdown_connections 2024-03-16T09:00:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.218+0000 7fabfa682700 1 -- 172.21.15.7:0/3250319520 wait complete. 2024-03-16T09:00:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.323+0000 7f3680902700 1 -- 172.21.15.7:0/1943676415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 msgr2=0x7f367c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.323+0000 7f3680902700 1 --2- 172.21.15.7:0/1943676415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 0x7f367c0fd420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f366c004940 tx=0x7f366c009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.323+0000 7f3680902700 1 -- 172.21.15.7:0/1943676415 shutdown_connections 2024-03-16T09:00:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.323+0000 7f3680902700 1 --2- 172.21.15.7:0/1943676415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 0x7f367c0fd420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.323+0000 7f3680902700 1 --2- 172.21.15.7:0/1943676415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c0fbd00 0x7f367c0fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.323+0000 7f3680902700 1 -- 172.21.15.7:0/1943676415 >> 172.21.15.7:0/1943676415 conn(0x7f367c0f7440 msgr2=0x7f367c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.323+0000 7f3680902700 1 -- 172.21.15.7:0/1943676415 shutdown_connections 2024-03-16T09:00:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.323+0000 7f3680902700 1 -- 172.21.15.7:0/1943676415 wait complete. 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3680902700 1 Processor -- start 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3680902700 1 -- start start 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3680902700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c0fbd00 0x7f367c066c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3680902700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 0x7f367c1126d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3680902700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c112c10 con 0x7f367c0fcfb0 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3680902700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c112d50 con 0x7f367c0fbd00 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3679d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 0x7f367c1126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3679d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 0x7f367c1126d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48224/0 (socket says 172.21.15.7:48224) 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f3679d9b700 1 -- 172.21.15.7:0/3683077427 learned_addr learned my addr 172.21.15.7:0/3683077427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.324+0000 7f367a59c700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c0fbd00 0x7f367c066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3679d9b700 1 -- 172.21.15.7:0/3683077427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c0fbd00 msgr2=0x7f367c066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3679d9b700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c0fbd00 0x7f367c066c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3679d9b700 1 -- 172.21.15.7:0/3683077427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c005040 con 0x7f367c0fcfb0 2024-03-16T09:00:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3679d9b700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 0x7f367c1126d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f366c009150 tx=0x7f366c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3672ffd700 1 -- 172.21.15.7:0/3683077427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366c00e040 con 0x7f367c0fcfb0 2024-03-16T09:00:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3672ffd700 1 -- 172.21.15.7:0/3683077427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f366c004290 con 0x7f367c0fcfb0 2024-03-16T09:00:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3672ffd700 1 -- 172.21.15.7:0/3683077427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366c01c630 con 0x7f367c0fcfb0 2024-03-16T09:00:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c112f50 con 0x7f367c0fcfb0 2024-03-16T09:00:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.325+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c1134e0 con 0x7f367c0fcfb0 2024-03-16T09:00:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.326+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367c04c710 con 0x7f367c0fcfb0 2024-03-16T09:00:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.327+0000 7f3672ffd700 1 -- 172.21.15.7:0/3683077427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f366c00bbf0 con 0x7f367c0fcfb0 2024-03-16T09:00:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.327+0000 7f3672ffd700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f36680692a0 0x7f366806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.327+0000 7f3672ffd700 1 -- 172.21.15.7:0/3683077427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f366c087d90 con 0x7f367c0fcfb0 2024-03-16T09:00:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.327+0000 7f367a59c700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f36680692a0 0x7f366806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.328+0000 7f367a59c700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f36680692a0 0x7f366806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3664009fd0 tx=0x7f366400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.335+0000 7f3672ffd700 1 -- 172.21.15.7:0/3683077427 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175445 (secure 0 0 0) 0x7f366c057ba0 con 0x7f367c0fcfb0 2024-03-16T09:00:26.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.549+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 --> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f367c101250 con 0x7f36680692a0 2024-03-16T09:00:26.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.550+0000 7f3672ffd700 1 -- 172.21.15.7:0/3683077427 <== mgr.14208 v2:172.21.15.7:6800/2519997150 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f367c101250 con 0x7f36680692a0 2024-03-16T09:00:26.552 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:00:26.552 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:00:26.553 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:00:26.553 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T09:00:26.553 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-03-16T09:00:26.553 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "0/2 daemons upgraded", 2024-03-16T09:00:26.553 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mgr daemons", 2024-03-16T09:00:26.553 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:00:26.553 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f36680692a0 msgr2=0x7f366806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f36680692a0 0x7f366806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3664009fd0 tx=0x7f366400b040 comp rx=0 tx=0).stop 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 msgr2=0x7f367c1126d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 0x7f367c1126d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f366c009150 tx=0x7f366c009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 shutdown_connections 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:6800/2519997150,v1:172.21.15.7:6801/2519997150] conn(0x7f36680692a0 0x7f366806b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f367c0fbd00 0x7f367c066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 --2- 172.21.15.7:0/3683077427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0fcfb0 0x7f367c1126d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.552+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 >> 172.21.15.7:0/3683077427 conn(0x7f367c0f7440 msgr2=0x7f367c0f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.553+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 shutdown_connections 2024-03-16T09:00:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:26.553+0000 7f3680902700 1 -- 172.21.15.7:0/3683077427 wait complete. 2024-03-16T09:00:26.719 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 93% (11490/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-16T09:00:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: pgmap v236: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:00:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:00:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:00:26.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3250319520' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:00:26.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:26 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:00:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: pgmap v236: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:00:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:00:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:00:26.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3250319520' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:00:26.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:26 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:00:27.388 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'. 2024-03-16T09:00:27.388 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-16T09:00:27.388 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-16T09:00:27.388 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-16T09:00:27.388 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-03-16T09:00:27.389 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at 3d78f5ba07b Merge PR #54581 into wip-batrick-testing-20240314.183243 2024-03-16T09:00:27.393 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-16T09:00:27.426 INFO:tasks.workunit.client.0.smithi007.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-16T09:00:27.428 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-16T09:00:27.428 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-16T09:00:27.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:27 smithi019 ceph-mon[36407]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:27.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:27 smithi019 ceph-mon[36407]: Upgrade: Updating mgr.smithi019.iuqxhi 2024-03-16T09:00:27.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:27 smithi019 ceph-mon[36407]: Deploying daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T09:00:27.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:27 smithi019 ceph-mon[36407]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:27.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:27 smithi007 ceph-mon[29470]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:27.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:27 smithi007 ceph-mon[29470]: Upgrade: Updating mgr.smithi019.iuqxhi 2024-03-16T09:00:27.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:27 smithi007 ceph-mon[29470]: Deploying daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T09:00:27.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:27 smithi007 ceph-mon[29470]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:28.235 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-16T09:00:28.309 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-16T09:00:28.348 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-16T09:00:28.349 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-16T09:00:28.350 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-16T09:00:28.388 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-16T09:00:28.391 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T09:00:28.391 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-16T09:00:28.446 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-16T09:00:28.449 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-16T09:00:28.449 DEBUG:teuthology.orchestra.run.smithi007:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-16T09:00:28.512 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir -p fsstress 2024-03-16T09:00:28.513 INFO:tasks.workunit.client.0.smithi007.stderr:+ pushd fsstress 2024-03-16T09:00:28.514 INFO:tasks.workunit.client.0.smithi007.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-16T09:00:28.514 INFO:tasks.workunit.client.0.smithi007.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-16T09:00:28.879 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:28 smithi019 ceph-mon[36407]: pgmap v237: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:28 smithi007 ceph-mon[29470]: pgmap v237: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:29.032 INFO:tasks.workunit.client.0.smithi007.stderr:+ tar xzf ltp-full.tgz 2024-03-16T09:00:30.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:30 smithi019 ceph-mon[36407]: pgmap v238: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:30.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:30 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:30.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:30 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:30.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:30 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:00:30.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:30 smithi007 ceph-mon[29470]: pgmap v238: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:00:30.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:30 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:30.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:30 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:30.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:30 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:00:32.315 INFO:tasks.workunit.client.1.smithi019.stderr:Updating files: 89% (11011/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-16T09:00:32.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:32 smithi019 ceph-mon[36407]: pgmap v239: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.5 KiB/s wr, 0 op/s 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr:Note: switching to '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'. 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr: 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr: 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr: 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr: git switch -c 2024-03-16T09:00:32.763 INFO:tasks.workunit.client.1.smithi019.stderr: 2024-03-16T09:00:32.764 INFO:tasks.workunit.client.1.smithi019.stderr:Or undo this operation with: 2024-03-16T09:00:32.764 INFO:tasks.workunit.client.1.smithi019.stderr: 2024-03-16T09:00:32.764 INFO:tasks.workunit.client.1.smithi019.stderr: git switch - 2024-03-16T09:00:32.764 INFO:tasks.workunit.client.1.smithi019.stderr: 2024-03-16T09:00:32.764 INFO:tasks.workunit.client.1.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-16T09:00:32.764 INFO:tasks.workunit.client.1.smithi019.stderr: 2024-03-16T09:00:32.764 INFO:tasks.workunit.client.1.smithi019.stderr:HEAD is now at 3d78f5ba07b Merge PR #54581 into wip-batrick-testing-20240314.183243 2024-03-16T09:00:32.767 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-16T09:00:32.851 INFO:tasks.workunit.client.1.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-16T09:00:32.853 INFO:tasks.workunit.client.1.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-16T09:00:32.853 INFO:tasks.workunit.client.1.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-16T09:00:32.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:32 smithi007 ceph-mon[29470]: pgmap v239: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.5 KiB/s wr, 0 op/s 2024-03-16T09:00:33.703 INFO:tasks.workunit.client.1.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-16T09:00:33.796 INFO:tasks.workunit.client.1.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-16T09:00:33.837 INFO:tasks.workunit.client.1.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-16T09:00:33.838 INFO:tasks.workunit.client.1.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-16T09:00:33.838 INFO:tasks.workunit.client.1.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-16T09:00:33.899 INFO:tasks.workunit.client.1.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-16T09:00:33.904 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T09:00:33.904 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-16T09:00:33.974 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-16T09:00:33.975 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-16T09:00:33.975 DEBUG:teuthology.orchestra.run.smithi019:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-16T09:00:34.065 INFO:tasks.workunit.client.1.smithi019.stderr:+ mkdir -p fsstress 2024-03-16T09:00:34.071 INFO:tasks.workunit.client.1.smithi019.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-16T09:00:34.071 INFO:tasks.workunit.client.1.smithi019.stderr:+ pushd fsstress 2024-03-16T09:00:34.071 INFO:tasks.workunit.client.1.smithi019.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-16T09:00:34.718 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:34 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:34.718 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:34 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:34.718 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:34 smithi019 ceph-mon[36407]: pgmap v240: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 348 KiB/s wr, 0 op/s 2024-03-16T09:00:34.718 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:34 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:34.718 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:34 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:34.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:34 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:34.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:34 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:34.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:34 smithi007 ceph-mon[29470]: pgmap v240: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 348 KiB/s wr, 0 op/s 2024-03-16T09:00:34.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:34 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:34.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:34 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:36.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:36 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:00:36.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:36 smithi019 ceph-mon[36407]: pgmap v241: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 348 KiB/s wr, 0 op/s 2024-03-16T09:00:36.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:36 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:00:36.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:36 smithi007 ceph-mon[29470]: pgmap v241: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 348 KiB/s wr, 0 op/s 2024-03-16T09:00:37.034 INFO:tasks.workunit.client.1.smithi019.stderr:+ tar xzf ltp-full.tgz 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: pgmap v242: 65 pgs: 65 active+clean; 27 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s wr, 166 op/s 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: Standby manager daemon smithi019.iuqxhi restarted 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T09:00:38.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr fail", "who": "smithi007.uamwtp"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: osdmap e47: 6 total, 6 up, 6 in 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "mgr fail", "who": "smithi007.uamwtp"}]': finished 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: mgrmap e20: smithi019.iuqxhi(active, starting, since 0.00773425s) 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr metadata", "who": "smithi019.iuqxhi", "id": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:00:39.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: Manager daemon smithi019.iuqxhi is now available 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.iuqxhi/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.iuqxhi/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:00:39.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.iuqxhi/trash_purge_schedule"}]: dispatch 2024-03-16T09:00:39.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:38 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.iuqxhi/trash_purge_schedule"}]: dispatch 2024-03-16T09:00:39.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: pgmap v242: 65 pgs: 65 active+clean; 27 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s wr, 166 op/s 2024-03-16T09:00:39.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:39.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: Standby manager daemon smithi019.iuqxhi restarted 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr fail", "who": "smithi007.uamwtp"}]: dispatch 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: osdmap e47: 6 total, 6 up, 6 in 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.14208 172.21.15.7:0/4234353702' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "mgr fail", "who": "smithi007.uamwtp"}]': finished 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: mgrmap e20: smithi019.iuqxhi(active, starting, since 0.00773425s) 2024-03-16T09:00:39.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr metadata", "who": "smithi019.iuqxhi", "id": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:00:39.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: Manager daemon smithi019.iuqxhi is now available 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.iuqxhi/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.iuqxhi/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.iuqxhi/trash_purge_schedule"}]: dispatch 2024-03-16T09:00:39.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:38 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi019.iuqxhi/trash_purge_schedule"}]: dispatch 2024-03-16T09:00:39.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:39 smithi007 ceph-mon[29470]: Deploying cephadm binary to smithi019 2024-03-16T09:00:39.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:39 smithi007 ceph-mon[29470]: Deploying cephadm binary to smithi007 2024-03-16T09:00:39.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:39 smithi007 ceph-mon[29470]: mgrmap e21: smithi019.iuqxhi(active, since 1.00898s) 2024-03-16T09:00:39.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:39 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:39.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:39 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:40.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:39 smithi019 ceph-mon[36407]: Deploying cephadm binary to smithi019 2024-03-16T09:00:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:39 smithi019 ceph-mon[36407]: Deploying cephadm binary to smithi007 2024-03-16T09:00:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:39 smithi019 ceph-mon[36407]: mgrmap e21: smithi019.iuqxhi(active, since 1.00898s) 2024-03-16T09:00:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:39 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:39 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:40.964 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:40 smithi007 ceph-mon[29470]: pgmap v3: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:00:41.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:40 smithi019 ceph-mon[36407]: pgmap v3: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:00:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:41 smithi007 ceph-mon[29470]: pgmap v4: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:00:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:41 smithi007 ceph-mon[29470]: [16/Mar/2024:09:00:40] ENGINE Bus STARTING 2024-03-16T09:00:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:41 smithi007 ceph-mon[29470]: [16/Mar/2024:09:00:40] ENGINE Serving on https://172.21.15.19:7150 2024-03-16T09:00:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:41 smithi007 ceph-mon[29470]: [16/Mar/2024:09:00:40] ENGINE Serving on http://172.21.15.19:8765 2024-03-16T09:00:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:41 smithi007 ceph-mon[29470]: [16/Mar/2024:09:00:40] ENGINE Bus STARTED 2024-03-16T09:00:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:41 smithi007 ceph-mon[29470]: mgrmap e22: smithi019.iuqxhi(active, since 2s) 2024-03-16T09:00:42.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:41 smithi019 ceph-mon[36407]: pgmap v4: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:00:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:41 smithi019 ceph-mon[36407]: [16/Mar/2024:09:00:40] ENGINE Bus STARTING 2024-03-16T09:00:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:41 smithi019 ceph-mon[36407]: [16/Mar/2024:09:00:40] ENGINE Serving on https://172.21.15.19:7150 2024-03-16T09:00:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:41 smithi019 ceph-mon[36407]: [16/Mar/2024:09:00:40] ENGINE Serving on http://172.21.15.19:8765 2024-03-16T09:00:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:41 smithi019 ceph-mon[36407]: [16/Mar/2024:09:00:40] ENGINE Bus STARTED 2024-03-16T09:00:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:41 smithi019 ceph-mon[36407]: mgrmap e22: smithi019.iuqxhi(active, since 2s) 2024-03-16T09:00:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:43 smithi007 ceph-mon[29470]: pgmap v5: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:00:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:43 smithi007 ceph-mon[29470]: mgrmap e23: smithi019.iuqxhi(active, since 4s) 2024-03-16T09:00:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:43 smithi019 ceph-mon[36407]: pgmap v5: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:00:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:43 smithi019 ceph-mon[36407]: mgrmap e23: smithi019.iuqxhi(active, since 4s) 2024-03-16T09:00:45.982 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:45 smithi007 ceph-mon[29470]: pgmap v6: 65 pgs: 65 active+clean; 73 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s wr, 66 op/s 2024-03-16T09:00:46.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:45 smithi019 ceph-mon[36407]: pgmap v6: 65 pgs: 65 active+clean; 73 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s wr, 66 op/s 2024-03-16T09:00:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:46 smithi007 ceph-mon[29470]: Standby manager daemon smithi007.uamwtp started 2024-03-16T09:00:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:46 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.7:0/4049100708' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.uamwtp/crt"}]: dispatch 2024-03-16T09:00:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:46 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.7:0/4049100708' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:00:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:46 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.7:0/4049100708' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.uamwtp/key"}]: dispatch 2024-03-16T09:00:46.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:46 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.7:0/4049100708' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:00:47.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:46 smithi019 ceph-mon[36407]: Standby manager daemon smithi007.uamwtp started 2024-03-16T09:00:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:46 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.7:0/4049100708' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.uamwtp/crt"}]: dispatch 2024-03-16T09:00:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:46 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.7:0/4049100708' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:00:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:46 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.7:0/4049100708' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.uamwtp/key"}]: dispatch 2024-03-16T09:00:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:46 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.7:0/4049100708' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:00:47.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:47 smithi019 ceph-mon[36407]: pgmap v7: 65 pgs: 65 active+clean; 89 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 3.9 MiB/s wr, 163 op/s 2024-03-16T09:00:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:47 smithi019 ceph-mon[36407]: mgrmap e24: smithi019.iuqxhi(active, since 8s), standbys: smithi007.uamwtp 2024-03-16T09:00:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:47 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T09:00:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:47 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:47 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:47 smithi007 ceph-mon[29470]: pgmap v7: 65 pgs: 65 active+clean; 89 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 3.9 MiB/s wr, 163 op/s 2024-03-16T09:00:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:47 smithi007 ceph-mon[29470]: mgrmap e24: smithi019.iuqxhi(active, since 8s), standbys: smithi007.uamwtp 2024-03-16T09:00:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:47 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T09:00:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:47 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:47 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:49.423 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:49 smithi007 ceph-mon[29470]: pgmap v8: 65 pgs: 65 active+clean; 89 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 3.0 MiB/s wr, 127 op/s 2024-03-16T09:00:49.424 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:49 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:49.424 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:49 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:49.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:49 smithi019 ceph-mon[36407]: pgmap v8: 65 pgs: 65 active+clean; 89 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 3.0 MiB/s wr, 127 op/s 2024-03-16T09:00:49.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:49 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:49.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:49 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:51.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:51 smithi007 ceph-mon[29470]: pgmap v9: 65 pgs: 65 active+clean; 100 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 110 op/s 2024-03-16T09:00:51.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:51 smithi019 ceph-mon[36407]: pgmap v9: 65 pgs: 65 active+clean; 100 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 110 op/s 2024-03-16T09:00:54.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:53 smithi019 ceph-mon[36407]: pgmap v10: 65 pgs: 65 active+clean; 100 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 101 op/s 2024-03-16T09:00:54.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:53 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:54.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:53 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:54.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:53 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:00:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:53 smithi007 ceph-mon[29470]: pgmap v10: 65 pgs: 65 active+clean; 100 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 101 op/s 2024-03-16T09:00:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:53 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:53 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:53 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: pgmap v11: 65 pgs: 65 active+clean; 114 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 211 op/s 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: Adjusting osd_memory_target on smithi007 to 1559M 2024-03-16T09:00:55.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:55 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:55.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: pgmap v11: 65 pgs: 65 active+clean; 114 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 4.5 MiB/s wr, 211 op/s 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: Adjusting osd_memory_target on smithi007 to 1559M 2024-03-16T09:00:55.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:55 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:00:56.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.730+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3436332388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fb440 msgr2=0x7fe69c0fb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:56.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.730+0000 7fe6a0b48700 1 --2- 172.21.15.7:0/3436332388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fb440 0x7fe69c0fb850 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe694004e00 tx=0x7fe69400adb0 comp rx=0 tx=0).stop 2024-03-16T09:00:56.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.731+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3436332388 shutdown_connections 2024-03-16T09:00:56.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.731+0000 7fe6a0b48700 1 --2- 172.21.15.7:0/3436332388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe69c0fc6f0 0x7fe69c0fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:56.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.731+0000 7fe6a0b48700 1 --2- 172.21.15.7:0/3436332388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fb440 0x7fe69c0fb850 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:56.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.731+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3436332388 >> 172.21.15.7:0/3436332388 conn(0x7fe69c0f6ba0 msgr2=0x7fe69c0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:56.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.731+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3436332388 shutdown_connections 2024-03-16T09:00:56.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.731+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3436332388 wait complete. 2024-03-16T09:00:56.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.733+0000 7fe6a0b48700 1 Processor -- start 2024-03-16T09:00:56.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.733+0000 7fe6a0b48700 1 -- start start 2024-03-16T09:00:56.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.733+0000 7fe6a0b48700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe69c0fb440 0x7fe69c10b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:56.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.733+0000 7fe6a0b48700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fc6f0 0x7fe69c10bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:56.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.733+0000 7fe6a0b48700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c10c280 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.733+0000 7fe6a0b48700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69c10cad0 con 0x7fe69c0fb440 2024-03-16T09:00:56.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.734+0000 7fe699d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fc6f0 0x7fe69c10bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:56.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.734+0000 7fe699d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fc6f0 0x7fe69c10bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47302/0 (socket says 172.21.15.7:47302) 2024-03-16T09:00:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.734+0000 7fe699d9b700 1 -- 172.21.15.7:0/3538869898 learned_addr learned my addr 172.21.15.7:0/3538869898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.734+0000 7fe69a59c700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe69c0fb440 0x7fe69c10b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.734+0000 7fe699d9b700 1 -- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe69c0fb440 msgr2=0x7fe69c10b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.734+0000 7fe699d9b700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe69c0fb440 0x7fe69c10b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.734+0000 7fe699d9b700 1 -- 172.21.15.7:0/3538869898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe694004a10 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.735+0000 7fe699d9b700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fc6f0 0x7fe69c10bc80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe69000fce0 tx=0x7fe6900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.735+0000 7fe68affd700 1 -- 172.21.15.7:0/3538869898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe69000fe90 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.735+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69c10cdb0 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.735+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69c10d2f0 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.735+0000 7fe68affd700 1 -- 172.21.15.7:0/3538869898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6900090d0 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.735+0000 7fe68affd700 1 -- 172.21.15.7:0/3538869898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6900126a0 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.737+0000 7fe68affd700 1 -- 172.21.15.7:0/3538869898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7fe690007ab0 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.738+0000 7fe68affd700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe68406b730 0x7fe68406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.738+0000 7fe68affd700 1 -- 172.21.15.7:0/3538869898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe69008c950 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.738+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c0052e0 con 0x7fe69c0fc6f0 2024-03-16T09:00:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.740+0000 7fe69a59c700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe68406b730 0x7fe68406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.741+0000 7fe69a59c700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe68406b730 0x7fe68406dbe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe69400afd0 tx=0x7fe69400adb0 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:56.747+0000 7fe68affd700 1 -- 172.21.15.7:0/3538869898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fe690059df0 con 0x7fe69c0fc6f0 2024-03-16T09:00:57.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.013+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe67c000bf0 con 0x7fe68406b730 2024-03-16T09:00:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.015+0000 7fe68affd700 1 -- 172.21.15.7:0/3538869898 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fe67c000bf0 con 0x7fe68406b730 2024-03-16T09:00:57.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe68406b730 msgr2=0x7fe68406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe68406b730 0x7fe68406dbe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe69400afd0 tx=0x7fe69400adb0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fc6f0 msgr2=0x7fe69c10bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fc6f0 0x7fe69c10bc80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe69000fce0 tx=0x7fe6900046b0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 shutdown_connections 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe68406b730 0x7fe68406dbe0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe69c0fb440 0x7fe69c10b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 --2- 172.21.15.7:0/3538869898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe69c0fc6f0 0x7fe69c10bc80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 >> 172.21.15.7:0/3538869898 conn(0x7fe69c0f6ba0 msgr2=0x7fe69c0f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 shutdown_connections 2024-03-16T09:00:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.017+0000 7fe6a0b48700 1 -- 172.21.15.7:0/3538869898 wait complete. 2024-03-16T09:00:57.030 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:00:57.097 INFO:tasks.workunit.client.0.smithi007.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-16T09:00:57.105 INFO:tasks.workunit.client.0.smithi007.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-16T09:00:57.105 INFO:tasks.workunit.client.0.smithi007.stderr:+ make 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.161+0000 7fcbdf267700 1 -- 172.21.15.7:0/4245891466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 msgr2=0x7fcbd81026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.161+0000 7fcbdf267700 1 --2- 172.21.15.7:0/4245891466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 0x7fcbd81026c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc004940 tx=0x7fcbcc009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.161+0000 7fcbdf267700 1 -- 172.21.15.7:0/4245891466 shutdown_connections 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.161+0000 7fcbdf267700 1 --2- 172.21.15.7:0/4245891466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 0x7fcbd81026c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.161+0000 7fcbdf267700 1 --2- 172.21.15.7:0/4245891466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fd9c0 0x7fcbd80ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.161+0000 7fcbdf267700 1 -- 172.21.15.7:0/4245891466 >> 172.21.15.7:0/4245891466 conn(0x7fcbd80f7480 msgr2=0x7fcbd80f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.162+0000 7fcbdf267700 1 -- 172.21.15.7:0/4245891466 shutdown_connections 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.162+0000 7fcbdf267700 1 -- 172.21.15.7:0/4245891466 wait complete. 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.162+0000 7fcbdf267700 1 Processor -- start 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.162+0000 7fcbdf267700 1 -- start start 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.162+0000 7fcbdf267700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fd9c0 0x7fcbd810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.162+0000 7fcbdf267700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 0x7fcbd810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdf267700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd810e540 con 0x7fcbd81002e0 2024-03-16T09:00:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdf267700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd810ed40 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdc802700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 0x7fcbd810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdd003700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fd9c0 0x7fcbd810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdc802700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 0x7fcbd810df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47326/0 (socket says 172.21.15.7:47326) 2024-03-16T09:00:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdd003700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fd9c0 0x7fcbd810da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:38850/0 (socket says 172.21.15.7:38850) 2024-03-16T09:00:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdc802700 1 -- 172.21.15.7:0/4043356761 learned_addr learned my addr 172.21.15.7:0/4043356761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdd003700 1 -- 172.21.15.7:0/4043356761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 msgr2=0x7fcbd810df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdd003700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 0x7fcbd810df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdd003700 1 -- 172.21.15.7:0/4043356761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc005040 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdc802700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 0x7fcbd810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:00:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdd003700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fd9c0 0x7fcbd810da00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8066960 tx=0x7fcbc800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbd5ffb700 1 -- 172.21.15.7:0/4043356761 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbc8004e90 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.163+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd810efc0 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.164+0000 7fcbd5ffb700 1 -- 172.21.15.7:0/4043356761 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbc800f040 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.164+0000 7fcbd5ffb700 1 -- 172.21.15.7:0/4043356761 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbc800ca20 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.164+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd810f500 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.165+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd805f5f0 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.165+0000 7fcbd5ffb700 1 -- 172.21.15.7:0/4043356761 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7fcbc800cbd0 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.166+0000 7fcbd5ffb700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fcbc006b660 0x7fcbc006db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.166+0000 7fcbd5ffb700 1 -- 172.21.15.7:0/4043356761 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fcbc8089cc0 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.166+0000 7fcbdc802700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fcbc006b660 0x7fcbc006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.166+0000 7fcbdc802700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fcbc006b660 0x7fcbc006db10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00b0d0 tx=0x7fcbcc009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.174+0000 7fcbd5ffb700 1 -- 172.21.15.7:0/4043356761 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fcbc8050040 con 0x7fcbd80fd9c0 2024-03-16T09:00:57.263 INFO:tasks.workunit.client.0.smithi007.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-16T09:00:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.467+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbd810f7d0 con 0x7fcbc006b660 2024-03-16T09:00:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.468+0000 7fcbd5ffb700 1 -- 172.21.15.7:0/4043356761 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fcbd810f7d0 con 0x7fcbc006b660 2024-03-16T09:00:57.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fcbc006b660 msgr2=0x7fcbc006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fcbc006b660 0x7fcbc006db10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00b0d0 tx=0x7fcbcc009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fd9c0 msgr2=0x7fcbd810da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fd9c0 0x7fcbd810da00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8066960 tx=0x7fcbc800b040 comp rx=0 tx=0).stop 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 shutdown_connections 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fcbc006b660 0x7fcbc006db10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbd80fd9c0 0x7fcbd810da00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 --2- 172.21.15.7:0/4043356761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd81002e0 0x7fcbd810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 >> 172.21.15.7:0/4043356761 conn(0x7fcbd80f7480 msgr2=0x7fcbd80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 shutdown_connections 2024-03-16T09:00:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.470+0000 7fcbdf267700 1 -- 172.21.15.7:0/4043356761 wait complete. 2024-03-16T09:00:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.582+0000 7fe11ced9700 1 -- 172.21.15.7:0/2736959022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 msgr2=0x7fe1180fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.582+0000 7fe11ced9700 1 --2- 172.21.15.7:0/2736959022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 0x7fe1180fe4b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe10c004940 tx=0x7fe10c009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.582+0000 7fe11ced9700 1 -- 172.21.15.7:0/2736959022 shutdown_connections 2024-03-16T09:00:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.582+0000 7fe11ced9700 1 --2- 172.21.15.7:0/2736959022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 0x7fe1180fe4b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.582+0000 7fe11ced9700 1 --2- 172.21.15.7:0/2736959022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1180fce40 0x7fe1180fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.582+0000 7fe11ced9700 1 -- 172.21.15.7:0/2736959022 >> 172.21.15.7:0/2736959022 conn(0x7fe1180f8570 msgr2=0x7fe1180fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.582+0000 7fe11ced9700 1 -- 172.21.15.7:0/2736959022 shutdown_connections 2024-03-16T09:00:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.582+0000 7fe11ced9700 1 -- 172.21.15.7:0/2736959022 wait complete. 2024-03-16T09:00:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11ced9700 1 Processor -- start 2024-03-16T09:00:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11ced9700 1 -- start start 2024-03-16T09:00:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11ced9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1180fce40 0x7fe118071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11ced9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 0x7fe118070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11ced9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180708a0 con 0x7fe1180fe040 2024-03-16T09:00:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11ced9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180709e0 con 0x7fe1180fce40 2024-03-16T09:00:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1180fce40 0x7fe118071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe115d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 0x7fe118070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1180fce40 0x7fe118071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:38872/0 (socket says 172.21.15.7:38872) 2024-03-16T09:00:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11659c700 1 -- 172.21.15.7:0/732838448 learned_addr learned my addr 172.21.15.7:0/732838448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe115d9b700 1 -- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1180fce40 msgr2=0x7fe118071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe115d9b700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1180fce40 0x7fe118071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe115d9b700 1 -- 172.21.15.7:0/732838448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c005040 con 0x7fe1180fe040 2024-03-16T09:00:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.583+0000 7fe11659c700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1180fce40 0x7fe118071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:00:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.584+0000 7fe115d9b700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 0x7fe118070310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00b970 tx=0x7fe10c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.584+0000 7fe106ffd700 1 -- 172.21.15.7:0/732838448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10c00e040 con 0x7fe1180fe040 2024-03-16T09:00:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.584+0000 7fe106ffd700 1 -- 172.21.15.7:0/732838448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe10c00bdc0 con 0x7fe1180fe040 2024-03-16T09:00:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.584+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe118070c60 con 0x7fe1180fe040 2024-03-16T09:00:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.584+0000 7fe106ffd700 1 -- 172.21.15.7:0/732838448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10c01c7f0 con 0x7fe1180fe040 2024-03-16T09:00:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.584+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe118115840 con 0x7fe1180fe040 2024-03-16T09:00:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.585+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11805f5f0 con 0x7fe1180fe040 2024-03-16T09:00:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.585+0000 7fe106ffd700 1 -- 172.21.15.7:0/732838448 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7fe10c013420 con 0x7fe1180fe040 2024-03-16T09:00:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.586+0000 7fe106ffd700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe10006b660 0x7fe10006db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.586+0000 7fe106ffd700 1 -- 172.21.15.7:0/732838448 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe10c08b790 con 0x7fe1180fe040 2024-03-16T09:00:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.586+0000 7fe11659c700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe10006b660 0x7fe10006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.587+0000 7fe11659c700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe10006b660 0x7fe10006db10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe118071b80 tx=0x7fe10800b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.591+0000 7fe106ffd700 1 -- 172.21.15.7:0/732838448 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fe10c055410 con 0x7fe1180fe040 2024-03-16T09:00:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:57 smithi007 ceph-mon[29470]: pgmap v12: 65 pgs: 65 active+clean; 135 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 383 op/s 2024-03-16T09:00:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.852+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe118102280 con 0x7fe10006b660 2024-03-16T09:00:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.865+0000 7fe106ffd700 1 -- 172.21.15.7:0/732838448 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fe118102280 con 0x7fe10006b660 2024-03-16T09:00:57.867 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:00:57.867 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (6m) 5s ago 7m 25.3M - 0.25.0 c8568f914cd2 d5db4ce74c0d 2024-03-16T09:00:57.867 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (8m) 5s ago 8m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (7m) 10s ago 7m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (6m) 5s ago 7m 90.7M - 9.4.7 954c08fa6188 4f8b149e57d8 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (2m) 5s ago 2m 11.3M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (2m) 5s ago 2m 140M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (2m) 10s ago 2m 59.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (2m) 10s ago 2m 11.9M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:9283 running (9m) 5s ago 9m 385M - 17.2.7-648-g1df27247 0bdcfccf14ce 376995a75ba3 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283 running (28s) 10s ago 6m 491M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 35bb11552f05 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (9m) 5s ago 9m 45.8M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (6m) 10s ago 6m 41.4M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (8m) 5s ago 7m 13.9M - 1.5.0 0da6a335fe13 9ca7275ea387 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (6m) 10s ago 6m 16.8M - 1.5.0 0da6a335fe13 9edea61eb01f 2024-03-16T09:00:57.868 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (6m) 5s ago 6m 100M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:00:57.869 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (5m) 5s ago 5m 87.9M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:00:57.869 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (5m) 5s ago 5m 82.8M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:00:57.869 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (4m) 10s ago 4m 95.0M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:00:57.869 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (4m) 10s ago 4m 82.6M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:00:57.869 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (3m) 10s ago 3m 86.5M 5313M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:00:57.869 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (6m) 5s ago 7m 59.7M - 2.43.0 a07b618ecd1d b457178ffb05 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe10006b660 msgr2=0x7fe10006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe10006b660 0x7fe10006db10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe118071b80 tx=0x7fe10800b040 comp rx=0 tx=0).stop 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 msgr2=0x7fe118070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 0x7fe118070310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00b970 tx=0x7fe10c009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 shutdown_connections 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fe10006b660 0x7fe10006db10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1180fce40 0x7fe118071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 --2- 172.21.15.7:0/732838448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180fe040 0x7fe118070310 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.868+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 >> 172.21.15.7:0/732838448 conn(0x7fe1180f8570 msgr2=0x7fe1180f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.869+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 shutdown_connections 2024-03-16T09:00:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.869+0000 7fe11ced9700 1 -- 172.21.15.7:0/732838448 wait complete. 2024-03-16T09:00:57.967 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:57 smithi019 ceph-mon[36407]: pgmap v12: 65 pgs: 65 active+clean; 135 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 383 op/s 2024-03-16T09:00:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.975+0000 7f123fee6700 1 -- 172.21.15.7:0/3117526435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12380fff00 msgr2=0x7f1238102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.975+0000 7f123fee6700 1 --2- 172.21.15.7:0/3117526435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12380fff00 0x7f1238102320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1228004940 tx=0x7f1228009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.975+0000 7f123fee6700 1 -- 172.21.15.7:0/3117526435 shutdown_connections 2024-03-16T09:00:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.975+0000 7f123fee6700 1 --2- 172.21.15.7:0/3117526435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12380fff00 0x7f1238102320 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 --2- 172.21.15.7:0/3117526435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12380662e0 0x7f12380ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 -- 172.21.15.7:0/3117526435 >> 172.21.15.7:0/3117526435 conn(0x7f12380f7430 msgr2=0x7f12380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 -- 172.21.15.7:0/3117526435 shutdown_connections 2024-03-16T09:00:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 -- 172.21.15.7:0/3117526435 wait complete. 2024-03-16T09:00:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 Processor -- start 2024-03-16T09:00:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 -- start start 2024-03-16T09:00:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12380662e0 0x7f123810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12380fff00 0x7f123810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123810e580 con 0x7f12380fff00 2024-03-16T09:00:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.976+0000 7f123fee6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123810ed80 con 0x7f12380662e0 2024-03-16T09:00:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123dc82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12380662e0 0x7f123810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123d481700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12380fff00 0x7f123810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123dc82700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12380662e0 0x7f123810da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:38890/0 (socket says 172.21.15.7:38890) 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123dc82700 1 -- 172.21.15.7:0/220959638 learned_addr learned my addr 172.21.15.7:0/220959638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123dc82700 1 -- 172.21.15.7:0/220959638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12380fff00 msgr2=0x7f123810df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123dc82700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12380fff00 0x7f123810df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123dc82700 1 -- 172.21.15.7:0/220959638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228005040 con 0x7f12380662e0 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123dc82700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12380662e0 0x7f123810da40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f123400eb40 tx=0x7f123400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f122e7fc700 1 -- 172.21.15.7:0/220959638 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1234007cc0 con 0x7f12380662e0 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123810f000 con 0x7f12380662e0 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f122e7fc700 1 -- 172.21.15.7:0/220959638 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f123400f040 con 0x7f12380662e0 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f122e7fc700 1 -- 172.21.15.7:0/220959638 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123400c9d0 con 0x7f12380662e0 2024-03-16T09:00:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.977+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123810f540 con 0x7f12380662e0 2024-03-16T09:00:57.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.979+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123805f5f0 con 0x7f12380662e0 2024-03-16T09:00:57.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.979+0000 7f122e7fc700 1 -- 172.21.15.7:0/220959638 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7f123400cc40 con 0x7f12380662e0 2024-03-16T09:00:57.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.979+0000 7f122e7fc700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f122406b660 0x7f122406db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:57.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.979+0000 7f122e7fc700 1 -- 172.21.15.7:0/220959638 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f1234089c80 con 0x7f12380662e0 2024-03-16T09:00:57.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.980+0000 7f123d481700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f122406b660 0x7f122406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:57.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.980+0000 7f123d481700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f122406b660 0x7f122406db10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f122800b0d0 tx=0x7f1228009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:57.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:57.984+0000 7f122e7fc700 1 -- 172.21.15.7:0/220959638 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f1234050040 con 0x7f12380662e0 2024-03-16T09:00:58.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.315+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f12381058d0 con 0x7f12380662e0 2024-03-16T09:00:58.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.316+0000 7f122e7fc700 1 -- 172.21.15.7:0/220959638 <== mon.1 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f1234057460 con 0x7f12380662e0 2024-03-16T09:00:58.318 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:00:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:00:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:00:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:00:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:00:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 1, 2024-03-16T09:00:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 13, 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:00:58.319 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:00:58.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f122406b660 msgr2=0x7f122406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f122406b660 0x7f122406db10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f122800b0d0 tx=0x7f1228009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12380662e0 msgr2=0x7f123810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12380662e0 0x7f123810da40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f123400eb40 tx=0x7f123400b040 comp rx=0 tx=0).stop 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 shutdown_connections 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f122406b660 0x7f122406db10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12380662e0 0x7f123810da40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 --2- 172.21.15.7:0/220959638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12380fff00 0x7f123810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 >> 172.21.15.7:0/220959638 conn(0x7f12380f7430 msgr2=0x7f12380f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 shutdown_connections 2024-03-16T09:00:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.318+0000 7f123fee6700 1 -- 172.21.15.7:0/220959638 wait complete. 2024-03-16T09:00:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 -- 172.21.15.7:0/3096805106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c8066af0 msgr2=0x7f75c80ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 --2- 172.21.15.7:0/3096805106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c8066af0 0x7f75c80ffa50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f75b8004940 tx=0x7f75b8009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 -- 172.21.15.7:0/3096805106 shutdown_connections 2024-03-16T09:00:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 --2- 172.21.15.7:0/3096805106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75c80fff90 0x7f75c8102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 --2- 172.21.15.7:0/3096805106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c8066af0 0x7f75c80ffa50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 -- 172.21.15.7:0/3096805106 >> 172.21.15.7:0/3096805106 conn(0x7f75c80f7440 msgr2=0x7f75c80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 -- 172.21.15.7:0/3096805106 shutdown_connections 2024-03-16T09:00:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 -- 172.21.15.7:0/3096805106 wait complete. 2024-03-16T09:00:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.439+0000 7f75cd9d9700 1 Processor -- start 2024-03-16T09:00:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75cd9d9700 1 -- start start 2024-03-16T09:00:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75cd9d9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75c8066af0 0x7f75c80fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75cd9d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c80fff90 0x7f75c80fb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75cd9d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c80fbee0 con 0x7f75c80fff90 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75cd9d9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c80fc020 con 0x7f75c8066af0 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75c67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c80fff90 0x7f75c80fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75c67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c80fff90 0x7f75c80fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47378/0 (socket says 172.21.15.7:47378) 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75c67fc700 1 -- 172.21.15.7:0/1251329021 learned_addr learned my addr 172.21.15.7:0/1251329021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75c6ffd700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75c8066af0 0x7f75c80fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75c67fc700 1 -- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75c8066af0 msgr2=0x7f75c80fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75c67fc700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75c8066af0 0x7f75c80fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75c67fc700 1 -- 172.21.15.7:0/1251329021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b8005040 con 0x7f75c80fff90 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.440+0000 7f75c67fc700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c80fff90 0x7f75c80fb950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00eb40 tx=0x7f75bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.441+0000 7f75af7fe700 1 -- 172.21.15.7:0/1251329021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc007cc0 con 0x7f75c80fff90 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.441+0000 7f75af7fe700 1 -- 172.21.15.7:0/1251329021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75bc00f040 con 0x7f75c80fff90 2024-03-16T09:00:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.441+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c80fc2a0 con 0x7f75c80fff90 2024-03-16T09:00:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.441+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c8115a50 con 0x7f75c80fff90 2024-03-16T09:00:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.441+0000 7f75af7fe700 1 -- 172.21.15.7:0/1251329021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc00ca10 con 0x7f75c80fff90 2024-03-16T09:00:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.442+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b40052e0 con 0x7f75c80fff90 2024-03-16T09:00:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.442+0000 7f75af7fe700 1 -- 172.21.15.7:0/1251329021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7f75bc00cb70 con 0x7f75c80fff90 2024-03-16T09:00:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.443+0000 7f75af7fe700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f75b006b660 0x7f75b006db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:00:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.443+0000 7f75af7fe700 1 -- 172.21.15.7:0/1251329021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f75bc089c60 con 0x7f75c80fff90 2024-03-16T09:00:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.443+0000 7f75c6ffd700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f75b006b660 0x7f75b006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:00:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.444+0000 7f75c6ffd700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f75b006b660 0x7f75b006db10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f75b8013740 tx=0x7f75b8009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:00:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.449+0000 7f75af7fe700 1 -- 172.21.15.7:0/1251329021 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f75bc050040 con 0x7f75c80fff90 2024-03-16T09:00:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.695+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75b4000bf0 con 0x7f75b006b660 2024-03-16T09:00:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.696+0000 7f75af7fe700 1 -- 172.21.15.7:0/1251329021 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f75b4000bf0 con 0x7f75b006b660 2024-03-16T09:00:58.699 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:00:58.699 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:00:58.700 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:00:58.700 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T09:00:58.700 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-03-16T09:00:58.700 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "1/2 daemons upgraded", 2024-03-16T09:00:58.700 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-03-16T09:00:58.701 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:00:58.701 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:00:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f75b006b660 msgr2=0x7f75b006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f75b006b660 0x7f75b006db10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f75b8013740 tx=0x7f75b8009e30 comp rx=0 tx=0).stop 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c80fff90 msgr2=0x7f75c80fb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c80fff90 0x7f75c80fb950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00eb40 tx=0x7f75bc00b040 comp rx=0 tx=0).stop 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 shutdown_connections 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f75b006b660 0x7f75b006db10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75c8066af0 0x7f75c80fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 --2- 172.21.15.7:0/1251329021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75c80fff90 0x7f75c80fb950 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 >> 172.21.15.7:0/1251329021 conn(0x7f75c80f7440 msgr2=0x7f75c80f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 shutdown_connections 2024-03-16T09:00:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:00:58.699+0000 7f75cd9d9700 1 -- 172.21.15.7:0/1251329021 wait complete. 2024-03-16T09:00:58.716 INFO:tasks.workunit.client.0.smithi007.stderr:++ readlink -f fsstress 2024-03-16T09:00:58.716 INFO:tasks.workunit.client.0.smithi007.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-16T09:00:58.716 INFO:tasks.workunit.client.0.smithi007.stderr:+ popd 2024-03-16T09:00:58.720 INFO:tasks.workunit.client.0.smithi007.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-16T09:00:58.720 INFO:tasks.workunit.client.0.smithi007.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-16T09:00:58.720 INFO:tasks.workunit.client.0.smithi007.stderr:+ popd 2024-03-16T09:00:58.720 INFO:tasks.workunit.client.0.smithi007.stderr:++ mktemp -d -p . 2024-03-16T09:00:58.722 INFO:tasks.workunit.client.0.smithi007.stderr:+ T=./tmp.u3zwYJdfBv 2024-03-16T09:00:58.722 INFO:tasks.workunit.client.0.smithi007.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.u3zwYJdfBv -l 1 -n 1000 -p 10 -v 2024-03-16T09:00:58.723 INFO:tasks.workunit.client.0.smithi007.stdout:seed = 1711167748 2024-03-16T09:00:58.726 INFO:tasks.workunit.client.0.smithi007.stdout:1/0: unlink - no file 2024-03-16T09:00:58.726 INFO:tasks.workunit.client.0.smithi007.stdout:1/1: getdents . 0 2024-03-16T09:00:58.726 INFO:tasks.workunit.client.0.smithi007.stdout:0/0: creat f0 x:0 0 0 2024-03-16T09:00:58.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/0: write - no filename 2024-03-16T09:00:58.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/1: chown . 31 1 2024-03-16T09:00:58.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/2: write - no filename 2024-03-16T09:00:58.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/3: fdatasync - no filename 2024-03-16T09:00:58.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/4: dwrite - no filename 2024-03-16T09:00:58.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/5: rename - no filename 2024-03-16T09:00:58.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/6: dwrite - no filename 2024-03-16T09:00:58.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/7: chown . 2988 1 2024-03-16T09:00:58.728 INFO:tasks.workunit.client.0.smithi007.stdout:2/8: stat - no entries 2024-03-16T09:00:58.728 INFO:tasks.workunit.client.0.smithi007.stdout:2/9: read - no filename 2024-03-16T09:00:58.728 INFO:tasks.workunit.client.0.smithi007.stdout:2/10: rename - no filename 2024-03-16T09:00:58.728 INFO:tasks.workunit.client.0.smithi007.stdout:2/11: truncate - no filename 2024-03-16T09:00:58.728 INFO:tasks.workunit.client.0.smithi007.stdout:2/12: rename - no filename 2024-03-16T09:00:58.728 INFO:tasks.workunit.client.0.smithi007.stdout:1/2: creat f0 x:0 0 0 2024-03-16T09:00:58.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/3: rmdir - no directory 2024-03-16T09:00:58.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/4: chown f0 1897 1 2024-03-16T09:00:58.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/5: read - f0 zero size 2024-03-16T09:00:58.729 INFO:tasks.workunit.client.0.smithi007.stdout:0/1: mknod c1 0 2024-03-16T09:00:58.729 INFO:tasks.workunit.client.0.smithi007.stdout:3/0: unlink - no file 2024-03-16T09:00:58.729 INFO:tasks.workunit.client.0.smithi007.stdout:2/13: creat f0 x:0 0 0 2024-03-16T09:00:58.730 INFO:tasks.workunit.client.0.smithi007.stdout:2/14: creat f1 x:0 0 0 2024-03-16T09:00:58.730 INFO:tasks.workunit.client.0.smithi007.stdout:2/15: truncate f1 26130 0 2024-03-16T09:00:58.730 INFO:tasks.workunit.client.0.smithi007.stdout:0/2: rename c1 to c2 0 2024-03-16T09:00:58.731 INFO:tasks.workunit.client.0.smithi007.stdout:4/0: dread - no filename 2024-03-16T09:00:58.731 INFO:tasks.workunit.client.0.smithi007.stdout:4/1: truncate - no filename 2024-03-16T09:00:58.731 INFO:tasks.workunit.client.0.smithi007.stdout:3/1: mknod c0 0 2024-03-16T09:00:58.731 INFO:tasks.workunit.client.0.smithi007.stdout:3/2: read - no filename 2024-03-16T09:00:58.732 INFO:tasks.workunit.client.0.smithi007.stdout:0/3: mkdir d3 0 2024-03-16T09:00:58.732 INFO:tasks.workunit.client.0.smithi007.stdout:0/4: truncate f0 623696 0 2024-03-16T09:00:58.732 INFO:tasks.workunit.client.0.smithi007.stdout:5/0: dwrite - no filename 2024-03-16T09:00:58.733 INFO:tasks.workunit.client.0.smithi007.stdout:4/2: creat f0 x:0 0 0 2024-03-16T09:00:58.733 INFO:tasks.workunit.client.0.smithi007.stdout:3/3: creat f1 x:0 0 0 2024-03-16T09:00:58.734 INFO:tasks.workunit.client.0.smithi007.stdout:6/0: chown . 6945 1 2024-03-16T09:00:58.734 INFO:tasks.workunit.client.0.smithi007.stdout:6/1: fsync - no filename 2024-03-16T09:00:58.734 INFO:tasks.workunit.client.0.smithi007.stdout:6/2: read - no filename 2024-03-16T09:00:58.735 INFO:tasks.workunit.client.0.smithi007.stdout:6/3: link - no file 2024-03-16T09:00:58.735 INFO:tasks.workunit.client.0.smithi007.stdout:6/4: dwrite - no filename 2024-03-16T09:00:58.735 INFO:tasks.workunit.client.0.smithi007.stdout:5/1: creat f0 x:0 0 0 2024-03-16T09:00:58.735 INFO:tasks.workunit.client.0.smithi007.stdout:5/2: chown f0 665 1 2024-03-16T09:00:58.736 INFO:tasks.workunit.client.0.smithi007.stdout:4/3: symlink l1 0 2024-03-16T09:00:58.736 INFO:tasks.workunit.client.0.smithi007.stdout:4/4: creat f2 x:0 0 0 2024-03-16T09:00:58.744 INFO:tasks.workunit.client.0.smithi007.stdout:5/3: mkdir d1 0 2024-03-16T09:00:58.745 INFO:tasks.workunit.client.0.smithi007.stdout:5/4: getdents d1 0 2024-03-16T09:00:58.745 INFO:tasks.workunit.client.0.smithi007.stdout:5/5: fdatasync f0 0 2024-03-16T09:00:58.745 INFO:tasks.workunit.client.0.smithi007.stdout:3/4: unlink f1 0 2024-03-16T09:00:58.745 INFO:tasks.workunit.client.0.smithi007.stdout:3/5: read - no filename 2024-03-16T09:00:58.745 INFO:tasks.workunit.client.0.smithi007.stdout:6/5: mknod c0 0 2024-03-16T09:00:58.745 INFO:tasks.workunit.client.0.smithi007.stdout:6/6: write - no filename 2024-03-16T09:00:58.747 INFO:tasks.workunit.client.0.smithi007.stdout:4/5: link f0 f3 0 2024-03-16T09:00:58.748 INFO:tasks.workunit.client.0.smithi007.stdout:4/6: write f2 [233842,91683] 0 2024-03-16T09:00:58.748 INFO:tasks.workunit.client.0.smithi007.stdout:4/7: write f2 [1342908,73987] 0 2024-03-16T09:00:58.752 INFO:tasks.workunit.client.0.smithi007.stdout:3/6: symlink l2 0 2024-03-16T09:00:58.752 INFO:tasks.workunit.client.0.smithi007.stdout:3/7: truncate - no filename 2024-03-16T09:00:58.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/8: truncate - no filename 2024-03-16T09:00:58.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/9: fsync - no filename 2024-03-16T09:00:58.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/10: chown c0 39 1 2024-03-16T09:00:58.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/11: dread - no filename 2024-03-16T09:00:58.754 INFO:tasks.workunit.client.0.smithi007.stdout:5/6: creat d1/f2 x:0 0 0 2024-03-16T09:00:58.754 INFO:tasks.workunit.client.0.smithi007.stdout:5/7: dread - d1/f2 zero size 2024-03-16T09:00:58.755 INFO:tasks.workunit.client.0.smithi007.stdout:6/7: mknod c1 0 2024-03-16T09:00:58.761 INFO:tasks.workunit.client.0.smithi007.stdout:3/12: unlink c0 0 2024-03-16T09:00:58.761 INFO:tasks.workunit.client.0.smithi007.stdout:3/13: creat f3 x:0 0 0 2024-03-16T09:00:58.763 INFO:tasks.workunit.client.0.smithi007.stdout:6/8: mkdir d2 0 2024-03-16T09:00:58.763 INFO:tasks.workunit.client.0.smithi007.stdout:6/9: readlink - no filename 2024-03-16T09:00:58.768 INFO:tasks.workunit.client.0.smithi007.stdout:6/10: rename c1 to d2/c3 0 2024-03-16T09:00:58.768 INFO:tasks.workunit.client.0.smithi007.stdout:6/11: dread - no filename 2024-03-16T09:00:58.788 INFO:tasks.workunit.client.0.smithi007.stdout:6/12: link c0 d2/c4 0 2024-03-16T09:00:58.789 INFO:tasks.workunit.client.0.smithi007.stdout:6/13: truncate - no filename 2024-03-16T09:00:58.794 INFO:tasks.workunit.client.0.smithi007.stdout:6/14: mknod d2/c5 0 2024-03-16T09:00:58.797 INFO:tasks.workunit.client.0.smithi007.stdout:6/15: mknod d2/c6 0 2024-03-16T09:00:58.797 INFO:tasks.workunit.client.0.smithi007.stdout:6/16: stat d2/c6 0 2024-03-16T09:00:58.797 INFO:tasks.workunit.client.0.smithi007.stdout:6/17: read - no filename 2024-03-16T09:00:58.797 INFO:tasks.workunit.client.0.smithi007.stdout:6/18: dwrite - no filename 2024-03-16T09:00:58.799 INFO:tasks.workunit.client.0.smithi007.stdout:6/19: creat d2/f7 x:0 0 0 2024-03-16T09:00:58.799 INFO:tasks.workunit.client.0.smithi007.stdout:6/20: dread - d2/f7 zero size 2024-03-16T09:00:58.889 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:58 smithi019 ceph-mon[36407]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:58.890 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:58 smithi019 ceph-mon[36407]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:58.890 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:58 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/220959638' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:00:58.977 INFO:tasks.workunit.client.0.smithi007.stdout:2/16: dwrite f0 [0,4194304] 0 2024-03-16T09:00:58.977 INFO:tasks.workunit.client.0.smithi007.stdout:2/17: creat f2 x:0 0 0 2024-03-16T09:00:58.993 INFO:tasks.workunit.client.0.smithi007.stdout:5/8: dwrite f0 [0,4194304] 0 2024-03-16T09:00:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:58 smithi007 ceph-mon[29470]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:58 smithi007 ceph-mon[29470]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:58 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/220959638' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:00:59.082 INFO:tasks.workunit.client.0.smithi007.stdout:9/0: dread - no filename 2024-03-16T09:00:59.082 INFO:tasks.workunit.client.0.smithi007.stdout:8/0: dread - no filename 2024-03-16T09:00:59.082 INFO:tasks.workunit.client.0.smithi007.stdout:7/0: dread - no filename 2024-03-16T09:00:59.082 INFO:tasks.workunit.client.0.smithi007.stdout:9/1: read - no filename 2024-03-16T09:00:59.082 INFO:tasks.workunit.client.0.smithi007.stdout:8/1: stat - no entries 2024-03-16T09:00:59.082 INFO:tasks.workunit.client.0.smithi007.stdout:9/2: dread - no filename 2024-03-16T09:00:59.082 INFO:tasks.workunit.client.0.smithi007.stdout:8/2: rename - no filename 2024-03-16T09:00:59.082 INFO:tasks.workunit.client.0.smithi007.stdout:8/3: read - no filename 2024-03-16T09:00:59.146 INFO:tasks.workunit.client.0.smithi007.stdout:4/8: dwrite f2 [0,4194304] 0 2024-03-16T09:00:59.146 INFO:tasks.workunit.client.0.smithi007.stdout:4/9: truncate f0 189416 0 2024-03-16T09:00:59.146 INFO:tasks.workunit.client.0.smithi007.stdout:4/10: rmdir - no directory 2024-03-16T09:00:59.146 INFO:tasks.workunit.client.0.smithi007.stdout:4/11: readlink l1 0 2024-03-16T09:00:59.212 INFO:tasks.workunit.client.0.smithi007.stdout:3/14: dwrite f3 [0,4194304] 0 2024-03-16T09:00:59.306 INFO:tasks.workunit.client.0.smithi007.stdout:2/18: symlink l3 0 2024-03-16T09:00:59.318 INFO:tasks.workunit.client.0.smithi007.stdout:2/19: write f0 [661923,71066] 0 2024-03-16T09:00:59.318 INFO:tasks.workunit.client.0.smithi007.stdout:2/20: rmdir - no directory 2024-03-16T09:00:59.318 INFO:tasks.workunit.client.0.smithi007.stdout:2/21: dread - f2 zero size 2024-03-16T09:00:59.318 INFO:tasks.workunit.client.0.smithi007.stdout:2/22: write f0 [2605885,115905] 0 2024-03-16T09:00:59.318 INFO:tasks.workunit.client.0.smithi007.stdout:2/23: write f0 [1575992,31109] 0 2024-03-16T09:00:59.318 INFO:tasks.workunit.client.0.smithi007.stdout:2/24: chown f2 3 1 2024-03-16T09:00:59.335 INFO:tasks.workunit.client.0.smithi007.stdout:5/9: mknod d1/c3 0 2024-03-16T09:00:59.664 INFO:tasks.workunit.client.0.smithi007.stdout:8/4: mkdir d0 0 2024-03-16T09:00:59.664 INFO:tasks.workunit.client.0.smithi007.stdout:8/5: rename d0 to d0/d1 22 2024-03-16T09:00:59.664 INFO:tasks.workunit.client.0.smithi007.stdout:9/3: symlink l0 0 2024-03-16T09:00:59.665 INFO:tasks.workunit.client.0.smithi007.stdout:9/4: chown l0 30948 1 2024-03-16T09:00:59.665 INFO:tasks.workunit.client.0.smithi007.stdout:9/5: stat l0 0 2024-03-16T09:00:59.665 INFO:tasks.workunit.client.0.smithi007.stdout:9/6: truncate - no filename 2024-03-16T09:00:59.665 INFO:tasks.workunit.client.0.smithi007.stdout:9/7: dwrite - no filename 2024-03-16T09:00:59.666 INFO:tasks.workunit.client.0.smithi007.stdout:7/1: mkdir d0 0 2024-03-16T09:00:59.666 INFO:tasks.workunit.client.0.smithi007.stdout:7/2: chown d0 91 1 2024-03-16T09:00:59.691 INFO:tasks.workunit.client.0.smithi007.stdout:4/12: mkdir d4 0 2024-03-16T09:00:59.691 INFO:tasks.workunit.client.0.smithi007.stdout:4/13: fdatasync f3 0 2024-03-16T09:00:59.709 INFO:tasks.workunit.client.0.smithi007.stdout:4/14: dwrite f0 [0,4194304] 0 2024-03-16T09:00:59.709 INFO:tasks.workunit.client.0.smithi007.stdout:4/15: stat d4 0 2024-03-16T09:00:59.710 INFO:tasks.workunit.client.0.smithi007.stdout:3/15: link l2 l4 0 2024-03-16T09:00:59.710 INFO:tasks.workunit.client.0.smithi007.stdout:3/16: creat f5 x:0 0 0 2024-03-16T09:00:59.710 INFO:tasks.workunit.client.0.smithi007.stdout:3/17: dread - f5 zero size 2024-03-16T09:00:59.711 INFO:tasks.workunit.client.0.smithi007.stdout:2/25: rename l3 to l4 0 2024-03-16T09:00:59.718 INFO:tasks.workunit.client.0.smithi007.stdout:8/6: mkdir d0/d2 0 2024-03-16T09:00:59.718 INFO:tasks.workunit.client.0.smithi007.stdout:8/7: dwrite - no filename 2024-03-16T09:00:59.718 INFO:tasks.workunit.client.0.smithi007.stdout:8/8: truncate - no filename 2024-03-16T09:00:59.718 INFO:tasks.workunit.client.0.smithi007.stdout:8/9: readlink - no filename 2024-03-16T09:00:59.719 INFO:tasks.workunit.client.0.smithi007.stdout:8/10: readlink - no filename 2024-03-16T09:00:59.719 INFO:tasks.workunit.client.0.smithi007.stdout:8/11: dread - no filename 2024-03-16T09:00:59.719 INFO:tasks.workunit.client.0.smithi007.stdout:8/12: dread - no filename 2024-03-16T09:00:59.719 INFO:tasks.workunit.client.0.smithi007.stdout:8/13: chown d0/d2 7 1 2024-03-16T09:00:59.719 INFO:tasks.workunit.client.0.smithi007.stdout:8/14: rename d0 to d0/d3 22 2024-03-16T09:00:59.721 INFO:tasks.workunit.client.0.smithi007.stdout:4/16: write f2 [832652,53581] 0 2024-03-16T09:00:59.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/3: creat d0/f1 x:0 0 0 2024-03-16T09:00:59.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/4: dread - d0/f1 zero size 2024-03-16T09:00:59.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/5: truncate d0/f1 746342 0 2024-03-16T09:00:59.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/18: write f3 [2551261,122858] 0 2024-03-16T09:00:59.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/19: write f3 [57329,35206] 0 2024-03-16T09:00:59.725 INFO:tasks.workunit.client.0.smithi007.stdout:2/26: symlink l5 0 2024-03-16T09:00:59.726 INFO:tasks.workunit.client.0.smithi007.stdout:8/15: symlink d0/l4 0 2024-03-16T09:00:59.726 INFO:tasks.workunit.client.0.smithi007.stdout:8/16: dwrite - no filename 2024-03-16T09:00:59.727 INFO:tasks.workunit.client.0.smithi007.stdout:8/17: write - no filename 2024-03-16T09:00:59.728 INFO:tasks.workunit.client.0.smithi007.stdout:3/20: mkdir d6 0 2024-03-16T09:00:59.730 INFO:tasks.workunit.client.0.smithi007.stdout:8/18: mknod d0/d2/c5 0 2024-03-16T09:00:59.734 INFO:tasks.workunit.client.0.smithi007.stdout:4/17: dread f3 [0,4194304] 0 2024-03-16T09:00:59.734 INFO:tasks.workunit.client.0.smithi007.stdout:8/19: mknod d0/d2/c6 0 2024-03-16T09:00:59.736 INFO:tasks.workunit.client.0.smithi007.stdout:8/20: symlink d0/l7 0 2024-03-16T09:00:59.736 INFO:tasks.workunit.client.0.smithi007.stdout:8/21: read - no filename 2024-03-16T09:00:59.736 INFO:tasks.workunit.client.0.smithi007.stdout:8/22: write - no filename 2024-03-16T09:00:59.736 INFO:tasks.workunit.client.0.smithi007.stdout:8/23: dwrite - no filename 2024-03-16T09:00:59.737 INFO:tasks.workunit.client.0.smithi007.stdout:8/24: dread - no filename 2024-03-16T09:00:59.737 INFO:tasks.workunit.client.0.smithi007.stdout:8/25: truncate - no filename 2024-03-16T09:00:59.738 INFO:tasks.workunit.client.0.smithi007.stdout:8/26: creat d0/f8 x:0 0 0 2024-03-16T09:00:59.738 INFO:tasks.workunit.client.0.smithi007.stdout:4/18: dread f3 [0,4194304] 0 2024-03-16T09:00:59.739 INFO:tasks.workunit.client.0.smithi007.stdout:8/27: symlink d0/l9 0 2024-03-16T09:00:59.739 INFO:tasks.workunit.client.0.smithi007.stdout:8/28: write d0/f8 [678131,90484] 0 2024-03-16T09:00:59.741 INFO:tasks.workunit.client.0.smithi007.stdout:8/29: link d0/f8 d0/d2/fa 0 2024-03-16T09:00:59.743 INFO:tasks.workunit.client.0.smithi007.stdout:8/30: mknod d0/cb 0 2024-03-16T09:00:59.768 INFO:tasks.workunit.client.0.smithi007.stdout:5/10: rmdir d1 39 2024-03-16T09:00:59.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/11: mknod d1/c4 0 2024-03-16T09:00:59.872 INFO:tasks.workunit.client.0.smithi007.stdout:9/8: getdents . 0 2024-03-16T09:00:59.872 INFO:tasks.workunit.client.0.smithi007.stdout:9/9: fdatasync - no filename 2024-03-16T09:00:59.873 INFO:tasks.workunit.client.0.smithi007.stdout:9/10: write - no filename 2024-03-16T09:00:59.874 INFO:tasks.workunit.client.0.smithi007.stdout:9/11: rename l0 to l1 0 2024-03-16T09:00:59.875 INFO:tasks.workunit.client.0.smithi007.stdout:9/12: mknod c2 0 2024-03-16T09:00:59.875 INFO:tasks.workunit.client.0.smithi007.stdout:9/13: dwrite - no filename 2024-03-16T09:00:59.877 INFO:tasks.workunit.client.0.smithi007.stdout:9/14: unlink c2 0 2024-03-16T09:00:59.877 INFO:tasks.workunit.client.0.smithi007.stdout:9/15: dwrite - no filename 2024-03-16T09:00:59.877 INFO:tasks.workunit.client.0.smithi007.stdout:9/16: truncate - no filename 2024-03-16T09:00:59.878 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:59 smithi019 ceph-mon[36407]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:59.878 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:59 smithi019 ceph-mon[36407]: pgmap v13: 65 pgs: 65 active+clean; 135 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 4.1 MiB/s wr, 316 op/s 2024-03-16T09:00:59.878 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:00:59 smithi019 ceph-mon[36407]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:59.879 INFO:tasks.workunit.client.0.smithi007.stdout:9/17: mknod c3 0 2024-03-16T09:00:59.881 INFO:tasks.workunit.client.0.smithi007.stdout:9/18: creat f4 x:0 0 0 2024-03-16T09:00:59.883 INFO:tasks.workunit.client.0.smithi007.stdout:9/19: mknod c5 0 2024-03-16T09:00:59.904 INFO:tasks.workunit.client.0.smithi007.stdout:7/6: getdents d0 0 2024-03-16T09:00:59.905 INFO:tasks.workunit.client.0.smithi007.stdout:7/7: symlink d0/l2 0 2024-03-16T09:00:59.906 INFO:tasks.workunit.client.0.smithi007.stdout:7/8: readlink d0/l2 0 2024-03-16T09:00:59.906 INFO:tasks.workunit.client.0.smithi007.stdout:7/9: creat d0/f3 x:0 0 0 2024-03-16T09:00:59.907 INFO:tasks.workunit.client.0.smithi007.stdout:3/21: fsync f5 0 2024-03-16T09:00:59.907 INFO:tasks.workunit.client.0.smithi007.stdout:3/22: getdents d6 0 2024-03-16T09:00:59.908 INFO:tasks.workunit.client.0.smithi007.stdout:7/10: dread d0/f1 [0,4194304] 0 2024-03-16T09:00:59.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/27: truncate f0 1437139 0 2024-03-16T09:00:59.910 INFO:tasks.workunit.client.0.smithi007.stdout:7/11: creat d0/f4 x:0 0 0 2024-03-16T09:00:59.914 INFO:tasks.workunit.client.0.smithi007.stdout:7/12: mknod d0/c5 0 2024-03-16T09:00:59.914 INFO:tasks.workunit.client.0.smithi007.stdout:7/13: write d0/f3 [555631,62319] 0 2024-03-16T09:00:59.914 INFO:tasks.workunit.client.0.smithi007.stdout:4/19: write f2 [4750178,4889] 0 2024-03-16T09:00:59.915 INFO:tasks.workunit.client.0.smithi007.stdout:8/31: fsync d0/f8 0 2024-03-16T09:00:59.915 INFO:tasks.workunit.client.0.smithi007.stdout:8/32: fsync d0/d2/fa 0 2024-03-16T09:00:59.915 INFO:tasks.workunit.client.0.smithi007.stdout:8/33: stat d0/l4 0 2024-03-16T09:00:59.917 INFO:tasks.workunit.client.0.smithi007.stdout:4/20: creat d4/f5 x:0 0 0 2024-03-16T09:00:59.917 INFO:tasks.workunit.client.0.smithi007.stdout:4/21: creat d4/f6 x:0 0 0 2024-03-16T09:00:59.918 INFO:tasks.workunit.client.0.smithi007.stdout:4/22: chown f0 911105782 1 2024-03-16T09:00:59.918 INFO:tasks.workunit.client.0.smithi007.stdout:8/34: mkdir d0/dc 0 2024-03-16T09:00:59.919 INFO:tasks.workunit.client.0.smithi007.stdout:5/12: getdents d1 0 2024-03-16T09:00:59.919 INFO:tasks.workunit.client.0.smithi007.stdout:5/13: rename d1 to d1/d5 22 2024-03-16T09:00:59.922 INFO:tasks.workunit.client.0.smithi007.stdout:9/20: rename l1 to l6 0 2024-03-16T09:00:59.922 INFO:tasks.workunit.client.0.smithi007.stdout:9/21: dread - f4 zero size 2024-03-16T09:00:59.922 INFO:tasks.workunit.client.0.smithi007.stdout:9/22: chown f4 1128597 1 2024-03-16T09:00:59.923 INFO:tasks.workunit.client.0.smithi007.stdout:4/23: dread f0 [0,4194304] 0 2024-03-16T09:00:59.923 INFO:tasks.workunit.client.0.smithi007.stdout:5/14: unlink d1/c3 0 2024-03-16T09:00:59.923 INFO:tasks.workunit.client.0.smithi007.stdout:3/23: dwrite f5 [0,4194304] 0 2024-03-16T09:00:59.925 INFO:tasks.workunit.client.0.smithi007.stdout:9/23: creat f7 x:0 0 0 2024-03-16T09:00:59.931 INFO:tasks.workunit.client.0.smithi007.stdout:7/14: dwrite d0/f1 [0,4194304] 0 2024-03-16T09:00:59.931 INFO:tasks.workunit.client.0.smithi007.stdout:7/15: dread - d0/f4 zero size 2024-03-16T09:00:59.931 INFO:tasks.workunit.client.0.smithi007.stdout:7/16: chown d0/f4 3 1 2024-03-16T09:00:59.931 INFO:tasks.workunit.client.0.smithi007.stdout:7/17: write d0/f4 [963519,96622] 0 2024-03-16T09:00:59.931 INFO:tasks.workunit.client.0.smithi007.stdout:9/24: creat f8 x:0 0 0 2024-03-16T09:00:59.933 INFO:tasks.workunit.client.0.smithi007.stdout:5/15: symlink d1/l6 0 2024-03-16T09:00:59.936 INFO:tasks.workunit.client.0.smithi007.stdout:8/35: rmdir d0 39 2024-03-16T09:00:59.937 INFO:tasks.workunit.client.0.smithi007.stdout:5/16: dread - d1/f2 zero size 2024-03-16T09:00:59.940 INFO:tasks.workunit.client.0.smithi007.stdout:4/24: dwrite d4/f5 [0,4194304] 0 2024-03-16T09:00:59.940 INFO:tasks.workunit.client.0.smithi007.stdout:3/24: rmdir d6 0 2024-03-16T09:00:59.941 INFO:tasks.workunit.client.0.smithi007.stdout:2/28: dread f0 [0,4194304] 0 2024-03-16T09:00:59.941 INFO:tasks.workunit.client.0.smithi007.stdout:2/29: creat f6 x:0 0 0 2024-03-16T09:00:59.942 INFO:tasks.workunit.client.0.smithi007.stdout:2/30: write f2 [403206,100211] 0 2024-03-16T09:00:59.942 INFO:tasks.workunit.client.0.smithi007.stdout:2/31: dread - f6 zero size 2024-03-16T09:00:59.942 INFO:tasks.workunit.client.0.smithi007.stdout:8/36: getdents d0 0 2024-03-16T09:00:59.949 INFO:tasks.workunit.client.0.smithi007.stdout:4/25: mkdir d4/d7 0 2024-03-16T09:00:59.951 INFO:tasks.workunit.client.0.smithi007.stdout:9/25: dwrite f8 [0,4194304] 0 2024-03-16T09:00:59.952 INFO:tasks.workunit.client.0.smithi007.stdout:8/37: mkdir d0/dc/dd 0 2024-03-16T09:00:59.953 INFO:tasks.workunit.client.0.smithi007.stdout:4/26: symlink d4/d7/l8 0 2024-03-16T09:00:59.954 INFO:tasks.workunit.client.0.smithi007.stdout:4/27: truncate d4/f6 534690 0 2024-03-16T09:00:59.954 INFO:tasks.workunit.client.0.smithi007.stdout:4/28: fsync d4/f6 0 2024-03-16T09:00:59.954 INFO:tasks.workunit.client.0.smithi007.stdout:4/29: rename d4 to d4/d7/d9 22 2024-03-16T09:00:59.954 INFO:tasks.workunit.client.0.smithi007.stdout:4/30: fdatasync d4/f5 0 2024-03-16T09:00:59.954 INFO:tasks.workunit.client.0.smithi007.stdout:4/31: creat d4/fa x:0 0 0 2024-03-16T09:00:59.955 INFO:tasks.workunit.client.0.smithi007.stdout:8/38: creat d0/fe x:0 0 0 2024-03-16T09:00:59.956 INFO:tasks.workunit.client.0.smithi007.stdout:8/39: fdatasync d0/fe 0 2024-03-16T09:00:59.956 INFO:tasks.workunit.client.0.smithi007.stdout:8/40: truncate d0/f8 1803646 0 2024-03-16T09:00:59.956 INFO:tasks.workunit.client.0.smithi007.stdout:8/41: dread - d0/fe zero size 2024-03-16T09:00:59.959 INFO:tasks.workunit.client.0.smithi007.stdout:9/26: mknod c9 0 2024-03-16T09:00:59.960 INFO:tasks.workunit.client.0.smithi007.stdout:9/27: creat fa x:0 0 0 2024-03-16T09:00:59.960 INFO:tasks.workunit.client.0.smithi007.stdout:8/42: write d0/d2/fa [1426699,72725] 0 2024-03-16T09:00:59.960 INFO:tasks.workunit.client.0.smithi007.stdout:8/43: chown d0/dc/dd 9 1 2024-03-16T09:00:59.960 INFO:tasks.workunit.client.0.smithi007.stdout:8/44: creat d0/ff x:0 0 0 2024-03-16T09:00:59.961 INFO:tasks.workunit.client.0.smithi007.stdout:8/45: dread - d0/fe zero size 2024-03-16T09:00:59.961 INFO:tasks.workunit.client.0.smithi007.stdout:5/17: dwrite f0 [0,4194304] 0 2024-03-16T09:00:59.961 INFO:tasks.workunit.client.0.smithi007.stdout:5/18: dread - d1/f2 zero size 2024-03-16T09:00:59.963 INFO:tasks.workunit.client.0.smithi007.stdout:3/25: dwrite f5 [0,4194304] 0 2024-03-16T09:00:59.964 INFO:tasks.workunit.client.0.smithi007.stdout:3/26: creat f7 x:0 0 0 2024-03-16T09:00:59.966 INFO:tasks.workunit.client.0.smithi007.stdout:5/19: mkdir d1/d7 0 2024-03-16T09:00:59.967 INFO:tasks.workunit.client.0.smithi007.stdout:2/32: getdents . 0 2024-03-16T09:00:59.969 INFO:tasks.workunit.client.0.smithi007.stdout:5/20: mkdir d1/d7/d8 0 2024-03-16T09:00:59.969 INFO:tasks.workunit.client.0.smithi007.stdout:5/21: readlink d1/l6 0 2024-03-16T09:00:59.974 INFO:tasks.workunit.client.0.smithi007.stdout:9/28: write f8 [473067,19015] 0 2024-03-16T09:00:59.974 INFO:tasks.workunit.client.0.smithi007.stdout:4/32: dwrite f0 [0,4194304] 0 2024-03-16T09:00:59.975 INFO:tasks.workunit.client.0.smithi007.stdout:2/33: fdatasync f2 0 2024-03-16T09:00:59.975 INFO:tasks.workunit.client.0.smithi007.stdout:2/34: chown f0 221023493 1 2024-03-16T09:00:59.976 INFO:tasks.workunit.client.0.smithi007.stdout:4/33: symlink d4/lb 0 2024-03-16T09:00:59.979 INFO:tasks.workunit.client.0.smithi007.stdout:4/34: creat d4/d7/fc x:0 0 0 2024-03-16T09:00:59.979 INFO:tasks.workunit.client.0.smithi007.stdout:4/35: chown f2 99 1 2024-03-16T09:00:59.981 INFO:tasks.workunit.client.0.smithi007.stdout:9/29: link c3 cb 0 2024-03-16T09:00:59.981 INFO:tasks.workunit.client.0.smithi007.stdout:9/30: write f4 [710513,104448] 0 2024-03-16T09:00:59.984 INFO:tasks.workunit.client.0.smithi007.stdout:8/46: dwrite d0/ff [0,4194304] 0 2024-03-16T09:00:59.984 INFO:tasks.workunit.client.0.smithi007.stdout:9/31: symlink lc 0 2024-03-16T09:00:59.988 INFO:tasks.workunit.client.0.smithi007.stdout:8/47: rename d0/f8 to d0/d2/f10 0 2024-03-16T09:00:59.989 INFO:tasks.workunit.client.0.smithi007.stdout:9/32: creat fd x:0 0 0 2024-03-16T09:00:59.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/27: dwrite f7 [0,4194304] 0 2024-03-16T09:00:59.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/28: readlink l4 0 2024-03-16T09:00:59.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/29: creat f8 x:0 0 0 2024-03-16T09:00:59.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/30: chown l4 57331630 1 2024-03-16T09:00:59.993 INFO:tasks.workunit.client.0.smithi007.stdout:4/36: getdents d4/d7 0 2024-03-16T09:00:59.993 INFO:tasks.workunit.client.0.smithi007.stdout:4/37: chown d4/d7/l8 9427 1 2024-03-16T09:00:59.993 INFO:tasks.workunit.client.0.smithi007.stdout:2/35: dwrite f2 [0,4194304] 0 2024-03-16T09:00:59.993 INFO:tasks.workunit.client.0.smithi007.stdout:2/36: rmdir - no directory 2024-03-16T09:00:59.993 INFO:tasks.workunit.client.0.smithi007.stdout:2/37: rmdir - no directory 2024-03-16T09:00:59.996 INFO:tasks.workunit.client.0.smithi007.stdout:4/38: creat d4/d7/fd x:0 0 0 2024-03-16T09:00:59.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:59 smithi007 ceph-mon[29470]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:00:59.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:59 smithi007 ceph-mon[29470]: pgmap v13: 65 pgs: 65 active+clean; 135 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 4.1 MiB/s wr, 316 op/s 2024-03-16T09:00:59.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:00:59 smithi007 ceph-mon[29470]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:00.002 INFO:tasks.workunit.client.0.smithi007.stdout:3/31: read f5 [2031796,54052] 0 2024-03-16T09:01:00.027 INFO:tasks.workunit.client.0.smithi007.stdout:9/33: dwrite fa [0,4194304] 0 2024-03-16T09:01:00.027 INFO:tasks.workunit.client.0.smithi007.stdout:4/39: dwrite f2 [4194304,4194304] 0 2024-03-16T09:01:00.028 INFO:tasks.workunit.client.0.smithi007.stdout:2/38: dwrite f6 [0,4194304] 0 2024-03-16T09:01:00.030 INFO:tasks.workunit.client.0.smithi007.stdout:2/39: creat f7 x:0 0 0 2024-03-16T09:01:00.030 INFO:tasks.workunit.client.0.smithi007.stdout:9/34: symlink le 0 2024-03-16T09:01:00.033 INFO:tasks.workunit.client.0.smithi007.stdout:9/35: rename fa to ff 0 2024-03-16T09:01:00.036 INFO:tasks.workunit.client.0.smithi007.stdout:9/36: symlink l10 0 2024-03-16T09:01:00.036 INFO:tasks.workunit.client.0.smithi007.stdout:9/37: fsync f8 0 2024-03-16T09:01:00.036 INFO:tasks.workunit.client.0.smithi007.stdout:9/38: fdatasync fd 0 2024-03-16T09:01:00.036 INFO:tasks.workunit.client.0.smithi007.stdout:9/39: write f4 [1031283,58844] 0 2024-03-16T09:01:00.037 INFO:tasks.workunit.client.0.smithi007.stdout:9/40: mknod c11 0 2024-03-16T09:01:00.041 INFO:tasks.workunit.client.0.smithi007.stdout:4/40: write f0 [522849,29565] 0 2024-03-16T09:01:00.043 INFO:tasks.workunit.client.0.smithi007.stdout:9/41: dread ff [0,4194304] 0 2024-03-16T09:01:00.044 INFO:tasks.workunit.client.0.smithi007.stdout:2/40: dread f2 [0,4194304] 0 2024-03-16T09:01:00.044 INFO:tasks.workunit.client.0.smithi007.stdout:2/41: fdatasync f1 0 2024-03-16T09:01:00.047 INFO:tasks.workunit.client.0.smithi007.stdout:2/42: dread f0 [0,4194304] 0 2024-03-16T09:01:00.048 INFO:tasks.workunit.client.0.smithi007.stdout:2/43: rename l5 to l8 0 2024-03-16T09:01:00.048 INFO:tasks.workunit.client.0.smithi007.stdout:2/44: chown l4 7 1 2024-03-16T09:01:00.048 INFO:tasks.workunit.client.0.smithi007.stdout:2/45: stat l4 0 2024-03-16T09:01:00.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/46: symlink l9 0 2024-03-16T09:01:00.050 INFO:tasks.workunit.client.0.smithi007.stdout:2/47: symlink la 0 2024-03-16T09:01:00.054 INFO:tasks.workunit.client.0.smithi007.stdout:4/41: dread f0 [0,4194304] 0 2024-03-16T09:01:00.054 INFO:tasks.workunit.client.0.smithi007.stdout:4/42: chown f0 4194242 1 2024-03-16T09:01:00.054 INFO:tasks.workunit.client.0.smithi007.stdout:4/43: creat d4/d7/fe x:0 0 0 2024-03-16T09:01:00.058 INFO:tasks.workunit.client.0.smithi007.stdout:2/48: rename l8 to lb 0 2024-03-16T09:01:00.059 INFO:tasks.workunit.client.0.smithi007.stdout:2/49: mkdir dc 0 2024-03-16T09:01:00.059 INFO:tasks.workunit.client.0.smithi007.stdout:2/50: fsync f1 0 2024-03-16T09:01:00.060 INFO:tasks.workunit.client.0.smithi007.stdout:2/51: creat dc/fd x:0 0 0 2024-03-16T09:01:00.060 INFO:tasks.workunit.client.0.smithi007.stdout:2/52: stat dc/fd 0 2024-03-16T09:01:00.060 INFO:tasks.workunit.client.0.smithi007.stdout:2/53: chown f1 6 1 2024-03-16T09:01:00.061 INFO:tasks.workunit.client.0.smithi007.stdout:2/54: symlink dc/le 0 2024-03-16T09:01:00.063 INFO:tasks.workunit.client.0.smithi007.stdout:9/42: dwrite f4 [0,4194304] 0 2024-03-16T09:01:00.071 INFO:tasks.workunit.client.0.smithi007.stdout:4/44: dwrite d4/d7/fd [0,4194304] 0 2024-03-16T09:01:00.071 INFO:tasks.workunit.client.0.smithi007.stdout:4/45: fsync f2 0 2024-03-16T09:01:00.072 INFO:tasks.workunit.client.0.smithi007.stdout:4/46: creat d4/ff x:0 0 0 2024-03-16T09:01:00.072 INFO:tasks.workunit.client.0.smithi007.stdout:9/43: truncate f4 2062941 0 2024-03-16T09:01:00.072 INFO:tasks.workunit.client.0.smithi007.stdout:9/44: chown fd 11333935 1 2024-03-16T09:01:00.072 INFO:tasks.workunit.client.0.smithi007.stdout:9/45: fdatasync f7 0 2024-03-16T09:01:00.072 INFO:tasks.workunit.client.0.smithi007.stdout:9/46: dread - fd zero size 2024-03-16T09:01:00.072 INFO:tasks.workunit.client.0.smithi007.stdout:9/47: write fd [1029049,28109] 0 2024-03-16T09:01:00.074 INFO:tasks.workunit.client.0.smithi007.stdout:9/48: symlink l12 0 2024-03-16T09:01:00.080 INFO:tasks.workunit.client.0.smithi007.stdout:2/55: dwrite f6 [0,4194304] 0 2024-03-16T09:01:00.084 INFO:tasks.workunit.client.0.smithi007.stdout:2/56: dread f0 [0,4194304] 0 2024-03-16T09:01:00.085 INFO:tasks.workunit.client.0.smithi007.stdout:2/57: getdents dc 0 2024-03-16T09:01:00.086 INFO:tasks.workunit.client.0.smithi007.stdout:2/58: mkdir dc/df 0 2024-03-16T09:01:00.086 INFO:tasks.workunit.client.0.smithi007.stdout:2/59: fdatasync f2 0 2024-03-16T09:01:00.092 INFO:tasks.workunit.client.0.smithi007.stdout:9/49: dwrite f7 [0,4194304] 0 2024-03-16T09:01:00.097 INFO:tasks.workunit.client.0.smithi007.stdout:9/50: dread f4 [0,4194304] 0 2024-03-16T09:01:00.097 INFO:tasks.workunit.client.0.smithi007.stdout:9/51: stat c11 0 2024-03-16T09:01:00.098 INFO:tasks.workunit.client.0.smithi007.stdout:9/52: write fd [590757,23704] 0 2024-03-16T09:01:00.098 INFO:tasks.workunit.client.0.smithi007.stdout:9/53: write f4 [2964300,5238] 0 2024-03-16T09:01:00.098 INFO:tasks.workunit.client.0.smithi007.stdout:9/54: chown f7 80874687 1 2024-03-16T09:01:00.101 INFO:tasks.workunit.client.0.smithi007.stdout:2/60: dwrite f1 [0,4194304] 0 2024-03-16T09:01:00.101 INFO:tasks.workunit.client.0.smithi007.stdout:2/61: chown dc 29826 1 2024-03-16T09:01:00.101 INFO:tasks.workunit.client.0.smithi007.stdout:2/62: write f7 [497200,43819] 0 2024-03-16T09:01:00.109 INFO:tasks.workunit.client.0.smithi007.stdout:2/63: read f1 [1824810,93982] 0 2024-03-16T09:01:00.110 INFO:tasks.workunit.client.0.smithi007.stdout:2/64: creat dc/f10 x:0 0 0 2024-03-16T09:01:00.111 INFO:tasks.workunit.client.0.smithi007.stdout:2/65: creat dc/f11 x:0 0 0 2024-03-16T09:01:00.119 INFO:tasks.workunit.client.0.smithi007.stdout:9/55: dwrite ff [4194304,4194304] 0 2024-03-16T09:01:00.119 INFO:tasks.workunit.client.0.smithi007.stdout:9/56: stat c5 0 2024-03-16T09:01:00.125 INFO:tasks.workunit.client.0.smithi007.stdout:9/57: truncate f4 146408 0 2024-03-16T09:01:00.132 INFO:tasks.workunit.client.0.smithi007.stdout:2/66: dwrite dc/f10 [0,4194304] 0 2024-03-16T09:01:00.135 INFO:tasks.workunit.client.0.smithi007.stdout:2/67: symlink dc/df/l12 0 2024-03-16T09:01:00.136 INFO:tasks.workunit.client.0.smithi007.stdout:2/68: symlink dc/l13 0 2024-03-16T09:01:00.137 INFO:tasks.workunit.client.0.smithi007.stdout:2/69: fsync f1 0 2024-03-16T09:01:00.137 INFO:tasks.workunit.client.0.smithi007.stdout:2/70: creat dc/df/f14 x:0 0 0 2024-03-16T09:01:00.139 INFO:tasks.workunit.client.0.smithi007.stdout:2/71: link f2 dc/f15 0 2024-03-16T09:01:00.145 INFO:tasks.workunit.client.0.smithi007.stdout:2/72: dread f1 [0,4194304] 0 2024-03-16T09:01:00.145 INFO:tasks.workunit.client.0.smithi007.stdout:2/73: truncate dc/f15 4966074 0 2024-03-16T09:01:00.146 INFO:tasks.workunit.client.0.smithi007.stdout:9/58: dwrite ff [4194304,4194304] 0 2024-03-16T09:01:00.151 INFO:tasks.workunit.client.0.smithi007.stdout:9/59: dread fd [0,4194304] 0 2024-03-16T09:01:00.153 INFO:tasks.workunit.client.0.smithi007.stdout:9/60: rename c11 to c13 0 2024-03-16T09:01:00.155 INFO:tasks.workunit.client.0.smithi007.stdout:9/61: link c3 c14 0 2024-03-16T09:01:00.157 INFO:tasks.workunit.client.0.smithi007.stdout:9/62: dread fd [0,4194304] 0 2024-03-16T09:01:00.157 INFO:tasks.workunit.client.0.smithi007.stdout:9/63: write fd [265961,77367] 0 2024-03-16T09:01:00.158 INFO:tasks.workunit.client.0.smithi007.stdout:9/64: dread f4 [0,4194304] 0 2024-03-16T09:01:00.159 INFO:tasks.workunit.client.0.smithi007.stdout:9/65: write f8 [5118577,42051] 0 2024-03-16T09:01:00.162 INFO:tasks.workunit.client.0.smithi007.stdout:9/66: dread f8 [4194304,4194304] 0 2024-03-16T09:01:00.164 INFO:tasks.workunit.client.0.smithi007.stdout:9/67: mknod c15 0 2024-03-16T09:01:00.164 INFO:tasks.workunit.client.0.smithi007.stdout:9/68: creat f16 x:0 0 0 2024-03-16T09:01:00.165 INFO:tasks.workunit.client.0.smithi007.stdout:9/69: creat f17 x:0 0 0 2024-03-16T09:01:00.166 INFO:tasks.workunit.client.0.smithi007.stdout:2/74: dwrite dc/f10 [0,4194304] 0 2024-03-16T09:01:00.166 INFO:tasks.workunit.client.0.smithi007.stdout:2/75: fdatasync dc/f11 0 2024-03-16T09:01:00.167 INFO:tasks.workunit.client.0.smithi007.stdout:2/76: mknod dc/c16 0 2024-03-16T09:01:00.170 INFO:tasks.workunit.client.0.smithi007.stdout:9/70: dread fd [0,4194304] 0 2024-03-16T09:01:00.171 INFO:tasks.workunit.client.0.smithi007.stdout:9/71: symlink l18 0 2024-03-16T09:01:00.172 INFO:tasks.workunit.client.0.smithi007.stdout:9/72: read - f17 zero size 2024-03-16T09:01:00.195 INFO:tasks.workunit.client.0.smithi007.stdout:2/77: dwrite dc/f15 [4194304,4194304] 0 2024-03-16T09:01:00.195 INFO:tasks.workunit.client.0.smithi007.stdout:9/73: dwrite f8 [4194304,4194304] 0 2024-03-16T09:01:00.197 INFO:tasks.workunit.client.0.smithi007.stdout:2/78: creat dc/f17 x:0 0 0 2024-03-16T09:01:00.197 INFO:tasks.workunit.client.0.smithi007.stdout:2/79: creat dc/df/f18 x:0 0 0 2024-03-16T09:01:00.199 INFO:tasks.workunit.client.0.smithi007.stdout:2/80: rename l9 to dc/df/l19 0 2024-03-16T09:01:00.204 INFO:tasks.workunit.client.0.smithi007.stdout:9/74: getdents . 0 2024-03-16T09:01:00.204 INFO:tasks.workunit.client.0.smithi007.stdout:9/75: readlink l6 0 2024-03-16T09:01:00.204 INFO:tasks.workunit.client.0.smithi007.stdout:9/76: chown l10 6309 1 2024-03-16T09:01:00.214 INFO:tasks.workunit.client.0.smithi007.stdout:9/77: dread ff [4194304,4194304] 0 2024-03-16T09:01:00.214 INFO:tasks.workunit.client.0.smithi007.stdout:9/78: dread - f16 zero size 2024-03-16T09:01:00.214 INFO:tasks.workunit.client.0.smithi007.stdout:9/79: chown c5 1689290 1 2024-03-16T09:01:00.217 INFO:tasks.workunit.client.0.smithi007.stdout:9/80: dread f8 [0,4194304] 0 2024-03-16T09:01:00.225 INFO:tasks.workunit.client.0.smithi007.stdout:9/81: write f7 [932330,65375] 0 2024-03-16T09:01:00.226 INFO:tasks.workunit.client.0.smithi007.stdout:9/82: mknod c19 0 2024-03-16T09:01:00.241 INFO:tasks.workunit.client.0.smithi007.stdout:9/83: dwrite f17 [0,4194304] 0 2024-03-16T09:01:00.242 INFO:tasks.workunit.client.0.smithi007.stdout:9/84: dread f4 [0,4194304] 0 2024-03-16T09:01:00.243 INFO:tasks.workunit.client.0.smithi007.stdout:9/85: creat f1a x:0 0 0 2024-03-16T09:01:00.243 INFO:tasks.workunit.client.0.smithi007.stdout:9/86: rmdir - no directory 2024-03-16T09:01:00.589 INFO:tasks.workunit.client.0.smithi007.stdout:6/21: sync 2024-03-16T09:01:00.589 INFO:tasks.workunit.client.0.smithi007.stdout:5/22: sync 2024-03-16T09:01:00.589 INFO:tasks.workunit.client.0.smithi007.stdout:0/5: sync 2024-03-16T09:01:00.589 INFO:tasks.workunit.client.0.smithi007.stdout:1/6: sync 2024-03-16T09:01:00.589 INFO:tasks.workunit.client.0.smithi007.stdout:5/23: fdatasync f0 0 2024-03-16T09:01:00.589 INFO:tasks.workunit.client.0.smithi007.stdout:6/22: fdatasync d2/f7 0 2024-03-16T09:01:00.589 INFO:tasks.workunit.client.0.smithi007.stdout:0/6: write f0 [833594,12415] 0 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:6/23: write d2/f7 [690882,26373] 0 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/7: creat f1 x:0 0 0 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:0/7: write f0 [1789437,4651] 0 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/8: write f0 [815672,88828] 0 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:7/18: sync 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:4/47: sync 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:4/48: truncate d4/d7/fc 8829 0 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:4/49: chown l1 272395 1 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:2/81: sync 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:8/48: sync 2024-03-16T09:01:00.590 INFO:tasks.workunit.client.0.smithi007.stdout:2/82: chown dc/c16 171 1 2024-03-16T09:01:00.591 INFO:tasks.workunit.client.0.smithi007.stdout:3/32: sync 2024-03-16T09:01:00.591 INFO:tasks.workunit.client.0.smithi007.stdout:9/87: sync 2024-03-16T09:01:00.591 INFO:tasks.workunit.client.0.smithi007.stdout:4/50: write d4/fa [308465,121368] 0 2024-03-16T09:01:00.591 INFO:tasks.workunit.client.0.smithi007.stdout:6/24: link c0 d2/c8 0 2024-03-16T09:01:00.591 INFO:tasks.workunit.client.0.smithi007.stdout:6/25: write d2/f7 [731084,35080] 0 2024-03-16T09:01:00.591 INFO:tasks.workunit.client.0.smithi007.stdout:4/51: read d4/d7/fc [4787,39351] 0 2024-03-16T09:01:00.592 INFO:tasks.workunit.client.0.smithi007.stdout:0/8: creat d3/f4 x:0 0 0 2024-03-16T09:01:00.592 INFO:tasks.workunit.client.0.smithi007.stdout:0/9: write f0 [2213818,38240] 0 2024-03-16T09:01:00.593 INFO:tasks.workunit.client.0.smithi007.stdout:1/9: mknod c2 0 2024-03-16T09:01:00.594 INFO:tasks.workunit.client.0.smithi007.stdout:2/83: creat dc/f1a x:0 0 0 2024-03-16T09:01:00.595 INFO:tasks.workunit.client.0.smithi007.stdout:6/26: link d2/c6 d2/c9 0 2024-03-16T09:01:00.595 INFO:tasks.workunit.client.0.smithi007.stdout:6/27: creat d2/fa x:0 0 0 2024-03-16T09:01:00.597 INFO:tasks.workunit.client.0.smithi007.stdout:4/52: rename d4/d7/fd to d4/f10 0 2024-03-16T09:01:00.597 INFO:tasks.workunit.client.0.smithi007.stdout:8/49: dread d0/ff [0,4194304] 0 2024-03-16T09:01:00.598 INFO:tasks.workunit.client.0.smithi007.stdout:0/10: mknod d3/c5 0 2024-03-16T09:01:00.598 INFO:tasks.workunit.client.0.smithi007.stdout:1/10: link c2 c3 0 2024-03-16T09:01:00.599 INFO:tasks.workunit.client.0.smithi007.stdout:5/24: write f0 [121711,119991] 0 2024-03-16T09:01:00.599 INFO:tasks.workunit.client.0.smithi007.stdout:5/25: write d1/f2 [763053,90239] 0 2024-03-16T09:01:00.600 INFO:tasks.workunit.client.0.smithi007.stdout:7/19: link d0/f4 d0/f6 0 2024-03-16T09:01:00.600 INFO:tasks.workunit.client.0.smithi007.stdout:2/84: symlink dc/df/l1b 0 2024-03-16T09:01:00.603 INFO:tasks.workunit.client.0.smithi007.stdout:8/50: truncate d0/d2/fa 528736 0 2024-03-16T09:01:00.603 INFO:tasks.workunit.client.0.smithi007.stdout:8/51: stat d0/d2/f10 0 2024-03-16T09:01:00.603 INFO:tasks.workunit.client.0.smithi007.stdout:8/52: readlink d0/l7 0 2024-03-16T09:01:00.604 INFO:tasks.workunit.client.0.smithi007.stdout:8/53: write d0/ff [5111624,113388] 0 2024-03-16T09:01:00.604 INFO:tasks.workunit.client.0.smithi007.stdout:8/54: dread - d0/fe zero size 2024-03-16T09:01:00.605 INFO:tasks.workunit.client.0.smithi007.stdout:0/11: rename f0 to d3/f6 0 2024-03-16T09:01:00.607 INFO:tasks.workunit.client.0.smithi007.stdout:7/20: truncate d0/f4 348311 0 2024-03-16T09:01:00.607 INFO:tasks.workunit.client.0.smithi007.stdout:3/33: dwrite f8 [0,4194304] 0 2024-03-16T09:01:00.608 INFO:tasks.workunit.client.0.smithi007.stdout:2/85: mknod dc/df/c1c 0 2024-03-16T09:01:00.608 INFO:tasks.workunit.client.0.smithi007.stdout:2/86: truncate f0 2352633 0 2024-03-16T09:01:00.611 INFO:tasks.workunit.client.0.smithi007.stdout:8/55: creat d0/dc/f11 x:0 0 0 2024-03-16T09:01:00.613 INFO:tasks.workunit.client.0.smithi007.stdout:5/26: rename d1/l6 to d1/d7/d8/l9 0 2024-03-16T09:01:00.615 INFO:tasks.workunit.client.0.smithi007.stdout:0/12: symlink d3/l7 0 2024-03-16T09:01:00.615 INFO:tasks.workunit.client.0.smithi007.stdout:7/21: creat d0/f7 x:0 0 0 2024-03-16T09:01:00.616 INFO:tasks.workunit.client.0.smithi007.stdout:1/11: dwrite f0 [0,4194304] 0 2024-03-16T09:01:00.622 INFO:tasks.workunit.client.0.smithi007.stdout:8/56: unlink d0/l7 0 2024-03-16T09:01:00.626 INFO:tasks.workunit.client.0.smithi007.stdout:5/27: dread d1/f2 [0,4194304] 0 2024-03-16T09:01:00.628 INFO:tasks.workunit.client.0.smithi007.stdout:6/28: getdents d2 0 2024-03-16T09:01:00.628 INFO:tasks.workunit.client.0.smithi007.stdout:6/29: fsync d2/f7 0 2024-03-16T09:01:00.628 INFO:tasks.workunit.client.0.smithi007.stdout:6/30: fdatasync d2/fa 0 2024-03-16T09:01:00.629 INFO:tasks.workunit.client.0.smithi007.stdout:1/12: dread f0 [0,4194304] 0 2024-03-16T09:01:00.629 INFO:tasks.workunit.client.0.smithi007.stdout:1/13: truncate f1 110876 0 2024-03-16T09:01:00.629 INFO:tasks.workunit.client.0.smithi007.stdout:1/14: dread f1 [0,4194304] 0 2024-03-16T09:01:00.632 INFO:tasks.workunit.client.0.smithi007.stdout:7/22: link d0/c5 d0/c8 0 2024-03-16T09:01:00.635 INFO:tasks.workunit.client.0.smithi007.stdout:3/34: dwrite f5 [4194304,4194304] 0 2024-03-16T09:01:00.637 INFO:tasks.workunit.client.0.smithi007.stdout:2/87: dwrite dc/df/f18 [0,4194304] 0 2024-03-16T09:01:00.638 INFO:tasks.workunit.client.0.smithi007.stdout:6/31: creat d2/fb x:0 0 0 2024-03-16T09:01:00.640 INFO:tasks.workunit.client.0.smithi007.stdout:0/13: dwrite d3/f6 [0,4194304] 0 2024-03-16T09:01:00.641 INFO:tasks.workunit.client.0.smithi007.stdout:0/14: write d3/f6 [4733489,9010] 0 2024-03-16T09:01:00.641 INFO:tasks.workunit.client.0.smithi007.stdout:5/28: rmdir d1 39 2024-03-16T09:01:00.643 INFO:tasks.workunit.client.0.smithi007.stdout:3/35: symlink l9 0 2024-03-16T09:01:00.645 INFO:tasks.workunit.client.0.smithi007.stdout:2/88: symlink dc/df/l1d 0 2024-03-16T09:01:00.646 INFO:tasks.workunit.client.0.smithi007.stdout:6/32: stat d2/c3 0 2024-03-16T09:01:00.646 INFO:tasks.workunit.client.0.smithi007.stdout:8/57: rmdir d0 39 2024-03-16T09:01:00.647 INFO:tasks.workunit.client.0.smithi007.stdout:7/23: mknod d0/c9 0 2024-03-16T09:01:00.647 INFO:tasks.workunit.client.0.smithi007.stdout:7/24: fdatasync d0/f7 0 2024-03-16T09:01:00.648 INFO:tasks.workunit.client.0.smithi007.stdout:0/15: write d3/f6 [983845,14176] 0 2024-03-16T09:01:00.649 INFO:tasks.workunit.client.0.smithi007.stdout:5/29: creat d1/fa x:0 0 0 2024-03-16T09:01:00.649 INFO:tasks.workunit.client.0.smithi007.stdout:5/30: write d1/fa [174323,84839] 0 2024-03-16T09:01:00.650 INFO:tasks.workunit.client.0.smithi007.stdout:6/33: dread - d2/fa zero size 2024-03-16T09:01:00.651 INFO:tasks.workunit.client.0.smithi007.stdout:8/58: creat d0/dc/f12 x:0 0 0 2024-03-16T09:01:00.651 INFO:tasks.workunit.client.0.smithi007.stdout:5/31: dread d1/f2 [0,4194304] 0 2024-03-16T09:01:00.653 INFO:tasks.workunit.client.0.smithi007.stdout:1/15: dwrite f0 [0,4194304] 0 2024-03-16T09:01:00.653 INFO:tasks.workunit.client.0.smithi007.stdout:1/16: truncate f1 1067263 0 2024-03-16T09:01:00.654 INFO:tasks.workunit.client.0.smithi007.stdout:6/34: write d2/f7 [957221,77564] 0 2024-03-16T09:01:00.655 INFO:tasks.workunit.client.0.smithi007.stdout:5/32: chown d1/c4 873448 1 2024-03-16T09:01:00.656 INFO:tasks.workunit.client.0.smithi007.stdout:8/59: write d0/ff [2254148,76417] 0 2024-03-16T09:01:00.657 INFO:tasks.workunit.client.0.smithi007.stdout:7/25: rename d0/f4 to d0/fa 0 2024-03-16T09:01:00.657 INFO:tasks.workunit.client.0.smithi007.stdout:7/26: fdatasync d0/f3 0 2024-03-16T09:01:00.657 INFO:tasks.workunit.client.0.smithi007.stdout:7/27: write d0/fa [863135,68502] 0 2024-03-16T09:01:00.658 INFO:tasks.workunit.client.0.smithi007.stdout:2/89: rename dc/le to dc/l1e 0 2024-03-16T09:01:00.660 INFO:tasks.workunit.client.0.smithi007.stdout:1/17: dread f1 [0,4194304] 0 2024-03-16T09:01:00.660 INFO:tasks.workunit.client.0.smithi007.stdout:0/16: dread d3/f6 [0,4194304] 0 2024-03-16T09:01:00.662 INFO:tasks.workunit.client.0.smithi007.stdout:8/60: symlink d0/dc/l13 0 2024-03-16T09:01:00.662 INFO:tasks.workunit.client.0.smithi007.stdout:3/36: dread f7 [0,4194304] 0 2024-03-16T09:01:00.662 INFO:tasks.workunit.client.0.smithi007.stdout:3/37: creat fa x:0 0 0 2024-03-16T09:01:00.662 INFO:tasks.workunit.client.0.smithi007.stdout:3/38: chown l4 10689 1 2024-03-16T09:01:00.662 INFO:tasks.workunit.client.0.smithi007.stdout:3/39: readlink l4 0 2024-03-16T09:01:00.663 INFO:tasks.workunit.client.0.smithi007.stdout:5/33: mknod d1/d7/d8/cb 0 2024-03-16T09:01:00.671 INFO:tasks.workunit.client.0.smithi007.stdout:5/34: creat d1/fc x:0 0 0 2024-03-16T09:01:00.671 INFO:tasks.workunit.client.0.smithi007.stdout:7/28: link d0/f3 d0/fb 0 2024-03-16T09:01:00.671 INFO:tasks.workunit.client.0.smithi007.stdout:7/29: chown d0/l2 3274553 1 2024-03-16T09:01:00.671 INFO:tasks.workunit.client.0.smithi007.stdout:5/35: write d1/fa [565340,108317] 0 2024-03-16T09:01:00.673 INFO:tasks.workunit.client.0.smithi007.stdout:0/17: link d3/f6 d3/f8 0 2024-03-16T09:01:00.673 INFO:tasks.workunit.client.0.smithi007.stdout:0/18: chown d3/f8 3 1 2024-03-16T09:01:00.673 INFO:tasks.workunit.client.0.smithi007.stdout:3/40: write f7 [1533157,54432] 0 2024-03-16T09:01:00.678 INFO:tasks.workunit.client.0.smithi007.stdout:3/41: unlink f3 0 2024-03-16T09:01:00.679 INFO:tasks.workunit.client.0.smithi007.stdout:5/36: mknod d1/cd 0 2024-03-16T09:01:00.679 INFO:tasks.workunit.client.0.smithi007.stdout:5/37: truncate d1/f2 1388053 0 2024-03-16T09:01:00.681 INFO:tasks.workunit.client.0.smithi007.stdout:3/42: symlink lb 0 2024-03-16T09:01:00.681 INFO:tasks.workunit.client.0.smithi007.stdout:3/43: chown lb 4193 1 2024-03-16T09:01:00.681 INFO:tasks.workunit.client.0.smithi007.stdout:3/44: write f7 [1230227,4014] 0 2024-03-16T09:01:00.681 INFO:tasks.workunit.client.0.smithi007.stdout:3/45: write fa [186465,130348] 0 2024-03-16T09:01:00.681 INFO:tasks.workunit.client.0.smithi007.stdout:3/46: readlink l9 0 2024-03-16T09:01:00.687 INFO:tasks.workunit.client.0.smithi007.stdout:6/35: dwrite d2/fb [0,4194304] 0 2024-03-16T09:01:00.687 INFO:tasks.workunit.client.0.smithi007.stdout:6/36: write d2/fa [829256,84932] 0 2024-03-16T09:01:00.692 INFO:tasks.workunit.client.0.smithi007.stdout:1/18: dwrite f1 [0,4194304] 0 2024-03-16T09:01:00.693 INFO:tasks.workunit.client.0.smithi007.stdout:6/37: symlink d2/lc 0 2024-03-16T09:01:00.695 INFO:tasks.workunit.client.0.smithi007.stdout:1/19: mkdir d4 0 2024-03-16T09:01:00.699 INFO:tasks.workunit.client.0.smithi007.stdout:0/19: dwrite d3/f8 [4194304,4194304] 0 2024-03-16T09:01:00.699 INFO:tasks.workunit.client.0.smithi007.stdout:0/20: fdatasync d3/f8 0 2024-03-16T09:01:00.699 INFO:tasks.workunit.client.0.smithi007.stdout:0/21: dread - d3/f4 zero size 2024-03-16T09:01:00.702 INFO:tasks.workunit.client.0.smithi007.stdout:2/90: dwrite dc/f10 [0,4194304] 0 2024-03-16T09:01:00.703 INFO:tasks.workunit.client.0.smithi007.stdout:0/22: mkdir d3/d9 0 2024-03-16T09:01:00.703 INFO:tasks.workunit.client.0.smithi007.stdout:0/23: chown d3/c5 1932 1 2024-03-16T09:01:00.703 INFO:tasks.workunit.client.0.smithi007.stdout:0/24: readlink d3/l7 0 2024-03-16T09:01:00.703 INFO:tasks.workunit.client.0.smithi007.stdout:0/25: chown d3/c5 27 1 2024-03-16T09:01:00.703 INFO:tasks.workunit.client.0.smithi007.stdout:7/30: dwrite d0/f7 [0,4194304] 0 2024-03-16T09:01:00.705 INFO:tasks.workunit.client.0.smithi007.stdout:6/38: dread d2/fa [0,4194304] 0 2024-03-16T09:01:00.707 INFO:tasks.workunit.client.0.smithi007.stdout:8/61: dwrite d0/d2/f10 [0,4194304] 0 2024-03-16T09:01:00.707 INFO:tasks.workunit.client.0.smithi007.stdout:8/62: read - d0/fe zero size 2024-03-16T09:01:00.711 INFO:tasks.workunit.client.0.smithi007.stdout:5/38: dread d1/fa [0,4194304] 0 2024-03-16T09:01:00.721 INFO:tasks.workunit.client.0.smithi007.stdout:1/20: dwrite f0 [0,4194304] 0 2024-03-16T09:01:00.722 INFO:tasks.workunit.client.0.smithi007.stdout:6/39: write d2/fb [1824723,97508] 0 2024-03-16T09:01:00.722 INFO:tasks.workunit.client.0.smithi007.stdout:2/91: mknod dc/c1f 0 2024-03-16T09:01:00.723 INFO:tasks.workunit.client.0.smithi007.stdout:2/92: write dc/f17 [731964,3398] 0 2024-03-16T09:01:00.724 INFO:tasks.workunit.client.0.smithi007.stdout:5/39: mkdir d1/d7/d8/de 0 2024-03-16T09:01:00.724 INFO:tasks.workunit.client.0.smithi007.stdout:0/26: dwrite d3/f6 [4194304,4194304] 0 2024-03-16T09:01:00.725 INFO:tasks.workunit.client.0.smithi007.stdout:7/31: mkdir d0/dc 0 2024-03-16T09:01:00.726 INFO:tasks.workunit.client.0.smithi007.stdout:1/21: rmdir d4 0 2024-03-16T09:01:00.731 INFO:tasks.workunit.client.0.smithi007.stdout:3/47: dwrite fa [0,4194304] 0 2024-03-16T09:01:00.731 INFO:tasks.workunit.client.0.smithi007.stdout:3/48: write f5 [1504170,25993] 0 2024-03-16T09:01:00.731 INFO:tasks.workunit.client.0.smithi007.stdout:8/63: dwrite d0/dc/f11 [0,4194304] 0 2024-03-16T09:01:00.737 INFO:tasks.workunit.client.0.smithi007.stdout:2/93: write dc/df/f18 [1310827,104006] 0 2024-03-16T09:01:00.737 INFO:tasks.workunit.client.0.smithi007.stdout:2/94: chown dc/c1f 0 1 2024-03-16T09:01:00.757 INFO:tasks.workunit.client.0.smithi007.stdout:6/40: write d2/fb [4910967,24180] 0 2024-03-16T09:01:00.757 INFO:tasks.workunit.client.0.smithi007.stdout:6/41: stat d2 0 2024-03-16T09:01:00.758 INFO:tasks.workunit.client.0.smithi007.stdout:5/40: mkdir d1/d7/df 0 2024-03-16T09:01:00.758 INFO:tasks.workunit.client.0.smithi007.stdout:5/41: read d1/f2 [1025866,14469] 0 2024-03-16T09:01:00.758 INFO:tasks.workunit.client.0.smithi007.stdout:5/42: stat d1/fa 0 2024-03-16T09:01:00.759 INFO:tasks.workunit.client.0.smithi007.stdout:6/42: read d2/fa [21569,92031] 0 2024-03-16T09:01:00.759 INFO:tasks.workunit.client.0.smithi007.stdout:0/27: mkdir d3/d9/da 0 2024-03-16T09:01:00.763 INFO:tasks.workunit.client.0.smithi007.stdout:7/32: rename d0/f7 to d0/dc/fd 0 2024-03-16T09:01:00.763 INFO:tasks.workunit.client.0.smithi007.stdout:7/33: write d0/f6 [1228677,37267] 0 2024-03-16T09:01:00.763 INFO:tasks.workunit.client.0.smithi007.stdout:7/34: write d0/fa [2248558,13157] 0 2024-03-16T09:01:00.764 INFO:tasks.workunit.client.0.smithi007.stdout:3/49: mknod cc 0 2024-03-16T09:01:00.764 INFO:tasks.workunit.client.0.smithi007.stdout:3/50: chown f8 325863734 1 2024-03-16T09:01:00.770 INFO:tasks.workunit.client.0.smithi007.stdout:8/64: link d0/l9 d0/l14 0 2024-03-16T09:01:00.776 INFO:tasks.workunit.client.0.smithi007.stdout:3/51: write f8 [1772660,29522] 0 2024-03-16T09:01:00.777 INFO:tasks.workunit.client.0.smithi007.stdout:6/43: dwrite d2/f7 [0,4194304] 0 2024-03-16T09:01:00.777 INFO:tasks.workunit.client.0.smithi007.stdout:6/44: chown d2/f7 4917692 1 2024-03-16T09:01:00.789 INFO:tasks.workunit.client.0.smithi007.stdout:8/65: dwrite d0/dc/f12 [0,4194304] 0 2024-03-16T09:01:00.789 INFO:tasks.workunit.client.0.smithi007.stdout:8/66: chown d0/dc/f11 3589 1 2024-03-16T09:01:00.802 INFO:tasks.workunit.client.0.smithi007.stdout:5/43: creat d1/f10 x:0 0 0 2024-03-16T09:01:00.804 INFO:tasks.workunit.client.0.smithi007.stdout:0/28: mkdir d3/d9/db 0 2024-03-16T09:01:00.806 INFO:tasks.workunit.client.0.smithi007.stdout:7/35: creat d0/dc/fe x:0 0 0 2024-03-16T09:01:00.809 INFO:tasks.workunit.client.0.smithi007.stdout:1/22: rename f1 to f5 0 2024-03-16T09:01:00.810 INFO:tasks.workunit.client.0.smithi007.stdout:1/23: rmdir - no directory 2024-03-16T09:01:00.812 INFO:tasks.workunit.client.0.smithi007.stdout:2/95: rmdir dc 39 2024-03-16T09:01:00.813 INFO:tasks.workunit.client.0.smithi007.stdout:0/29: dread d3/f6 [4194304,4194304] 0 2024-03-16T09:01:00.814 INFO:tasks.workunit.client.0.smithi007.stdout:3/52: symlink ld 0 2024-03-16T09:01:00.814 INFO:tasks.workunit.client.0.smithi007.stdout:3/53: fsync fa 0 2024-03-16T09:01:00.818 INFO:tasks.workunit.client.0.smithi007.stdout:6/45: chown d2/c6 1449 1 2024-03-16T09:01:00.818 INFO:tasks.workunit.client.0.smithi007.stdout:6/46: fsync d2/fa 0 2024-03-16T09:01:00.818 INFO:tasks.workunit.client.0.smithi007.stdout:6/47: chown d2/lc 8 1 2024-03-16T09:01:00.819 INFO:tasks.workunit.client.0.smithi007.stdout:8/67: stat d0/l14 0 2024-03-16T09:01:00.821 INFO:tasks.workunit.client.0.smithi007.stdout:0/30: dread d3/f6 [4194304,4194304] 0 2024-03-16T09:01:00.821 INFO:tasks.workunit.client.0.smithi007.stdout:0/31: fdatasync d3/f6 0 2024-03-16T09:01:00.821 INFO:tasks.workunit.client.0.smithi007.stdout:7/36: symlink d0/dc/lf 0 2024-03-16T09:01:00.823 INFO:tasks.workunit.client.0.smithi007.stdout:1/24: mknod c6 0 2024-03-16T09:01:00.823 INFO:tasks.workunit.client.0.smithi007.stdout:1/25: write f0 [4955333,19796] 0 2024-03-16T09:01:00.823 INFO:tasks.workunit.client.0.smithi007.stdout:1/26: creat f7 x:0 0 0 2024-03-16T09:01:00.824 INFO:tasks.workunit.client.0.smithi007.stdout:5/44: dwrite d1/fa [0,4194304] 0 2024-03-16T09:01:00.825 INFO:tasks.workunit.client.0.smithi007.stdout:2/96: creat dc/f20 x:0 0 0 2024-03-16T09:01:00.825 INFO:tasks.workunit.client.0.smithi007.stdout:2/97: chown la 70977 1 2024-03-16T09:01:00.825 INFO:tasks.workunit.client.0.smithi007.stdout:2/98: write dc/f20 [931446,99227] 0 2024-03-16T09:01:00.830 INFO:tasks.workunit.client.0.smithi007.stdout:3/54: symlink le 0 2024-03-16T09:01:00.830 INFO:tasks.workunit.client.0.smithi007.stdout:3/55: fdatasync f7 0 2024-03-16T09:01:00.830 INFO:tasks.workunit.client.0.smithi007.stdout:2/99: fdatasync dc/f20 0 2024-03-16T09:01:00.830 INFO:tasks.workunit.client.0.smithi007.stdout:2/100: read dc/df/f18 [3519018,63366] 0 2024-03-16T09:01:00.836 INFO:tasks.workunit.client.0.smithi007.stdout:8/68: unlink d0/d2/f10 0 2024-03-16T09:01:00.836 INFO:tasks.workunit.client.0.smithi007.stdout:0/32: creat d3/d9/db/fc x:0 0 0 2024-03-16T09:01:00.836 INFO:tasks.workunit.client.0.smithi007.stdout:1/27: dread f5 [0,4194304] 0 2024-03-16T09:01:00.836 INFO:tasks.workunit.client.0.smithi007.stdout:7/37: write d0/dc/fd [3864158,35038] 0 2024-03-16T09:01:00.840 INFO:tasks.workunit.client.0.smithi007.stdout:5/45: unlink d1/c4 0 2024-03-16T09:01:00.847 INFO:tasks.workunit.client.0.smithi007.stdout:2/101: symlink dc/df/l21 0 2024-03-16T09:01:00.847 INFO:tasks.workunit.client.0.smithi007.stdout:5/46: dread f0 [0,4194304] 0 2024-03-16T09:01:00.849 INFO:tasks.workunit.client.0.smithi007.stdout:6/48: dwrite d2/fa [0,4194304] 0 2024-03-16T09:01:00.850 INFO:tasks.workunit.client.0.smithi007.stdout:2/102: creat dc/df/f22 x:0 0 0 2024-03-16T09:01:00.850 INFO:tasks.workunit.client.0.smithi007.stdout:2/103: chown dc/l1e 2 1 2024-03-16T09:01:00.851 INFO:tasks.workunit.client.0.smithi007.stdout:7/38: mknod d0/c10 0 2024-03-16T09:01:00.851 INFO:tasks.workunit.client.0.smithi007.stdout:7/39: chown d0/f1 6 1 2024-03-16T09:01:00.857 INFO:tasks.workunit.client.0.smithi007.stdout:5/47: mknod d1/d7/d8/de/c11 0 2024-03-16T09:01:00.857 INFO:tasks.workunit.client.0.smithi007.stdout:8/69: dread d0/d2/fa [0,4194304] 0 2024-03-16T09:01:00.857 INFO:tasks.workunit.client.0.smithi007.stdout:8/70: readlink d0/l14 0 2024-03-16T09:01:00.857 INFO:tasks.workunit.client.0.smithi007.stdout:8/71: fdatasync d0/d2/fa 0 2024-03-16T09:01:00.857 INFO:tasks.workunit.client.0.smithi007.stdout:0/33: dwrite d3/f4 [0,4194304] 0 2024-03-16T09:01:00.859 INFO:tasks.workunit.client.0.smithi007.stdout:2/104: creat dc/f23 x:0 0 0 2024-03-16T09:01:00.860 INFO:tasks.workunit.client.0.smithi007.stdout:7/40: creat d0/f11 x:0 0 0 2024-03-16T09:01:00.862 INFO:tasks.workunit.client.0.smithi007.stdout:5/48: creat d1/d7/f12 x:0 0 0 2024-03-16T09:01:00.864 INFO:tasks.workunit.client.0.smithi007.stdout:6/49: write d2/fa [1009551,64926] 0 2024-03-16T09:01:00.864 INFO:tasks.workunit.client.0.smithi007.stdout:6/50: write d2/fa [3260831,29029] 0 2024-03-16T09:01:00.868 INFO:tasks.workunit.client.0.smithi007.stdout:2/105: dread f2 [0,4194304] 0 2024-03-16T09:01:00.868 INFO:tasks.workunit.client.0.smithi007.stdout:8/72: rename d0/d2/c6 to d0/c15 0 2024-03-16T09:01:00.873 INFO:tasks.workunit.client.0.smithi007.stdout:3/56: truncate f5 3254821 0 2024-03-16T09:01:00.873 INFO:tasks.workunit.client.0.smithi007.stdout:3/57: chown l4 14 1 2024-03-16T09:01:00.873 INFO:tasks.workunit.client.0.smithi007.stdout:3/58: write f7 [4780346,100520] 0 2024-03-16T09:01:00.876 INFO:tasks.workunit.client.0.smithi007.stdout:6/51: dread d2/fa [0,4194304] 0 2024-03-16T09:01:00.876 INFO:tasks.workunit.client.0.smithi007.stdout:1/28: truncate f0 3945232 0 2024-03-16T09:01:00.879 INFO:tasks.workunit.client.0.smithi007.stdout:0/34: dwrite d3/f8 [0,4194304] 0 2024-03-16T09:01:00.888 INFO:tasks.workunit.client.0.smithi007.stdout:5/49: mknod d1/d7/d8/c13 0 2024-03-16T09:01:00.888 INFO:tasks.workunit.client.0.smithi007.stdout:5/50: write d1/d7/f12 [327218,87140] 0 2024-03-16T09:01:00.892 INFO:tasks.workunit.client.0.smithi007.stdout:2/106: mkdir dc/df/d24 0 2024-03-16T09:01:00.892 INFO:tasks.workunit.client.0.smithi007.stdout:2/107: read f0 [367490,24999] 0 2024-03-16T09:01:00.892 INFO:tasks.workunit.client.0.smithi007.stdout:2/108: fdatasync dc/f23 0 2024-03-16T09:01:00.893 INFO:tasks.workunit.client.0.smithi007.stdout:1/29: dwrite f5 [0,4194304] 0 2024-03-16T09:01:00.893 INFO:tasks.workunit.client.0.smithi007.stdout:8/73: unlink d0/l4 0 2024-03-16T09:01:00.897 INFO:tasks.workunit.client.0.smithi007.stdout:3/59: link l2 lf 0 2024-03-16T09:01:00.897 INFO:tasks.workunit.client.0.smithi007.stdout:3/60: creat f10 x:0 0 0 2024-03-16T09:01:00.897 INFO:tasks.workunit.client.0.smithi007.stdout:3/61: chown cc 1577027416 1 2024-03-16T09:01:00.897 INFO:tasks.workunit.client.0.smithi007.stdout:3/62: chown fa 66 1 2024-03-16T09:01:00.897 INFO:tasks.workunit.client.0.smithi007.stdout:6/52: creat d2/fd x:0 0 0 2024-03-16T09:01:00.899 INFO:tasks.workunit.client.0.smithi007.stdout:0/35: mknod d3/d9/cd 0 2024-03-16T09:01:00.901 INFO:tasks.workunit.client.0.smithi007.stdout:7/41: truncate d0/dc/fd 716268 0 2024-03-16T09:01:00.901 INFO:tasks.workunit.client.0.smithi007.stdout:7/42: fdatasync d0/fb 0 2024-03-16T09:01:00.903 INFO:tasks.workunit.client.0.smithi007.stdout:2/109: read dc/f15 [6272266,9284] 0 2024-03-16T09:01:00.906 INFO:tasks.workunit.client.0.smithi007.stdout:1/30: mknod c8 0 2024-03-16T09:01:00.910 INFO:tasks.workunit.client.0.smithi007.stdout:9/88: sync 2024-03-16T09:01:00.910 INFO:tasks.workunit.client.0.smithi007.stdout:9/89: chown ff 180148475 1 2024-03-16T09:01:00.910 INFO:tasks.workunit.client.0.smithi007.stdout:9/90: creat f1b x:0 0 0 2024-03-16T09:01:00.910 INFO:tasks.workunit.client.0.smithi007.stdout:9/91: chown f17 210221 1 2024-03-16T09:01:00.910 INFO:tasks.workunit.client.0.smithi007.stdout:9/92: creat f1c x:0 0 0 2024-03-16T09:01:00.910 INFO:tasks.workunit.client.0.smithi007.stdout:9/93: readlink l18 0 2024-03-16T09:01:00.911 INFO:tasks.workunit.client.0.smithi007.stdout:9/94: chown c9 26 1 2024-03-16T09:01:00.911 INFO:tasks.workunit.client.0.smithi007.stdout:9/95: creat f1d x:0 0 0 2024-03-16T09:01:00.911 INFO:tasks.workunit.client.0.smithi007.stdout:4/53: sync 2024-03-16T09:01:00.911 INFO:tasks.workunit.client.0.smithi007.stdout:4/54: stat d4/ff 0 2024-03-16T09:01:00.911 INFO:tasks.workunit.client.0.smithi007.stdout:4/55: chown d4/f6 88 1 2024-03-16T09:01:00.911 INFO:tasks.workunit.client.0.smithi007.stdout:4/56: write d4/f6 [1289948,32854] 0 2024-03-16T09:01:00.911 INFO:tasks.workunit.client.0.smithi007.stdout:8/74: rename d0/dc/f11 to d0/f16 0 2024-03-16T09:01:00.911 INFO:tasks.workunit.client.0.smithi007.stdout:8/75: readlink d0/l14 0 2024-03-16T09:01:00.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/63: symlink l11 0 2024-03-16T09:01:00.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/64: creat f12 x:0 0 0 2024-03-16T09:01:00.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/65: write f10 [9229,73433] 0 2024-03-16T09:01:00.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/66: dread - f12 zero size 2024-03-16T09:01:00.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/67: chown fa 720817 1 2024-03-16T09:01:00.916 INFO:tasks.workunit.client.0.smithi007.stdout:6/53: creat d2/fe x:0 0 0 2024-03-16T09:01:00.916 INFO:tasks.workunit.client.0.smithi007.stdout:6/54: write d2/fd [640687,14909] 0 2024-03-16T09:01:00.916 INFO:tasks.workunit.client.0.smithi007.stdout:6/55: chown d2/c3 1114050940 1 2024-03-16T09:01:00.918 INFO:tasks.workunit.client.0.smithi007.stdout:7/43: symlink d0/dc/l12 0 2024-03-16T09:01:00.919 INFO:tasks.workunit.client.0.smithi007.stdout:5/51: rmdir d1/d7 39 2024-03-16T09:01:00.919 INFO:tasks.workunit.client.0.smithi007.stdout:5/52: stat f0 0 2024-03-16T09:01:00.921 INFO:tasks.workunit.client.0.smithi007.stdout:5/53: write d1/f2 [134093,4995] 0 2024-03-16T09:01:00.921 INFO:tasks.workunit.client.0.smithi007.stdout:7/44: dread d0/f3 [0,4194304] 0 2024-03-16T09:01:00.921 INFO:tasks.workunit.client.0.smithi007.stdout:2/110: truncate dc/fd 583516 0 2024-03-16T09:01:00.921 INFO:tasks.workunit.client.0.smithi007.stdout:7/45: read d0/f6 [975365,79024] 0 2024-03-16T09:01:00.922 INFO:tasks.workunit.client.0.smithi007.stdout:3/68: fdatasync f7 0 2024-03-16T09:01:00.923 INFO:tasks.workunit.client.0.smithi007.stdout:7/46: read d0/f6 [103183,91416] 0 2024-03-16T09:01:00.924 INFO:tasks.workunit.client.0.smithi007.stdout:2/111: dread f7 [0,4194304] 0 2024-03-16T09:01:00.925 INFO:tasks.workunit.client.0.smithi007.stdout:2/112: chown dc/df/c1c 1094585 1 2024-03-16T09:01:00.925 INFO:tasks.workunit.client.0.smithi007.stdout:4/57: dread d4/f5 [0,4194304] 0 2024-03-16T09:01:00.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/36: dwrite d3/f6 [4194304,4194304] 0 2024-03-16T09:01:00.928 INFO:tasks.workunit.client.0.smithi007.stdout:8/76: creat d0/d2/f17 x:0 0 0 2024-03-16T09:01:00.928 INFO:tasks.workunit.client.0.smithi007.stdout:1/31: dwrite f7 [0,4194304] 0 2024-03-16T09:01:00.929 INFO:tasks.workunit.client.0.smithi007.stdout:1/32: readlink - no filename 2024-03-16T09:01:00.930 INFO:tasks.workunit.client.0.smithi007.stdout:3/69: dread f7 [4194304,4194304] 0 2024-03-16T09:01:00.934 INFO:tasks.workunit.client.0.smithi007.stdout:5/54: dread d1/fa [0,4194304] 0 2024-03-16T09:01:00.938 INFO:tasks.workunit.client.0.smithi007.stdout:7/47: unlink d0/f1 0 2024-03-16T09:01:00.938 INFO:tasks.workunit.client.0.smithi007.stdout:8/77: dread d0/f16 [0,4194304] 0 2024-03-16T09:01:00.938 INFO:tasks.workunit.client.0.smithi007.stdout:8/78: readlink d0/dc/l13 0 2024-03-16T09:01:00.939 INFO:tasks.workunit.client.0.smithi007.stdout:2/113: unlink dc/df/f22 0 2024-03-16T09:01:00.941 INFO:tasks.workunit.client.0.smithi007.stdout:4/58: dread d4/f5 [0,4194304] 0 2024-03-16T09:01:00.941 INFO:tasks.workunit.client.0.smithi007.stdout:4/59: read d4/d7/fc [7478,30586] 0 2024-03-16T09:01:00.943 INFO:tasks.workunit.client.0.smithi007.stdout:1/33: mkdir d9 0 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:3/70: link fa f13 0 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/96: fsync f1d 0 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/97: rmdir - no directory 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/98: creat f1e x:0 0 0 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/99: write f1a [413744,88475] 0 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/100: chown f1b 12949 1 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/101: chown f8 13526 1 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/102: truncate f4 261288 0 2024-03-16T09:01:00.945 INFO:tasks.workunit.client.0.smithi007.stdout:9/103: truncate f1e 791427 0 2024-03-16T09:01:00.946 INFO:tasks.workunit.client.0.smithi007.stdout:0/37: dwrite d3/f6 [0,4194304] 0 2024-03-16T09:01:00.946 INFO:tasks.workunit.client.0.smithi007.stdout:6/56: symlink d2/lf 0 2024-03-16T09:01:00.949 INFO:tasks.workunit.client.0.smithi007.stdout:7/48: unlink d0/fb 0 2024-03-16T09:01:00.950 INFO:tasks.workunit.client.0.smithi007.stdout:8/79: stat d0/d2/fa 0 2024-03-16T09:01:00.953 INFO:tasks.workunit.client.0.smithi007.stdout:2/114: truncate dc/df/f18 2218759 0 2024-03-16T09:01:00.955 INFO:tasks.workunit.client.0.smithi007.stdout:4/60: mknod d4/c11 0 2024-03-16T09:01:00.955 INFO:tasks.workunit.client.0.smithi007.stdout:4/61: readlink l1 0 2024-03-16T09:01:00.955 INFO:tasks.workunit.client.0.smithi007.stdout:4/62: write f2 [1610692,40797] 0 2024-03-16T09:01:00.955 INFO:tasks.workunit.client.0.smithi007.stdout:4/63: chown d4/c11 288360 1 2024-03-16T09:01:00.957 INFO:tasks.workunit.client.0.smithi007.stdout:1/34: write f7 [569230,94905] 0 2024-03-16T09:01:00.957 INFO:tasks.workunit.client.0.smithi007.stdout:1/35: readlink - no filename 2024-03-16T09:01:00.959 INFO:tasks.workunit.client.0.smithi007.stdout:9/104: link f8 f1f 0 2024-03-16T09:01:00.960 INFO:tasks.workunit.client.0.smithi007.stdout:0/38: mkdir d3/d9/db/de 0 2024-03-16T09:01:00.960 INFO:tasks.workunit.client.0.smithi007.stdout:3/71: write fa [3942800,20431] 0 2024-03-16T09:01:00.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/55: mkdir d1/d14 0 2024-03-16T09:01:00.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/56: stat d1/f2 0 2024-03-16T09:01:00.968 INFO:tasks.workunit.client.0.smithi007.stdout:5/57: dread d1/f2 [0,4194304] 0 2024-03-16T09:01:00.975 INFO:tasks.workunit.client.0.smithi007.stdout:6/57: getdents d2 0 2024-03-16T09:01:00.975 INFO:tasks.workunit.client.0.smithi007.stdout:6/58: truncate d2/f7 4842026 0 2024-03-16T09:01:00.975 INFO:tasks.workunit.client.0.smithi007.stdout:6/59: write d2/fb [5566138,111498] 0 2024-03-16T09:01:00.975 INFO:tasks.workunit.client.0.smithi007.stdout:6/60: getdents d2 0 2024-03-16T09:01:00.977 INFO:tasks.workunit.client.0.smithi007.stdout:7/49: dread d0/f3 [0,4194304] 0 2024-03-16T09:01:00.977 INFO:tasks.workunit.client.0.smithi007.stdout:0/39: write d3/f8 [6839376,45912] 0 2024-03-16T09:01:00.980 INFO:tasks.workunit.client.0.smithi007.stdout:8/80: mkdir d0/d2/d18 0 2024-03-16T09:01:00.981 INFO:tasks.workunit.client.0.smithi007.stdout:2/115: rename dc/c1f to dc/c25 0 2024-03-16T09:01:00.983 INFO:tasks.workunit.client.0.smithi007.stdout:6/61: mkdir d2/d10 0 2024-03-16T09:01:00.984 INFO:tasks.workunit.client.0.smithi007.stdout:3/72: getdents . 0 2024-03-16T09:01:00.984 INFO:tasks.workunit.client.0.smithi007.stdout:3/73: chown f10 22962 1 2024-03-16T09:01:00.984 INFO:tasks.workunit.client.0.smithi007.stdout:5/58: dread d1/d7/f12 [0,4194304] 0 2024-03-16T09:01:00.990 INFO:tasks.workunit.client.0.smithi007.stdout:4/64: dwrite d4/f6 [0,4194304] 0 2024-03-16T09:01:00.990 INFO:tasks.workunit.client.0.smithi007.stdout:9/105: dwrite f17 [0,4194304] 0 2024-03-16T09:01:01.001 INFO:tasks.workunit.client.0.smithi007.stdout:0/40: rename d3/c5 to d3/d9/da/cf 0 2024-03-16T09:01:01.002 INFO:tasks.workunit.client.0.smithi007.stdout:0/41: stat d3/f8 0 2024-03-16T09:01:01.002 INFO:tasks.workunit.client.0.smithi007.stdout:0/42: write d3/f6 [6148534,65748] 0 2024-03-16T09:01:01.002 INFO:tasks.workunit.client.0.smithi007.stdout:0/43: read d3/f6 [6368094,32539] 0 2024-03-16T09:01:01.005 INFO:tasks.workunit.client.0.smithi007.stdout:8/81: creat d0/d2/d18/f19 x:0 0 0 2024-03-16T09:01:01.014 INFO:tasks.workunit.client.0.smithi007.stdout:5/59: truncate d1/fa 3129401 0 2024-03-16T09:01:01.015 INFO:tasks.workunit.client.0.smithi007.stdout:5/60: write f0 [3159809,99756] 0 2024-03-16T09:01:01.015 INFO:tasks.workunit.client.0.smithi007.stdout:5/61: dread - d1/fc zero size 2024-03-16T09:01:01.018 INFO:tasks.workunit.client.0.smithi007.stdout:7/50: link d0/c10 d0/c13 0 2024-03-16T09:01:01.026 INFO:tasks.workunit.client.0.smithi007.stdout:4/65: unlink f3 0 2024-03-16T09:01:01.027 INFO:tasks.workunit.client.0.smithi007.stdout:4/66: creat d4/f12 x:0 0 0 2024-03-16T09:01:01.027 INFO:tasks.workunit.client.0.smithi007.stdout:4/67: readlink l1 0 2024-03-16T09:01:01.032 INFO:tasks.workunit.client.0.smithi007.stdout:9/106: symlink l20 0 2024-03-16T09:01:01.033 INFO:tasks.workunit.client.0.smithi007.stdout:6/62: dwrite d2/fa [0,4194304] 0 2024-03-16T09:01:01.038 INFO:tasks.workunit.client.0.smithi007.stdout:5/62: write d1/fa [126875,116460] 0 2024-03-16T09:01:01.038 INFO:tasks.workunit.client.0.smithi007.stdout:5/63: chown d1/d7/d8/cb 27511 1 2024-03-16T09:01:01.039 INFO:tasks.workunit.client.0.smithi007.stdout:0/44: mkdir d3/d9/da/d10 0 2024-03-16T09:01:01.041 INFO:tasks.workunit.client.0.smithi007.stdout:8/82: write d0/d2/f17 [432701,40808] 0 2024-03-16T09:01:01.041 INFO:tasks.workunit.client.0.smithi007.stdout:8/83: chown d0/cb 79725045 1 2024-03-16T09:01:01.043 INFO:tasks.workunit.client.0.smithi007.stdout:2/116: symlink dc/l26 0 2024-03-16T09:01:01.046 INFO:tasks.workunit.client.0.smithi007.stdout:4/68: dwrite d4/f12 [0,4194304] 0 2024-03-16T09:01:01.048 INFO:tasks.workunit.client.0.smithi007.stdout:2/117: dread f2 [0,4194304] 0 2024-03-16T09:01:01.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/118: dread - dc/f11 zero size 2024-03-16T09:01:01.054 INFO:tasks.workunit.client.0.smithi007.stdout:9/107: link l10 l21 0 2024-03-16T09:01:01.060 INFO:tasks.workunit.client.0.smithi007.stdout:6/63: dread d2/fd [0,4194304] 0 2024-03-16T09:01:01.060 INFO:tasks.workunit.client.0.smithi007.stdout:6/64: fsync d2/fa 0 2024-03-16T09:01:01.062 INFO:tasks.workunit.client.0.smithi007.stdout:5/64: link d1/f10 d1/d14/f15 0 2024-03-16T09:01:01.063 INFO:tasks.workunit.client.0.smithi007.stdout:5/65: dread d1/d7/f12 [0,4194304] 0 2024-03-16T09:01:01.065 INFO:tasks.workunit.client.0.smithi007.stdout:0/45: creat d3/d9/f11 x:0 0 0 2024-03-16T09:01:01.066 INFO:tasks.workunit.client.0.smithi007.stdout:3/74: truncate fa 72000 0 2024-03-16T09:01:01.066 INFO:tasks.workunit.client.0.smithi007.stdout:3/75: chown l9 702 1 2024-03-16T09:01:01.066 INFO:tasks.workunit.client.0.smithi007.stdout:3/76: chown f7 4 1 2024-03-16T09:01:01.067 INFO:tasks.workunit.client.0.smithi007.stdout:8/84: chown d0/d2/fa 14393554 1 2024-03-16T09:01:01.067 INFO:tasks.workunit.client.0.smithi007.stdout:8/85: fdatasync d0/d2/fa 0 2024-03-16T09:01:01.070 INFO:tasks.workunit.client.0.smithi007.stdout:4/69: mknod d4/c13 0 2024-03-16T09:01:01.070 INFO:tasks.workunit.client.0.smithi007.stdout:9/108: dwrite f1e [0,4194304] 0 2024-03-16T09:01:01.070 INFO:tasks.workunit.client.0.smithi007.stdout:9/109: fsync f4 0 2024-03-16T09:01:01.072 INFO:tasks.workunit.client.0.smithi007.stdout:8/86: dread d0/f16 [0,4194304] 0 2024-03-16T09:01:01.072 INFO:tasks.workunit.client.0.smithi007.stdout:2/119: truncate f1 1205143 0 2024-03-16T09:01:01.073 INFO:tasks.workunit.client.0.smithi007.stdout:0/46: read d3/f4 [549785,86324] 0 2024-03-16T09:01:01.074 INFO:tasks.workunit.client.0.smithi007.stdout:7/51: symlink d0/l14 0 2024-03-16T09:01:01.075 INFO:tasks.workunit.client.0.smithi007.stdout:7/52: write d0/f3 [1315630,402] 0 2024-03-16T09:01:01.075 INFO:tasks.workunit.client.0.smithi007.stdout:7/53: read d0/fa [238402,19550] 0 2024-03-16T09:01:01.081 INFO:tasks.workunit.client.0.smithi007.stdout:6/65: getdents d2 0 2024-03-16T09:01:01.087 INFO:tasks.workunit.client.0.smithi007.stdout:5/66: rmdir d1/d7/d8/de 39 2024-03-16T09:01:01.087 INFO:tasks.workunit.client.0.smithi007.stdout:5/67: write d1/fc [160032,31728] 0 2024-03-16T09:01:01.091 INFO:tasks.workunit.client.0.smithi007.stdout:3/77: creat f14 x:0 0 0 2024-03-16T09:01:01.091 INFO:tasks.workunit.client.0.smithi007.stdout:3/78: write fa [212176,78426] 0 2024-03-16T09:01:01.092 INFO:tasks.workunit.client.0.smithi007.stdout:9/110: dwrite f7 [0,4194304] 0 2024-03-16T09:01:01.092 INFO:tasks.workunit.client.0.smithi007.stdout:9/111: creat f22 x:0 0 0 2024-03-16T09:01:01.093 INFO:tasks.workunit.client.0.smithi007.stdout:9/112: dread - f16 zero size 2024-03-16T09:01:01.093 INFO:tasks.workunit.client.0.smithi007.stdout:9/113: readlink l18 0 2024-03-16T09:01:01.093 INFO:tasks.workunit.client.0.smithi007.stdout:9/114: creat f23 x:0 0 0 2024-03-16T09:01:01.093 INFO:tasks.workunit.client.0.smithi007.stdout:9/115: chown f1e 813500 1 2024-03-16T09:01:01.095 INFO:tasks.workunit.client.0.smithi007.stdout:4/70: mkdir d4/d14 0 2024-03-16T09:01:01.104 INFO:tasks.workunit.client.0.smithi007.stdout:8/87: mknod d0/d2/d18/c1a 0 2024-03-16T09:01:01.107 INFO:tasks.workunit.client.0.smithi007.stdout:0/47: creat d3/d9/db/de/f12 x:0 0 0 2024-03-16T09:01:01.107 INFO:tasks.workunit.client.0.smithi007.stdout:0/48: creat d3/d9/db/de/f13 x:0 0 0 2024-03-16T09:01:01.108 INFO:tasks.workunit.client.0.smithi007.stdout:2/120: dwrite dc/f20 [0,4194304] 0 2024-03-16T09:01:01.108 INFO:tasks.workunit.client.0.smithi007.stdout:2/121: truncate dc/f1a 593837 0 2024-03-16T09:01:01.108 INFO:tasks.workunit.client.0.smithi007.stdout:2/122: readlink dc/l26 0 2024-03-16T09:01:01.111 INFO:tasks.workunit.client.0.smithi007.stdout:7/54: link d0/f11 d0/dc/f15 0 2024-03-16T09:01:01.111 INFO:tasks.workunit.client.0.smithi007.stdout:7/55: readlink d0/dc/lf 0 2024-03-16T09:01:01.115 INFO:tasks.workunit.client.0.smithi007.stdout:6/66: dwrite d2/fe [0,4194304] 0 2024-03-16T09:01:01.117 INFO:tasks.workunit.client.0.smithi007.stdout:3/79: unlink lb 0 2024-03-16T09:01:01.124 INFO:tasks.workunit.client.0.smithi007.stdout:9/116: dwrite f16 [0,4194304] 0 2024-03-16T09:01:01.125 INFO:tasks.workunit.client.0.smithi007.stdout:4/71: mknod d4/d7/c15 0 2024-03-16T09:01:01.125 INFO:tasks.workunit.client.0.smithi007.stdout:5/68: dwrite f0 [4194304,4194304] 0 2024-03-16T09:01:01.126 INFO:tasks.workunit.client.0.smithi007.stdout:5/69: stat d1/cd 0 2024-03-16T09:01:01.128 INFO:tasks.workunit.client.0.smithi007.stdout:8/88: symlink d0/l1b 0 2024-03-16T09:01:01.129 INFO:tasks.workunit.client.0.smithi007.stdout:8/89: dread d0/d2/f17 [0,4194304] 0 2024-03-16T09:01:01.134 INFO:tasks.workunit.client.0.smithi007.stdout:0/49: mknod d3/d9/da/d10/c14 0 2024-03-16T09:01:01.134 INFO:tasks.workunit.client.0.smithi007.stdout:0/50: read d3/f4 [456218,64258] 0 2024-03-16T09:01:01.136 INFO:tasks.workunit.client.0.smithi007.stdout:3/80: dwrite f8 [0,4194304] 0 2024-03-16T09:01:01.138 INFO:tasks.workunit.client.0.smithi007.stdout:3/81: dread fa [0,4194304] 0 2024-03-16T09:01:01.138 INFO:tasks.workunit.client.0.smithi007.stdout:3/82: read - f12 zero size 2024-03-16T09:01:01.145 INFO:tasks.workunit.client.0.smithi007.stdout:3/83: dread f7 [0,4194304] 0 2024-03-16T09:01:01.150 INFO:tasks.workunit.client.0.smithi007.stdout:8/90: dwrite d0/f16 [0,4194304] 0 2024-03-16T09:01:01.150 INFO:tasks.workunit.client.0.smithi007.stdout:3/84: dread f7 [0,4194304] 0 2024-03-16T09:01:01.183 INFO:tasks.workunit.client.0.smithi007.stdout:6/67: mkdir d2/d11 0 2024-03-16T09:01:01.190 INFO:tasks.workunit.client.0.smithi007.stdout:9/117: unlink f4 0 2024-03-16T09:01:01.194 INFO:tasks.workunit.client.0.smithi007.stdout:5/70: mkdir d1/d7/d16 0 2024-03-16T09:01:01.194 INFO:tasks.workunit.client.0.smithi007.stdout:5/71: write d1/f10 [459775,15340] 0 2024-03-16T09:01:01.196 INFO:tasks.workunit.client.0.smithi007.stdout:0/51: symlink d3/d9/da/l15 0 2024-03-16T09:01:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:2/123: mkdir dc/d27 0 2024-03-16T09:01:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:2/124: dread - dc/f11 zero size 2024-03-16T09:01:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:3/85: mkdir d15 0 2024-03-16T09:01:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:7/56: rename d0/c13 to d0/c16 0 2024-03-16T09:01:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:9/118: dwrite f22 [0,4194304] 0 2024-03-16T09:01:01.211 INFO:tasks.workunit.client.0.smithi007.stdout:6/68: mkdir d2/d10/d12 0 2024-03-16T09:01:01.212 INFO:tasks.workunit.client.0.smithi007.stdout:3/86: dread f7 [0,4194304] 0 2024-03-16T09:01:01.212 INFO:tasks.workunit.client.0.smithi007.stdout:3/87: chown f5 44457748 1 2024-03-16T09:01:01.212 INFO:tasks.workunit.client.0.smithi007.stdout:3/88: write f8 [5084785,36124] 0 2024-03-16T09:01:01.214 INFO:tasks.workunit.client.0.smithi007.stdout:4/72: getdents d4 0 2024-03-16T09:01:01.214 INFO:tasks.workunit.client.0.smithi007.stdout:2/125: dread f6 [0,4194304] 0 2024-03-16T09:01:01.214 INFO:tasks.workunit.client.0.smithi007.stdout:2/126: write dc/f23 [129863,90243] 0 2024-03-16T09:01:01.218 INFO:tasks.workunit.client.0.smithi007.stdout:5/72: link d1/d14/f15 d1/d14/f17 0 2024-03-16T09:01:01.218 INFO:tasks.workunit.client.0.smithi007.stdout:5/73: write d1/f2 [1807235,45317] 0 2024-03-16T09:01:01.218 INFO:tasks.workunit.client.0.smithi007.stdout:5/74: fdatasync d1/d7/f12 0 2024-03-16T09:01:01.220 INFO:tasks.workunit.client.0.smithi007.stdout:0/52: rmdir d3/d9/da/d10 39 2024-03-16T09:01:01.220 INFO:tasks.workunit.client.0.smithi007.stdout:0/53: fsync d3/d9/f11 0 2024-03-16T09:01:01.222 INFO:tasks.workunit.client.0.smithi007.stdout:8/91: truncate d0/d2/f17 117235 0 2024-03-16T09:01:01.223 INFO:tasks.workunit.client.0.smithi007.stdout:3/89: dread f5 [0,4194304] 0 2024-03-16T09:01:01.223 INFO:tasks.workunit.client.0.smithi007.stdout:3/90: chown ld 345 1 2024-03-16T09:01:01.223 INFO:tasks.workunit.client.0.smithi007.stdout:3/91: chown d15 25973 1 2024-03-16T09:01:01.223 INFO:tasks.workunit.client.0.smithi007.stdout:3/92: write f8 [4854140,44722] 0 2024-03-16T09:01:01.225 INFO:tasks.workunit.client.0.smithi007.stdout:6/69: read d2/fe [4142711,56096] 0 2024-03-16T09:01:01.225 INFO:tasks.workunit.client.0.smithi007.stdout:7/57: mknod d0/dc/c17 0 2024-03-16T09:01:01.230 INFO:tasks.workunit.client.0.smithi007.stdout:6/70: dread d2/fe [0,4194304] 0 2024-03-16T09:01:01.230 INFO:tasks.workunit.client.0.smithi007.stdout:6/71: stat d2/fb 0 2024-03-16T09:01:01.231 INFO:tasks.workunit.client.0.smithi007.stdout:7/58: dread d0/fa [0,4194304] 0 2024-03-16T09:01:01.231 INFO:tasks.workunit.client.0.smithi007.stdout:4/73: dwrite d4/f5 [0,4194304] 0 2024-03-16T09:01:01.243 INFO:tasks.workunit.client.0.smithi007.stdout:3/93: dwrite f7 [0,4194304] 0 2024-03-16T09:01:01.244 INFO:tasks.workunit.client.0.smithi007.stdout:9/119: mkdir d24 0 2024-03-16T09:01:01.244 INFO:tasks.workunit.client.0.smithi007.stdout:9/120: stat f23 0 2024-03-16T09:01:01.245 INFO:tasks.workunit.client.0.smithi007.stdout:2/127: mkdir dc/df/d24/d28 0 2024-03-16T09:01:01.245 INFO:tasks.workunit.client.0.smithi007.stdout:2/128: write dc/f23 [1170096,24655] 0 2024-03-16T09:01:01.251 INFO:tasks.workunit.client.0.smithi007.stdout:9/121: read ff [4823875,90991] 0 2024-03-16T09:01:01.251 INFO:tasks.workunit.client.0.smithi007.stdout:0/54: creat d3/d9/da/d10/f16 x:0 0 0 2024-03-16T09:01:01.251 INFO:tasks.workunit.client.0.smithi007.stdout:0/55: dread - d3/d9/f11 zero size 2024-03-16T09:01:01.255 INFO:tasks.workunit.client.0.smithi007.stdout:1/36: sync 2024-03-16T09:01:01.255 INFO:tasks.workunit.client.0.smithi007.stdout:1/37: fdatasync f5 0 2024-03-16T09:01:01.256 INFO:tasks.workunit.client.0.smithi007.stdout:6/72: mkdir d2/d13 0 2024-03-16T09:01:01.257 INFO:tasks.workunit.client.0.smithi007.stdout:7/59: creat d0/f18 x:0 0 0 2024-03-16T09:01:01.259 INFO:tasks.workunit.client.0.smithi007.stdout:4/74: mknod d4/c16 0 2024-03-16T09:01:01.259 INFO:tasks.workunit.client.0.smithi007.stdout:7/60: dread d0/f6 [0,4194304] 0 2024-03-16T09:01:01.260 INFO:tasks.workunit.client.0.smithi007.stdout:3/94: creat d15/f16 x:0 0 0 2024-03-16T09:01:01.263 INFO:tasks.workunit.client.0.smithi007.stdout:7/61: dread d0/f3 [0,4194304] 0 2024-03-16T09:01:01.266 INFO:tasks.workunit.client.0.smithi007.stdout:8/92: rename d0/l14 to d0/d2/l1c 0 2024-03-16T09:01:01.266 INFO:tasks.workunit.client.0.smithi007.stdout:8/93: chown d0/f16 9 1 2024-03-16T09:01:01.267 INFO:tasks.workunit.client.0.smithi007.stdout:9/122: write f1f [7417800,32193] 0 2024-03-16T09:01:01.268 INFO:tasks.workunit.client.0.smithi007.stdout:6/73: mknod d2/d10/c14 0 2024-03-16T09:01:01.268 INFO:tasks.workunit.client.0.smithi007.stdout:2/129: dwrite f7 [0,4194304] 0 2024-03-16T09:01:01.269 INFO:tasks.workunit.client.0.smithi007.stdout:4/75: rmdir d4 39 2024-03-16T09:01:01.274 INFO:tasks.workunit.client.0.smithi007.stdout:3/95: mknod d15/c17 0 2024-03-16T09:01:01.274 INFO:tasks.workunit.client.0.smithi007.stdout:0/56: rename c2 to d3/d9/db/de/c17 0 2024-03-16T09:01:01.275 INFO:tasks.workunit.client.0.smithi007.stdout:0/57: readlink d3/l7 0 2024-03-16T09:01:01.275 INFO:tasks.workunit.client.0.smithi007.stdout:0/58: readlink d3/d9/da/l15 0 2024-03-16T09:01:01.275 INFO:tasks.workunit.client.0.smithi007.stdout:0/59: read d3/f4 [3896901,50552] 0 2024-03-16T09:01:01.275 INFO:tasks.workunit.client.0.smithi007.stdout:0/60: truncate d3/d9/db/de/f13 735575 0 2024-03-16T09:01:01.277 INFO:tasks.workunit.client.0.smithi007.stdout:8/94: dread d0/dc/f12 [0,4194304] 0 2024-03-16T09:01:01.280 INFO:tasks.workunit.client.0.smithi007.stdout:9/123: dread f22 [0,4194304] 0 2024-03-16T09:01:01.280 INFO:tasks.workunit.client.0.smithi007.stdout:9/124: stat f17 0 2024-03-16T09:01:01.280 INFO:tasks.workunit.client.0.smithi007.stdout:5/75: truncate d1/f2 1430137 0 2024-03-16T09:01:01.282 INFO:tasks.workunit.client.0.smithi007.stdout:6/74: mkdir d2/d10/d12/d15 0 2024-03-16T09:01:01.286 INFO:tasks.workunit.client.0.smithi007.stdout:4/76: read d4/f12 [1186731,52556] 0 2024-03-16T09:01:01.287 INFO:tasks.workunit.client.0.smithi007.stdout:1/38: rename c8 to d9/ca 0 2024-03-16T09:01:01.287 INFO:tasks.workunit.client.0.smithi007.stdout:1/39: stat f7 0 2024-03-16T09:01:01.292 INFO:tasks.workunit.client.0.smithi007.stdout:6/75: mknod d2/d10/d12/c16 0 2024-03-16T09:01:01.293 INFO:tasks.workunit.client.0.smithi007.stdout:6/76: write d2/fa [4583935,101040] 0 2024-03-16T09:01:01.296 INFO:tasks.workunit.client.0.smithi007.stdout:4/77: link d4/ff d4/f17 0 2024-03-16T09:01:01.298 INFO:tasks.workunit.client.0.smithi007.stdout:8/95: read d0/d2/fa [2020729,25122] 0 2024-03-16T09:01:01.299 INFO:tasks.workunit.client.0.smithi007.stdout:8/96: write d0/d2/d18/f19 [169819,62561] 0 2024-03-16T09:01:01.299 INFO:tasks.workunit.client.0.smithi007.stdout:0/61: getdents d3/d9/db 0 2024-03-16T09:01:01.301 INFO:tasks.workunit.client.0.smithi007.stdout:9/125: write f17 [4155258,94417] 0 2024-03-16T09:01:01.302 INFO:tasks.workunit.client.0.smithi007.stdout:6/77: rename d2/c5 to d2/d10/d12/c17 0 2024-03-16T09:01:01.304 INFO:tasks.workunit.client.0.smithi007.stdout:8/97: rename d0/c15 to d0/c1d 0 2024-03-16T09:01:01.304 INFO:tasks.workunit.client.0.smithi007.stdout:8/98: truncate d0/d2/f17 401035 0 2024-03-16T09:01:01.304 INFO:tasks.workunit.client.0.smithi007.stdout:8/99: readlink d0/d2/l1c 0 2024-03-16T09:01:01.306 INFO:tasks.workunit.client.0.smithi007.stdout:7/62: dwrite d0/dc/fe [0,4194304] 0 2024-03-16T09:01:01.306 INFO:tasks.workunit.client.0.smithi007.stdout:7/63: dread - d0/f18 zero size 2024-03-16T09:01:01.309 INFO:tasks.workunit.client.0.smithi007.stdout:0/62: mkdir d3/d9/da/d18 0 2024-03-16T09:01:01.310 INFO:tasks.workunit.client.0.smithi007.stdout:2/130: dwrite dc/df/f14 [0,4194304] 0 2024-03-16T09:01:01.311 INFO:tasks.workunit.client.0.smithi007.stdout:8/100: dread d0/dc/f12 [0,4194304] 0 2024-03-16T09:01:01.311 INFO:tasks.workunit.client.0.smithi007.stdout:8/101: fdatasync d0/d2/f17 0 2024-03-16T09:01:01.311 INFO:tasks.workunit.client.0.smithi007.stdout:9/126: fsync f17 0 2024-03-16T09:01:01.311 INFO:tasks.workunit.client.0.smithi007.stdout:1/40: dwrite f5 [0,4194304] 0 2024-03-16T09:01:01.311 INFO:tasks.workunit.client.0.smithi007.stdout:3/96: dwrite f10 [0,4194304] 0 2024-03-16T09:01:01.311 INFO:tasks.workunit.client.0.smithi007.stdout:1/41: chown d9/ca 948610 1 2024-03-16T09:01:01.313 INFO:tasks.workunit.client.0.smithi007.stdout:6/78: rename d2/lc to d2/d11/l18 0 2024-03-16T09:01:01.321 INFO:tasks.workunit.client.0.smithi007.stdout:5/76: dread d1/f2 [0,4194304] 0 2024-03-16T09:01:01.321 INFO:tasks.workunit.client.0.smithi007.stdout:5/77: chown d1/fc 741423336 1 2024-03-16T09:01:01.321 INFO:tasks.workunit.client.0.smithi007.stdout:4/78: dwrite f2 [0,4194304] 0 2024-03-16T09:01:01.322 INFO:tasks.workunit.client.0.smithi007.stdout:4/79: chown d4/d7/l8 17 1 2024-03-16T09:01:01.324 INFO:tasks.workunit.client.0.smithi007.stdout:6/79: dread d2/f7 [0,4194304] 0 2024-03-16T09:01:01.326 INFO:tasks.workunit.client.0.smithi007.stdout:0/63: creat d3/d9/da/d18/f19 x:0 0 0 2024-03-16T09:01:01.337 INFO:tasks.workunit.client.0.smithi007.stdout:1/42: symlink d9/lb 0 2024-03-16T09:01:01.341 INFO:tasks.workunit.client.0.smithi007.stdout:5/78: rmdir d1/d7/df 0 2024-03-16T09:01:01.341 INFO:tasks.workunit.client.0.smithi007.stdout:5/79: chown d1/fc 5829741 1 2024-03-16T09:01:01.341 INFO:tasks.workunit.client.0.smithi007.stdout:5/80: write f0 [99189,79627] 0 2024-03-16T09:01:01.343 INFO:tasks.workunit.client.0.smithi007.stdout:4/80: creat d4/f18 x:0 0 0 2024-03-16T09:01:01.343 INFO:tasks.workunit.client.0.smithi007.stdout:7/64: mkdir d0/d19 0 2024-03-16T09:01:01.343 INFO:tasks.workunit.client.0.smithi007.stdout:7/65: creat d0/dc/f1a x:0 0 0 2024-03-16T09:01:01.344 INFO:tasks.workunit.client.0.smithi007.stdout:6/80: mkdir d2/d10/d19 0 2024-03-16T09:01:01.345 INFO:tasks.workunit.client.0.smithi007.stdout:0/64: truncate d3/f8 7093620 0 2024-03-16T09:01:01.345 INFO:tasks.workunit.client.0.smithi007.stdout:0/65: truncate d3/d9/da/d10/f16 1006018 0 2024-03-16T09:01:01.345 INFO:tasks.workunit.client.0.smithi007.stdout:0/66: dread - d3/d9/db/fc zero size 2024-03-16T09:01:01.346 INFO:tasks.workunit.client.0.smithi007.stdout:8/102: symlink d0/d2/l1e 0 2024-03-16T09:01:01.346 INFO:tasks.workunit.client.0.smithi007.stdout:3/97: rmdir d15 39 2024-03-16T09:01:01.347 INFO:tasks.workunit.client.0.smithi007.stdout:5/81: symlink d1/d7/l18 0 2024-03-16T09:01:01.347 INFO:tasks.workunit.client.0.smithi007.stdout:5/82: readlink d1/d7/l18 0 2024-03-16T09:01:01.347 INFO:tasks.workunit.client.0.smithi007.stdout:4/81: truncate d4/f12 1779198 0 2024-03-16T09:01:01.348 INFO:tasks.workunit.client.0.smithi007.stdout:5/83: dread d1/d14/f15 [0,4194304] 0 2024-03-16T09:01:01.348 INFO:tasks.workunit.client.0.smithi007.stdout:6/81: symlink d2/d13/l1a 0 2024-03-16T09:01:01.349 INFO:tasks.workunit.client.0.smithi007.stdout:0/67: mkdir d3/d9/da/d10/d1a 0 2024-03-16T09:01:01.349 INFO:tasks.workunit.client.0.smithi007.stdout:8/103: creat d0/f1f x:0 0 0 2024-03-16T09:01:01.350 INFO:tasks.workunit.client.0.smithi007.stdout:3/98: mkdir d15/d18 0 2024-03-16T09:01:01.351 INFO:tasks.workunit.client.0.smithi007.stdout:4/82: rename f0 to d4/d14/f19 0 2024-03-16T09:01:01.352 INFO:tasks.workunit.client.0.smithi007.stdout:4/83: write d4/d7/fe [214363,34281] 0 2024-03-16T09:01:01.352 INFO:tasks.workunit.client.0.smithi007.stdout:7/66: mkdir d0/d1b 0 2024-03-16T09:01:01.353 INFO:tasks.workunit.client.0.smithi007.stdout:6/82: link d2/c6 d2/d10/d12/d15/c1b 0 2024-03-16T09:01:01.353 INFO:tasks.workunit.client.0.smithi007.stdout:6/83: fdatasync d2/fb 0 2024-03-16T09:01:01.353 INFO:tasks.workunit.client.0.smithi007.stdout:6/84: write d2/fd [1158940,10720] 0 2024-03-16T09:01:01.354 INFO:tasks.workunit.client.0.smithi007.stdout:0/68: creat d3/d9/da/d10/d1a/f1b x:0 0 0 2024-03-16T09:01:01.354 INFO:tasks.workunit.client.0.smithi007.stdout:3/99: creat d15/d18/f19 x:0 0 0 2024-03-16T09:01:01.355 INFO:tasks.workunit.client.0.smithi007.stdout:4/84: truncate d4/f10 2480939 0 2024-03-16T09:01:01.355 INFO:tasks.workunit.client.0.smithi007.stdout:7/67: creat d0/f1c x:0 0 0 2024-03-16T09:01:01.355 INFO:tasks.workunit.client.0.smithi007.stdout:7/68: chown d0/dc/c17 394 1 2024-03-16T09:01:01.356 INFO:tasks.workunit.client.0.smithi007.stdout:0/69: mkdir d3/d9/db/d1c 0 2024-03-16T09:01:01.358 INFO:tasks.workunit.client.0.smithi007.stdout:0/70: unlink d3/d9/da/d10/f16 0 2024-03-16T09:01:01.358 INFO:tasks.workunit.client.0.smithi007.stdout:0/71: fsync d3/d9/db/fc 0 2024-03-16T09:01:01.359 INFO:tasks.workunit.client.0.smithi007.stdout:0/72: mknod d3/d9/da/d10/d1a/c1d 0 2024-03-16T09:01:01.360 INFO:tasks.workunit.client.0.smithi007.stdout:5/84: write f0 [5101279,125266] 0 2024-03-16T09:01:01.360 INFO:tasks.workunit.client.0.smithi007.stdout:5/85: truncate d1/d7/f12 871687 0 2024-03-16T09:01:01.362 INFO:tasks.workunit.client.0.smithi007.stdout:5/86: rmdir d1/d7 39 2024-03-16T09:01:01.363 INFO:tasks.workunit.client.0.smithi007.stdout:0/73: getdents d3 0 2024-03-16T09:01:01.364 INFO:tasks.workunit.client.0.smithi007.stdout:0/74: stat d3/d9/db/fc 0 2024-03-16T09:01:01.364 INFO:tasks.workunit.client.0.smithi007.stdout:0/75: fdatasync d3/d9/db/fc 0 2024-03-16T09:01:01.370 INFO:tasks.workunit.client.0.smithi007.stdout:3/100: dread fa [0,4194304] 0 2024-03-16T09:01:01.384 INFO:tasks.workunit.client.0.smithi007.stdout:4/85: write f2 [5127794,121486] 0 2024-03-16T09:01:01.384 INFO:tasks.workunit.client.0.smithi007.stdout:9/127: dwrite f1a [0,4194304] 0 2024-03-16T09:01:01.387 INFO:tasks.workunit.client.0.smithi007.stdout:9/128: truncate f16 55932 0 2024-03-16T09:01:01.387 INFO:tasks.workunit.client.0.smithi007.stdout:9/129: truncate f1b 72891 0 2024-03-16T09:01:01.388 INFO:tasks.workunit.client.0.smithi007.stdout:9/130: chown c9 132 1 2024-03-16T09:01:01.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/131: creat d24/f25 x:0 0 0 2024-03-16T09:01:01.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/132: creat d24/f26 x:0 0 0 2024-03-16T09:01:01.390 INFO:tasks.workunit.client.0.smithi007.stdout:6/85: dwrite d2/fa [4194304,4194304] 0 2024-03-16T09:01:01.396 INFO:tasks.workunit.client.0.smithi007.stdout:6/86: symlink d2/d10/l1c 0 2024-03-16T09:01:01.396 INFO:tasks.workunit.client.0.smithi007.stdout:6/87: fdatasync d2/fd 0 2024-03-16T09:01:01.399 INFO:tasks.workunit.client.0.smithi007.stdout:9/133: dread f17 [0,4194304] 0 2024-03-16T09:01:01.399 INFO:tasks.workunit.client.0.smithi007.stdout:9/134: creat d24/f27 x:0 0 0 2024-03-16T09:01:01.400 INFO:tasks.workunit.client.0.smithi007.stdout:6/88: mknod d2/d13/c1d 0 2024-03-16T09:01:01.401 INFO:tasks.workunit.client.0.smithi007.stdout:6/89: truncate d2/f7 1851615 0 2024-03-16T09:01:01.406 INFO:tasks.workunit.client.0.smithi007.stdout:1/43: dwrite f0 [0,4194304] 0 2024-03-16T09:01:01.406 INFO:tasks.workunit.client.0.smithi007.stdout:7/69: dwrite d0/fa [0,4194304] 0 2024-03-16T09:01:01.407 INFO:tasks.workunit.client.0.smithi007.stdout:5/87: dwrite d1/f10 [0,4194304] 0 2024-03-16T09:01:01.407 INFO:tasks.workunit.client.0.smithi007.stdout:8/104: dwrite d0/f16 [0,4194304] 0 2024-03-16T09:01:01.407 INFO:tasks.workunit.client.0.smithi007.stdout:8/105: fdatasync d0/f1f 0 2024-03-16T09:01:01.407 INFO:tasks.workunit.client.0.smithi007.stdout:3/101: dwrite f7 [0,4194304] 0 2024-03-16T09:01:01.408 INFO:tasks.workunit.client.0.smithi007.stdout:1/44: mknod d9/cc 0 2024-03-16T09:01:01.409 INFO:tasks.workunit.client.0.smithi007.stdout:7/70: creat d0/d19/f1d x:0 0 0 2024-03-16T09:01:01.415 INFO:tasks.workunit.client.0.smithi007.stdout:3/102: mkdir d15/d18/d1a 0 2024-03-16T09:01:01.415 INFO:tasks.workunit.client.0.smithi007.stdout:5/88: read d1/d14/f15 [1315526,81465] 0 2024-03-16T09:01:01.415 INFO:tasks.workunit.client.0.smithi007.stdout:3/103: chown cc 1 1 2024-03-16T09:01:01.416 INFO:tasks.workunit.client.0.smithi007.stdout:9/135: dwrite f1b [0,4194304] 0 2024-03-16T09:01:01.416 INFO:tasks.workunit.client.0.smithi007.stdout:1/45: mkdir d9/dd 0 2024-03-16T09:01:01.417 INFO:tasks.workunit.client.0.smithi007.stdout:7/71: mkdir d0/dc/d1e 0 2024-03-16T09:01:01.417 INFO:tasks.workunit.client.0.smithi007.stdout:7/72: dread - d0/f11 zero size 2024-03-16T09:01:01.418 INFO:tasks.workunit.client.0.smithi007.stdout:8/106: rmdir d0/d2/d18 39 2024-03-16T09:01:01.419 INFO:tasks.workunit.client.0.smithi007.stdout:5/89: mkdir d1/d14/d19 0 2024-03-16T09:01:01.420 INFO:tasks.workunit.client.0.smithi007.stdout:5/90: fdatasync d1/f2 0 2024-03-16T09:01:01.426 INFO:tasks.workunit.client.0.smithi007.stdout:7/73: unlink d0/f1c 0 2024-03-16T09:01:01.426 INFO:tasks.workunit.client.0.smithi007.stdout:7/74: readlink d0/dc/lf 0 2024-03-16T09:01:01.427 INFO:tasks.workunit.client.0.smithi007.stdout:1/46: write f0 [2730177,98210] 0 2024-03-16T09:01:01.427 INFO:tasks.workunit.client.0.smithi007.stdout:1/47: fdatasync f7 0 2024-03-16T09:01:01.430 INFO:tasks.workunit.client.0.smithi007.stdout:7/75: rmdir d0/d1b 0 2024-03-16T09:01:01.430 INFO:tasks.workunit.client.0.smithi007.stdout:5/91: dread d1/fc [0,4194304] 0 2024-03-16T09:01:01.431 INFO:tasks.workunit.client.0.smithi007.stdout:5/92: write d1/f10 [4118161,10444] 0 2024-03-16T09:01:01.431 INFO:tasks.workunit.client.0.smithi007.stdout:1/48: rename d9/cc to d9/dd/ce 0 2024-03-16T09:01:01.431 INFO:tasks.workunit.client.0.smithi007.stdout:1/49: chown c2 5865954 1 2024-03-16T09:01:01.433 INFO:tasks.workunit.client.0.smithi007.stdout:7/76: mkdir d0/d19/d1f 0 2024-03-16T09:01:01.433 INFO:tasks.workunit.client.0.smithi007.stdout:7/77: chown d0/d19 4192673 1 2024-03-16T09:01:01.434 INFO:tasks.workunit.client.0.smithi007.stdout:5/93: creat d1/d7/d8/de/f1a x:0 0 0 2024-03-16T09:01:01.435 INFO:tasks.workunit.client.0.smithi007.stdout:4/86: getdents d4/d14 0 2024-03-16T09:01:01.438 INFO:tasks.workunit.client.0.smithi007.stdout:8/107: dwrite d0/dc/f12 [0,4194304] 0 2024-03-16T09:01:01.438 INFO:tasks.workunit.client.0.smithi007.stdout:8/108: truncate d0/d2/d18/f19 968001 0 2024-03-16T09:01:01.438 INFO:tasks.workunit.client.0.smithi007.stdout:5/94: mkdir d1/d7/d16/d1b 0 2024-03-16T09:01:01.438 INFO:tasks.workunit.client.0.smithi007.stdout:5/95: truncate d1/d7/f12 992089 0 2024-03-16T09:01:01.439 INFO:tasks.workunit.client.0.smithi007.stdout:0/76: fsync d3/d9/da/d10/d1a/f1b 0 2024-03-16T09:01:01.442 INFO:tasks.workunit.client.0.smithi007.stdout:7/78: write d0/dc/fe [1090616,5763] 0 2024-03-16T09:01:01.442 INFO:tasks.workunit.client.0.smithi007.stdout:8/109: symlink d0/l20 0 2024-03-16T09:01:01.442 INFO:tasks.workunit.client.0.smithi007.stdout:7/79: chown d0/c9 54917940 1 2024-03-16T09:01:01.444 INFO:tasks.workunit.client.0.smithi007.stdout:4/87: getdents d4 0 2024-03-16T09:01:01.450 INFO:tasks.workunit.client.0.smithi007.stdout:4/88: unlink d4/c11 0 2024-03-16T09:01:01.452 INFO:tasks.workunit.client.0.smithi007.stdout:7/80: dread d0/fa [0,4194304] 0 2024-03-16T09:01:01.452 INFO:tasks.workunit.client.0.smithi007.stdout:7/81: creat d0/d19/f20 x:0 0 0 2024-03-16T09:01:01.452 INFO:tasks.workunit.client.0.smithi007.stdout:7/82: read - d0/f18 zero size 2024-03-16T09:01:01.455 INFO:tasks.workunit.client.0.smithi007.stdout:4/89: dread f2 [4194304,4194304] 0 2024-03-16T09:01:01.456 INFO:tasks.workunit.client.0.smithi007.stdout:4/90: stat d4/f18 0 2024-03-16T09:01:01.456 INFO:tasks.workunit.client.0.smithi007.stdout:4/91: truncate d4/f18 533689 0 2024-03-16T09:01:01.458 INFO:tasks.workunit.client.0.smithi007.stdout:9/136: getdents d24 0 2024-03-16T09:01:01.459 INFO:tasks.workunit.client.0.smithi007.stdout:7/83: write d0/fa [2778439,86141] 0 2024-03-16T09:01:01.459 INFO:tasks.workunit.client.0.smithi007.stdout:7/84: fsync d0/f11 0 2024-03-16T09:01:01.459 INFO:tasks.workunit.client.0.smithi007.stdout:7/85: chown d0/f18 22060639 1 2024-03-16T09:01:01.460 INFO:tasks.workunit.client.0.smithi007.stdout:9/137: mknod d24/c28 0 2024-03-16T09:01:01.460 INFO:tasks.workunit.client.0.smithi007.stdout:9/138: fdatasync f1a 0 2024-03-16T09:01:01.461 INFO:tasks.workunit.client.0.smithi007.stdout:8/110: dwrite d0/d2/f17 [0,4194304] 0 2024-03-16T09:01:01.465 INFO:tasks.workunit.client.0.smithi007.stdout:9/139: truncate f1a 226375 0 2024-03-16T09:01:01.467 INFO:tasks.workunit.client.0.smithi007.stdout:8/111: symlink d0/dc/dd/l21 0 2024-03-16T09:01:01.467 INFO:tasks.workunit.client.0.smithi007.stdout:8/112: truncate d0/d2/fa 4766931 0 2024-03-16T09:01:01.467 INFO:tasks.workunit.client.0.smithi007.stdout:8/113: truncate d0/f1f 412527 0 2024-03-16T09:01:01.471 INFO:tasks.workunit.client.0.smithi007.stdout:9/140: link d24/f27 d24/f29 0 2024-03-16T09:01:01.471 INFO:tasks.workunit.client.0.smithi007.stdout:3/104: getdents d15/d18 0 2024-03-16T09:01:01.471 INFO:tasks.workunit.client.0.smithi007.stdout:9/141: fdatasync f7 0 2024-03-16T09:01:01.473 INFO:tasks.workunit.client.0.smithi007.stdout:3/105: creat d15/d18/d1a/f1b x:0 0 0 2024-03-16T09:01:01.473 INFO:tasks.workunit.client.0.smithi007.stdout:3/106: truncate d15/f16 368282 0 2024-03-16T09:01:01.474 INFO:tasks.workunit.client.0.smithi007.stdout:5/96: rename d1/d14 to d1/d7/d1c 0 2024-03-16T09:01:01.476 INFO:tasks.workunit.client.0.smithi007.stdout:6/90: dwrite d2/f7 [0,4194304] 0 2024-03-16T09:01:01.476 INFO:tasks.workunit.client.0.smithi007.stdout:6/91: chown d2/d11 2293 1 2024-03-16T09:01:01.476 INFO:tasks.workunit.client.0.smithi007.stdout:3/107: unlink le 0 2024-03-16T09:01:01.480 INFO:tasks.workunit.client.0.smithi007.stdout:3/108: symlink d15/l1c 0 2024-03-16T09:01:01.482 INFO:tasks.workunit.client.0.smithi007.stdout:7/86: dwrite d0/f6 [0,4194304] 0 2024-03-16T09:01:01.482 INFO:tasks.workunit.client.0.smithi007.stdout:7/87: fdatasync d0/dc/fd 0 2024-03-16T09:01:01.482 INFO:tasks.workunit.client.0.smithi007.stdout:7/88: dread - d0/d19/f1d zero size 2024-03-16T09:01:01.482 INFO:tasks.workunit.client.0.smithi007.stdout:7/89: write d0/fa [4968970,14710] 0 2024-03-16T09:01:01.483 INFO:tasks.workunit.client.0.smithi007.stdout:7/90: dread - d0/d19/f1d zero size 2024-03-16T09:01:01.483 INFO:tasks.workunit.client.0.smithi007.stdout:8/114: write d0/f16 [1169285,83160] 0 2024-03-16T09:01:01.483 INFO:tasks.workunit.client.0.smithi007.stdout:1/50: getdents d9/dd 0 2024-03-16T09:01:01.486 INFO:tasks.workunit.client.0.smithi007.stdout:3/109: unlink l11 0 2024-03-16T09:01:01.488 INFO:tasks.workunit.client.0.smithi007.stdout:1/51: symlink d9/dd/lf 0 2024-03-16T09:01:01.490 INFO:tasks.workunit.client.0.smithi007.stdout:3/110: unlink f12 0 2024-03-16T09:01:01.491 INFO:tasks.workunit.client.0.smithi007.stdout:3/111: readlink l4 0 2024-03-16T09:01:01.491 INFO:tasks.workunit.client.0.smithi007.stdout:3/112: rmdir d15 39 2024-03-16T09:01:01.492 INFO:tasks.workunit.client.0.smithi007.stdout:9/142: dwrite f8 [8388608,4194304] 0 2024-03-16T09:01:01.494 INFO:tasks.workunit.client.0.smithi007.stdout:3/113: creat d15/d18/f1d x:0 0 0 2024-03-16T09:01:01.494 INFO:tasks.workunit.client.0.smithi007.stdout:3/114: readlink l9 0 2024-03-16T09:01:01.494 INFO:tasks.workunit.client.0.smithi007.stdout:3/115: stat f7 0 2024-03-16T09:01:01.494 INFO:tasks.workunit.client.0.smithi007.stdout:3/116: fdatasync d15/f16 0 2024-03-16T09:01:01.495 INFO:tasks.workunit.client.0.smithi007.stdout:3/117: fdatasync fa 0 2024-03-16T09:01:01.496 INFO:tasks.workunit.client.0.smithi007.stdout:5/97: truncate d1/d7/d1c/f17 1343008 0 2024-03-16T09:01:01.496 INFO:tasks.workunit.client.0.smithi007.stdout:5/98: chown d1/d7/d16 794738920 1 2024-03-16T09:01:01.497 INFO:tasks.workunit.client.0.smithi007.stdout:3/118: truncate f10 596348 0 2024-03-16T09:01:01.497 INFO:tasks.workunit.client.0.smithi007.stdout:1/52: dread f7 [0,4194304] 0 2024-03-16T09:01:01.499 INFO:tasks.workunit.client.0.smithi007.stdout:9/143: rename f22 to d24/f2a 0 2024-03-16T09:01:01.499 INFO:tasks.workunit.client.0.smithi007.stdout:9/144: stat l12 0 2024-03-16T09:01:01.502 INFO:tasks.workunit.client.0.smithi007.stdout:3/119: creat d15/f1e x:0 0 0 2024-03-16T09:01:01.503 INFO:tasks.workunit.client.0.smithi007.stdout:3/120: dread - d15/d18/d1a/f1b zero size 2024-03-16T09:01:01.503 INFO:tasks.workunit.client.0.smithi007.stdout:8/115: dread d0/f16 [0,4194304] 0 2024-03-16T09:01:01.505 INFO:tasks.workunit.client.0.smithi007.stdout:9/145: symlink d24/l2b 0 2024-03-16T09:01:01.506 INFO:tasks.workunit.client.0.smithi007.stdout:8/116: write d0/f1f [140838,67521] 0 2024-03-16T09:01:01.506 INFO:tasks.workunit.client.0.smithi007.stdout:8/117: creat d0/f22 x:0 0 0 2024-03-16T09:01:01.506 INFO:tasks.workunit.client.0.smithi007.stdout:8/118: chown d0/d2/d18/c1a 785298 1 2024-03-16T09:01:01.509 INFO:tasks.workunit.client.0.smithi007.stdout:7/91: dwrite d0/dc/f15 [0,4194304] 0 2024-03-16T09:01:01.513 INFO:tasks.workunit.client.0.smithi007.stdout:8/119: truncate d0/d2/f17 4007106 0 2024-03-16T09:01:01.513 INFO:tasks.workunit.client.0.smithi007.stdout:8/120: creat d0/dc/f23 x:0 0 0 2024-03-16T09:01:01.518 INFO:tasks.workunit.client.0.smithi007.stdout:7/92: mkdir d0/dc/d1e/d21 0 2024-03-16T09:01:01.523 INFO:tasks.workunit.client.0.smithi007.stdout:8/121: symlink d0/d2/d18/l24 0 2024-03-16T09:01:01.524 INFO:tasks.workunit.client.0.smithi007.stdout:6/92: truncate d2/f7 1743434 0 2024-03-16T09:01:01.524 INFO:tasks.workunit.client.0.smithi007.stdout:6/93: chown d2 7513470 1 2024-03-16T09:01:01.524 INFO:tasks.workunit.client.0.smithi007.stdout:6/94: chown d2/d10/d12/d15/c1b 192450107 1 2024-03-16T09:01:01.525 INFO:tasks.workunit.client.0.smithi007.stdout:7/93: mknod d0/dc/d1e/c22 0 2024-03-16T09:01:01.525 INFO:tasks.workunit.client.0.smithi007.stdout:7/94: read - d0/d19/f1d zero size 2024-03-16T09:01:01.527 INFO:tasks.workunit.client.0.smithi007.stdout:6/95: mkdir d2/d11/d1e 0 2024-03-16T09:01:01.527 INFO:tasks.workunit.client.0.smithi007.stdout:6/96: write d2/fb [1607627,30375] 0 2024-03-16T09:01:01.529 INFO:tasks.workunit.client.0.smithi007.stdout:7/95: symlink d0/dc/d1e/l23 0 2024-03-16T09:01:01.529 INFO:tasks.workunit.client.0.smithi007.stdout:7/96: chown d0/d19/f20 918256 1 2024-03-16T09:01:01.529 INFO:tasks.workunit.client.0.smithi007.stdout:7/97: truncate d0/f11 5064315 0 2024-03-16T09:01:01.529 INFO:tasks.workunit.client.0.smithi007.stdout:6/97: mkdir d2/d13/d1f 0 2024-03-16T09:01:01.532 INFO:tasks.workunit.client.0.smithi007.stdout:7/98: rename d0/dc/d1e/d21 to d0/dc/d24 0 2024-03-16T09:01:01.532 INFO:tasks.workunit.client.0.smithi007.stdout:7/99: write d0/d19/f20 [941663,40350] 0 2024-03-16T09:01:01.533 INFO:tasks.workunit.client.0.smithi007.stdout:1/53: dwrite f5 [0,4194304] 0 2024-03-16T09:01:01.533 INFO:tasks.workunit.client.0.smithi007.stdout:3/121: dwrite d15/f1e [0,4194304] 0 2024-03-16T09:01:01.533 INFO:tasks.workunit.client.0.smithi007.stdout:6/98: rmdir d2/d10 39 2024-03-16T09:01:01.538 INFO:tasks.workunit.client.0.smithi007.stdout:1/54: symlink d9/l10 0 2024-03-16T09:01:01.538 INFO:tasks.workunit.client.0.smithi007.stdout:5/99: dread d1/d7/d1c/f15 [0,4194304] 0 2024-03-16T09:01:01.541 INFO:tasks.workunit.client.0.smithi007.stdout:6/99: creat d2/f20 x:0 0 0 2024-03-16T09:01:01.542 INFO:tasks.workunit.client.0.smithi007.stdout:7/100: getdents d0/d19 0 2024-03-16T09:01:01.547 INFO:tasks.workunit.client.0.smithi007.stdout:6/100: rename d2/fe to d2/d11/f21 0 2024-03-16T09:01:01.548 INFO:tasks.workunit.client.0.smithi007.stdout:9/146: truncate f1f 9995581 0 2024-03-16T09:01:01.548 INFO:tasks.workunit.client.0.smithi007.stdout:9/147: write f17 [4794630,86801] 0 2024-03-16T09:01:01.550 INFO:tasks.workunit.client.0.smithi007.stdout:7/101: unlink d0/c10 0 2024-03-16T09:01:01.550 INFO:tasks.workunit.client.0.smithi007.stdout:7/102: chown d0/dc/d1e/c22 14072 1 2024-03-16T09:01:01.551 INFO:tasks.workunit.client.0.smithi007.stdout:1/55: symlink d9/dd/l11 0 2024-03-16T09:01:01.552 INFO:tasks.workunit.client.0.smithi007.stdout:6/101: rmdir d2/d13 39 2024-03-16T09:01:01.554 INFO:tasks.workunit.client.0.smithi007.stdout:9/148: creat d24/f2c x:0 0 0 2024-03-16T09:01:01.557 INFO:tasks.workunit.client.0.smithi007.stdout:1/56: creat d9/f12 x:0 0 0 2024-03-16T09:01:01.558 INFO:tasks.workunit.client.0.smithi007.stdout:3/122: dwrite fa [0,4194304] 0 2024-03-16T09:01:01.558 INFO:tasks.workunit.client.0.smithi007.stdout:3/123: write f14 [965682,111472] 0 2024-03-16T09:01:01.559 INFO:tasks.workunit.client.0.smithi007.stdout:6/102: creat d2/d10/d12/d15/f22 x:0 0 0 2024-03-16T09:01:01.559 INFO:tasks.workunit.client.0.smithi007.stdout:5/100: dwrite d1/d7/f12 [0,4194304] 0 2024-03-16T09:01:01.560 INFO:tasks.workunit.client.0.smithi007.stdout:9/149: truncate d24/f25 214412 0 2024-03-16T09:01:01.560 INFO:tasks.workunit.client.0.smithi007.stdout:9/150: write f16 [573004,93040] 0 2024-03-16T09:01:01.560 INFO:tasks.workunit.client.0.smithi007.stdout:9/151: truncate f1c 730858 0 2024-03-16T09:01:01.561 INFO:tasks.workunit.client.0.smithi007.stdout:1/57: chown d9/dd/ce 28 1 2024-03-16T09:01:01.566 INFO:tasks.workunit.client.0.smithi007.stdout:7/103: rename d0/dc/d1e to d0/dc/d24/d25 0 2024-03-16T09:01:01.566 INFO:tasks.workunit.client.0.smithi007.stdout:7/104: chown d0/dc/d24/d25/l23 212035 1 2024-03-16T09:01:01.567 INFO:tasks.workunit.client.0.smithi007.stdout:5/101: mkdir d1/d7/d8/d1d 0 2024-03-16T09:01:01.570 INFO:tasks.workunit.client.0.smithi007.stdout:9/152: unlink f23 0 2024-03-16T09:01:01.570 INFO:tasks.workunit.client.0.smithi007.stdout:8/122: write d0/d2/f17 [3021171,81406] 0 2024-03-16T09:01:01.571 INFO:tasks.workunit.client.0.smithi007.stdout:7/105: dread d0/d19/f20 [0,4194304] 0 2024-03-16T09:01:01.572 INFO:tasks.workunit.client.0.smithi007.stdout:9/153: dread fd [0,4194304] 0 2024-03-16T09:01:01.572 INFO:tasks.workunit.client.0.smithi007.stdout:1/58: unlink d9/dd/l11 0 2024-03-16T09:01:01.572 INFO:tasks.workunit.client.0.smithi007.stdout:1/59: chown d9/dd/lf 1886078 1 2024-03-16T09:01:01.573 INFO:tasks.workunit.client.0.smithi007.stdout:8/123: dread d0/d2/d18/f19 [0,4194304] 0 2024-03-16T09:01:01.573 INFO:tasks.workunit.client.0.smithi007.stdout:3/124: getdents d15/d18 0 2024-03-16T09:01:01.575 INFO:tasks.workunit.client.0.smithi007.stdout:5/102: mkdir d1/d7/d16/d1e 0 2024-03-16T09:01:01.575 INFO:tasks.workunit.client.0.smithi007.stdout:5/103: write d1/fa [315917,36276] 0 2024-03-16T09:01:01.576 INFO:tasks.workunit.client.0.smithi007.stdout:5/104: chown d1/fc 797 1 2024-03-16T09:01:01.578 INFO:tasks.workunit.client.0.smithi007.stdout:6/103: dwrite d2/f7 [0,4194304] 0 2024-03-16T09:01:01.578 INFO:tasks.workunit.client.0.smithi007.stdout:7/106: creat d0/d19/f26 x:0 0 0 2024-03-16T09:01:01.578 INFO:tasks.workunit.client.0.smithi007.stdout:7/107: creat d0/d19/f27 x:0 0 0 2024-03-16T09:01:01.578 INFO:tasks.workunit.client.0.smithi007.stdout:7/108: write d0/d19/f26 [415509,22837] 0 2024-03-16T09:01:01.580 INFO:tasks.workunit.client.0.smithi007.stdout:9/154: unlink c13 0 2024-03-16T09:01:01.580 INFO:tasks.workunit.client.0.smithi007.stdout:9/155: stat d24/l2b 0 2024-03-16T09:01:01.580 INFO:tasks.workunit.client.0.smithi007.stdout:9/156: fsync d24/f27 0 2024-03-16T09:01:01.582 INFO:tasks.workunit.client.0.smithi007.stdout:8/124: mknod d0/c25 0 2024-03-16T09:01:01.582 INFO:tasks.workunit.client.0.smithi007.stdout:8/125: read d0/ff [1308314,65921] 0 2024-03-16T09:01:01.583 INFO:tasks.workunit.client.0.smithi007.stdout:8/126: fdatasync d0/dc/f12 0 2024-03-16T09:01:01.585 INFO:tasks.workunit.client.0.smithi007.stdout:3/125: mkdir d15/d18/d1f 0 2024-03-16T09:01:01.587 INFO:tasks.workunit.client.0.smithi007.stdout:5/105: creat d1/d7/d8/d1d/f1f x:0 0 0 2024-03-16T09:01:01.589 INFO:tasks.workunit.client.0.smithi007.stdout:6/104: truncate d2/fa 1765704 0 2024-03-16T09:01:01.593 INFO:tasks.workunit.client.0.smithi007.stdout:1/60: dwrite f7 [0,4194304] 0 2024-03-16T09:01:01.598 INFO:tasks.workunit.client.0.smithi007.stdout:9/157: truncate d24/f2a 3296938 0 2024-03-16T09:01:01.599 INFO:tasks.workunit.client.0.smithi007.stdout:9/158: write d24/f2c [425424,87408] 0 2024-03-16T09:01:01.599 INFO:tasks.workunit.client.0.smithi007.stdout:9/159: write f16 [1611867,27013] 0 2024-03-16T09:01:01.599 INFO:tasks.workunit.client.0.smithi007.stdout:9/160: readlink l20 0 2024-03-16T09:01:01.601 INFO:tasks.workunit.client.0.smithi007.stdout:8/127: mkdir d0/d26 0 2024-03-16T09:01:01.601 INFO:tasks.workunit.client.0.smithi007.stdout:8/128: read d0/d2/fa [3153830,94485] 0 2024-03-16T09:01:01.601 INFO:tasks.workunit.client.0.smithi007.stdout:8/129: stat d0/dc/dd 0 2024-03-16T09:01:01.601 INFO:tasks.workunit.client.0.smithi007.stdout:8/130: chown d0/l9 13774822 1 2024-03-16T09:01:01.601 INFO:tasks.workunit.client.0.smithi007.stdout:8/131: read - d0/dc/f23 zero size 2024-03-16T09:01:01.601 INFO:tasks.workunit.client.0.smithi007.stdout:8/132: write d0/f16 [2707404,27210] 0 2024-03-16T09:01:01.602 INFO:tasks.workunit.client.0.smithi007.stdout:1/61: read f5 [2676640,23803] 0 2024-03-16T09:01:01.602 INFO:tasks.workunit.client.0.smithi007.stdout:1/62: creat d9/f13 x:0 0 0 2024-03-16T09:01:01.602 INFO:tasks.workunit.client.0.smithi007.stdout:1/63: fdatasync f7 0 2024-03-16T09:01:01.620 INFO:tasks.workunit.client.0.smithi007.stdout:5/106: symlink d1/d7/d1c/l20 0 2024-03-16T09:01:01.620 INFO:tasks.workunit.client.0.smithi007.stdout:5/107: readlink d1/d7/d8/l9 0 2024-03-16T09:01:01.620 INFO:tasks.workunit.client.0.smithi007.stdout:5/108: write d1/f10 [625997,77464] 0 2024-03-16T09:01:01.621 INFO:tasks.workunit.client.0.smithi007.stdout:6/105: symlink d2/d10/d12/l23 0 2024-03-16T09:01:01.621 INFO:tasks.workunit.client.0.smithi007.stdout:6/106: write d2/f20 [1033382,97108] 0 2024-03-16T09:01:01.629 INFO:tasks.workunit.client.0.smithi007.stdout:6/107: dread d2/f20 [0,4194304] 0 2024-03-16T09:01:01.629 INFO:tasks.workunit.client.0.smithi007.stdout:8/133: mknod d0/dc/c27 0 2024-03-16T09:01:01.632 INFO:tasks.workunit.client.0.smithi007.stdout:1/64: creat d9/dd/f14 x:0 0 0 2024-03-16T09:01:01.632 INFO:tasks.workunit.client.0.smithi007.stdout:1/65: fdatasync f0 0 2024-03-16T09:01:01.633 INFO:tasks.workunit.client.0.smithi007.stdout:5/109: symlink d1/d7/d16/d1e/l21 0 2024-03-16T09:01:01.633 INFO:tasks.workunit.client.0.smithi007.stdout:7/109: rmdir d0/dc 39 2024-03-16T09:01:01.636 INFO:tasks.workunit.client.0.smithi007.stdout:3/126: dwrite d15/d18/d1a/f1b [0,4194304] 0 2024-03-16T09:01:01.636 INFO:tasks.workunit.client.0.smithi007.stdout:3/127: readlink l9 0 2024-03-16T09:01:01.636 INFO:tasks.workunit.client.0.smithi007.stdout:3/128: write f8 [5893320,111585] 0 2024-03-16T09:01:01.636 INFO:tasks.workunit.client.0.smithi007.stdout:9/161: link d24/f27 d24/f2d 0 2024-03-16T09:01:01.639 INFO:tasks.workunit.client.0.smithi007.stdout:8/134: rename d0/d2/l1e to d0/dc/l28 0 2024-03-16T09:01:01.639 INFO:tasks.workunit.client.0.smithi007.stdout:5/110: rename d1 to d1/d7/d16/d22 22 2024-03-16T09:01:01.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/66: stat d9/ca 0 2024-03-16T09:01:01.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/67: chown d9/f13 202 1 2024-03-16T09:01:01.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/68: write f0 [5177254,43663] 0 2024-03-16T09:01:01.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/69: creat d9/f15 x:0 0 0 2024-03-16T09:01:01.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/70: write f0 [5793750,35618] 0 2024-03-16T09:01:01.643 INFO:tasks.workunit.client.0.smithi007.stdout:3/129: chown d15/d18/f1d 0 1 2024-03-16T09:01:01.645 INFO:tasks.workunit.client.0.smithi007.stdout:8/135: rmdir d0/d2/d18 39 2024-03-16T09:01:01.647 INFO:tasks.workunit.client.0.smithi007.stdout:5/111: creat d1/d7/d1c/f23 x:0 0 0 2024-03-16T09:01:01.650 INFO:tasks.workunit.client.0.smithi007.stdout:6/108: dwrite d2/fa [0,4194304] 0 2024-03-16T09:01:01.654 INFO:tasks.workunit.client.0.smithi007.stdout:3/130: getdents d15/d18 0 2024-03-16T09:01:01.654 INFO:tasks.workunit.client.0.smithi007.stdout:7/110: creat d0/f28 x:0 0 0 2024-03-16T09:01:01.655 INFO:tasks.workunit.client.0.smithi007.stdout:9/162: symlink d24/l2e 0 2024-03-16T09:01:01.656 INFO:tasks.workunit.client.0.smithi007.stdout:8/136: rmdir d0/d2 39 2024-03-16T09:01:01.658 INFO:tasks.workunit.client.0.smithi007.stdout:1/71: symlink d9/dd/l16 0 2024-03-16T09:01:01.660 INFO:tasks.workunit.client.0.smithi007.stdout:6/109: rmdir d2/d10/d19 0 2024-03-16T09:01:01.662 INFO:tasks.workunit.client.0.smithi007.stdout:8/137: dread d0/ff [0,4194304] 0 2024-03-16T09:01:01.665 INFO:tasks.workunit.client.0.smithi007.stdout:9/163: truncate d24/f2a 129742 0 2024-03-16T09:01:01.667 INFO:tasks.workunit.client.0.smithi007.stdout:6/110: rmdir d2/d10/d12 39 2024-03-16T09:01:01.668 INFO:tasks.workunit.client.0.smithi007.stdout:8/138: getdents d0/d2/d18 0 2024-03-16T09:01:01.671 INFO:tasks.workunit.client.0.smithi007.stdout:1/72: getdents d9 0 2024-03-16T09:01:01.671 INFO:tasks.workunit.client.0.smithi007.stdout:6/111: dread d2/fd [0,4194304] 0 2024-03-16T09:01:01.671 INFO:tasks.workunit.client.0.smithi007.stdout:8/139: symlink d0/d2/l29 0 2024-03-16T09:01:01.671 INFO:tasks.workunit.client.0.smithi007.stdout:8/140: creat d0/dc/f2a x:0 0 0 2024-03-16T09:01:01.672 INFO:tasks.workunit.client.0.smithi007.stdout:3/131: dwrite d15/f16 [0,4194304] 0 2024-03-16T09:01:01.674 INFO:tasks.workunit.client.0.smithi007.stdout:9/164: dread f1f [8388608,4194304] 0 2024-03-16T09:01:01.675 INFO:tasks.workunit.client.0.smithi007.stdout:9/165: truncate f1d 411114 0 2024-03-16T09:01:01.675 INFO:tasks.workunit.client.0.smithi007.stdout:5/112: dwrite d1/d7/d1c/f15 [0,4194304] 0 2024-03-16T09:01:01.677 INFO:tasks.workunit.client.0.smithi007.stdout:7/111: write d0/dc/f1a [789045,111819] 0 2024-03-16T09:01:01.680 INFO:tasks.workunit.client.0.smithi007.stdout:6/112: getdents d2/d10 0 2024-03-16T09:01:01.680 INFO:tasks.workunit.client.0.smithi007.stdout:6/113: write d2/d11/f21 [3322570,8003] 0 2024-03-16T09:01:01.681 INFO:tasks.workunit.client.0.smithi007.stdout:6/114: chown d2/d10/d12/d15 4 1 2024-03-16T09:01:01.685 INFO:tasks.workunit.client.0.smithi007.stdout:8/141: rmdir d0 39 2024-03-16T09:01:01.696 INFO:tasks.workunit.client.0.smithi007.stdout:5/113: rename d1/d7/d1c/l20 to d1/d7/d8/de/l24 0 2024-03-16T09:01:01.697 INFO:tasks.workunit.client.0.smithi007.stdout:7/112: rmdir d0/dc/d24/d25 39 2024-03-16T09:01:01.697 INFO:tasks.workunit.client.0.smithi007.stdout:5/114: write d1/f2 [1303227,128438] 0 2024-03-16T09:01:01.697 INFO:tasks.workunit.client.0.smithi007.stdout:1/73: dwrite f7 [0,4194304] 0 2024-03-16T09:01:01.699 INFO:tasks.workunit.client.0.smithi007.stdout:3/132: truncate d15/f16 3581588 0 2024-03-16T09:01:01.699 INFO:tasks.workunit.client.0.smithi007.stdout:9/166: dwrite f8 [4194304,4194304] 0 2024-03-16T09:01:01.699 INFO:tasks.workunit.client.0.smithi007.stdout:6/115: mkdir d2/d13/d24 0 2024-03-16T09:01:01.700 INFO:tasks.workunit.client.0.smithi007.stdout:8/142: mknod d0/dc/dd/c2b 0 2024-03-16T09:01:01.701 INFO:tasks.workunit.client.0.smithi007.stdout:7/113: fsync d0/d19/f27 0 2024-03-16T09:01:01.710 INFO:tasks.workunit.client.0.smithi007.stdout:3/133: mkdir d15/d20 0 2024-03-16T09:01:01.711 INFO:tasks.workunit.client.0.smithi007.stdout:3/134: truncate d15/d18/f1d 88665 0 2024-03-16T09:01:01.711 INFO:tasks.workunit.client.0.smithi007.stdout:3/135: fsync f5 0 2024-03-16T09:01:01.713 INFO:tasks.workunit.client.0.smithi007.stdout:1/74: mknod d9/dd/c17 0 2024-03-16T09:01:01.713 INFO:tasks.workunit.client.0.smithi007.stdout:1/75: chown c3 412 1 2024-03-16T09:01:01.714 INFO:tasks.workunit.client.0.smithi007.stdout:3/136: mknod d15/d18/c21 0 2024-03-16T09:01:01.714 INFO:tasks.workunit.client.0.smithi007.stdout:9/167: mknod d24/c2f 0 2024-03-16T09:01:01.715 INFO:tasks.workunit.client.0.smithi007.stdout:1/76: creat d9/f18 x:0 0 0 2024-03-16T09:01:01.720 INFO:tasks.workunit.client.0.smithi007.stdout:7/114: dread d0/f6 [0,4194304] 0 2024-03-16T09:01:01.723 INFO:tasks.workunit.client.0.smithi007.stdout:8/143: dwrite d0/d2/f17 [4194304,4194304] 0 2024-03-16T09:01:01.724 INFO:tasks.workunit.client.0.smithi007.stdout:8/144: dread d0/f1f [0,4194304] 0 2024-03-16T09:01:01.725 INFO:tasks.workunit.client.0.smithi007.stdout:6/116: dread d2/d11/f21 [0,4194304] 0 2024-03-16T09:01:01.725 INFO:tasks.workunit.client.0.smithi007.stdout:5/115: dwrite d1/fc [0,4194304] 0 2024-03-16T09:01:01.725 INFO:tasks.workunit.client.0.smithi007.stdout:5/116: chown d1/d7/d8/d1d 100645 1 2024-03-16T09:01:01.727 INFO:tasks.workunit.client.0.smithi007.stdout:7/115: read d0/fa [2090624,41953] 0 2024-03-16T09:01:01.728 INFO:tasks.workunit.client.0.smithi007.stdout:8/145: creat d0/dc/dd/f2c x:0 0 0 2024-03-16T09:01:01.729 INFO:tasks.workunit.client.0.smithi007.stdout:6/117: creat d2/d11/f25 x:0 0 0 2024-03-16T09:01:01.729 INFO:tasks.workunit.client.0.smithi007.stdout:6/118: getdents d2/d13/d1f 0 2024-03-16T09:01:01.730 INFO:tasks.workunit.client.0.smithi007.stdout:1/77: write f7 [1642512,58521] 0 2024-03-16T09:01:01.730 INFO:tasks.workunit.client.0.smithi007.stdout:1/78: creat d9/f19 x:0 0 0 2024-03-16T09:01:01.730 INFO:tasks.workunit.client.0.smithi007.stdout:2/131: sync 2024-03-16T09:01:01.730 INFO:tasks.workunit.client.0.smithi007.stdout:2/132: chown dc/f17 472103 1 2024-03-16T09:01:01.732 INFO:tasks.workunit.client.0.smithi007.stdout:0/77: sync 2024-03-16T09:01:01.732 INFO:tasks.workunit.client.0.smithi007.stdout:4/92: sync 2024-03-16T09:01:01.732 INFO:tasks.workunit.client.0.smithi007.stdout:6/119: dread d2/f20 [0,4194304] 0 2024-03-16T09:01:01.732 INFO:tasks.workunit.client.0.smithi007.stdout:6/120: creat d2/d11/f26 x:0 0 0 2024-03-16T09:01:01.732 INFO:tasks.workunit.client.0.smithi007.stdout:5/117: rmdir d1/d7/d16 39 2024-03-16T09:01:01.736 INFO:tasks.workunit.client.0.smithi007.stdout:7/116: write d0/f3 [2029506,66685] 0 2024-03-16T09:01:01.737 INFO:tasks.workunit.client.0.smithi007.stdout:3/137: read d15/f16 [2459662,114552] 0 2024-03-16T09:01:01.737 INFO:tasks.workunit.client.0.smithi007.stdout:8/146: mkdir d0/d26/d2d 0 2024-03-16T09:01:01.737 INFO:tasks.workunit.client.0.smithi007.stdout:8/147: getdents d0/d26/d2d 0 2024-03-16T09:01:01.737 INFO:tasks.workunit.client.0.smithi007.stdout:3/138: write f5 [2597787,1563] 0 2024-03-16T09:01:01.737 INFO:tasks.workunit.client.0.smithi007.stdout:8/148: chown d0/d26/d2d 218 1 2024-03-16T09:01:01.738 INFO:tasks.workunit.client.0.smithi007.stdout:9/168: rmdir d24 39 2024-03-16T09:01:01.738 INFO:tasks.workunit.client.0.smithi007.stdout:7/117: dread d0/dc/fd [0,4194304] 0 2024-03-16T09:01:01.738 INFO:tasks.workunit.client.0.smithi007.stdout:7/118: fsync d0/dc/fe 0 2024-03-16T09:01:01.739 INFO:tasks.workunit.client.0.smithi007.stdout:1/79: mknod d9/c1a 0 2024-03-16T09:01:01.739 INFO:tasks.workunit.client.0.smithi007.stdout:1/80: read - d9/f19 zero size 2024-03-16T09:01:01.739 INFO:tasks.workunit.client.0.smithi007.stdout:1/81: write d9/dd/f14 [2540,42234] 0 2024-03-16T09:01:01.739 INFO:tasks.workunit.client.0.smithi007.stdout:7/119: dread d0/d19/f20 [0,4194304] 0 2024-03-16T09:01:01.741 INFO:tasks.workunit.client.0.smithi007.stdout:2/133: rename f1 to dc/df/f29 0 2024-03-16T09:01:01.747 INFO:tasks.workunit.client.0.smithi007.stdout:6/121: symlink d2/d10/d12/d15/l27 0 2024-03-16T09:01:01.747 INFO:tasks.workunit.client.0.smithi007.stdout:6/122: creat d2/d10/d12/d15/f28 x:0 0 0 2024-03-16T09:01:01.748 INFO:tasks.workunit.client.0.smithi007.stdout:3/139: rmdir d15/d20 0 2024-03-16T09:01:01.754 INFO:tasks.workunit.client.0.smithi007.stdout:1/82: unlink d9/dd/lf 0 2024-03-16T09:01:01.757 INFO:tasks.workunit.client.0.smithi007.stdout:7/120: mknod d0/dc/c29 0 2024-03-16T09:01:01.757 INFO:tasks.workunit.client.0.smithi007.stdout:7/121: dread - d0/d19/f1d zero size 2024-03-16T09:01:01.757 INFO:tasks.workunit.client.0.smithi007.stdout:3/140: write f8 [173930,67867] 0 2024-03-16T09:01:01.759 INFO:tasks.workunit.client.0.smithi007.stdout:7/122: write d0/f3 [1083680,127759] 0 2024-03-16T09:01:01.760 INFO:tasks.workunit.client.0.smithi007.stdout:8/149: rename d0/l20 to d0/dc/l2e 0 2024-03-16T09:01:01.763 INFO:tasks.workunit.client.0.smithi007.stdout:6/123: mkdir d2/d29 0 2024-03-16T09:01:01.763 INFO:tasks.workunit.client.0.smithi007.stdout:6/124: chown d2/d11/f25 6547 1 2024-03-16T09:01:01.763 INFO:tasks.workunit.client.0.smithi007.stdout:5/118: symlink d1/l25 0 2024-03-16T09:01:01.764 INFO:tasks.workunit.client.0.smithi007.stdout:9/169: link lc d24/l30 0 2024-03-16T09:01:01.764 INFO:tasks.workunit.client.0.smithi007.stdout:9/170: write d24/f27 [454579,25550] 0 2024-03-16T09:01:01.765 INFO:tasks.workunit.client.0.smithi007.stdout:3/141: truncate f10 327311 0 2024-03-16T09:01:01.765 INFO:tasks.workunit.client.0.smithi007.stdout:3/142: chown d15/d18/c21 1844699 1 2024-03-16T09:01:01.766 INFO:tasks.workunit.client.0.smithi007.stdout:6/125: symlink d2/d13/d1f/l2a 0 2024-03-16T09:01:01.766 INFO:tasks.workunit.client.0.smithi007.stdout:6/126: truncate d2/d10/d12/d15/f28 637209 0 2024-03-16T09:01:01.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/119: link d1/f10 d1/d7/d1c/d19/f26 0 2024-03-16T09:01:01.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/120: read d1/fa [2074188,90644] 0 2024-03-16T09:01:01.770 INFO:tasks.workunit.client.0.smithi007.stdout:3/143: creat d15/d18/f22 x:0 0 0 2024-03-16T09:01:01.773 INFO:tasks.workunit.client.0.smithi007.stdout:6/127: link d2/d11/l18 d2/l2b 0 2024-03-16T09:01:01.773 INFO:tasks.workunit.client.0.smithi007.stdout:6/128: chown d2/d11 236 1 2024-03-16T09:01:01.773 INFO:tasks.workunit.client.0.smithi007.stdout:6/129: fdatasync d2/fd 0 2024-03-16T09:01:01.773 INFO:tasks.workunit.client.0.smithi007.stdout:6/130: truncate d2/d11/f25 739240 0 2024-03-16T09:01:01.774 INFO:tasks.workunit.client.0.smithi007.stdout:3/144: unlink d15/d18/f1d 0 2024-03-16T09:01:01.774 INFO:tasks.workunit.client.0.smithi007.stdout:3/145: creat d15/d18/f23 x:0 0 0 2024-03-16T09:01:01.776 INFO:tasks.workunit.client.0.smithi007.stdout:6/131: rename d2/d13/c1d to d2/d13/c2c 0 2024-03-16T09:01:01.776 INFO:tasks.workunit.client.0.smithi007.stdout:6/132: getdents d2/d10 0 2024-03-16T09:01:01.778 INFO:tasks.workunit.client.0.smithi007.stdout:3/146: symlink d15/d18/l24 0 2024-03-16T09:01:01.778 INFO:tasks.workunit.client.0.smithi007.stdout:3/147: chown d15/d18/d1a 3804436 1 2024-03-16T09:01:01.779 INFO:tasks.workunit.client.0.smithi007.stdout:6/133: mknod d2/d13/c2d 0 2024-03-16T09:01:01.780 INFO:tasks.workunit.client.0.smithi007.stdout:4/93: dwrite d4/d14/f19 [0,4194304] 0 2024-03-16T09:01:01.783 INFO:tasks.workunit.client.0.smithi007.stdout:4/94: mkdir d4/d14/d1a 0 2024-03-16T09:01:01.784 INFO:tasks.workunit.client.0.smithi007.stdout:2/134: dwrite dc/df/f18 [0,4194304] 0 2024-03-16T09:01:01.787 INFO:tasks.workunit.client.0.smithi007.stdout:1/83: dwrite d9/f18 [0,4194304] 0 2024-03-16T09:01:01.788 INFO:tasks.workunit.client.0.smithi007.stdout:2/135: rename dc/l13 to dc/l2a 0 2024-03-16T09:01:01.788 INFO:tasks.workunit.client.0.smithi007.stdout:2/136: creat dc/df/f2b x:0 0 0 2024-03-16T09:01:01.788 INFO:tasks.workunit.client.0.smithi007.stdout:2/137: fsync dc/f10 0 2024-03-16T09:01:01.803 INFO:tasks.workunit.client.0.smithi007.stdout:4/95: write f2 [3273262,68295] 0 2024-03-16T09:01:01.804 INFO:tasks.workunit.client.0.smithi007.stdout:4/96: creat d4/d14/f1b x:0 0 0 2024-03-16T09:01:01.805 INFO:tasks.workunit.client.0.smithi007.stdout:4/97: rename d4/d14/d1a to d4/d14/d1c 0 2024-03-16T09:01:01.806 INFO:tasks.workunit.client.0.smithi007.stdout:2/138: dread dc/df/f14 [0,4194304] 0 2024-03-16T09:01:01.806 INFO:tasks.workunit.client.0.smithi007.stdout:4/98: creat d4/d7/f1d x:0 0 0 2024-03-16T09:01:01.807 INFO:tasks.workunit.client.0.smithi007.stdout:2/139: mknod dc/df/d24/c2c 0 2024-03-16T09:01:01.807 INFO:tasks.workunit.client.0.smithi007.stdout:2/140: write dc/df/f18 [4377109,39758] 0 2024-03-16T09:01:01.807 INFO:tasks.workunit.client.0.smithi007.stdout:2/141: creat dc/df/f2d x:0 0 0 2024-03-16T09:01:01.808 INFO:tasks.workunit.client.0.smithi007.stdout:4/99: mkdir d4/d14/d1c/d1e 0 2024-03-16T09:01:01.811 INFO:tasks.workunit.client.0.smithi007.stdout:9/171: dwrite ff [0,4194304] 0 2024-03-16T09:01:01.811 INFO:tasks.workunit.client.0.smithi007.stdout:6/134: dwrite d2/d11/f25 [0,4194304] 0 2024-03-16T09:01:01.811 INFO:tasks.workunit.client.0.smithi007.stdout:6/135: chown d2/c3 1 1 2024-03-16T09:01:01.811 INFO:tasks.workunit.client.0.smithi007.stdout:6/136: write d2/fb [1036174,55399] 0 2024-03-16T09:01:01.812 INFO:tasks.workunit.client.0.smithi007.stdout:5/121: dwrite d1/d7/d8/d1d/f1f [0,4194304] 0 2024-03-16T09:01:01.812 INFO:tasks.workunit.client.0.smithi007.stdout:5/122: fsync d1/d7/d8/de/f1a 0 2024-03-16T09:01:01.812 INFO:tasks.workunit.client.0.smithi007.stdout:2/142: mkdir dc/df/d24/d28/d2e 0 2024-03-16T09:01:01.812 INFO:tasks.workunit.client.0.smithi007.stdout:4/100: chown d4/f17 776 1 2024-03-16T09:01:01.813 INFO:tasks.workunit.client.0.smithi007.stdout:9/172: truncate d24/f2c 500594 0 2024-03-16T09:01:01.813 INFO:tasks.workunit.client.0.smithi007.stdout:6/137: dread d2/f20 [0,4194304] 0 2024-03-16T09:01:01.813 INFO:tasks.workunit.client.0.smithi007.stdout:6/138: chown d2/lf 14908460 1 2024-03-16T09:01:01.813 INFO:tasks.workunit.client.0.smithi007.stdout:6/139: chown d2/d10/d12/c16 142695 1 2024-03-16T09:01:01.814 INFO:tasks.workunit.client.0.smithi007.stdout:9/173: dread d24/f25 [0,4194304] 0 2024-03-16T09:01:01.818 INFO:tasks.workunit.client.0.smithi007.stdout:1/84: rmdir d9/dd 39 2024-03-16T09:01:01.818 INFO:tasks.workunit.client.0.smithi007.stdout:1/85: chown d9 2299805 1 2024-03-16T09:01:01.820 INFO:tasks.workunit.client.0.smithi007.stdout:5/123: mknod d1/d7/d1c/c27 0 2024-03-16T09:01:01.821 INFO:tasks.workunit.client.0.smithi007.stdout:6/140: dread d2/f7 [0,4194304] 0 2024-03-16T09:01:01.828 INFO:tasks.workunit.client.0.smithi007.stdout:4/101: chown d4/ff 8254 1 2024-03-16T09:01:01.829 INFO:tasks.workunit.client.0.smithi007.stdout:2/143: mknod dc/df/c2f 0 2024-03-16T09:01:01.831 INFO:tasks.workunit.client.0.smithi007.stdout:9/174: unlink f1a 0 2024-03-16T09:01:01.831 INFO:tasks.workunit.client.0.smithi007.stdout:9/175: chown f7 36355620 1 2024-03-16T09:01:01.834 INFO:tasks.workunit.client.0.smithi007.stdout:6/141: mknod d2/c2e 0 2024-03-16T09:01:01.834 INFO:tasks.workunit.client.0.smithi007.stdout:6/142: chown d2/d11/f21 111 1 2024-03-16T09:01:01.835 INFO:tasks.workunit.client.0.smithi007.stdout:3/148: rmdir d15 39 2024-03-16T09:01:01.837 INFO:tasks.workunit.client.0.smithi007.stdout:2/144: symlink dc/df/l30 0 2024-03-16T09:01:01.837 INFO:tasks.workunit.client.0.smithi007.stdout:2/145: write dc/f11 [356449,120709] 0 2024-03-16T09:01:01.838 INFO:tasks.workunit.client.0.smithi007.stdout:2/146: chown dc/df/d24 0 1 2024-03-16T09:01:01.838 INFO:tasks.workunit.client.0.smithi007.stdout:2/147: fdatasync dc/f1a 0 2024-03-16T09:01:01.838 INFO:tasks.workunit.client.0.smithi007.stdout:9/176: unlink f1f 0 2024-03-16T09:01:01.838 INFO:tasks.workunit.client.0.smithi007.stdout:2/148: dread dc/f1a [0,4194304] 0 2024-03-16T09:01:01.839 INFO:tasks.workunit.client.0.smithi007.stdout:1/86: mkdir d9/dd/d1b 0 2024-03-16T09:01:01.840 INFO:tasks.workunit.client.0.smithi007.stdout:5/124: getdents d1/d7/d8/de 0 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:2/149: write f6 [1421428,56338] 0 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:6/143: getdents d2/d13/d1f 0 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:3/149: rmdir d15/d18 39 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:5/125: mkdir d1/d7/d8/d28 0 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:1/87: rename d9 to d9/dd/d1c 22 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:1/88: chown f5 30 1 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:1/89: write d9/f13 [345366,33385] 0 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:1/90: chown d9/f13 20 1 2024-03-16T09:01:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:1/91: write d9/f19 [999803,112075] 0 2024-03-16T09:01:01.852 INFO:tasks.workunit.client.0.smithi007.stdout:4/102: write d4/f6 [3801440,21955] 0 2024-03-16T09:01:01.852 INFO:tasks.workunit.client.0.smithi007.stdout:4/103: chown d4/d7 77788930 1 2024-03-16T09:01:01.852 INFO:tasks.workunit.client.0.smithi007.stdout:3/150: creat d15/d18/d1a/f25 x:0 0 0 2024-03-16T09:01:01.852 INFO:tasks.workunit.client.0.smithi007.stdout:2/150: dread dc/fd [0,4194304] 0 2024-03-16T09:01:01.852 INFO:tasks.workunit.client.0.smithi007.stdout:6/144: link d2/f7 d2/d11/d1e/f2f 0 2024-03-16T09:01:01.854 INFO:tasks.workunit.client.0.smithi007.stdout:5/126: rename d1/l25 to d1/d7/d8/d28/l29 0 2024-03-16T09:01:01.855 INFO:tasks.workunit.client.0.smithi007.stdout:4/104: mknod d4/d14/d1c/c1f 0 2024-03-16T09:01:01.856 INFO:tasks.workunit.client.0.smithi007.stdout:9/177: dread f8 [4194304,4194304] 0 2024-03-16T09:01:01.857 INFO:tasks.workunit.client.0.smithi007.stdout:3/151: rmdir d15 39 2024-03-16T09:01:01.857 INFO:tasks.workunit.client.0.smithi007.stdout:3/152: chown f14 55507187 1 2024-03-16T09:01:01.860 INFO:tasks.workunit.client.0.smithi007.stdout:6/145: symlink d2/d11/l30 0 2024-03-16T09:01:01.860 INFO:tasks.workunit.client.0.smithi007.stdout:6/146: truncate d2/d10/d12/d15/f22 1040049 0 2024-03-16T09:01:01.861 INFO:tasks.workunit.client.0.smithi007.stdout:5/127: truncate d1/d7/d1c/f15 3534474 0 2024-03-16T09:01:01.861 INFO:tasks.workunit.client.0.smithi007.stdout:5/128: write d1/fa [1489769,22611] 0 2024-03-16T09:01:01.864 INFO:tasks.workunit.client.0.smithi007.stdout:4/105: rename d4/d7/fc to d4/d7/f20 0 2024-03-16T09:01:01.865 INFO:tasks.workunit.client.0.smithi007.stdout:9/178: creat d24/f31 x:0 0 0 2024-03-16T09:01:01.867 INFO:tasks.workunit.client.0.smithi007.stdout:1/92: dwrite d9/f12 [0,4194304] 0 2024-03-16T09:01:01.868 INFO:tasks.workunit.client.0.smithi007.stdout:3/153: link f8 d15/d18/d1f/f26 0 2024-03-16T09:01:01.868 INFO:tasks.workunit.client.0.smithi007.stdout:3/154: write f10 [432757,40532] 0 2024-03-16T09:01:01.870 INFO:tasks.workunit.client.0.smithi007.stdout:6/147: symlink d2/d10/d12/l31 0 2024-03-16T09:01:01.871 INFO:tasks.workunit.client.0.smithi007.stdout:6/148: write d2/d11/f26 [642557,112687] 0 2024-03-16T09:01:01.873 INFO:tasks.workunit.client.0.smithi007.stdout:9/179: creat d24/f32 x:0 0 0 2024-03-16T09:01:01.873 INFO:tasks.workunit.client.0.smithi007.stdout:9/180: write d24/f2c [1289675,15517] 0 2024-03-16T09:01:01.875 INFO:tasks.workunit.client.0.smithi007.stdout:1/93: dread f0 [0,4194304] 0 2024-03-16T09:01:01.876 INFO:tasks.workunit.client.0.smithi007.stdout:2/151: dwrite dc/f10 [0,4194304] 0 2024-03-16T09:01:01.885 INFO:tasks.workunit.client.0.smithi007.stdout:3/155: rename l9 to d15/d18/l27 0 2024-03-16T09:01:01.887 INFO:tasks.workunit.client.0.smithi007.stdout:9/181: creat d24/f33 x:0 0 0 2024-03-16T09:01:01.890 INFO:tasks.workunit.client.0.smithi007.stdout:1/94: rename d9/l10 to d9/l1d 0 2024-03-16T09:01:01.892 INFO:tasks.workunit.client.0.smithi007.stdout:2/152: rename dc/l1e to dc/df/d24/d28/l31 0 2024-03-16T09:01:01.893 INFO:tasks.workunit.client.0.smithi007.stdout:6/149: dwrite d2/d10/d12/d15/f22 [0,4194304] 0 2024-03-16T09:01:01.894 INFO:tasks.workunit.client.0.smithi007.stdout:2/153: dread dc/f17 [0,4194304] 0 2024-03-16T09:01:01.894 INFO:tasks.workunit.client.0.smithi007.stdout:2/154: getdents dc/df/d24/d28/d2e 0 2024-03-16T09:01:01.903 INFO:tasks.workunit.client.0.smithi007.stdout:3/156: dwrite d15/d18/d1f/f26 [0,4194304] 0 2024-03-16T09:01:01.903 INFO:tasks.workunit.client.0.smithi007.stdout:9/182: write f7 [4168103,44367] 0 2024-03-16T09:01:01.904 INFO:tasks.workunit.client.0.smithi007.stdout:2/155: read dc/df/f14 [3652133,48424] 0 2024-03-16T09:01:01.904 INFO:tasks.workunit.client.0.smithi007.stdout:6/150: link d2/lf d2/d11/l32 0 2024-03-16T09:01:01.906 INFO:tasks.workunit.client.0.smithi007.stdout:3/157: creat d15/f28 x:0 0 0 2024-03-16T09:01:01.906 INFO:tasks.workunit.client.0.smithi007.stdout:3/158: write d15/d18/f19 [530422,99443] 0 2024-03-16T09:01:01.906 INFO:tasks.workunit.client.0.smithi007.stdout:3/159: fsync d15/d18/f23 0 2024-03-16T09:01:01.907 INFO:tasks.workunit.client.0.smithi007.stdout:2/156: mknod dc/d27/c32 0 2024-03-16T09:01:01.910 INFO:tasks.workunit.client.0.smithi007.stdout:6/151: rename d2/d10/c14 to d2/d13/d24/c33 0 2024-03-16T09:01:01.911 INFO:tasks.workunit.client.0.smithi007.stdout:2/157: link lb dc/df/l33 0 2024-03-16T09:01:01.911 INFO:tasks.workunit.client.0.smithi007.stdout:2/158: creat dc/df/f34 x:0 0 0 2024-03-16T09:01:01.915 INFO:tasks.workunit.client.0.smithi007.stdout:1/95: dwrite d9/f18 [0,4194304] 0 2024-03-16T09:01:01.915 INFO:tasks.workunit.client.0.smithi007.stdout:6/152: truncate d2/d11/f21 1552383 0 2024-03-16T09:01:01.920 INFO:tasks.workunit.client.0.smithi007.stdout:6/153: mkdir d2/d13/d1f/d34 0 2024-03-16T09:01:01.920 INFO:tasks.workunit.client.0.smithi007.stdout:3/160: rename d15/d18/d1a to d15/d18/d1f/d29 0 2024-03-16T09:01:01.920 INFO:tasks.workunit.client.0.smithi007.stdout:3/161: chown d15/d18/f22 0 1 2024-03-16T09:01:01.922 INFO:tasks.workunit.client.0.smithi007.stdout:6/154: creat d2/d10/d12/f35 x:0 0 0 2024-03-16T09:01:01.922 INFO:tasks.workunit.client.0.smithi007.stdout:1/96: dread d9/f13 [0,4194304] 0 2024-03-16T09:01:01.922 INFO:tasks.workunit.client.0.smithi007.stdout:1/97: write d9/dd/f14 [162358,87216] 0 2024-03-16T09:01:01.924 INFO:tasks.workunit.client.0.smithi007.stdout:6/155: link d2/fa d2/d10/f36 0 2024-03-16T09:01:01.925 INFO:tasks.workunit.client.0.smithi007.stdout:1/98: mkdir d9/d1e 0 2024-03-16T09:01:01.933 INFO:tasks.workunit.client.0.smithi007.stdout:1/99: dread d9/f12 [0,4194304] 0 2024-03-16T09:01:01.933 INFO:tasks.workunit.client.0.smithi007.stdout:1/100: chown f7 101166110 1 2024-03-16T09:01:01.934 INFO:tasks.workunit.client.0.smithi007.stdout:1/101: truncate d9/f19 1854408 0 2024-03-16T09:01:01.934 INFO:tasks.workunit.client.0.smithi007.stdout:9/183: dwrite d24/f27 [0,4194304] 0 2024-03-16T09:01:01.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/159: dwrite dc/df/f34 [0,4194304] 0 2024-03-16T09:01:01.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/160: readlink dc/df/l12 0 2024-03-16T09:01:01.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/161: readlink lb 0 2024-03-16T09:01:01.935 INFO:tasks.workunit.client.0.smithi007.stdout:3/162: read f13 [2332111,88937] 0 2024-03-16T09:01:01.940 INFO:tasks.workunit.client.0.smithi007.stdout:6/156: dread d2/fb [0,4194304] 0 2024-03-16T09:01:01.944 INFO:tasks.workunit.client.0.smithi007.stdout:1/102: dread f7 [0,4194304] 0 2024-03-16T09:01:01.944 INFO:tasks.workunit.client.0.smithi007.stdout:3/163: creat d15/d18/d1f/d29/f2a x:0 0 0 2024-03-16T09:01:01.946 INFO:tasks.workunit.client.0.smithi007.stdout:6/157: mknod d2/d29/c37 0 2024-03-16T09:01:01.952 INFO:tasks.workunit.client.0.smithi007.stdout:6/158: getdents d2/d10/d12/d15 0 2024-03-16T09:01:01.954 INFO:tasks.workunit.client.0.smithi007.stdout:9/184: dwrite f1e [4194304,4194304] 0 2024-03-16T09:01:01.954 INFO:tasks.workunit.client.0.smithi007.stdout:9/185: write f1c [953760,124448] 0 2024-03-16T09:01:01.957 INFO:tasks.workunit.client.0.smithi007.stdout:9/186: creat d24/f34 x:0 0 0 2024-03-16T09:01:01.960 INFO:tasks.workunit.client.0.smithi007.stdout:9/187: symlink d24/l35 0 2024-03-16T09:01:01.962 INFO:tasks.workunit.client.0.smithi007.stdout:9/188: mknod d24/c36 0 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 9.5 MiB/s wr, 1.05k op/s 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: Adjusting osd_memory_target on smithi019 to 2583M 2024-03-16T09:01:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:01.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:01:01.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:01:01.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T09:01:01.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:01 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T09:01:01.964 INFO:tasks.workunit.client.0.smithi007.stdout:6/159: read d2/fa [2445143,24577] 0 2024-03-16T09:01:01.965 INFO:tasks.workunit.client.0.smithi007.stdout:9/189: rename d24/l2e to d24/l37 0 2024-03-16T09:01:01.966 INFO:tasks.workunit.client.0.smithi007.stdout:6/160: creat d2/d13/d24/f38 x:0 0 0 2024-03-16T09:01:01.966 INFO:tasks.workunit.client.0.smithi007.stdout:6/161: fsync d2/fd 0 2024-03-16T09:01:01.967 INFO:tasks.workunit.client.0.smithi007.stdout:9/190: creat d24/f38 x:0 0 0 2024-03-16T09:01:01.967 INFO:tasks.workunit.client.0.smithi007.stdout:9/191: truncate d24/f2a 395756 0 2024-03-16T09:01:01.968 INFO:tasks.workunit.client.0.smithi007.stdout:6/162: mknod d2/d13/d1f/d34/c39 0 2024-03-16T09:01:01.969 INFO:tasks.workunit.client.0.smithi007.stdout:9/192: creat d24/f39 x:0 0 0 2024-03-16T09:01:01.969 INFO:tasks.workunit.client.0.smithi007.stdout:9/193: stat d24/c28 0 2024-03-16T09:01:01.974 INFO:tasks.workunit.client.0.smithi007.stdout:3/164: dwrite d15/d18/f22 [0,4194304] 0 2024-03-16T09:01:01.974 INFO:tasks.workunit.client.0.smithi007.stdout:1/103: dwrite d9/f12 [0,4194304] 0 2024-03-16T09:01:01.974 INFO:tasks.workunit.client.0.smithi007.stdout:1/104: fsync d9/f18 0 2024-03-16T09:01:01.975 INFO:tasks.workunit.client.0.smithi007.stdout:1/105: read - d9/f15 zero size 2024-03-16T09:01:01.975 INFO:tasks.workunit.client.0.smithi007.stdout:1/106: readlink d9/l1d 0 2024-03-16T09:01:01.975 INFO:tasks.workunit.client.0.smithi007.stdout:1/107: chown d9/dd/ce 1283396 1 2024-03-16T09:01:01.976 INFO:tasks.workunit.client.0.smithi007.stdout:3/165: getdents d15/d18/d1f 0 2024-03-16T09:01:01.977 INFO:tasks.workunit.client.0.smithi007.stdout:3/166: mknod d15/d18/c2b 0 2024-03-16T09:01:01.978 INFO:tasks.workunit.client.0.smithi007.stdout:9/194: dread f1e [0,4194304] 0 2024-03-16T09:01:01.982 INFO:tasks.workunit.client.0.smithi007.stdout:9/195: symlink d24/l3a 0 2024-03-16T09:01:01.984 INFO:tasks.workunit.client.0.smithi007.stdout:9/196: symlink d24/l3b 0 2024-03-16T09:01:01.987 INFO:tasks.workunit.client.0.smithi007.stdout:9/197: unlink d24/f32 0 2024-03-16T09:01:01.994 INFO:tasks.workunit.client.0.smithi007.stdout:9/198: getdents d24 0 2024-03-16T09:01:01.995 INFO:tasks.workunit.client.0.smithi007.stdout:9/199: creat d24/f3c x:0 0 0 2024-03-16T09:01:01.995 INFO:tasks.workunit.client.0.smithi007.stdout:1/108: dwrite d9/f19 [0,4194304] 0 2024-03-16T09:01:01.996 INFO:tasks.workunit.client.0.smithi007.stdout:9/200: creat d24/f3d x:0 0 0 2024-03-16T09:01:01.998 INFO:tasks.workunit.client.0.smithi007.stdout:6/163: dwrite d2/fa [0,4194304] 0 2024-03-16T09:01:01.998 INFO:tasks.workunit.client.0.smithi007.stdout:1/109: mknod d9/dd/d1b/c1f 0 2024-03-16T09:01:01.998 INFO:tasks.workunit.client.0.smithi007.stdout:1/110: truncate d9/f15 522152 0 2024-03-16T09:01:01.998 INFO:tasks.workunit.client.0.smithi007.stdout:3/167: dwrite d15/d18/d1f/d29/f2a [0,4194304] 0 2024-03-16T09:01:01.998 INFO:tasks.workunit.client.0.smithi007.stdout:3/168: fdatasync f10 0 2024-03-16T09:01:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 9.5 MiB/s wr, 1.05k op/s 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: Adjusting osd_memory_target on smithi019 to 2583M 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:01:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T09:01:02.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:01 smithi007 ceph-mon[29470]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T09:01:02.000 INFO:tasks.workunit.client.0.smithi007.stdout:6/164: creat d2/d10/f3a x:0 0 0 2024-03-16T09:01:02.000 INFO:tasks.workunit.client.0.smithi007.stdout:1/111: rmdir d9/d1e 0 2024-03-16T09:01:02.004 INFO:tasks.workunit.client.0.smithi007.stdout:3/169: rename d15/f28 to d15/f2c 0 2024-03-16T09:01:02.008 INFO:tasks.workunit.client.0.smithi007.stdout:1/112: symlink d9/l20 0 2024-03-16T09:01:02.009 INFO:tasks.workunit.client.0.smithi007.stdout:1/113: creat d9/f21 x:0 0 0 2024-03-16T09:01:02.012 INFO:tasks.workunit.client.0.smithi007.stdout:9/201: truncate f1c 1056858 0 2024-03-16T09:01:02.015 INFO:tasks.workunit.client.0.smithi007.stdout:9/202: rmdir d24 39 2024-03-16T09:01:02.017 INFO:tasks.workunit.client.0.smithi007.stdout:1/114: dread f7 [0,4194304] 0 2024-03-16T09:01:02.017 INFO:tasks.workunit.client.0.smithi007.stdout:1/115: fsync d9/f18 0 2024-03-16T09:01:02.017 INFO:tasks.workunit.client.0.smithi007.stdout:1/116: creat d9/f22 x:0 0 0 2024-03-16T09:01:02.017 INFO:tasks.workunit.client.0.smithi007.stdout:1/117: write f7 [4168780,89627] 0 2024-03-16T09:01:02.017 INFO:tasks.workunit.client.0.smithi007.stdout:9/203: rename l18 to d24/l3e 0 2024-03-16T09:01:02.018 INFO:tasks.workunit.client.0.smithi007.stdout:6/165: dwrite d2/f20 [0,4194304] 0 2024-03-16T09:01:02.019 INFO:tasks.workunit.client.0.smithi007.stdout:1/118: link d9/ca d9/c23 0 2024-03-16T09:01:02.020 INFO:tasks.workunit.client.0.smithi007.stdout:3/170: dwrite d15/d18/d1f/d29/f2a [0,4194304] 0 2024-03-16T09:01:02.020 INFO:tasks.workunit.client.0.smithi007.stdout:3/171: stat d15/d18/d1f/d29/f1b 0 2024-03-16T09:01:02.021 INFO:tasks.workunit.client.0.smithi007.stdout:9/204: creat d24/f3f x:0 0 0 2024-03-16T09:01:02.025 INFO:tasks.workunit.client.0.smithi007.stdout:6/166: mkdir d2/d11/d3b 0 2024-03-16T09:01:02.027 INFO:tasks.workunit.client.0.smithi007.stdout:1/119: link d9/f18 d9/f24 0 2024-03-16T09:01:02.028 INFO:tasks.workunit.client.0.smithi007.stdout:6/167: dread d2/fd [0,4194304] 0 2024-03-16T09:01:02.029 INFO:tasks.workunit.client.0.smithi007.stdout:3/172: creat d15/d18/d1f/d29/f2d x:0 0 0 2024-03-16T09:01:02.032 INFO:tasks.workunit.client.0.smithi007.stdout:3/173: link d15/d18/d1f/f26 d15/d18/d1f/f2e 0 2024-03-16T09:01:02.032 INFO:tasks.workunit.client.0.smithi007.stdout:6/168: dread d2/d11/f21 [0,4194304] 0 2024-03-16T09:01:02.035 INFO:tasks.workunit.client.0.smithi007.stdout:6/169: symlink d2/d13/l3c 0 2024-03-16T09:01:02.035 INFO:tasks.workunit.client.0.smithi007.stdout:6/170: write d2/fd [1643144,5454] 0 2024-03-16T09:01:02.035 INFO:tasks.workunit.client.0.smithi007.stdout:6/171: fdatasync d2/d10/d12/d15/f28 0 2024-03-16T09:01:02.035 INFO:tasks.workunit.client.0.smithi007.stdout:6/172: readlink d2/d11/l30 0 2024-03-16T09:01:02.044 INFO:tasks.workunit.client.0.smithi007.stdout:3/174: truncate d15/d18/d1f/f2e 2572696 0 2024-03-16T09:01:02.048 INFO:tasks.workunit.client.0.smithi007.stdout:3/175: mknod d15/c2f 0 2024-03-16T09:01:02.048 INFO:tasks.workunit.client.0.smithi007.stdout:3/176: chown d15/f16 4623706 1 2024-03-16T09:01:02.062 INFO:tasks.workunit.client.0.smithi007.stdout:6/173: dwrite d2/d11/f26 [0,4194304] 0 2024-03-16T09:01:02.064 INFO:tasks.workunit.client.0.smithi007.stdout:6/174: mkdir d2/d10/d3d 0 2024-03-16T09:01:02.066 INFO:tasks.workunit.client.0.smithi007.stdout:1/120: dwrite d9/f24 [0,4194304] 0 2024-03-16T09:01:02.066 INFO:tasks.workunit.client.0.smithi007.stdout:1/121: chown d9/f22 154 1 2024-03-16T09:01:02.066 INFO:tasks.workunit.client.0.smithi007.stdout:6/175: getdents d2/d13 0 2024-03-16T09:01:02.067 INFO:tasks.workunit.client.0.smithi007.stdout:1/122: rename d9/f15 to d9/dd/d1b/f25 0 2024-03-16T09:01:02.069 INFO:tasks.workunit.client.0.smithi007.stdout:9/205: dwrite d24/f3f [0,4194304] 0 2024-03-16T09:01:02.069 INFO:tasks.workunit.client.0.smithi007.stdout:9/206: fsync f16 0 2024-03-16T09:01:02.072 INFO:tasks.workunit.client.0.smithi007.stdout:3/177: dwrite f7 [0,4194304] 0 2024-03-16T09:01:02.081 INFO:tasks.workunit.client.0.smithi007.stdout:9/207: read - d24/f31 zero size 2024-03-16T09:01:02.081 INFO:tasks.workunit.client.0.smithi007.stdout:9/208: readlink l21 0 2024-03-16T09:01:02.084 INFO:tasks.workunit.client.0.smithi007.stdout:3/178: unlink d15/d18/d1f/d29/f2a 0 2024-03-16T09:01:02.084 INFO:tasks.workunit.client.0.smithi007.stdout:3/179: fsync d15/d18/f22 0 2024-03-16T09:01:02.084 INFO:tasks.workunit.client.0.smithi007.stdout:6/176: dwrite d2/fb [0,4194304] 0 2024-03-16T09:01:02.084 INFO:tasks.workunit.client.0.smithi007.stdout:9/209: fsync d24/f33 0 2024-03-16T09:01:02.084 INFO:tasks.workunit.client.0.smithi007.stdout:9/210: chown l12 16072715 1 2024-03-16T09:01:02.084 INFO:tasks.workunit.client.0.smithi007.stdout:9/211: chown f8 55865794 1 2024-03-16T09:01:02.085 INFO:tasks.workunit.client.0.smithi007.stdout:3/180: mknod d15/c30 0 2024-03-16T09:01:02.086 INFO:tasks.workunit.client.0.smithi007.stdout:6/177: mknod d2/d13/d24/c3e 0 2024-03-16T09:01:02.087 INFO:tasks.workunit.client.0.smithi007.stdout:9/212: dread - d24/f38 zero size 2024-03-16T09:01:02.088 INFO:tasks.workunit.client.0.smithi007.stdout:6/178: mkdir d2/d13/d3f 0 2024-03-16T09:01:02.090 INFO:tasks.workunit.client.0.smithi007.stdout:9/213: mknod d24/c40 0 2024-03-16T09:01:02.091 INFO:tasks.workunit.client.0.smithi007.stdout:9/214: truncate d24/f2a 858593 0 2024-03-16T09:01:02.093 INFO:tasks.workunit.client.0.smithi007.stdout:9/215: symlink d24/l41 0 2024-03-16T09:01:02.136 INFO:tasks.workunit.client.0.smithi007.stdout:8/150: sync 2024-03-16T09:01:02.136 INFO:tasks.workunit.client.0.smithi007.stdout:7/123: sync 2024-03-16T09:01:02.136 INFO:tasks.workunit.client.0.smithi007.stdout:0/78: sync 2024-03-16T09:01:02.136 INFO:tasks.workunit.client.0.smithi007.stdout:0/79: dread - d3/d9/da/d18/f19 zero size 2024-03-16T09:01:02.136 INFO:tasks.workunit.client.0.smithi007.stdout:8/151: getdents d0 0 2024-03-16T09:01:02.137 INFO:tasks.workunit.client.0.smithi007.stdout:8/152: getdents d0/d26 0 2024-03-16T09:01:02.139 INFO:tasks.workunit.client.0.smithi007.stdout:8/153: link d0/f22 d0/d26/f2f 0 2024-03-16T09:01:02.141 INFO:tasks.workunit.client.0.smithi007.stdout:8/154: rename d0/dc/f2a to d0/d26/f30 0 2024-03-16T09:01:02.159 INFO:tasks.workunit.client.0.smithi007.stdout:0/80: dwrite d3/d9/f11 [0,4194304] 0 2024-03-16T09:01:02.160 INFO:tasks.workunit.client.0.smithi007.stdout:7/124: dwrite d0/fa [0,4194304] 0 2024-03-16T09:01:02.160 INFO:tasks.workunit.client.0.smithi007.stdout:7/125: write d0/d19/f20 [2020501,6298] 0 2024-03-16T09:01:02.160 INFO:tasks.workunit.client.0.smithi007.stdout:8/155: dwrite d0/d2/d18/f19 [0,4194304] 0 2024-03-16T09:01:02.160 INFO:tasks.workunit.client.0.smithi007.stdout:8/156: dread - d0/dc/dd/f2c zero size 2024-03-16T09:01:02.163 INFO:tasks.workunit.client.0.smithi007.stdout:7/126: symlink d0/dc/d24/l2a 0 2024-03-16T09:01:02.171 INFO:tasks.workunit.client.0.smithi007.stdout:7/127: dread d0/f6 [0,4194304] 0 2024-03-16T09:01:02.173 INFO:tasks.workunit.client.0.smithi007.stdout:7/128: truncate d0/dc/fd 262259 0 2024-03-16T09:01:02.176 INFO:tasks.workunit.client.0.smithi007.stdout:0/81: dwrite d3/d9/db/de/f13 [0,4194304] 0 2024-03-16T09:01:02.188 INFO:tasks.workunit.client.0.smithi007.stdout:8/157: dwrite d0/d2/fa [0,4194304] 0 2024-03-16T09:01:02.188 INFO:tasks.workunit.client.0.smithi007.stdout:8/158: truncate d0/dc/dd/f2c 25969 0 2024-03-16T09:01:02.201 INFO:tasks.workunit.client.0.smithi007.stdout:7/129: dwrite d0/d19/f27 [0,4194304] 0 2024-03-16T09:01:02.201 INFO:tasks.workunit.client.0.smithi007.stdout:7/130: write d0/dc/fe [3480437,55243] 0 2024-03-16T09:01:02.204 INFO:tasks.workunit.client.0.smithi007.stdout:0/82: dwrite d3/d9/db/fc [0,4194304] 0 2024-03-16T09:01:02.205 INFO:tasks.workunit.client.0.smithi007.stdout:7/131: rename d0/f28 to d0/d19/f2b 0 2024-03-16T09:01:02.205 INFO:tasks.workunit.client.0.smithi007.stdout:7/132: chown d0/dc/d24/d25/c22 1484 1 2024-03-16T09:01:02.207 INFO:tasks.workunit.client.0.smithi007.stdout:8/159: dwrite d0/dc/f23 [0,4194304] 0 2024-03-16T09:01:02.209 INFO:tasks.workunit.client.0.smithi007.stdout:0/83: creat d3/d9/da/f1e x:0 0 0 2024-03-16T09:01:02.209 INFO:tasks.workunit.client.0.smithi007.stdout:0/84: readlink d3/d9/da/l15 0 2024-03-16T09:01:02.209 INFO:tasks.workunit.client.0.smithi007.stdout:7/133: dread - d0/f18 zero size 2024-03-16T09:01:02.209 INFO:tasks.workunit.client.0.smithi007.stdout:7/134: chown d0/dc/d24 9 1 2024-03-16T09:01:02.213 INFO:tasks.workunit.client.0.smithi007.stdout:8/160: creat d0/f31 x:0 0 0 2024-03-16T09:01:02.214 INFO:tasks.workunit.client.0.smithi007.stdout:0/85: mkdir d3/d1f 0 2024-03-16T09:01:02.214 INFO:tasks.workunit.client.0.smithi007.stdout:0/86: write d3/f8 [4619751,52384] 0 2024-03-16T09:01:02.214 INFO:tasks.workunit.client.0.smithi007.stdout:0/87: chown d3/d9/da/d10/d1a 3797282 1 2024-03-16T09:01:02.214 INFO:tasks.workunit.client.0.smithi007.stdout:0/88: creat d3/d9/da/d10/d1a/f20 x:0 0 0 2024-03-16T09:01:02.217 INFO:tasks.workunit.client.0.smithi007.stdout:0/89: rmdir d3/d9/db 39 2024-03-16T09:01:02.219 INFO:tasks.workunit.client.0.smithi007.stdout:8/161: creat d0/d26/d2d/f32 x:0 0 0 2024-03-16T09:01:02.219 INFO:tasks.workunit.client.0.smithi007.stdout:8/162: chown d0/dc/dd/c2b 8092 1 2024-03-16T09:01:02.225 INFO:tasks.workunit.client.0.smithi007.stdout:8/163: dread d0/d2/f17 [0,4194304] 0 2024-03-16T09:01:02.226 INFO:tasks.workunit.client.0.smithi007.stdout:8/164: mknod d0/dc/c33 0 2024-03-16T09:01:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:0/90: dread d3/f8 [0,4194304] 0 2024-03-16T09:01:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:0/91: write d3/d9/db/de/f12 [790739,71195] 0 2024-03-16T09:01:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:0/92: truncate d3/d9/da/f1e 467071 0 2024-03-16T09:01:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:8/165: rename d0/c1d to d0/c34 0 2024-03-16T09:01:02.228 INFO:tasks.workunit.client.0.smithi007.stdout:7/135: dwrite d0/dc/f1a [0,4194304] 0 2024-03-16T09:01:02.229 INFO:tasks.workunit.client.0.smithi007.stdout:8/166: creat d0/dc/f35 x:0 0 0 2024-03-16T09:01:02.234 INFO:tasks.workunit.client.0.smithi007.stdout:0/93: rename d3/d9/db/de to d3/d9/da/d18/d21 0 2024-03-16T09:01:02.238 INFO:tasks.workunit.client.0.smithi007.stdout:8/167: read d0/d2/d18/f19 [1645966,11469] 0 2024-03-16T09:01:02.238 INFO:tasks.workunit.client.0.smithi007.stdout:8/168: write d0/d2/d18/f19 [3409943,31778] 0 2024-03-16T09:01:02.245 INFO:tasks.workunit.client.0.smithi007.stdout:7/136: dwrite d0/f3 [0,4194304] 0 2024-03-16T09:01:02.245 INFO:tasks.workunit.client.0.smithi007.stdout:7/137: readlink d0/dc/d24/d25/l23 0 2024-03-16T09:01:02.245 INFO:tasks.workunit.client.0.smithi007.stdout:7/138: truncate d0/d19/f27 4817241 0 2024-03-16T09:01:02.245 INFO:tasks.workunit.client.0.smithi007.stdout:7/139: stat d0/fa 0 2024-03-16T09:01:02.247 INFO:tasks.workunit.client.0.smithi007.stdout:7/140: readlink d0/l14 0 2024-03-16T09:01:02.251 INFO:tasks.workunit.client.0.smithi007.stdout:8/169: read d0/d2/fa [3857531,90838] 0 2024-03-16T09:01:02.252 INFO:tasks.workunit.client.0.smithi007.stdout:7/141: write d0/fa [1028060,101948] 0 2024-03-16T09:01:02.256 INFO:tasks.workunit.client.0.smithi007.stdout:0/94: dwrite d3/f8 [0,4194304] 0 2024-03-16T09:01:02.278 INFO:tasks.workunit.client.0.smithi007.stdout:8/170: dwrite d0/dc/f23 [0,4194304] 0 2024-03-16T09:01:02.279 INFO:tasks.workunit.client.0.smithi007.stdout:0/95: dwrite d3/d9/da/d18/f19 [0,4194304] 0 2024-03-16T09:01:02.279 INFO:tasks.workunit.client.0.smithi007.stdout:7/142: dwrite d0/f6 [0,4194304] 0 2024-03-16T09:01:02.281 INFO:tasks.workunit.client.0.smithi007.stdout:0/96: truncate d3/d9/da/d18/d21/f13 1772214 0 2024-03-16T09:01:02.282 INFO:tasks.workunit.client.0.smithi007.stdout:7/143: unlink d0/f6 0 2024-03-16T09:01:02.284 INFO:tasks.workunit.client.0.smithi007.stdout:0/97: fdatasync d3/d9/da/d18/d21/f12 0 2024-03-16T09:01:02.288 INFO:tasks.workunit.client.0.smithi007.stdout:0/98: creat d3/f22 x:0 0 0 2024-03-16T09:01:02.288 INFO:tasks.workunit.client.0.smithi007.stdout:0/99: chown d3/l7 0 1 2024-03-16T09:01:02.288 INFO:tasks.workunit.client.0.smithi007.stdout:0/100: write d3/f8 [6086685,109760] 0 2024-03-16T09:01:02.289 INFO:tasks.workunit.client.0.smithi007.stdout:0/101: mkdir d3/d9/da/d18/d23 0 2024-03-16T09:01:02.292 INFO:tasks.workunit.client.0.smithi007.stdout:8/171: dwrite d0/fe [0,4194304] 0 2024-03-16T09:01:02.292 INFO:tasks.workunit.client.0.smithi007.stdout:8/172: stat d0/d2/f17 0 2024-03-16T09:01:02.295 INFO:tasks.workunit.client.0.smithi007.stdout:8/173: rename d0/c25 to d0/d26/d2d/c36 0 2024-03-16T09:01:02.295 INFO:tasks.workunit.client.0.smithi007.stdout:8/174: fdatasync d0/fe 0 2024-03-16T09:01:02.296 INFO:tasks.workunit.client.0.smithi007.stdout:8/175: fdatasync d0/fe 0 2024-03-16T09:01:02.296 INFO:tasks.workunit.client.0.smithi007.stdout:8/176: getdents d0/d2/d18 0 2024-03-16T09:01:02.307 INFO:tasks.workunit.client.0.smithi007.stdout:8/177: dwrite d0/f1f [0,4194304] 0 2024-03-16T09:01:02.311 INFO:tasks.workunit.client.0.smithi007.stdout:8/178: link d0/d2/d18/l24 d0/l37 0 2024-03-16T09:01:02.311 INFO:tasks.workunit.client.0.smithi007.stdout:8/179: write d0/f31 [453272,480] 0 2024-03-16T09:01:02.312 INFO:tasks.workunit.client.0.smithi007.stdout:8/180: link d0/d2/l29 d0/d2/d18/l38 0 2024-03-16T09:01:02.313 INFO:tasks.workunit.client.0.smithi007.stdout:8/181: write d0/f22 [343507,95111] 0 2024-03-16T09:01:02.314 INFO:tasks.workunit.client.0.smithi007.stdout:8/182: rename d0/d2/d18/f19 to d0/d26/f39 0 2024-03-16T09:01:02.314 INFO:tasks.workunit.client.0.smithi007.stdout:8/183: read - d0/d26/f30 zero size 2024-03-16T09:01:02.314 INFO:tasks.workunit.client.0.smithi007.stdout:8/184: creat d0/dc/f3a x:0 0 0 2024-03-16T09:01:02.317 INFO:tasks.workunit.client.0.smithi007.stdout:8/185: dread d0/d26/f2f [0,4194304] 0 2024-03-16T09:01:02.319 INFO:tasks.workunit.client.0.smithi007.stdout:8/186: rename d0/d2/f17 to d0/d2/d18/f3b 0 2024-03-16T09:01:02.319 INFO:tasks.workunit.client.0.smithi007.stdout:8/187: creat d0/f3c x:0 0 0 2024-03-16T09:01:02.319 INFO:tasks.workunit.client.0.smithi007.stdout:8/188: chown d0 0 1 2024-03-16T09:01:02.321 INFO:tasks.workunit.client.0.smithi007.stdout:8/189: rename d0/dc/dd/l21 to d0/dc/l3d 0 2024-03-16T09:01:02.321 INFO:tasks.workunit.client.0.smithi007.stdout:8/190: stat d0/cb 0 2024-03-16T09:01:02.322 INFO:tasks.workunit.client.0.smithi007.stdout:8/191: unlink d0/f1f 0 2024-03-16T09:01:02.323 INFO:tasks.workunit.client.0.smithi007.stdout:8/192: symlink d0/l3e 0 2024-03-16T09:01:02.324 INFO:tasks.workunit.client.0.smithi007.stdout:8/193: creat d0/d2/d18/f3f x:0 0 0 2024-03-16T09:01:02.324 INFO:tasks.workunit.client.0.smithi007.stdout:8/194: readlink d0/dc/l28 0 2024-03-16T09:01:02.341 INFO:tasks.workunit.client.0.smithi007.stdout:8/195: dwrite d0/f22 [0,4194304] 0 2024-03-16T09:01:02.347 INFO:tasks.workunit.client.0.smithi007.stdout:8/196: write d0/fe [2414833,121069] 0 2024-03-16T09:01:02.356 INFO:tasks.workunit.client.0.smithi007.stdout:8/197: dread d0/dc/f23 [0,4194304] 0 2024-03-16T09:01:02.356 INFO:tasks.workunit.client.0.smithi007.stdout:8/198: creat d0/d2/d18/f40 x:0 0 0 2024-03-16T09:01:02.358 INFO:tasks.workunit.client.0.smithi007.stdout:8/199: symlink d0/d26/d2d/l41 0 2024-03-16T09:01:02.358 INFO:tasks.workunit.client.0.smithi007.stdout:8/200: dread - d0/d2/d18/f3f zero size 2024-03-16T09:01:02.359 INFO:tasks.workunit.client.0.smithi007.stdout:8/201: link d0/dc/c27 d0/d26/d2d/c42 0 2024-03-16T09:01:02.359 INFO:tasks.workunit.client.0.smithi007.stdout:8/202: truncate d0/dc/f3a 339584 0 2024-03-16T09:01:02.360 INFO:tasks.workunit.client.0.smithi007.stdout:8/203: read d0/ff [4773367,130989] 0 2024-03-16T09:01:02.361 INFO:tasks.workunit.client.0.smithi007.stdout:8/204: mkdir d0/d26/d2d/d43 0 2024-03-16T09:01:02.361 INFO:tasks.workunit.client.0.smithi007.stdout:8/205: dread - d0/d26/d2d/f32 zero size 2024-03-16T09:01:02.361 INFO:tasks.workunit.client.0.smithi007.stdout:8/206: readlink d0/l37 0 2024-03-16T09:01:02.362 INFO:tasks.workunit.client.0.smithi007.stdout:8/207: symlink d0/l44 0 2024-03-16T09:01:02.363 INFO:tasks.workunit.client.0.smithi007.stdout:8/208: rename d0/dc/dd/c2b to d0/d26/d2d/c45 0 2024-03-16T09:01:02.365 INFO:tasks.workunit.client.0.smithi007.stdout:8/209: mknod d0/d2/c46 0 2024-03-16T09:01:02.366 INFO:tasks.workunit.client.0.smithi007.stdout:8/210: unlink d0/d26/d2d/c42 0 2024-03-16T09:01:02.368 INFO:tasks.workunit.client.0.smithi007.stdout:8/211: symlink d0/dc/dd/l47 0 2024-03-16T09:01:02.374 INFO:tasks.workunit.client.0.smithi007.stdout:8/212: write d0/d26/f2f [2385852,117673] 0 2024-03-16T09:01:02.374 INFO:tasks.workunit.client.0.smithi007.stdout:8/213: creat d0/d2/d18/f48 x:0 0 0 2024-03-16T09:01:02.376 INFO:tasks.workunit.client.0.smithi007.stdout:8/214: rename d0/d2/d18/f40 to d0/d2/f49 0 2024-03-16T09:01:02.376 INFO:tasks.workunit.client.0.smithi007.stdout:8/215: stat d0/d2/d18/f48 0 2024-03-16T09:01:02.377 INFO:tasks.workunit.client.0.smithi007.stdout:8/216: mknod d0/d2/d18/c4a 0 2024-03-16T09:01:02.377 INFO:tasks.workunit.client.0.smithi007.stdout:8/217: read d0/ff [986786,19170] 0 2024-03-16T09:01:02.378 INFO:tasks.workunit.client.0.smithi007.stdout:8/218: stat d0/d26/d2d 0 2024-03-16T09:01:02.378 INFO:tasks.workunit.client.0.smithi007.stdout:8/219: creat d0/d2/f4b x:0 0 0 2024-03-16T09:01:02.387 INFO:tasks.workunit.client.0.smithi007.stdout:8/220: dread d0/dc/f12 [0,4194304] 0 2024-03-16T09:01:02.392 INFO:tasks.workunit.client.0.smithi007.stdout:8/221: getdents d0/d26/d2d 0 2024-03-16T09:01:02.392 INFO:tasks.workunit.client.0.smithi007.stdout:8/222: write d0/f22 [4761469,90192] 0 2024-03-16T09:01:02.393 INFO:tasks.workunit.client.0.smithi007.stdout:8/223: mkdir d0/d26/d2d/d43/d4c 0 2024-03-16T09:01:02.406 INFO:tasks.workunit.client.0.smithi007.stdout:8/224: dwrite d0/d2/f49 [0,4194304] 0 2024-03-16T09:01:02.409 INFO:tasks.workunit.client.0.smithi007.stdout:8/225: getdents d0/d26/d2d/d43 0 2024-03-16T09:01:02.414 INFO:tasks.workunit.client.0.smithi007.stdout:8/226: truncate d0/d2/fa 2828602 0 2024-03-16T09:01:02.419 INFO:tasks.workunit.client.0.smithi007.stdout:8/227: write d0/dc/f12 [663598,89971] 0 2024-03-16T09:01:02.419 INFO:tasks.workunit.client.0.smithi007.stdout:8/228: write d0/d2/d18/f48 [564070,16057] 0 2024-03-16T09:01:02.419 INFO:tasks.workunit.client.0.smithi007.stdout:8/229: dread - d0/d2/f4b zero size 2024-03-16T09:01:02.420 INFO:tasks.workunit.client.0.smithi007.stdout:8/230: mknod d0/dc/c4d 0 2024-03-16T09:01:02.421 INFO:tasks.workunit.client.0.smithi007.stdout:8/231: rename d0/d2/f49 to d0/d2/d18/f4e 0 2024-03-16T09:01:02.422 INFO:tasks.workunit.client.0.smithi007.stdout:8/232: mknod d0/d26/c4f 0 2024-03-16T09:01:02.422 INFO:tasks.workunit.client.0.smithi007.stdout:8/233: mkdir d0/dc/d50 0 2024-03-16T09:01:02.424 INFO:tasks.workunit.client.0.smithi007.stdout:8/234: symlink d0/d26/d2d/d43/l51 0 2024-03-16T09:01:02.424 INFO:tasks.workunit.client.0.smithi007.stdout:8/235: creat d0/d2/d18/f52 x:0 0 0 2024-03-16T09:01:02.424 INFO:tasks.workunit.client.0.smithi007.stdout:8/236: write d0/f3c [1042639,64494] 0 2024-03-16T09:01:02.425 INFO:tasks.workunit.client.0.smithi007.stdout:8/237: symlink d0/d26/d2d/d43/l53 0 2024-03-16T09:01:02.432 INFO:tasks.workunit.client.0.smithi007.stdout:8/238: mkdir d0/dc/d54 0 2024-03-16T09:01:02.435 INFO:tasks.workunit.client.0.smithi007.stdout:8/239: mknod d0/d26/d2d/c55 0 2024-03-16T09:01:02.435 INFO:tasks.workunit.client.0.smithi007.stdout:8/240: write d0/dc/f3a [353561,104061] 0 2024-03-16T09:01:02.435 INFO:tasks.workunit.client.0.smithi007.stdout:8/241: stat d0/d26/d2d/l41 0 2024-03-16T09:01:02.435 INFO:tasks.workunit.client.0.smithi007.stdout:8/242: truncate d0/d26/d2d/f32 537096 0 2024-03-16T09:01:02.452 INFO:tasks.workunit.client.0.smithi007.stdout:8/243: dwrite d0/dc/f12 [0,4194304] 0 2024-03-16T09:01:02.453 INFO:tasks.workunit.client.0.smithi007.stdout:8/244: symlink d0/dc/d54/l56 0 2024-03-16T09:01:02.454 INFO:tasks.workunit.client.0.smithi007.stdout:8/245: link d0/d2/c46 d0/d26/d2d/d43/c57 0 2024-03-16T09:01:02.454 INFO:tasks.workunit.client.0.smithi007.stdout:8/246: chown d0/dc/f35 3962 1 2024-03-16T09:01:02.474 INFO:tasks.workunit.client.0.smithi007.stdout:8/247: dwrite d0/d26/d2d/f32 [0,4194304] 0 2024-03-16T09:01:02.476 INFO:tasks.workunit.client.0.smithi007.stdout:8/248: fdatasync d0/f3c 0 2024-03-16T09:01:02.476 INFO:tasks.workunit.client.0.smithi007.stdout:8/249: fsync d0/f3c 0 2024-03-16T09:01:02.478 INFO:tasks.workunit.client.0.smithi007.stdout:8/250: dread d0/f3c [0,4194304] 0 2024-03-16T09:01:02.479 INFO:tasks.workunit.client.0.smithi007.stdout:8/251: creat d0/d2/f58 x:0 0 0 2024-03-16T09:01:02.480 INFO:tasks.workunit.client.0.smithi007.stdout:8/252: rename d0/d2/l29 to d0/dc/l59 0 2024-03-16T09:01:02.482 INFO:tasks.workunit.client.0.smithi007.stdout:8/253: dread d0/f3c [0,4194304] 0 2024-03-16T09:01:02.512 INFO:tasks.workunit.client.0.smithi007.stdout:5/129: sync 2024-03-16T09:01:02.513 INFO:tasks.workunit.client.0.smithi007.stdout:2/162: sync 2024-03-16T09:01:02.513 INFO:tasks.workunit.client.0.smithi007.stdout:1/123: sync 2024-03-16T09:01:02.513 INFO:tasks.workunit.client.0.smithi007.stdout:2/163: chown dc/f20 90 1 2024-03-16T09:01:02.513 INFO:tasks.workunit.client.0.smithi007.stdout:2/164: fdatasync dc/f11 0 2024-03-16T09:01:02.513 INFO:tasks.workunit.client.0.smithi007.stdout:1/124: fsync d9/dd/f14 0 2024-03-16T09:01:02.513 INFO:tasks.workunit.client.0.smithi007.stdout:1/125: fdatasync d9/f12 0 2024-03-16T09:01:02.513 INFO:tasks.workunit.client.0.smithi007.stdout:4/106: sync 2024-03-16T09:01:02.513 INFO:tasks.workunit.client.0.smithi007.stdout:4/107: fdatasync d4/f6 0 2024-03-16T09:01:02.514 INFO:tasks.workunit.client.0.smithi007.stdout:5/130: mkdir d1/d7/d16/d1e/d2a 0 2024-03-16T09:01:02.514 INFO:tasks.workunit.client.0.smithi007.stdout:5/131: truncate d1/d7/d8/de/f1a 399141 0 2024-03-16T09:01:02.514 INFO:tasks.workunit.client.0.smithi007.stdout:5/132: chown d1/d7/d1c/f23 12674 1 2024-03-16T09:01:02.514 INFO:tasks.workunit.client.0.smithi007.stdout:2/165: mkdir dc/df/d24/d28/d2e/d35 0 2024-03-16T09:01:02.514 INFO:tasks.workunit.client.0.smithi007.stdout:2/166: dread - dc/df/f2d zero size 2024-03-16T09:01:02.516 INFO:tasks.workunit.client.0.smithi007.stdout:5/133: truncate d1/d7/f12 1973922 0 2024-03-16T09:01:02.516 INFO:tasks.workunit.client.0.smithi007.stdout:5/134: readlink d1/d7/d8/d28/l29 0 2024-03-16T09:01:02.518 INFO:tasks.workunit.client.0.smithi007.stdout:2/167: truncate dc/f10 5932 0 2024-03-16T09:01:02.520 INFO:tasks.workunit.client.0.smithi007.stdout:5/135: creat d1/d7/d16/f2b x:0 0 0 2024-03-16T09:01:02.521 INFO:tasks.workunit.client.0.smithi007.stdout:4/108: dread d4/f10 [0,4194304] 0 2024-03-16T09:01:02.521 INFO:tasks.workunit.client.0.smithi007.stdout:4/109: dread - d4/d14/f1b zero size 2024-03-16T09:01:02.523 INFO:tasks.workunit.client.0.smithi007.stdout:2/168: unlink dc/l2a 0 2024-03-16T09:01:02.523 INFO:tasks.workunit.client.0.smithi007.stdout:2/169: fsync f2 0 2024-03-16T09:01:02.523 INFO:tasks.workunit.client.0.smithi007.stdout:2/170: write dc/df/f18 [4585063,11220] 0 2024-03-16T09:01:02.523 INFO:tasks.workunit.client.0.smithi007.stdout:2/171: stat dc/df/d24/c2c 0 2024-03-16T09:01:02.523 INFO:tasks.workunit.client.0.smithi007.stdout:2/172: readlink la 0 2024-03-16T09:01:02.524 INFO:tasks.workunit.client.0.smithi007.stdout:4/110: symlink d4/d14/d1c/l21 0 2024-03-16T09:01:02.524 INFO:tasks.workunit.client.0.smithi007.stdout:4/111: read f2 [421386,123164] 0 2024-03-16T09:01:02.526 INFO:tasks.workunit.client.0.smithi007.stdout:2/173: truncate dc/f20 2008909 0 2024-03-16T09:01:02.529 INFO:tasks.workunit.client.0.smithi007.stdout:2/174: unlink f2 0 2024-03-16T09:01:02.529 INFO:tasks.workunit.client.0.smithi007.stdout:2/175: fsync dc/f23 0 2024-03-16T09:01:02.531 INFO:tasks.workunit.client.0.smithi007.stdout:4/112: link d4/f18 d4/f22 0 2024-03-16T09:01:02.532 INFO:tasks.workunit.client.0.smithi007.stdout:2/176: dread dc/f10 [0,4194304] 0 2024-03-16T09:01:02.532 INFO:tasks.workunit.client.0.smithi007.stdout:4/113: mkdir d4/d14/d1c/d23 0 2024-03-16T09:01:02.533 INFO:tasks.workunit.client.0.smithi007.stdout:5/136: dwrite d1/f10 [0,4194304] 0 2024-03-16T09:01:02.537 INFO:tasks.workunit.client.0.smithi007.stdout:2/177: truncate dc/f15 5349674 0 2024-03-16T09:01:02.537 INFO:tasks.workunit.client.0.smithi007.stdout:4/114: rmdir d4/d7 39 2024-03-16T09:01:02.539 INFO:tasks.workunit.client.0.smithi007.stdout:2/178: dread dc/f17 [0,4194304] 0 2024-03-16T09:01:02.540 INFO:tasks.workunit.client.0.smithi007.stdout:2/179: mknod dc/df/d24/d28/c36 0 2024-03-16T09:01:02.540 INFO:tasks.workunit.client.0.smithi007.stdout:2/180: chown dc/f17 114233719 1 2024-03-16T09:01:02.540 INFO:tasks.workunit.client.0.smithi007.stdout:2/181: chown f6 59 1 2024-03-16T09:01:02.541 INFO:tasks.workunit.client.0.smithi007.stdout:2/182: mknod dc/df/d24/d28/d2e/d35/c37 0 2024-03-16T09:01:02.542 INFO:tasks.workunit.client.0.smithi007.stdout:2/183: read f6 [1598107,127922] 0 2024-03-16T09:01:02.542 INFO:tasks.workunit.client.0.smithi007.stdout:2/184: write dc/f1a [364013,111161] 0 2024-03-16T09:01:02.544 INFO:tasks.workunit.client.0.smithi007.stdout:2/185: rename f0 to dc/df/d24/d28/d2e/d35/f38 0 2024-03-16T09:01:02.547 INFO:tasks.workunit.client.0.smithi007.stdout:2/186: truncate dc/df/f2b 976575 0 2024-03-16T09:01:02.547 INFO:tasks.workunit.client.0.smithi007.stdout:5/137: dwrite d1/d7/d1c/f15 [4194304,4194304] 0 2024-03-16T09:01:02.548 INFO:tasks.workunit.client.0.smithi007.stdout:4/115: write d4/f5 [2030643,34591] 0 2024-03-16T09:01:02.553 INFO:tasks.workunit.client.0.smithi007.stdout:2/187: creat dc/df/d24/d28/f39 x:0 0 0 2024-03-16T09:01:02.561 INFO:tasks.workunit.client.0.smithi007.stdout:2/188: creat dc/f3a x:0 0 0 2024-03-16T09:01:02.561 INFO:tasks.workunit.client.0.smithi007.stdout:2/189: readlink dc/df/l12 0 2024-03-16T09:01:02.562 INFO:tasks.workunit.client.0.smithi007.stdout:2/190: mknod dc/df/c3b 0 2024-03-16T09:01:02.569 INFO:tasks.workunit.client.0.smithi007.stdout:4/116: dwrite d4/f6 [0,4194304] 0 2024-03-16T09:01:02.569 INFO:tasks.workunit.client.0.smithi007.stdout:4/117: write d4/d7/fe [1164488,8766] 0 2024-03-16T09:01:02.570 INFO:tasks.workunit.client.0.smithi007.stdout:4/118: symlink d4/l24 0 2024-03-16T09:01:02.576 INFO:tasks.workunit.client.0.smithi007.stdout:4/119: dread d4/f10 [0,4194304] 0 2024-03-16T09:01:02.576 INFO:tasks.workunit.client.0.smithi007.stdout:4/120: write d4/d14/f19 [4910103,90292] 0 2024-03-16T09:01:02.576 INFO:tasks.workunit.client.0.smithi007.stdout:2/191: write dc/df/f34 [3543590,82821] 0 2024-03-16T09:01:02.720 INFO:tasks.workunit.client.0.smithi007.stdout:3/181: sync 2024-03-16T09:01:02.720 INFO:tasks.workunit.client.0.smithi007.stdout:3/182: read f13 [119994,111192] 0 2024-03-16T09:01:02.721 INFO:tasks.workunit.client.0.smithi007.stdout:3/183: truncate d15/d18/d1f/d29/f1b 857134 0 2024-03-16T09:01:02.721 INFO:tasks.workunit.client.0.smithi007.stdout:3/184: creat d15/d18/d1f/d29/f31 x:0 0 0 2024-03-16T09:01:02.723 INFO:tasks.workunit.client.0.smithi007.stdout:3/185: rename d15/d18/d1f/d29/f2d to d15/f32 0 2024-03-16T09:01:02.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/186: rmdir d15/d18 39 2024-03-16T09:01:02.725 INFO:tasks.workunit.client.0.smithi007.stdout:3/187: creat d15/d18/f33 x:0 0 0 2024-03-16T09:01:02.725 INFO:tasks.workunit.client.0.smithi007.stdout:3/188: mknod d15/d18/d1f/c34 0 2024-03-16T09:01:02.725 INFO:tasks.workunit.client.0.smithi007.stdout:3/189: write d15/d18/d1f/d29/f31 [337283,69041] 0 2024-03-16T09:01:02.726 INFO:tasks.workunit.client.0.smithi007.stdout:3/190: creat d15/d18/f35 x:0 0 0 2024-03-16T09:01:02.727 INFO:tasks.workunit.client.0.smithi007.stdout:3/191: mknod d15/c36 0 2024-03-16T09:01:02.727 INFO:tasks.workunit.client.0.smithi007.stdout:3/192: rename d15 to d15/d18/d37 22 2024-03-16T09:01:02.727 INFO:tasks.workunit.client.0.smithi007.stdout:3/193: write f5 [1840780,54776] 0 2024-03-16T09:01:02.728 INFO:tasks.workunit.client.0.smithi007.stdout:3/194: mknod d15/c38 0 2024-03-16T09:01:02.729 INFO:tasks.workunit.client.0.smithi007.stdout:3/195: creat d15/d18/d1f/f39 x:0 0 0 2024-03-16T09:01:02.747 INFO:tasks.workunit.client.0.smithi007.stdout:3/196: dwrite d15/f16 [0,4194304] 0 2024-03-16T09:01:02.748 INFO:tasks.workunit.client.0.smithi007.stdout:3/197: fdatasync f5 0 2024-03-16T09:01:02.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/198: rmdir d15/d18 39 2024-03-16T09:01:02.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/199: dread - d15/d18/f33 zero size 2024-03-16T09:01:02.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/200: chown d15/d18/f33 842 1 2024-03-16T09:01:02.753 INFO:tasks.workunit.client.0.smithi007.stdout:3/201: symlink d15/d18/d1f/d29/l3a 0 2024-03-16T09:01:02.754 INFO:tasks.workunit.client.0.smithi007.stdout:3/202: mkdir d15/d18/d1f/d3b 0 2024-03-16T09:01:02.755 INFO:tasks.workunit.client.0.smithi007.stdout:3/203: chown d15/d18/d1f/f2e 2277 1 2024-03-16T09:01:02.755 INFO:tasks.workunit.client.0.smithi007.stdout:3/204: symlink d15/d18/d1f/d29/l3c 0 2024-03-16T09:01:02.773 INFO:tasks.workunit.client.0.smithi007.stdout:3/205: dwrite d15/f1e [0,4194304] 0 2024-03-16T09:01:02.773 INFO:tasks.workunit.client.0.smithi007.stdout:3/206: truncate f5 4072527 0 2024-03-16T09:01:02.776 INFO:tasks.workunit.client.0.smithi007.stdout:3/207: rmdir d15 39 2024-03-16T09:01:02.776 INFO:tasks.workunit.client.0.smithi007.stdout:3/208: getdents d15/d18 0 2024-03-16T09:01:02.778 INFO:tasks.workunit.client.0.smithi007.stdout:3/209: rename d15/l1c to d15/d18/l3d 0 2024-03-16T09:01:02.794 INFO:tasks.workunit.client.0.smithi007.stdout:3/210: dwrite d15/f16 [0,4194304] 0 2024-03-16T09:01:02.798 INFO:tasks.workunit.client.0.smithi007.stdout:3/211: truncate f5 1741240 0 2024-03-16T09:01:02.800 INFO:tasks.workunit.client.0.smithi007.stdout:3/212: mkdir d15/d18/d1f/d29/d3e 0 2024-03-16T09:01:02.805 INFO:tasks.workunit.client.0.smithi007.stdout:3/213: dread fa [0,4194304] 0 2024-03-16T09:01:02.806 INFO:tasks.workunit.client.0.smithi007.stdout:3/214: creat d15/d18/d1f/d3b/f3f x:0 0 0 2024-03-16T09:01:02.807 INFO:tasks.workunit.client.0.smithi007.stdout:3/215: stat d15/d18/l3d 0 2024-03-16T09:01:02.808 INFO:tasks.workunit.client.0.smithi007.stdout:3/216: fsync d15/d18/f22 0 2024-03-16T09:01:02.808 INFO:tasks.workunit.client.0.smithi007.stdout:3/217: write d15/d18/d1f/d29/f25 [806009,3421] 0 2024-03-16T09:01:02.809 INFO:tasks.workunit.client.0.smithi007.stdout:3/218: truncate d15/d18/f19 1541973 0 2024-03-16T09:01:02.831 INFO:tasks.workunit.client.0.smithi007.stdout:3/219: dwrite d15/d18/f23 [0,4194304] 0 2024-03-16T09:01:02.834 INFO:tasks.workunit.client.0.smithi007.stdout:3/220: symlink d15/l40 0 2024-03-16T09:01:02.835 INFO:tasks.workunit.client.0.smithi007.stdout:3/221: creat d15/d18/d1f/f41 x:0 0 0 2024-03-16T09:01:02.837 INFO:tasks.workunit.client.0.smithi007.stdout:3/222: unlink f14 0 2024-03-16T09:01:02.838 INFO:tasks.workunit.client.0.smithi007.stdout:3/223: creat d15/d18/f42 x:0 0 0 2024-03-16T09:01:02.839 INFO:tasks.workunit.client.0.smithi007.stdout:3/224: mkdir d15/d18/d1f/d29/d43 0 2024-03-16T09:01:02.841 INFO:tasks.workunit.client.0.smithi007.stdout:3/225: rename d15/f32 to d15/d18/d1f/d3b/f44 0 2024-03-16T09:01:02.843 INFO:tasks.workunit.client.0.smithi007.stdout:3/226: rename d15/d18/d1f/f26 to d15/d18/d1f/d3b/f45 0 2024-03-16T09:01:02.845 INFO:tasks.workunit.client.0.smithi007.stdout:3/227: mknod d15/d18/c46 0 2024-03-16T09:01:02.846 INFO:tasks.workunit.client.0.smithi007.stdout:3/228: chown d15/d18/d1f/d3b/f44 161935136 1 2024-03-16T09:01:02.849 INFO:tasks.workunit.client.0.smithi007.stdout:3/229: rename f7 to d15/d18/d1f/d3b/f47 0 2024-03-16T09:01:02.849 INFO:tasks.workunit.client.0.smithi007.stdout:3/230: chown d15/d18/d1f/f39 6133926 1 2024-03-16T09:01:02.849 INFO:tasks.workunit.client.0.smithi007.stdout:3/231: write f13 [2927228,116201] 0 2024-03-16T09:01:02.849 INFO:tasks.workunit.client.0.smithi007.stdout:3/232: write d15/d18/f35 [4451,70256] 0 2024-03-16T09:01:02.849 INFO:tasks.workunit.client.0.smithi007.stdout:3/233: readlink l4 0 2024-03-16T09:01:02.850 INFO:tasks.workunit.client.0.smithi007.stdout:3/234: link cc d15/d18/d1f/d3b/c48 0 2024-03-16T09:01:02.851 INFO:tasks.workunit.client.0.smithi007.stdout:6/179: sync 2024-03-16T09:01:02.851 INFO:tasks.workunit.client.0.smithi007.stdout:9/216: sync 2024-03-16T09:01:02.852 INFO:tasks.workunit.client.0.smithi007.stdout:3/235: unlink d15/d18/f42 0 2024-03-16T09:01:02.852 INFO:tasks.workunit.client.0.smithi007.stdout:3/236: write fa [2815192,16759] 0 2024-03-16T09:01:02.852 INFO:tasks.workunit.client.0.smithi007.stdout:9/217: write f16 [518281,120784] 0 2024-03-16T09:01:02.852 INFO:tasks.workunit.client.0.smithi007.stdout:9/218: dread - d24/f3d zero size 2024-03-16T09:01:02.854 INFO:tasks.workunit.client.0.smithi007.stdout:6/180: unlink d2/lf 0 2024-03-16T09:01:02.856 INFO:tasks.workunit.client.0.smithi007.stdout:6/181: unlink d2/c4 0 2024-03-16T09:01:02.856 INFO:tasks.workunit.client.0.smithi007.stdout:6/182: readlink d2/d11/l30 0 2024-03-16T09:01:02.856 INFO:tasks.workunit.client.0.smithi007.stdout:9/219: link c5 d24/c42 0 2024-03-16T09:01:02.858 INFO:tasks.workunit.client.0.smithi007.stdout:6/183: creat d2/d11/d1e/f40 x:0 0 0 2024-03-16T09:01:02.858 INFO:tasks.workunit.client.0.smithi007.stdout:6/184: chown d2/d10/l1c 21829869 1 2024-03-16T09:01:02.861 INFO:tasks.workunit.client.0.smithi007.stdout:9/220: dread f16 [0,4194304] 0 2024-03-16T09:01:02.862 INFO:tasks.workunit.client.0.smithi007.stdout:9/221: dread f16 [0,4194304] 0 2024-03-16T09:01:02.866 INFO:tasks.workunit.client.0.smithi007.stdout:6/185: read d2/d11/f25 [18118,30957] 0 2024-03-16T09:01:02.867 INFO:tasks.workunit.client.0.smithi007.stdout:6/186: stat d2/d10/d12/l23 0 2024-03-16T09:01:02.868 INFO:tasks.workunit.client.0.smithi007.stdout:6/187: rename d2/d13/c2c to d2/d10/d12/c41 0 2024-03-16T09:01:02.874 INFO:tasks.workunit.client.0.smithi007.stdout:6/188: write d2/d11/d1e/f2f [102880,121015] 0 2024-03-16T09:01:02.875 INFO:tasks.workunit.client.0.smithi007.stdout:6/189: truncate d2/d11/f21 1221845 0 2024-03-16T09:01:02.876 INFO:tasks.workunit.client.0.smithi007.stdout:6/190: mknod d2/d11/d1e/c42 0 2024-03-16T09:01:02.876 INFO:tasks.workunit.client.0.smithi007.stdout:6/191: fdatasync d2/d11/f26 0 2024-03-16T09:01:02.882 INFO:tasks.workunit.client.0.smithi007.stdout:6/192: dread d2/fd [0,4194304] 0 2024-03-16T09:01:02.882 INFO:tasks.workunit.client.0.smithi007.stdout:6/193: write d2/d10/f3a [729100,63990] 0 2024-03-16T09:01:02.888 INFO:tasks.workunit.client.0.smithi007.stdout:6/194: read d2/fb [728641,74949] 0 2024-03-16T09:01:02.894 INFO:tasks.workunit.client.0.smithi007.stdout:8/254: sync 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:7/144: sync 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:4/121: sync 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:5/138: sync 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:1/126: sync 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:0/102: sync 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:8/255: stat d0/d2/d18/f48 0 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:1/127: chown d9/dd/d1b/c1f 1413291 1 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:1/128: fsync d9/f12 0 2024-03-16T09:01:02.895 INFO:tasks.workunit.client.0.smithi007.stdout:2/192: sync 2024-03-16T09:01:02.897 INFO:tasks.workunit.client.0.smithi007.stdout:4/122: rmdir d4/d14/d1c/d23 0 2024-03-16T09:01:02.897 INFO:tasks.workunit.client.0.smithi007.stdout:4/123: truncate d4/ff 783922 0 2024-03-16T09:01:02.898 INFO:tasks.workunit.client.0.smithi007.stdout:7/145: link d0/d19/f26 d0/d19/d1f/f2c 0 2024-03-16T09:01:02.898 INFO:tasks.workunit.client.0.smithi007.stdout:8/256: stat d0/dc/l59 0 2024-03-16T09:01:02.899 INFO:tasks.workunit.client.0.smithi007.stdout:0/103: rename d3/d9/da/cf to d3/d9/da/d18/c24 0 2024-03-16T09:01:02.899 INFO:tasks.workunit.client.0.smithi007.stdout:0/104: stat d3/d9/da/d18/d21/f12 0 2024-03-16T09:01:02.900 INFO:tasks.workunit.client.0.smithi007.stdout:5/139: symlink d1/d7/l2c 0 2024-03-16T09:01:02.900 INFO:tasks.workunit.client.0.smithi007.stdout:2/193: dread - dc/f3a zero size 2024-03-16T09:01:02.902 INFO:tasks.workunit.client.0.smithi007.stdout:8/257: link d0/d2/d18/c1a d0/dc/d54/c5a 0 2024-03-16T09:01:02.902 INFO:tasks.workunit.client.0.smithi007.stdout:8/258: write d0/d2/fa [1823897,119190] 0 2024-03-16T09:01:02.902 INFO:tasks.workunit.client.0.smithi007.stdout:8/259: write d0/d26/f30 [671962,117994] 0 2024-03-16T09:01:02.903 INFO:tasks.workunit.client.0.smithi007.stdout:1/129: rename d9/ca to d9/dd/d1b/c26 0 2024-03-16T09:01:02.904 INFO:tasks.workunit.client.0.smithi007.stdout:0/105: mkdir d3/d9/da/d18/d21/d25 0 2024-03-16T09:01:02.904 INFO:tasks.workunit.client.0.smithi007.stdout:0/106: creat d3/d9/f26 x:0 0 0 2024-03-16T09:01:02.904 INFO:tasks.workunit.client.0.smithi007.stdout:0/107: chown d3/d9/f11 46 1 2024-03-16T09:01:02.904 INFO:tasks.workunit.client.0.smithi007.stdout:2/194: symlink dc/df/d24/d28/l3c 0 2024-03-16T09:01:02.905 INFO:tasks.workunit.client.0.smithi007.stdout:8/260: creat d0/d26/f5b x:0 0 0 2024-03-16T09:01:02.906 INFO:tasks.workunit.client.0.smithi007.stdout:1/130: dread d9/dd/f14 [0,4194304] 0 2024-03-16T09:01:02.906 INFO:tasks.workunit.client.0.smithi007.stdout:0/108: mkdir d3/d9/da/d18/d27 0 2024-03-16T09:01:02.908 INFO:tasks.workunit.client.0.smithi007.stdout:8/261: symlink d0/l5c 0 2024-03-16T09:01:02.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/195: mknod dc/d27/c3d 0 2024-03-16T09:01:02.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/196: write dc/df/f2d [53530,59190] 0 2024-03-16T09:01:02.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/197: chown dc/df/l30 3480529 1 2024-03-16T09:01:02.917 INFO:tasks.workunit.client.0.smithi007.stdout:7/146: dwrite d0/dc/fe [0,4194304] 0 2024-03-16T09:01:02.917 INFO:tasks.workunit.client.0.smithi007.stdout:7/147: creat d0/d19/f2d x:0 0 0 2024-03-16T09:01:02.920 INFO:tasks.workunit.client.0.smithi007.stdout:7/148: symlink d0/d19/d1f/l2e 0 2024-03-16T09:01:02.922 INFO:tasks.workunit.client.0.smithi007.stdout:1/131: rename d9/dd/d1b/c26 to d9/c27 0 2024-03-16T09:01:02.922 INFO:tasks.workunit.client.0.smithi007.stdout:1/132: write f7 [222708,18380] 0 2024-03-16T09:01:02.923 INFO:tasks.workunit.client.0.smithi007.stdout:2/198: rename dc to dc/df/d24/d28/d3e 22 2024-03-16T09:01:02.924 INFO:tasks.workunit.client.0.smithi007.stdout:7/149: dread d0/dc/fd [0,4194304] 0 2024-03-16T09:01:02.927 INFO:tasks.workunit.client.0.smithi007.stdout:2/199: creat dc/f3f x:0 0 0 2024-03-16T09:01:02.928 INFO:tasks.workunit.client.0.smithi007.stdout:7/150: creat d0/dc/f2f x:0 0 0 2024-03-16T09:01:02.928 INFO:tasks.workunit.client.0.smithi007.stdout:1/133: dread f5 [0,4194304] 0 2024-03-16T09:01:02.929 INFO:tasks.workunit.client.0.smithi007.stdout:1/134: readlink d9/l20 0 2024-03-16T09:01:02.929 INFO:tasks.workunit.client.0.smithi007.stdout:7/151: symlink d0/dc/d24/l30 0 2024-03-16T09:01:02.929 INFO:tasks.workunit.client.0.smithi007.stdout:1/135: read d9/f13 [195618,6256] 0 2024-03-16T09:01:02.930 INFO:tasks.workunit.client.0.smithi007.stdout:7/152: fsync d0/f11 0 2024-03-16T09:01:02.931 INFO:tasks.workunit.client.0.smithi007.stdout:1/136: mkdir d9/dd/d1b/d28 0 2024-03-16T09:01:02.931 INFO:tasks.workunit.client.0.smithi007.stdout:1/137: write d9/f22 [931553,116107] 0 2024-03-16T09:01:02.931 INFO:tasks.workunit.client.0.smithi007.stdout:1/138: stat d9/dd/d1b 0 2024-03-16T09:01:02.932 INFO:tasks.workunit.client.0.smithi007.stdout:1/139: mkdir d9/dd/d1b/d28/d29 0 2024-03-16T09:01:02.934 INFO:tasks.workunit.client.0.smithi007.stdout:5/140: dwrite d1/fc [0,4194304] 0 2024-03-16T09:01:02.934 INFO:tasks.workunit.client.0.smithi007.stdout:1/140: link d9/dd/d1b/f25 d9/dd/d1b/d28/d29/f2a 0 2024-03-16T09:01:02.934 INFO:tasks.workunit.client.0.smithi007.stdout:1/141: fdatasync f0 0 2024-03-16T09:01:02.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/262: dwrite d0/f22 [4194304,4194304] 0 2024-03-16T09:01:02.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/263: readlink d0/l9 0 2024-03-16T09:01:02.935 INFO:tasks.workunit.client.0.smithi007.stdout:4/124: dwrite d4/fa [0,4194304] 0 2024-03-16T09:01:02.935 INFO:tasks.workunit.client.0.smithi007.stdout:2/200: dread f7 [0,4194304] 0 2024-03-16T09:01:02.936 INFO:tasks.workunit.client.0.smithi007.stdout:5/141: symlink d1/d7/d16/d1b/l2d 0 2024-03-16T09:01:02.936 INFO:tasks.workunit.client.0.smithi007.stdout:5/142: readlink d1/d7/d16/d1b/l2d 0 2024-03-16T09:01:02.936 INFO:tasks.workunit.client.0.smithi007.stdout:5/143: dread - d1/d7/d16/f2b zero size 2024-03-16T09:01:02.937 INFO:tasks.workunit.client.0.smithi007.stdout:8/264: creat d0/d2/f5d x:0 0 0 2024-03-16T09:01:02.937 INFO:tasks.workunit.client.0.smithi007.stdout:8/265: stat d0/dc/c4d 0 2024-03-16T09:01:02.941 INFO:tasks.workunit.client.0.smithi007.stdout:1/142: dread f5 [0,4194304] 0 2024-03-16T09:01:02.942 INFO:tasks.workunit.client.0.smithi007.stdout:4/125: rename d4/f18 to d4/d14/d1c/d1e/f25 0 2024-03-16T09:01:02.943 INFO:tasks.workunit.client.0.smithi007.stdout:2/201: creat dc/df/d24/d28/d2e/d35/f40 x:0 0 0 2024-03-16T09:01:02.943 INFO:tasks.workunit.client.0.smithi007.stdout:2/202: fdatasync dc/df/f2b 0 2024-03-16T09:01:02.944 INFO:tasks.workunit.client.0.smithi007.stdout:5/144: link d1/d7/d8/de/l24 d1/d7/d8/d1d/l2e 0 2024-03-16T09:01:02.944 INFO:tasks.workunit.client.0.smithi007.stdout:5/145: stat d1/fc 0 2024-03-16T09:01:02.945 INFO:tasks.workunit.client.0.smithi007.stdout:8/266: mknod d0/d26/d2d/d43/c5e 0 2024-03-16T09:01:02.946 INFO:tasks.workunit.client.0.smithi007.stdout:7/153: dwrite d0/dc/fd [0,4194304] 0 2024-03-16T09:01:02.947 INFO:tasks.workunit.client.0.smithi007.stdout:1/143: link d9/f18 d9/dd/d1b/d28/f2b 0 2024-03-16T09:01:02.948 INFO:tasks.workunit.client.0.smithi007.stdout:4/126: creat d4/d14/d1c/f26 x:0 0 0 2024-03-16T09:01:02.948 INFO:tasks.workunit.client.0.smithi007.stdout:4/127: readlink d4/d7/l8 0 2024-03-16T09:01:02.949 INFO:tasks.workunit.client.0.smithi007.stdout:4/128: dread d4/f22 [0,4194304] 0 2024-03-16T09:01:02.951 INFO:tasks.workunit.client.0.smithi007.stdout:5/146: truncate d1/d7/d1c/f17 621675 0 2024-03-16T09:01:02.955 INFO:tasks.workunit.client.0.smithi007.stdout:7/154: mknod d0/dc/c31 0 2024-03-16T09:01:02.957 INFO:tasks.workunit.client.0.smithi007.stdout:1/144: creat d9/dd/d1b/f2c x:0 0 0 2024-03-16T09:01:02.959 INFO:tasks.workunit.client.0.smithi007.stdout:4/129: write d4/fa [2430867,43165] 0 2024-03-16T09:01:02.959 INFO:tasks.workunit.client.0.smithi007.stdout:8/267: mkdir d0/d2/d5f 0 2024-03-16T09:01:02.960 INFO:tasks.workunit.client.0.smithi007.stdout:5/147: mknod d1/c2f 0 2024-03-16T09:01:02.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:01:02.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:02 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.964 INFO:tasks.workunit.client.0.smithi007.stdout:8/268: truncate d0/d2/d18/f3b 7597030 0 2024-03-16T09:01:02.966 INFO:tasks.workunit.client.0.smithi007.stdout:5/148: mknod d1/d7/d1c/c30 0 2024-03-16T09:01:02.967 INFO:tasks.workunit.client.0.smithi007.stdout:4/130: mknod d4/c27 0 2024-03-16T09:01:02.968 INFO:tasks.workunit.client.0.smithi007.stdout:8/269: symlink d0/dc/l60 0 2024-03-16T09:01:02.969 INFO:tasks.workunit.client.0.smithi007.stdout:4/131: creat d4/f28 x:0 0 0 2024-03-16T09:01:02.969 INFO:tasks.workunit.client.0.smithi007.stdout:1/145: dread d9/f12 [0,4194304] 0 2024-03-16T09:01:02.970 INFO:tasks.workunit.client.0.smithi007.stdout:1/146: chown d9/dd/l16 21 1 2024-03-16T09:01:02.970 INFO:tasks.workunit.client.0.smithi007.stdout:8/270: creat d0/d26/d2d/f61 x:0 0 0 2024-03-16T09:01:02.972 INFO:tasks.workunit.client.0.smithi007.stdout:4/132: creat d4/d14/d1c/f29 x:0 0 0 2024-03-16T09:01:02.972 INFO:tasks.workunit.client.0.smithi007.stdout:4/133: dread - d4/d14/d1c/f26 zero size 2024-03-16T09:01:02.972 INFO:tasks.workunit.client.0.smithi007.stdout:4/134: dread - d4/d14/f1b zero size 2024-03-16T09:01:02.972 INFO:tasks.workunit.client.0.smithi007.stdout:4/135: read - d4/d14/d1c/f26 zero size 2024-03-16T09:01:02.973 INFO:tasks.workunit.client.0.smithi007.stdout:1/147: mkdir d9/dd/d1b/d2d 0 2024-03-16T09:01:02.974 INFO:tasks.workunit.client.0.smithi007.stdout:1/148: mknod d9/dd/d1b/d28/d29/c2e 0 2024-03-16T09:01:02.982 INFO:tasks.workunit.client.0.smithi007.stdout:7/155: dwrite d0/dc/f1a [0,4194304] 0 2024-03-16T09:01:02.987 INFO:tasks.workunit.client.0.smithi007.stdout:8/271: dread d0/d26/d2d/f32 [0,4194304] 0 2024-03-16T09:01:02.987 INFO:tasks.workunit.client.0.smithi007.stdout:2/203: dwrite dc/f10 [0,4194304] 0 2024-03-16T09:01:02.988 INFO:tasks.workunit.client.0.smithi007.stdout:2/204: truncate dc/f17 1735005 0 2024-03-16T09:01:02.989 INFO:tasks.workunit.client.0.smithi007.stdout:8/272: mknod d0/d26/d2d/d43/c62 0 2024-03-16T09:01:02.990 INFO:tasks.workunit.client.0.smithi007.stdout:2/205: rename dc/d27/c3d to dc/df/c41 0 2024-03-16T09:01:02.990 INFO:tasks.workunit.client.0.smithi007.stdout:2/206: stat dc/df/f2d 0 2024-03-16T09:01:02.991 INFO:tasks.workunit.client.0.smithi007.stdout:8/273: dread d0/dc/f3a [0,4194304] 0 2024-03-16T09:01:02.997 INFO:tasks.workunit.client.0.smithi007.stdout:2/207: dread dc/f23 [0,4194304] 0 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:02.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:02 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:03.004 INFO:tasks.workunit.client.0.smithi007.stdout:8/274: write d0/d2/d18/f4e [3288650,45631] 0 2024-03-16T09:01:03.004 INFO:tasks.workunit.client.0.smithi007.stdout:4/136: dwrite f2 [0,4194304] 0 2024-03-16T09:01:03.004 INFO:tasks.workunit.client.0.smithi007.stdout:7/156: dwrite d0/d19/f2b [0,4194304] 0 2024-03-16T09:01:03.004 INFO:tasks.workunit.client.0.smithi007.stdout:8/275: write d0/d2/d18/f4e [633107,84090] 0 2024-03-16T09:01:03.006 INFO:tasks.workunit.client.0.smithi007.stdout:2/208: write dc/f20 [813275,14067] 0 2024-03-16T09:01:03.007 INFO:tasks.workunit.client.0.smithi007.stdout:7/157: mknod d0/dc/d24/d25/c32 0 2024-03-16T09:01:03.009 INFO:tasks.workunit.client.0.smithi007.stdout:4/137: symlink d4/l2a 0 2024-03-16T09:01:03.011 INFO:tasks.workunit.client.0.smithi007.stdout:7/158: creat d0/f33 x:0 0 0 2024-03-16T09:01:03.012 INFO:tasks.workunit.client.0.smithi007.stdout:8/276: symlink d0/d26/l63 0 2024-03-16T09:01:03.016 INFO:tasks.workunit.client.0.smithi007.stdout:8/277: truncate d0/d26/f39 3220836 0 2024-03-16T09:01:03.021 INFO:tasks.workunit.client.0.smithi007.stdout:8/278: mkdir d0/d26/d2d/d64 0 2024-03-16T09:01:03.021 INFO:tasks.workunit.client.0.smithi007.stdout:8/279: fsync d0/d26/f5b 0 2024-03-16T09:01:03.021 INFO:tasks.workunit.client.0.smithi007.stdout:8/280: dread - d0/d2/f4b zero size 2024-03-16T09:01:03.021 INFO:tasks.workunit.client.0.smithi007.stdout:8/281: truncate d0/d2/d18/f3f 375912 0 2024-03-16T09:01:03.022 INFO:tasks.workunit.client.0.smithi007.stdout:7/159: rename d0/c9 to d0/c34 0 2024-03-16T09:01:03.023 INFO:tasks.workunit.client.0.smithi007.stdout:7/160: symlink d0/d19/l35 0 2024-03-16T09:01:03.027 INFO:tasks.workunit.client.0.smithi007.stdout:7/161: dread d0/d19/d1f/f2c [0,4194304] 0 2024-03-16T09:01:03.027 INFO:tasks.workunit.client.0.smithi007.stdout:7/162: stat d0/d19/l35 0 2024-03-16T09:01:03.028 INFO:tasks.workunit.client.0.smithi007.stdout:2/209: dwrite dc/df/f34 [0,4194304] 0 2024-03-16T09:01:03.028 INFO:tasks.workunit.client.0.smithi007.stdout:2/210: chown dc/df/f18 1735 1 2024-03-16T09:01:03.034 INFO:tasks.workunit.client.0.smithi007.stdout:7/163: dread d0/d19/f27 [0,4194304] 0 2024-03-16T09:01:03.034 INFO:tasks.workunit.client.0.smithi007.stdout:7/164: fdatasync d0/fa 0 2024-03-16T09:01:03.053 INFO:tasks.workunit.client.0.smithi007.stdout:8/282: dwrite d0/f16 [4194304,4194304] 0 2024-03-16T09:01:03.053 INFO:tasks.workunit.client.0.smithi007.stdout:2/211: dwrite dc/df/d24/d28/d2e/d35/f38 [0,4194304] 0 2024-03-16T09:01:03.055 INFO:tasks.workunit.client.0.smithi007.stdout:8/283: symlink d0/d2/l65 0 2024-03-16T09:01:03.055 INFO:tasks.workunit.client.0.smithi007.stdout:8/284: stat d0/dc/dd/f2c 0 2024-03-16T09:01:03.055 INFO:tasks.workunit.client.0.smithi007.stdout:8/285: dread - d0/d26/d2d/f61 zero size 2024-03-16T09:01:03.069 INFO:tasks.workunit.client.0.smithi007.stdout:2/212: dwrite dc/f20 [0,4194304] 0 2024-03-16T09:01:03.071 INFO:tasks.workunit.client.0.smithi007.stdout:2/213: unlink dc/df/f2b 0 2024-03-16T09:01:03.076 INFO:tasks.workunit.client.0.smithi007.stdout:2/214: rmdir dc 39 2024-03-16T09:01:03.076 INFO:tasks.workunit.client.0.smithi007.stdout:2/215: truncate dc/f3f 285947 0 2024-03-16T09:01:03.076 INFO:tasks.workunit.client.0.smithi007.stdout:2/216: write dc/df/d24/d28/f39 [334994,31610] 0 2024-03-16T09:01:03.077 INFO:tasks.workunit.client.0.smithi007.stdout:2/217: symlink dc/l42 0 2024-03-16T09:01:03.077 INFO:tasks.workunit.client.0.smithi007.stdout:2/218: fsync dc/f20 0 2024-03-16T09:01:03.078 INFO:tasks.workunit.client.0.smithi007.stdout:2/219: mknod dc/df/d24/d28/d2e/c43 0 2024-03-16T09:01:03.079 INFO:tasks.workunit.client.0.smithi007.stdout:3/237: sync 2024-03-16T09:01:03.079 INFO:tasks.workunit.client.0.smithi007.stdout:2/220: mknod dc/c44 0 2024-03-16T09:01:03.080 INFO:tasks.workunit.client.0.smithi007.stdout:3/238: mkdir d15/d49 0 2024-03-16T09:01:03.090 INFO:tasks.workunit.client.0.smithi007.stdout:2/221: dread dc/df/f34 [0,4194304] 0 2024-03-16T09:01:03.099 INFO:tasks.workunit.client.0.smithi007.stdout:3/239: dwrite d15/d18/f23 [0,4194304] 0 2024-03-16T09:01:03.100 INFO:tasks.workunit.client.0.smithi007.stdout:3/240: symlink d15/d18/d1f/l4a 0 2024-03-16T09:01:03.101 INFO:tasks.workunit.client.0.smithi007.stdout:2/222: dwrite dc/f3f [0,4194304] 0 2024-03-16T09:01:03.104 INFO:tasks.workunit.client.0.smithi007.stdout:2/223: creat dc/df/d24/d28/d2e/f45 x:0 0 0 2024-03-16T09:01:03.104 INFO:tasks.workunit.client.0.smithi007.stdout:2/224: write dc/df/d24/d28/d2e/f45 [233175,27587] 0 2024-03-16T09:01:03.112 INFO:tasks.workunit.client.0.smithi007.stdout:3/241: dread d15/f1e [0,4194304] 0 2024-03-16T09:01:03.113 INFO:tasks.workunit.client.0.smithi007.stdout:3/242: mknod d15/d49/c4b 0 2024-03-16T09:01:03.114 INFO:tasks.workunit.client.0.smithi007.stdout:3/243: truncate d15/f1e 899796 0 2024-03-16T09:01:03.115 INFO:tasks.workunit.client.0.smithi007.stdout:3/244: mkdir d15/d18/d1f/d29/d3e/d4c 0 2024-03-16T09:01:03.118 INFO:tasks.workunit.client.0.smithi007.stdout:2/225: dwrite dc/df/f29 [0,4194304] 0 2024-03-16T09:01:03.119 INFO:tasks.workunit.client.0.smithi007.stdout:2/226: truncate dc/df/d24/d28/f39 282099 0 2024-03-16T09:01:03.120 INFO:tasks.workunit.client.0.smithi007.stdout:2/227: mknod dc/df/d24/d28/d2e/d35/c46 0 2024-03-16T09:01:03.122 INFO:tasks.workunit.client.0.smithi007.stdout:2/228: read dc/fd [344135,130060] 0 2024-03-16T09:01:03.123 INFO:tasks.workunit.client.0.smithi007.stdout:2/229: stat lb 0 2024-03-16T09:01:03.124 INFO:tasks.workunit.client.0.smithi007.stdout:3/245: read d15/f1e [366859,5736] 0 2024-03-16T09:01:03.126 INFO:tasks.workunit.client.0.smithi007.stdout:3/246: creat d15/f4d x:0 0 0 2024-03-16T09:01:03.138 INFO:tasks.workunit.client.0.smithi007.stdout:2/230: read dc/df/d24/d28/d2e/d35/f38 [2991611,48393] 0 2024-03-16T09:01:03.138 INFO:tasks.workunit.client.0.smithi007.stdout:2/231: dread - dc/f3a zero size 2024-03-16T09:01:03.138 INFO:tasks.workunit.client.0.smithi007.stdout:2/232: fsync dc/fd 0 2024-03-16T09:01:03.139 INFO:tasks.workunit.client.0.smithi007.stdout:3/247: read d15/f16 [3065033,70873] 0 2024-03-16T09:01:03.139 INFO:tasks.workunit.client.0.smithi007.stdout:2/233: rename dc/df/l19 to dc/df/l47 0 2024-03-16T09:01:03.139 INFO:tasks.workunit.client.0.smithi007.stdout:2/234: truncate dc/df/f2d 888836 0 2024-03-16T09:01:03.140 INFO:tasks.workunit.client.0.smithi007.stdout:3/248: rename f8 to d15/d18/d1f/d29/d43/f4e 0 2024-03-16T09:01:03.141 INFO:tasks.workunit.client.0.smithi007.stdout:2/235: mkdir dc/df/d24/d28/d48 0 2024-03-16T09:01:03.142 INFO:tasks.workunit.client.0.smithi007.stdout:3/249: creat d15/d18/f4f x:0 0 0 2024-03-16T09:01:03.142 INFO:tasks.workunit.client.0.smithi007.stdout:2/236: mkdir dc/df/d24/d28/d2e/d35/d49 0 2024-03-16T09:01:03.144 INFO:tasks.workunit.client.0.smithi007.stdout:2/237: truncate dc/df/f34 578230 0 2024-03-16T09:01:03.162 INFO:tasks.workunit.client.0.smithi007.stdout:3/250: dwrite d15/d18/f19 [0,4194304] 0 2024-03-16T09:01:03.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:03 smithi007 ceph-mon[29470]: pgmap v15: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 8.5 MiB/s wr, 1.04k op/s 2024-03-16T09:01:03.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:03 smithi007 ceph-mon[29470]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:01:03.819 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:03 smithi007 ceph-mon[29470]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:01:03.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:03 smithi007 ceph-mon[29470]: Reconfiguring node-exporter.smithi007 (dependencies changed)... 2024-03-16T09:01:03.820 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:03 smithi007 ceph-mon[29470]: Reconfiguring daemon node-exporter.smithi007 on smithi007 2024-03-16T09:01:03.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:03 smithi019 ceph-mon[36407]: pgmap v15: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 8.5 MiB/s wr, 1.04k op/s 2024-03-16T09:01:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:03 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:01:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:03 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:01:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:03 smithi019 ceph-mon[36407]: Reconfiguring node-exporter.smithi007 (dependencies changed)... 2024-03-16T09:01:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:03 smithi019 ceph-mon[36407]: Reconfiguring daemon node-exporter.smithi007 on smithi007 2024-03-16T09:01:04.678 INFO:tasks.workunit.client.1.smithi019.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-16T09:01:04.681 INFO:tasks.workunit.client.1.smithi019.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-16T09:01:04.681 INFO:tasks.workunit.client.1.smithi019.stderr:+ make 2024-03-16T09:01:05.132 INFO:tasks.workunit.client.1.smithi019.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-16T09:01:05.292 INFO:tasks.workunit.client.0.smithi007.stdout:5/149: sync 2024-03-16T09:01:05.292 INFO:tasks.workunit.client.0.smithi007.stdout:8/286: sync 2024-03-16T09:01:05.292 INFO:tasks.workunit.client.0.smithi007.stdout:3/251: sync 2024-03-16T09:01:05.292 INFO:tasks.workunit.client.0.smithi007.stdout:2/238: sync 2024-03-16T09:01:05.293 INFO:tasks.workunit.client.0.smithi007.stdout:4/138: sync 2024-03-16T09:01:05.293 INFO:tasks.workunit.client.0.smithi007.stdout:1/149: sync 2024-03-16T09:01:05.293 INFO:tasks.workunit.client.0.smithi007.stdout:0/109: sync 2024-03-16T09:01:05.293 INFO:tasks.workunit.client.0.smithi007.stdout:9/222: sync 2024-03-16T09:01:05.293 INFO:tasks.workunit.client.0.smithi007.stdout:6/195: sync 2024-03-16T09:01:05.293 INFO:tasks.workunit.client.0.smithi007.stdout:6/196: chown d2/d13/d1f/d34/c39 394819 1 2024-03-16T09:01:05.293 INFO:tasks.workunit.client.0.smithi007.stdout:6/197: chown d2/d13/d24 45826693 1 2024-03-16T09:01:05.293 INFO:tasks.workunit.client.0.smithi007.stdout:7/165: sync 2024-03-16T09:01:05.312 INFO:tasks.workunit.client.0.smithi007.stdout:2/239: dwrite dc/df/f2d [0,4194304] 0 2024-03-16T09:01:05.312 INFO:tasks.workunit.client.0.smithi007.stdout:2/240: chown dc/f20 22093502 1 2024-03-16T09:01:05.312 INFO:tasks.workunit.client.0.smithi007.stdout:2/241: stat dc/df/d24/d28/d2e 0 2024-03-16T09:01:05.313 INFO:tasks.workunit.client.0.smithi007.stdout:2/242: dread dc/df/d24/d28/d2e/f45 [0,4194304] 0 2024-03-16T09:01:05.317 INFO:tasks.workunit.client.0.smithi007.stdout:3/252: creat d15/d49/f50 x:0 0 0 2024-03-16T09:01:05.317 INFO:tasks.workunit.client.0.smithi007.stdout:1/150: dwrite d9/f24 [0,4194304] 0 2024-03-16T09:01:05.318 INFO:tasks.workunit.client.0.smithi007.stdout:1/151: getdents d9/dd/d1b/d2d 0 2024-03-16T09:01:05.318 INFO:tasks.workunit.client.0.smithi007.stdout:8/287: creat d0/d2/f66 x:0 0 0 2024-03-16T09:01:05.319 INFO:tasks.workunit.client.0.smithi007.stdout:4/139: symlink d4/d7/l2b 0 2024-03-16T09:01:05.321 INFO:tasks.workunit.client.0.smithi007.stdout:0/110: link d3/f8 d3/f28 0 2024-03-16T09:01:05.323 INFO:tasks.workunit.client.0.smithi007.stdout:9/223: creat d24/f43 x:0 0 0 2024-03-16T09:01:05.323 INFO:tasks.workunit.client.0.smithi007.stdout:9/224: fdatasync f1d 0 2024-03-16T09:01:05.323 INFO:tasks.workunit.client.0.smithi007.stdout:9/225: dread - d24/f43 zero size 2024-03-16T09:01:05.323 INFO:tasks.workunit.client.0.smithi007.stdout:7/166: dwrite d0/d19/f2d [0,4194304] 0 2024-03-16T09:01:05.570 INFO:tasks.workunit.client.1.smithi019.stderr:++ readlink -f fsstress 2024-03-16T09:01:05.571 INFO:tasks.workunit.client.1.smithi019.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-16T09:01:05.571 INFO:tasks.workunit.client.1.smithi019.stderr:+ popd 2024-03-16T09:01:05.571 INFO:tasks.workunit.client.1.smithi019.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-16T09:01:05.571 INFO:tasks.workunit.client.1.smithi019.stderr:+ popd 2024-03-16T09:01:05.572 INFO:tasks.workunit.client.1.smithi019.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-16T09:01:05.572 INFO:tasks.workunit.client.1.smithi019.stderr:++ mktemp -d -p . 2024-03-16T09:01:05.573 INFO:tasks.workunit.client.1.smithi019.stderr:+ T=./tmp.ONg0mYJCmf 2024-03-16T09:01:05.573 INFO:tasks.workunit.client.1.smithi019.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.ONg0mYJCmf -l 1 -n 1000 -p 10 -v 2024-03-16T09:01:05.574 INFO:tasks.workunit.client.1.smithi019.stdout:seed = 1711119629 2024-03-16T09:01:05.577 INFO:tasks.workunit.client.1.smithi019.stdout:0/0: dwrite - no filename 2024-03-16T09:01:05.579 INFO:tasks.workunit.client.1.smithi019.stdout:1/0: write - no filename 2024-03-16T09:01:05.579 INFO:tasks.workunit.client.1.smithi019.stdout:1/1: dwrite - no filename 2024-03-16T09:01:05.579 INFO:tasks.workunit.client.1.smithi019.stdout:1/2: rename - no filename 2024-03-16T09:01:05.579 INFO:tasks.workunit.client.1.smithi019.stdout:1/3: chown . 132590816 1 2024-03-16T09:01:05.579 INFO:tasks.workunit.client.1.smithi019.stdout:1/4: read - no filename 2024-03-16T09:01:05.579 INFO:tasks.workunit.client.1.smithi019.stdout:1/5: chown . 0 1 2024-03-16T09:01:05.579 INFO:tasks.workunit.client.1.smithi019.stdout:1/6: rename - no filename 2024-03-16T09:01:05.580 INFO:tasks.workunit.client.1.smithi019.stdout:0/1: mknod c0 0 2024-03-16T09:01:05.581 INFO:tasks.workunit.client.1.smithi019.stdout:2/0: chown . 796390 1 2024-03-16T09:01:05.581 INFO:tasks.workunit.client.1.smithi019.stdout:1/7: creat f0 x:0 0 0 2024-03-16T09:01:05.581 INFO:tasks.workunit.client.1.smithi019.stdout:1/8: dread - f0 zero size 2024-03-16T09:01:05.582 INFO:tasks.workunit.client.1.smithi019.stdout:1/9: dread - f0 zero size 2024-03-16T09:01:05.582 INFO:tasks.workunit.client.1.smithi019.stdout:0/2: link c0 c1 0 2024-03-16T09:01:05.583 INFO:tasks.workunit.client.1.smithi019.stdout:0/3: read - no filename 2024-03-16T09:01:05.584 INFO:tasks.workunit.client.1.smithi019.stdout:3/0: truncate - no filename 2024-03-16T09:01:05.584 INFO:tasks.workunit.client.1.smithi019.stdout:2/1: mknod c0 0 2024-03-16T09:01:05.586 INFO:tasks.workunit.client.1.smithi019.stdout:1/10: mknod c1 0 2024-03-16T09:01:05.586 INFO:tasks.workunit.client.1.smithi019.stdout:0/4: stat c0 0 2024-03-16T09:01:05.588 INFO:tasks.workunit.client.1.smithi019.stdout:4/0: truncate - no filename 2024-03-16T09:01:05.588 INFO:tasks.workunit.client.1.smithi019.stdout:4/1: dwrite - no filename 2024-03-16T09:01:05.589 INFO:tasks.workunit.client.1.smithi019.stdout:3/1: creat f0 x:0 0 0 2024-03-16T09:01:05.590 INFO:tasks.workunit.client.1.smithi019.stdout:2/2: creat f1 x:0 0 0 2024-03-16T09:01:05.590 INFO:tasks.workunit.client.1.smithi019.stdout:0/5: stat c1 0 2024-03-16T09:01:05.592 INFO:tasks.workunit.client.1.smithi019.stdout:5/0: dread - no filename 2024-03-16T09:01:05.592 INFO:tasks.workunit.client.1.smithi019.stdout:5/1: rename - no filename 2024-03-16T09:01:05.592 INFO:tasks.workunit.client.1.smithi019.stdout:5/2: dread - no filename 2024-03-16T09:01:05.592 INFO:tasks.workunit.client.1.smithi019.stdout:5/3: write - no filename 2024-03-16T09:01:05.592 INFO:tasks.workunit.client.1.smithi019.stdout:5/4: rename - no filename 2024-03-16T09:01:05.593 INFO:tasks.workunit.client.1.smithi019.stdout:5/5: write - no filename 2024-03-16T09:01:05.593 INFO:tasks.workunit.client.1.smithi019.stdout:5/6: dwrite - no filename 2024-03-16T09:01:05.593 INFO:tasks.workunit.client.1.smithi019.stdout:4/2: creat f0 x:0 0 0 2024-03-16T09:01:05.594 INFO:tasks.workunit.client.1.smithi019.stdout:4/3: read - f0 zero size 2024-03-16T09:01:05.594 INFO:tasks.workunit.client.1.smithi019.stdout:0/6: creat f2 x:0 0 0 2024-03-16T09:01:05.597 INFO:tasks.workunit.client.1.smithi019.stdout:4/4: rename f0 to f1 0 2024-03-16T09:01:05.597 INFO:tasks.workunit.client.1.smithi019.stdout:5/7: mknod c0 0 2024-03-16T09:01:05.597 INFO:tasks.workunit.client.1.smithi019.stdout:5/8: write - no filename 2024-03-16T09:01:05.599 INFO:tasks.workunit.client.1.smithi019.stdout:5/9: unlink c0 0 2024-03-16T09:01:05.599 INFO:tasks.workunit.client.1.smithi019.stdout:5/10: fsync - no filename 2024-03-16T09:01:05.599 INFO:tasks.workunit.client.1.smithi019.stdout:5/11: link - no file 2024-03-16T09:01:05.599 INFO:tasks.workunit.client.1.smithi019.stdout:5/12: unlink - no file 2024-03-16T09:01:05.599 INFO:tasks.workunit.client.1.smithi019.stdout:5/13: chown . 42196137 1 2024-03-16T09:01:05.599 INFO:tasks.workunit.client.1.smithi019.stdout:7/0: dread - no filename 2024-03-16T09:01:05.599 INFO:tasks.workunit.client.1.smithi019.stdout:7/1: getdents . 0 2024-03-16T09:01:05.600 INFO:tasks.workunit.client.1.smithi019.stdout:6/0: symlink l0 0 2024-03-16T09:01:05.601 INFO:tasks.workunit.client.1.smithi019.stdout:5/14: creat f1 x:0 0 0 2024-03-16T09:01:05.601 INFO:tasks.workunit.client.1.smithi019.stdout:5/15: creat f2 x:0 0 0 2024-03-16T09:01:05.601 INFO:tasks.workunit.client.1.smithi019.stdout:5/16: chown f1 14 1 2024-03-16T09:01:05.601 INFO:tasks.workunit.client.1.smithi019.stdout:5/17: dread - f1 zero size 2024-03-16T09:01:05.601 INFO:tasks.workunit.client.1.smithi019.stdout:5/18: chown f1 902 1 2024-03-16T09:01:05.602 INFO:tasks.workunit.client.1.smithi019.stdout:5/19: creat f3 x:0 0 0 2024-03-16T09:01:05.602 INFO:tasks.workunit.client.1.smithi019.stdout:5/20: stat f2 0 2024-03-16T09:01:05.602 INFO:tasks.workunit.client.1.smithi019.stdout:5/21: rmdir - no directory 2024-03-16T09:01:05.602 INFO:tasks.workunit.client.1.smithi019.stdout:5/22: truncate f3 162980 0 2024-03-16T09:01:05.602 INFO:tasks.workunit.client.1.smithi019.stdout:8/0: truncate - no filename 2024-03-16T09:01:05.602 INFO:tasks.workunit.client.1.smithi019.stdout:7/2: mknod c0 0 2024-03-16T09:01:05.602 INFO:tasks.workunit.client.1.smithi019.stdout:7/3: dwrite - no filename 2024-03-16T09:01:05.603 INFO:tasks.workunit.client.1.smithi019.stdout:5/23: dread f3 [0,4194304] 0 2024-03-16T09:01:05.603 INFO:tasks.workunit.client.1.smithi019.stdout:6/1: mknod c1 0 2024-03-16T09:01:05.608 INFO:tasks.workunit.client.1.smithi019.stdout:8/1: creat f0 x:0 0 0 2024-03-16T09:01:05.610 INFO:tasks.workunit.client.1.smithi019.stdout:9/0: dwrite - no filename 2024-03-16T09:01:05.610 INFO:tasks.workunit.client.1.smithi019.stdout:9/1: dwrite - no filename 2024-03-16T09:01:05.610 INFO:tasks.workunit.client.1.smithi019.stdout:9/2: write - no filename 2024-03-16T09:01:05.610 INFO:tasks.workunit.client.1.smithi019.stdout:9/3: chown . 169 1 2024-03-16T09:01:05.613 INFO:tasks.workunit.client.1.smithi019.stdout:6/2: link c1 c2 0 2024-03-16T09:01:05.613 INFO:tasks.workunit.client.1.smithi019.stdout:6/3: chown l0 5955 1 2024-03-16T09:01:05.615 INFO:tasks.workunit.client.1.smithi019.stdout:8/2: symlink l1 0 2024-03-16T09:01:05.615 INFO:tasks.workunit.client.1.smithi019.stdout:8/3: chown f0 0 1 2024-03-16T09:01:05.615 INFO:tasks.workunit.client.1.smithi019.stdout:8/4: creat f2 x:0 0 0 2024-03-16T09:01:05.616 INFO:tasks.workunit.client.1.smithi019.stdout:8/5: truncate f0 463078 0 2024-03-16T09:01:05.616 INFO:tasks.workunit.client.1.smithi019.stdout:9/4: creat f0 x:0 0 0 2024-03-16T09:01:05.619 INFO:tasks.workunit.client.1.smithi019.stdout:6/4: creat f3 x:0 0 0 2024-03-16T09:01:05.619 INFO:tasks.workunit.client.1.smithi019.stdout:6/5: fdatasync f3 0 2024-03-16T09:01:05.619 INFO:tasks.workunit.client.1.smithi019.stdout:6/6: creat f4 x:0 0 0 2024-03-16T09:01:05.619 INFO:tasks.workunit.client.1.smithi019.stdout:0/7: getdents . 0 2024-03-16T09:01:05.623 INFO:tasks.workunit.client.1.smithi019.stdout:4/5: link f1 f2 0 2024-03-16T09:01:05.623 INFO:tasks.workunit.client.1.smithi019.stdout:3/2: dwrite f0 [0,4194304] 0 2024-03-16T09:01:05.625 INFO:tasks.workunit.client.1.smithi019.stdout:5/24: dwrite f2 [0,4194304] 0 2024-03-16T09:01:05.625 INFO:tasks.workunit.client.1.smithi019.stdout:1/11: dwrite f0 [0,4194304] 0 2024-03-16T09:01:05.626 INFO:tasks.workunit.client.1.smithi019.stdout:4/6: rename f1 to f3 0 2024-03-16T09:01:05.627 INFO:tasks.workunit.client.1.smithi019.stdout:4/7: fdatasync f2 0 2024-03-16T09:01:05.628 INFO:tasks.workunit.client.1.smithi019.stdout:4/8: chown f2 1097 1 2024-03-16T09:01:05.630 INFO:tasks.workunit.client.1.smithi019.stdout:5/25: link f1 f4 0 2024-03-16T09:01:05.633 INFO:tasks.workunit.client.1.smithi019.stdout:2/3: dwrite f1 [0,4194304] 0 2024-03-16T09:01:05.633 INFO:tasks.workunit.client.1.smithi019.stdout:2/4: creat f2 x:0 0 0 2024-03-16T09:01:05.634 INFO:tasks.workunit.client.1.smithi019.stdout:6/7: unlink f4 0 2024-03-16T09:01:05.634 INFO:tasks.workunit.client.1.smithi019.stdout:9/5: dwrite f0 [0,4194304] 0 2024-03-16T09:01:05.635 INFO:tasks.workunit.client.1.smithi019.stdout:4/9: mkdir d4 0 2024-03-16T09:01:05.635 INFO:tasks.workunit.client.1.smithi019.stdout:4/10: write f2 [181321,107176] 0 2024-03-16T09:01:05.635 INFO:tasks.workunit.client.1.smithi019.stdout:5/26: symlink l5 0 2024-03-16T09:01:05.637 INFO:tasks.workunit.client.1.smithi019.stdout:3/3: read f0 [4096460,109583] 0 2024-03-16T09:01:05.640 INFO:tasks.workunit.client.1.smithi019.stdout:7/4: getdents . 0 2024-03-16T09:01:05.646 INFO:tasks.workunit.client.1.smithi019.stdout:0/8: dwrite f2 [0,4194304] 0 2024-03-16T09:01:05.647 INFO:tasks.workunit.client.1.smithi019.stdout:1/12: dwrite f0 [0,4194304] 0 2024-03-16T09:01:05.649 INFO:tasks.workunit.client.1.smithi019.stdout:9/6: mkdir d1 0 2024-03-16T09:01:05.650 INFO:tasks.workunit.client.1.smithi019.stdout:8/6: getdents . 0 2024-03-16T09:01:05.651 INFO:tasks.workunit.client.1.smithi019.stdout:8/7: dread f0 [0,4194304] 0 2024-03-16T09:01:05.652 INFO:tasks.workunit.client.1.smithi019.stdout:8/8: write f2 [752207,115329] 0 2024-03-16T09:01:05.653 INFO:tasks.workunit.client.1.smithi019.stdout:5/27: mkdir d6 0 2024-03-16T09:01:05.653 INFO:tasks.workunit.client.1.smithi019.stdout:5/28: chown l5 151921911 1 2024-03-16T09:01:05.656 INFO:tasks.workunit.client.1.smithi019.stdout:7/5: rename c0 to c1 0 2024-03-16T09:01:05.656 INFO:tasks.workunit.client.1.smithi019.stdout:7/6: truncate - no filename 2024-03-16T09:01:05.656 INFO:tasks.workunit.client.1.smithi019.stdout:7/7: rmdir - no directory 2024-03-16T09:01:05.656 INFO:tasks.workunit.client.1.smithi019.stdout:7/8: dread - no filename 2024-03-16T09:01:05.656 INFO:tasks.workunit.client.1.smithi019.stdout:7/9: write - no filename 2024-03-16T09:01:05.656 INFO:tasks.workunit.client.1.smithi019.stdout:7/10: write - no filename 2024-03-16T09:01:05.657 INFO:tasks.workunit.client.1.smithi019.stdout:7/11: truncate - no filename 2024-03-16T09:01:05.657 INFO:tasks.workunit.client.1.smithi019.stdout:7/12: fsync - no filename 2024-03-16T09:01:05.657 INFO:tasks.workunit.client.1.smithi019.stdout:7/13: fsync - no filename 2024-03-16T09:01:05.658 INFO:tasks.workunit.client.1.smithi019.stdout:0/9: write f2 [1351171,96274] 0 2024-03-16T09:01:05.658 INFO:tasks.workunit.client.1.smithi019.stdout:5/29: dread f3 [0,4194304] 0 2024-03-16T09:01:05.662 INFO:tasks.workunit.client.1.smithi019.stdout:9/7: write f0 [2982804,84009] 0 2024-03-16T09:01:05.663 INFO:tasks.workunit.client.1.smithi019.stdout:6/8: dwrite f3 [0,4194304] 0 2024-03-16T09:01:05.663 INFO:tasks.workunit.client.1.smithi019.stdout:6/9: chown l0 2910936 1 2024-03-16T09:01:05.663 INFO:tasks.workunit.client.1.smithi019.stdout:3/4: dwrite f0 [0,4194304] 0 2024-03-16T09:01:05.664 INFO:tasks.workunit.client.1.smithi019.stdout:2/5: truncate f1 3667938 0 2024-03-16T09:01:05.666 INFO:tasks.workunit.client.0.smithi007.stdout:2/243: mkdir dc/df/d24/d28/d2e/d4a 0 2024-03-16T09:01:05.667 INFO:tasks.workunit.client.1.smithi019.stdout:8/9: creat f3 x:0 0 0 2024-03-16T09:01:05.667 INFO:tasks.workunit.client.0.smithi007.stdout:3/253: mknod d15/d18/d1f/d3b/c51 0 2024-03-16T09:01:05.670 INFO:tasks.workunit.client.0.smithi007.stdout:8/288: creat d0/d2/f67 x:0 0 0 2024-03-16T09:01:05.671 INFO:tasks.workunit.client.0.smithi007.stdout:0/111: mknod d3/d9/da/d18/d27/c29 0 2024-03-16T09:01:05.673 INFO:tasks.workunit.client.0.smithi007.stdout:6/198: link d2/d13/c2d d2/d11/c43 0 2024-03-16T09:01:05.674 INFO:tasks.workunit.client.1.smithi019.stdout:7/14: rename c1 to c2 0 2024-03-16T09:01:05.674 INFO:tasks.workunit.client.0.smithi007.stdout:7/167: symlink d0/d19/l36 0 2024-03-16T09:01:05.676 INFO:tasks.workunit.client.0.smithi007.stdout:6/199: creat d2/d13/d3f/f44 x:0 0 0 2024-03-16T09:01:05.676 INFO:tasks.workunit.client.1.smithi019.stdout:5/30: dwrite f2 [0,4194304] 0 2024-03-16T09:01:05.676 INFO:tasks.workunit.client.1.smithi019.stdout:5/31: truncate f4 496663 0 2024-03-16T09:01:05.677 INFO:tasks.workunit.client.0.smithi007.stdout:0/112: dread d3/d9/da/d18/d21/f12 [0,4194304] 0 2024-03-16T09:01:05.677 INFO:tasks.workunit.client.0.smithi007.stdout:0/113: chown d3/d9/db/d1c 135 1 2024-03-16T09:01:05.678 INFO:tasks.workunit.client.0.smithi007.stdout:7/168: mknod d0/dc/c37 0 2024-03-16T09:01:05.678 INFO:tasks.workunit.client.1.smithi019.stdout:3/5: dread f0 [0,4194304] 0 2024-03-16T09:01:05.679 INFO:tasks.workunit.client.1.smithi019.stdout:3/6: stat f0 0 2024-03-16T09:01:05.679 INFO:tasks.workunit.client.1.smithi019.stdout:3/7: fdatasync f0 0 2024-03-16T09:01:05.679 INFO:tasks.workunit.client.0.smithi007.stdout:6/200: creat d2/d10/d12/d15/f45 x:0 0 0 2024-03-16T09:01:05.679 INFO:tasks.workunit.client.0.smithi007.stdout:6/201: stat d2/fb 0 2024-03-16T09:01:05.680 INFO:tasks.workunit.client.1.smithi019.stdout:0/10: creat f3 x:0 0 0 2024-03-16T09:01:05.681 INFO:tasks.workunit.client.0.smithi007.stdout:0/114: creat d3/d9/db/d1c/f2a x:0 0 0 2024-03-16T09:01:05.683 INFO:tasks.workunit.client.0.smithi007.stdout:7/169: truncate d0/f3 2412410 0 2024-03-16T09:01:05.684 INFO:tasks.workunit.client.0.smithi007.stdout:2/244: write dc/f20 [3833761,18168] 0 2024-03-16T09:01:05.684 INFO:tasks.workunit.client.0.smithi007.stdout:6/202: rmdir d2/d13/d1f/d34 39 2024-03-16T09:01:05.684 INFO:tasks.workunit.client.0.smithi007.stdout:6/203: chown d2/c3 3351 1 2024-03-16T09:01:05.686 INFO:tasks.workunit.client.1.smithi019.stdout:1/13: unlink f0 0 2024-03-16T09:01:05.687 INFO:tasks.workunit.client.1.smithi019.stdout:1/14: truncate - no filename 2024-03-16T09:01:05.687 INFO:tasks.workunit.client.1.smithi019.stdout:1/15: dwrite - no filename 2024-03-16T09:01:05.687 INFO:tasks.workunit.client.1.smithi019.stdout:2/6: write f1 [4003320,5505] 0 2024-03-16T09:01:05.688 INFO:tasks.workunit.client.0.smithi007.stdout:7/170: mkdir d0/dc/d38 0 2024-03-16T09:01:05.693 INFO:tasks.workunit.client.0.smithi007.stdout:2/245: read f7 [2965453,102870] 0 2024-03-16T09:01:05.694 INFO:tasks.workunit.client.1.smithi019.stdout:1/16: unlink c1 0 2024-03-16T09:01:05.698 INFO:tasks.workunit.client.0.smithi007.stdout:7/171: mknod d0/d19/d1f/c39 0 2024-03-16T09:01:05.702 INFO:tasks.workunit.client.1.smithi019.stdout:8/10: getdents . 0 2024-03-16T09:01:05.702 INFO:tasks.workunit.client.1.smithi019.stdout:8/11: write f3 [936933,121448] 0 2024-03-16T09:01:05.702 INFO:tasks.workunit.client.1.smithi019.stdout:8/12: chown l1 3 1 2024-03-16T09:01:05.702 INFO:tasks.workunit.client.0.smithi007.stdout:2/246: rename dc/df/d24/d28/c36 to dc/df/d24/d28/d2e/d4a/c4b 0 2024-03-16T09:01:05.708 INFO:tasks.workunit.client.0.smithi007.stdout:2/247: rename dc/df/d24/d28/d48 to dc/df/d24/d28/d4c 0 2024-03-16T09:01:05.710 INFO:tasks.workunit.client.0.smithi007.stdout:2/248: creat dc/df/d24/d28/d2e/d35/d49/f4d x:0 0 0 2024-03-16T09:01:05.711 INFO:tasks.workunit.client.1.smithi019.stdout:8/13: rename f3 to f4 0 2024-03-16T09:01:05.715 INFO:tasks.workunit.client.1.smithi019.stdout:8/14: creat f5 x:0 0 0 2024-03-16T09:01:05.716 INFO:tasks.workunit.client.0.smithi007.stdout:2/249: creat dc/df/f4e x:0 0 0 2024-03-16T09:01:05.716 INFO:tasks.workunit.client.0.smithi007.stdout:2/250: write dc/df/f4e [893013,62407] 0 2024-03-16T09:01:05.720 INFO:tasks.workunit.client.1.smithi019.stdout:6/10: dwrite f3 [0,4194304] 0 2024-03-16T09:01:05.723 INFO:tasks.workunit.client.0.smithi007.stdout:2/251: rename dc/f1a to dc/d27/f4f 0 2024-03-16T09:01:05.723 INFO:tasks.workunit.client.0.smithi007.stdout:2/252: fdatasync dc/d27/f4f 0 2024-03-16T09:01:05.724 INFO:tasks.workunit.client.0.smithi007.stdout:2/253: rmdir dc/df/d24 39 2024-03-16T09:01:05.729 INFO:tasks.workunit.client.1.smithi019.stdout:6/11: link l0 l5 0 2024-03-16T09:01:05.731 INFO:tasks.workunit.client.1.smithi019.stdout:6/12: mknod c6 0 2024-03-16T09:01:05.732 INFO:tasks.workunit.client.1.smithi019.stdout:6/13: rmdir - no directory 2024-03-16T09:01:05.734 INFO:tasks.workunit.client.1.smithi019.stdout:6/14: creat f7 x:0 0 0 2024-03-16T09:01:05.734 INFO:tasks.workunit.client.1.smithi019.stdout:6/15: dread - f7 zero size 2024-03-16T09:01:05.734 INFO:tasks.workunit.client.1.smithi019.stdout:6/16: mknod c8 0 2024-03-16T09:01:05.735 INFO:tasks.workunit.client.1.smithi019.stdout:6/17: write f7 [911752,61336] 0 2024-03-16T09:01:05.771 INFO:tasks.workunit.client.1.smithi019.stdout:3/8: dwrite f0 [0,4194304] 0 2024-03-16T09:01:05.789 INFO:tasks.workunit.client.1.smithi019.stdout:3/9: symlink l1 0 2024-03-16T09:01:05.797 INFO:tasks.workunit.client.1.smithi019.stdout:4/11: truncate f3 191517 0 2024-03-16T09:01:05.809 INFO:tasks.workunit.client.0.smithi007.stdout:6/204: dread d2/d11/d1e/f2f [0,4194304] 0 2024-03-16T09:01:05.810 INFO:tasks.workunit.client.0.smithi007.stdout:3/254: dwrite d15/d18/d1f/d29/f31 [0,4194304] 0 2024-03-16T09:01:05.811 INFO:tasks.workunit.client.0.smithi007.stdout:6/205: getdents d2 0 2024-03-16T09:01:05.813 INFO:tasks.workunit.client.0.smithi007.stdout:3/255: chown d15/d18/l24 60 1 2024-03-16T09:01:05.815 INFO:tasks.workunit.client.0.smithi007.stdout:6/206: fsync d2/d11/f21 0 2024-03-16T09:01:05.815 INFO:tasks.workunit.client.0.smithi007.stdout:6/207: write d2/d10/d12/f35 [692573,89076] 0 2024-03-16T09:01:05.815 INFO:tasks.workunit.client.0.smithi007.stdout:6/208: fdatasync d2/fb 0 2024-03-16T09:01:05.821 INFO:tasks.workunit.client.0.smithi007.stdout:3/256: creat d15/d18/d1f/d29/d3e/d4c/f52 x:0 0 0 2024-03-16T09:01:05.821 INFO:tasks.workunit.client.0.smithi007.stdout:6/209: creat d2/f46 x:0 0 0 2024-03-16T09:01:05.821 INFO:tasks.workunit.client.0.smithi007.stdout:6/210: creat d2/d11/f47 x:0 0 0 2024-03-16T09:01:05.822 INFO:tasks.workunit.client.0.smithi007.stdout:6/211: chown c0 783 1 2024-03-16T09:01:05.823 INFO:tasks.workunit.client.1.smithi019.stdout:8/15: read f4 [804712,40599] 0 2024-03-16T09:01:05.825 INFO:tasks.workunit.client.0.smithi007.stdout:0/115: dwrite d3/d9/da/d18/f19 [0,4194304] 0 2024-03-16T09:01:05.829 INFO:tasks.workunit.client.0.smithi007.stdout:0/116: rmdir d3/d9/da/d18/d21 39 2024-03-16T09:01:05.865 INFO:tasks.workunit.client.1.smithi019.stdout:5/32: dwrite f3 [0,4194304] 0 2024-03-16T09:01:05.865 INFO:tasks.workunit.client.1.smithi019.stdout:5/33: chown f1 493 1 2024-03-16T09:01:05.895 INFO:tasks.workunit.client.1.smithi019.stdout:7/15: rename c2 to c3 0 2024-03-16T09:01:05.896 INFO:tasks.workunit.client.1.smithi019.stdout:7/16: write - no filename 2024-03-16T09:01:05.896 INFO:tasks.workunit.client.1.smithi019.stdout:7/17: chown c3 22 1 2024-03-16T09:01:05.896 INFO:tasks.workunit.client.1.smithi019.stdout:7/18: write - no filename 2024-03-16T09:01:05.896 INFO:tasks.workunit.client.1.smithi019.stdout:7/19: truncate - no filename 2024-03-16T09:01:05.899 INFO:tasks.workunit.client.1.smithi019.stdout:4/12: fsync f2 0 2024-03-16T09:01:05.940 INFO:tasks.workunit.client.1.smithi019.stdout:0/11: dread f2 [0,4194304] 0 2024-03-16T09:01:05.940 INFO:tasks.workunit.client.1.smithi019.stdout:0/12: write f2 [3889714,64891] 0 2024-03-16T09:01:05.941 INFO:tasks.workunit.client.1.smithi019.stdout:0/13: rmdir - no directory 2024-03-16T09:01:05.941 INFO:tasks.workunit.client.1.smithi019.stdout:0/14: creat f4 x:0 0 0 2024-03-16T09:01:05.965 INFO:tasks.workunit.client.1.smithi019.stdout:3/10: write f0 [2139039,95704] 0 2024-03-16T09:01:05.971 INFO:tasks.workunit.client.0.smithi007.stdout:2/254: dwrite dc/df/d24/d28/f39 [0,4194304] 0 2024-03-16T09:01:05.975 INFO:tasks.workunit.client.0.smithi007.stdout:2/255: mknod dc/df/d24/d28/d2e/d35/c50 0 2024-03-16T09:01:05.975 INFO:tasks.workunit.client.0.smithi007.stdout:2/256: readlink dc/l42 0 2024-03-16T09:01:05.975 INFO:tasks.workunit.client.0.smithi007.stdout:2/257: read - dc/df/d24/d28/d2e/d35/f40 zero size 2024-03-16T09:01:05.990 INFO:tasks.workunit.client.0.smithi007.stdout:3/257: dwrite d15/d18/f35 [0,4194304] 0 2024-03-16T09:01:05.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:05 smithi007 ceph-mon[29470]: pgmap v16: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 8.5 MiB/s wr, 1.04k op/s 2024-03-16T09:01:05.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:05 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:05.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:05 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:06.012 INFO:tasks.workunit.client.0.smithi007.stdout:7/172: dwrite d0/d19/f2d [0,4194304] 0 2024-03-16T09:01:06.016 INFO:tasks.workunit.client.0.smithi007.stdout:7/173: unlink d0/c34 0 2024-03-16T09:01:06.034 INFO:tasks.workunit.client.1.smithi019.stdout:5/34: dread f1 [0,4194304] 0 2024-03-16T09:01:06.040 INFO:tasks.workunit.client.0.smithi007.stdout:2/258: dread dc/df/d24/d28/d2e/f45 [0,4194304] 0 2024-03-16T09:01:06.040 INFO:tasks.workunit.client.0.smithi007.stdout:7/174: write d0/dc/f15 [2750451,120906] 0 2024-03-16T09:01:06.041 INFO:tasks.workunit.client.0.smithi007.stdout:7/175: readlink d0/dc/d24/l2a 0 2024-03-16T09:01:06.043 INFO:tasks.workunit.client.0.smithi007.stdout:7/176: mkdir d0/dc/d24/d25/d3a 0 2024-03-16T09:01:06.051 INFO:tasks.workunit.client.0.smithi007.stdout:0/117: write d3/d9/db/fc [1400498,87374] 0 2024-03-16T09:01:06.051 INFO:tasks.workunit.client.0.smithi007.stdout:0/118: read - d3/f22 zero size 2024-03-16T09:01:06.051 INFO:tasks.workunit.client.0.smithi007.stdout:0/119: write d3/d9/f26 [549431,56394] 0 2024-03-16T09:01:06.053 INFO:tasks.workunit.client.0.smithi007.stdout:0/120: rename d3/d9/da/d18/f19 to d3/d9/da/d18/f2b 0 2024-03-16T09:01:06.062 INFO:tasks.workunit.client.1.smithi019.stdout:4/13: dread f3 [0,4194304] 0 2024-03-16T09:01:06.069 INFO:tasks.workunit.client.0.smithi007.stdout:0/121: write d3/f6 [1112764,4295] 0 2024-03-16T09:01:06.069 INFO:tasks.workunit.client.0.smithi007.stdout:0/122: write d3/f28 [2370927,127843] 0 2024-03-16T09:01:06.072 INFO:tasks.workunit.client.0.smithi007.stdout:0/123: stat d3/d9/da/d18/d21/c17 0 2024-03-16T09:01:06.072 INFO:tasks.workunit.client.0.smithi007.stdout:0/124: readlink d3/l7 0 2024-03-16T09:01:06.073 INFO:tasks.workunit.client.0.smithi007.stdout:0/125: fdatasync d3/d9/da/d18/f2b 0 2024-03-16T09:01:06.073 INFO:tasks.workunit.client.0.smithi007.stdout:0/126: read - d3/d9/da/d10/d1a/f1b zero size 2024-03-16T09:01:06.074 INFO:tasks.workunit.client.0.smithi007.stdout:0/127: getdents d3/d9/da/d18/d21 0 2024-03-16T09:01:06.112 INFO:tasks.workunit.client.1.smithi019.stdout:9/8: truncate f0 1463254 0 2024-03-16T09:01:06.112 INFO:tasks.workunit.client.1.smithi019.stdout:9/9: chown f0 1693665 1 2024-03-16T09:01:06.112 INFO:tasks.workunit.client.1.smithi019.stdout:6/18: fdatasync f7 0 2024-03-16T09:01:06.112 INFO:tasks.workunit.client.1.smithi019.stdout:3/11: fsync f0 0 2024-03-16T09:01:06.159 INFO:tasks.workunit.client.0.smithi007.stdout:7/177: dread d0/dc/fd [0,4194304] 0 2024-03-16T09:01:06.161 INFO:tasks.workunit.client.0.smithi007.stdout:7/178: creat d0/dc/d24/f3b x:0 0 0 2024-03-16T09:01:06.163 INFO:tasks.workunit.client.0.smithi007.stdout:7/179: unlink d0/d19/f1d 0 2024-03-16T09:01:06.164 INFO:tasks.workunit.client.0.smithi007.stdout:7/180: creat d0/f3c x:0 0 0 2024-03-16T09:01:06.170 INFO:tasks.workunit.client.0.smithi007.stdout:6/212: dwrite d2/d10/d12/d15/f45 [0,4194304] 0 2024-03-16T09:01:06.180 INFO:tasks.workunit.client.0.smithi007.stdout:6/213: read d2/d10/d12/d15/f22 [4171142,33647] 0 2024-03-16T09:01:06.181 INFO:tasks.workunit.client.0.smithi007.stdout:6/214: write d2/d11/f47 [673823,125611] 0 2024-03-16T09:01:06.181 INFO:tasks.workunit.client.0.smithi007.stdout:6/215: read - d2/d11/d1e/f40 zero size 2024-03-16T09:01:06.181 INFO:tasks.workunit.client.1.smithi019.stdout:6/19: dread f3 [0,4194304] 0 2024-03-16T09:01:06.181 INFO:tasks.workunit.client.1.smithi019.stdout:6/20: creat f9 x:0 0 0 2024-03-16T09:01:06.181 INFO:tasks.workunit.client.1.smithi019.stdout:6/21: dread - f9 zero size 2024-03-16T09:01:06.183 INFO:tasks.workunit.client.0.smithi007.stdout:6/216: mknod d2/d11/d3b/c48 0 2024-03-16T09:01:06.184 INFO:tasks.workunit.client.1.smithi019.stdout:8/16: dread f4 [0,4194304] 0 2024-03-16T09:01:06.211 INFO:tasks.workunit.client.0.smithi007.stdout:0/128: truncate d3/d9/da/d18/f2b 3010963 0 2024-03-16T09:01:06.212 INFO:tasks.workunit.client.1.smithi019.stdout:0/15: symlink l5 0 2024-03-16T09:01:06.212 INFO:tasks.workunit.client.1.smithi019.stdout:0/16: dread - f4 zero size 2024-03-16T09:01:06.212 INFO:tasks.workunit.client.1.smithi019.stdout:0/17: dread - f4 zero size 2024-03-16T09:01:06.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:05 smithi019 ceph-mon[36407]: pgmap v16: 65 pgs: 65 active+clean; 195 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 8.5 MiB/s wr, 1.04k op/s 2024-03-16T09:01:06.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:05 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:05 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:06.213 INFO:tasks.workunit.client.0.smithi007.stdout:0/129: mkdir d3/d9/db/d2c 0 2024-03-16T09:01:06.213 INFO:tasks.workunit.client.0.smithi007.stdout:0/130: stat d3/f6 0 2024-03-16T09:01:06.213 INFO:tasks.workunit.client.0.smithi007.stdout:0/131: creat d3/d9/da/d10/d1a/f2d x:0 0 0 2024-03-16T09:01:06.228 INFO:tasks.workunit.client.0.smithi007.stdout:0/132: dwrite d3/d9/da/d18/d21/f13 [0,4194304] 0 2024-03-16T09:01:06.228 INFO:tasks.workunit.client.0.smithi007.stdout:0/133: chown d3/d9/da/d18 0 1 2024-03-16T09:01:06.229 INFO:tasks.workunit.client.0.smithi007.stdout:0/134: fsync d3/f8 0 2024-03-16T09:01:06.229 INFO:tasks.workunit.client.0.smithi007.stdout:0/135: readlink d3/d9/da/l15 0 2024-03-16T09:01:06.229 INFO:tasks.workunit.client.0.smithi007.stdout:0/136: symlink d3/d9/da/d10/l2e 0 2024-03-16T09:01:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:2/259: rename l4 to dc/df/d24/d28/d2e/d4a/l51 0 2024-03-16T09:01:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:2/260: readlink dc/df/l30 0 2024-03-16T09:01:06.257 INFO:tasks.workunit.client.0.smithi007.stdout:0/137: dwrite d3/f8 [0,4194304] 0 2024-03-16T09:01:06.260 INFO:tasks.workunit.client.0.smithi007.stdout:2/261: dwrite dc/f11 [0,4194304] 0 2024-03-16T09:01:06.260 INFO:tasks.workunit.client.0.smithi007.stdout:2/262: stat dc/df/f14 0 2024-03-16T09:01:06.263 INFO:tasks.workunit.client.0.smithi007.stdout:2/263: link dc/df/f29 dc/df/d24/d28/d2e/f52 0 2024-03-16T09:01:06.265 INFO:tasks.workunit.client.0.smithi007.stdout:2/264: mknod dc/df/d24/d28/d2e/d4a/c53 0 2024-03-16T09:01:06.274 INFO:tasks.workunit.client.1.smithi019.stdout:7/20: unlink c3 0 2024-03-16T09:01:06.277 INFO:tasks.workunit.client.0.smithi007.stdout:2/265: write dc/df/f2d [1782267,83001] 0 2024-03-16T09:01:06.278 INFO:tasks.workunit.client.0.smithi007.stdout:2/266: fdatasync dc/fd 0 2024-03-16T09:01:06.278 INFO:tasks.workunit.client.0.smithi007.stdout:0/138: dwrite d3/d9/f26 [0,4194304] 0 2024-03-16T09:01:06.278 INFO:tasks.workunit.client.0.smithi007.stdout:0/139: readlink d3/d9/da/l15 0 2024-03-16T09:01:06.279 INFO:tasks.workunit.client.0.smithi007.stdout:2/267: symlink dc/df/d24/l54 0 2024-03-16T09:01:06.280 INFO:tasks.workunit.client.0.smithi007.stdout:0/140: creat d3/d9/db/d2c/f2f x:0 0 0 2024-03-16T09:01:06.281 INFO:tasks.workunit.client.0.smithi007.stdout:2/268: symlink dc/df/d24/d28/d2e/d35/d49/l55 0 2024-03-16T09:01:06.283 INFO:tasks.workunit.client.0.smithi007.stdout:0/141: unlink d3/l7 0 2024-03-16T09:01:06.285 INFO:tasks.workunit.client.0.smithi007.stdout:2/269: rename dc/df/d24/d28/d2e/d4a/l51 to dc/df/d24/d28/d2e/d35/d49/l56 0 2024-03-16T09:01:06.286 INFO:tasks.workunit.client.0.smithi007.stdout:2/270: creat dc/df/d24/d28/d4c/f57 x:0 0 0 2024-03-16T09:01:06.286 INFO:tasks.workunit.client.0.smithi007.stdout:2/271: chown dc/fd 3931609 1 2024-03-16T09:01:06.287 INFO:tasks.workunit.client.0.smithi007.stdout:2/272: creat dc/df/d24/d28/d4c/f58 x:0 0 0 2024-03-16T09:01:06.287 INFO:tasks.workunit.client.0.smithi007.stdout:2/273: write dc/df/f14 [2743448,126700] 0 2024-03-16T09:01:06.287 INFO:tasks.workunit.client.0.smithi007.stdout:2/274: truncate dc/f20 4624031 0 2024-03-16T09:01:06.287 INFO:tasks.workunit.client.0.smithi007.stdout:2/275: fdatasync dc/df/d24/d28/d2e/d35/f38 0 2024-03-16T09:01:06.289 INFO:tasks.workunit.client.0.smithi007.stdout:2/276: rename dc/df/d24/d28/d2e/d35/f38 to dc/df/d24/d28/d4c/f59 0 2024-03-16T09:01:06.289 INFO:tasks.workunit.client.0.smithi007.stdout:2/277: creat dc/df/f5a x:0 0 0 2024-03-16T09:01:06.293 INFO:tasks.workunit.client.0.smithi007.stdout:0/142: dread d3/d9/f11 [0,4194304] 0 2024-03-16T09:01:06.293 INFO:tasks.workunit.client.0.smithi007.stdout:0/143: creat d3/d9/da/d10/d1a/f30 x:0 0 0 2024-03-16T09:01:06.293 INFO:tasks.workunit.client.0.smithi007.stdout:0/144: readlink d3/d9/da/l15 0 2024-03-16T09:01:06.294 INFO:tasks.workunit.client.0.smithi007.stdout:0/145: mkdir d3/d9/d31 0 2024-03-16T09:01:06.295 INFO:tasks.workunit.client.0.smithi007.stdout:0/146: creat d3/d9/da/d18/d23/f32 x:0 0 0 2024-03-16T09:01:06.297 INFO:tasks.workunit.client.0.smithi007.stdout:0/147: mkdir d3/d9/da/d18/d21/d25/d33 0 2024-03-16T09:01:06.311 INFO:tasks.workunit.client.0.smithi007.stdout:2/278: dwrite dc/f11 [0,4194304] 0 2024-03-16T09:01:06.311 INFO:tasks.workunit.client.0.smithi007.stdout:2/279: creat dc/df/d24/d28/d2e/d35/d49/f5b x:0 0 0 2024-03-16T09:01:06.311 INFO:tasks.workunit.client.0.smithi007.stdout:2/280: creat dc/df/d24/d28/d2e/f5c x:0 0 0 2024-03-16T09:01:06.312 INFO:tasks.workunit.client.0.smithi007.stdout:2/281: read dc/df/f2d [3413122,73415] 0 2024-03-16T09:01:06.312 INFO:tasks.workunit.client.0.smithi007.stdout:2/282: dread - dc/f3a zero size 2024-03-16T09:01:06.314 INFO:tasks.workunit.client.0.smithi007.stdout:2/283: rename dc/c44 to dc/df/d24/d28/d2e/d35/d49/c5d 0 2024-03-16T09:01:06.314 INFO:tasks.workunit.client.0.smithi007.stdout:2/284: chown dc/df/d24/d28/d4c/f59 191692342 1 2024-03-16T09:01:06.315 INFO:tasks.workunit.client.0.smithi007.stdout:0/148: dwrite d3/d9/da/d18/d21/f12 [0,4194304] 0 2024-03-16T09:01:06.317 INFO:tasks.workunit.client.0.smithi007.stdout:0/149: mkdir d3/d9/da/d18/d27/d34 0 2024-03-16T09:01:06.317 INFO:tasks.workunit.client.0.smithi007.stdout:0/150: chown d3/d9/da/d18/d21/d25/d33 95 1 2024-03-16T09:01:06.317 INFO:tasks.workunit.client.0.smithi007.stdout:0/151: dread - d3/d9/da/d18/d23/f32 zero size 2024-03-16T09:01:06.318 INFO:tasks.workunit.client.0.smithi007.stdout:0/152: mknod d3/d9/da/c35 0 2024-03-16T09:01:06.318 INFO:tasks.workunit.client.0.smithi007.stdout:0/153: write d3/d9/db/fc [1020291,61742] 0 2024-03-16T09:01:06.319 INFO:tasks.workunit.client.0.smithi007.stdout:0/154: mkdir d3/d9/da/d18/d23/d36 0 2024-03-16T09:01:06.319 INFO:tasks.workunit.client.0.smithi007.stdout:0/155: dread - d3/d9/db/d1c/f2a zero size 2024-03-16T09:01:06.325 INFO:tasks.workunit.client.0.smithi007.stdout:0/156: read d3/d9/f26 [3690084,68996] 0 2024-03-16T09:01:06.326 INFO:tasks.workunit.client.0.smithi007.stdout:2/285: rename dc/df/d24/d28/d2e/d35/d49 to dc/df/d24/d28/d4c/d5e 0 2024-03-16T09:01:06.326 INFO:tasks.workunit.client.0.smithi007.stdout:2/286: chown dc/df/l30 35 1 2024-03-16T09:01:06.328 INFO:tasks.workunit.client.0.smithi007.stdout:2/287: creat dc/df/d24/d28/d2e/d35/f5f x:0 0 0 2024-03-16T09:01:06.329 INFO:tasks.workunit.client.0.smithi007.stdout:2/288: mknod dc/df/c60 0 2024-03-16T09:01:06.330 INFO:tasks.workunit.client.0.smithi007.stdout:2/289: truncate dc/f10 4038377 0 2024-03-16T09:01:06.331 INFO:tasks.workunit.client.0.smithi007.stdout:2/290: rmdir dc/df/d24 39 2024-03-16T09:01:06.333 INFO:tasks.workunit.client.0.smithi007.stdout:0/157: truncate d3/f6 1535673 0 2024-03-16T09:01:06.333 INFO:tasks.workunit.client.0.smithi007.stdout:0/158: chown d3/d9/da 69133 1 2024-03-16T09:01:06.339 INFO:tasks.workunit.client.0.smithi007.stdout:2/291: write dc/f3f [709560,75666] 0 2024-03-16T09:01:06.341 INFO:tasks.workunit.client.0.smithi007.stdout:2/292: mknod dc/df/d24/d28/d2e/d35/c61 0 2024-03-16T09:01:06.353 INFO:tasks.workunit.client.0.smithi007.stdout:0/159: dwrite d3/d9/db/d1c/f2a [0,4194304] 0 2024-03-16T09:01:06.353 INFO:tasks.workunit.client.0.smithi007.stdout:0/160: fsync d3/d9/da/d10/d1a/f30 0 2024-03-16T09:01:06.354 INFO:tasks.workunit.client.0.smithi007.stdout:0/161: truncate d3/d9/db/d2c/f2f 5123 0 2024-03-16T09:01:06.354 INFO:tasks.workunit.client.0.smithi007.stdout:0/162: write d3/d9/da/d10/d1a/f2d [997150,17331] 0 2024-03-16T09:01:06.355 INFO:tasks.workunit.client.0.smithi007.stdout:0/163: truncate d3/d9/f11 468375 0 2024-03-16T09:01:06.356 INFO:tasks.workunit.client.0.smithi007.stdout:0/164: read - d3/d9/da/d18/d23/f32 zero size 2024-03-16T09:01:06.356 INFO:tasks.workunit.client.0.smithi007.stdout:2/293: write dc/f10 [1477360,12148] 0 2024-03-16T09:01:06.357 INFO:tasks.workunit.client.0.smithi007.stdout:2/294: symlink dc/l62 0 2024-03-16T09:01:06.359 INFO:tasks.workunit.client.0.smithi007.stdout:2/295: creat dc/df/d24/f63 x:0 0 0 2024-03-16T09:01:06.363 INFO:tasks.workunit.client.0.smithi007.stdout:2/296: creat dc/df/d24/d28/d4c/f64 x:0 0 0 2024-03-16T09:01:06.363 INFO:tasks.workunit.client.0.smithi007.stdout:2/297: fdatasync dc/f3f 0 2024-03-16T09:01:06.364 INFO:tasks.workunit.client.0.smithi007.stdout:2/298: creat dc/d27/f65 x:0 0 0 2024-03-16T09:01:06.365 INFO:tasks.workunit.client.0.smithi007.stdout:2/299: dread - dc/df/d24/d28/d2e/f5c zero size 2024-03-16T09:01:06.365 INFO:tasks.workunit.client.0.smithi007.stdout:2/300: creat dc/df/d24/d28/f66 x:0 0 0 2024-03-16T09:01:06.367 INFO:tasks.workunit.client.0.smithi007.stdout:2/301: rename dc/df/f2d to dc/df/d24/d28/d4c/f67 0 2024-03-16T09:01:06.369 INFO:tasks.workunit.client.0.smithi007.stdout:2/302: unlink dc/df/f4e 0 2024-03-16T09:01:06.369 INFO:tasks.workunit.client.0.smithi007.stdout:2/303: dread - dc/df/d24/f63 zero size 2024-03-16T09:01:06.369 INFO:tasks.workunit.client.0.smithi007.stdout:2/304: chown dc/df/d24/d28/d4c/f57 176916360 1 2024-03-16T09:01:06.381 INFO:tasks.workunit.client.0.smithi007.stdout:2/305: dwrite dc/df/d24/d28/f66 [0,4194304] 0 2024-03-16T09:01:06.383 INFO:tasks.workunit.client.0.smithi007.stdout:2/306: rename dc/f20 to dc/f68 0 2024-03-16T09:01:06.384 INFO:tasks.workunit.client.0.smithi007.stdout:2/307: symlink dc/df/d24/d28/d4c/d5e/l69 0 2024-03-16T09:01:06.572 INFO:tasks.workunit.client.1.smithi019.stdout:5/35: creat d6/f7 x:0 0 0 2024-03-16T09:01:06.572 INFO:tasks.workunit.client.1.smithi019.stdout:5/36: stat d6/f7 0 2024-03-16T09:01:06.572 INFO:tasks.workunit.client.1.smithi019.stdout:5/37: truncate f2 4365621 0 2024-03-16T09:01:06.796 INFO:tasks.workunit.client.1.smithi019.stdout:4/14: symlink d4/l5 0 2024-03-16T09:01:06.797 INFO:tasks.workunit.client.1.smithi019.stdout:4/15: truncate f3 1238527 0 2024-03-16T09:01:06.797 INFO:tasks.workunit.client.1.smithi019.stdout:4/16: truncate f3 1493225 0 2024-03-16T09:01:06.811 INFO:tasks.workunit.client.1.smithi019.stdout:4/17: dwrite f2 [0,4194304] 0 2024-03-16T09:01:06.948 INFO:tasks.workunit.client.1.smithi019.stdout:9/10: creat d1/f2 x:0 0 0 2024-03-16T09:01:06.951 INFO:tasks.workunit.client.1.smithi019.stdout:3/12: link f0 f2 0 2024-03-16T09:01:06.956 INFO:tasks.workunit.client.1.smithi019.stdout:3/13: dread f0 [0,4194304] 0 2024-03-16T09:01:06.974 INFO:tasks.workunit.client.1.smithi019.stdout:3/14: dwrite f0 [0,4194304] 0 2024-03-16T09:01:06.980 INFO:tasks.workunit.client.1.smithi019.stdout:3/15: dread f2 [0,4194304] 0 2024-03-16T09:01:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:06 smithi007 ceph-mon[29470]: Reconfiguring alertmanager.smithi007 (dependencies changed)... 2024-03-16T09:01:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:06 smithi007 ceph-mon[29470]: Reconfiguring daemon alertmanager.smithi007 on smithi007 2024-03-16T09:01:07.017 INFO:tasks.workunit.client.1.smithi019.stdout:6/22: mknod ca 0 2024-03-16T09:01:07.017 INFO:tasks.workunit.client.1.smithi019.stdout:6/23: dread - f9 zero size 2024-03-16T09:01:07.017 INFO:tasks.workunit.client.1.smithi019.stdout:6/24: creat fb x:0 0 0 2024-03-16T09:01:07.028 INFO:tasks.workunit.client.1.smithi019.stdout:8/17: mkdir d6 0 2024-03-16T09:01:07.030 INFO:tasks.workunit.client.1.smithi019.stdout:8/18: dread f2 [0,4194304] 0 2024-03-16T09:01:07.105 INFO:tasks.workunit.client.1.smithi019.stdout:0/18: symlink l6 0 2024-03-16T09:01:07.106 INFO:tasks.workunit.client.1.smithi019.stdout:0/19: stat f4 0 2024-03-16T09:01:07.106 INFO:tasks.workunit.client.1.smithi019.stdout:0/20: write f2 [4685088,77520] 0 2024-03-16T09:01:07.188 INFO:tasks.workunit.client.1.smithi019.stdout:7/21: creat f4 x:0 0 0 2024-03-16T09:01:07.188 INFO:tasks.workunit.client.1.smithi019.stdout:7/22: truncate f4 991271 0 2024-03-16T09:01:07.192 INFO:tasks.workunit.client.1.smithi019.stdout:4/18: unlink f3 0 2024-03-16T09:01:07.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/11: rename d1/f2 to d1/f3 0 2024-03-16T09:01:07.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/12: rename d1 to d1/d4 22 2024-03-16T09:01:07.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/13: chown f0 1492452585 1 2024-03-16T09:01:07.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/14: chown d1 26489 1 2024-03-16T09:01:07.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/15: dread - d1/f3 zero size 2024-03-16T09:01:07.197 INFO:tasks.workunit.client.1.smithi019.stdout:8/19: creat d6/f7 x:0 0 0 2024-03-16T09:01:07.198 INFO:tasks.workunit.client.1.smithi019.stdout:8/20: creat d6/f8 x:0 0 0 2024-03-16T09:01:07.198 INFO:tasks.workunit.client.1.smithi019.stdout:8/21: creat d6/f9 x:0 0 0 2024-03-16T09:01:07.198 INFO:tasks.workunit.client.1.smithi019.stdout:8/22: fdatasync f0 0 2024-03-16T09:01:07.198 INFO:tasks.workunit.client.1.smithi019.stdout:0/21: mkdir d7 0 2024-03-16T09:01:07.198 INFO:tasks.workunit.client.1.smithi019.stdout:0/22: dread - f3 zero size 2024-03-16T09:01:07.199 INFO:tasks.workunit.client.1.smithi019.stdout:5/38: getdents d6 0 2024-03-16T09:01:07.200 INFO:tasks.workunit.client.1.smithi019.stdout:4/19: mkdir d4/d6 0 2024-03-16T09:01:07.201 INFO:tasks.workunit.client.1.smithi019.stdout:9/16: mknod d1/c5 0 2024-03-16T09:01:07.201 INFO:tasks.workunit.client.1.smithi019.stdout:9/17: rename d1 to d1/d6 22 2024-03-16T09:01:07.203 INFO:tasks.workunit.client.1.smithi019.stdout:7/23: dwrite f4 [0,4194304] 0 2024-03-16T09:01:07.203 INFO:tasks.workunit.client.1.smithi019.stdout:7/24: fdatasync f4 0 2024-03-16T09:01:07.203 INFO:tasks.workunit.client.1.smithi019.stdout:7/25: creat f5 x:0 0 0 2024-03-16T09:01:07.203 INFO:tasks.workunit.client.1.smithi019.stdout:7/26: creat f6 x:0 0 0 2024-03-16T09:01:07.203 INFO:tasks.workunit.client.1.smithi019.stdout:7/27: chown f6 176113 1 2024-03-16T09:01:07.203 INFO:tasks.workunit.client.1.smithi019.stdout:7/28: read - f6 zero size 2024-03-16T09:01:07.205 INFO:tasks.workunit.client.1.smithi019.stdout:0/23: mkdir d7/d8 0 2024-03-16T09:01:07.205 INFO:tasks.workunit.client.1.smithi019.stdout:0/24: read f2 [2344355,28123] 0 2024-03-16T09:01:07.205 INFO:tasks.workunit.client.1.smithi019.stdout:0/25: fdatasync f3 0 2024-03-16T09:01:07.207 INFO:tasks.workunit.client.1.smithi019.stdout:4/20: mkdir d4/d6/d7 0 2024-03-16T09:01:07.208 INFO:tasks.workunit.client.1.smithi019.stdout:9/18: mkdir d1/d7 0 2024-03-16T09:01:07.212 INFO:tasks.workunit.client.1.smithi019.stdout:9/19: dread f0 [0,4194304] 0 2024-03-16T09:01:07.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:06 smithi019 ceph-mon[36407]: Reconfiguring alertmanager.smithi007 (dependencies changed)... 2024-03-16T09:01:07.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:06 smithi019 ceph-mon[36407]: Reconfiguring daemon alertmanager.smithi007 on smithi007 2024-03-16T09:01:07.213 INFO:tasks.workunit.client.1.smithi019.stdout:8/23: rename d6/f8 to d6/fa 0 2024-03-16T09:01:07.214 INFO:tasks.workunit.client.1.smithi019.stdout:8/24: chown d6 14738484 1 2024-03-16T09:01:07.214 INFO:tasks.workunit.client.1.smithi019.stdout:0/26: rmdir d7 39 2024-03-16T09:01:07.216 INFO:tasks.workunit.client.1.smithi019.stdout:5/39: dwrite f2 [4194304,4194304] 0 2024-03-16T09:01:07.217 INFO:tasks.workunit.client.1.smithi019.stdout:5/40: read - d6/f7 zero size 2024-03-16T09:01:07.217 INFO:tasks.workunit.client.1.smithi019.stdout:9/20: symlink d1/l8 0 2024-03-16T09:01:07.218 INFO:tasks.workunit.client.1.smithi019.stdout:8/25: mknod d6/cb 0 2024-03-16T09:01:07.218 INFO:tasks.workunit.client.1.smithi019.stdout:8/26: write d6/fa [535165,51644] 0 2024-03-16T09:01:07.221 INFO:tasks.workunit.client.1.smithi019.stdout:9/21: link d1/l8 d1/l9 0 2024-03-16T09:01:07.221 INFO:tasks.workunit.client.1.smithi019.stdout:9/22: truncate d1/f3 688335 0 2024-03-16T09:01:07.223 INFO:tasks.workunit.client.1.smithi019.stdout:9/23: symlink d1/d7/la 0 2024-03-16T09:01:07.223 INFO:tasks.workunit.client.1.smithi019.stdout:9/24: truncate f0 2056301 0 2024-03-16T09:01:07.225 INFO:tasks.workunit.client.1.smithi019.stdout:9/25: mkdir d1/d7/db 0 2024-03-16T09:01:07.227 INFO:tasks.workunit.client.1.smithi019.stdout:4/21: write f2 [2819640,64952] 0 2024-03-16T09:01:07.227 INFO:tasks.workunit.client.1.smithi019.stdout:4/22: rename d4 to d4/d6/d7/d8 22 2024-03-16T09:01:07.227 INFO:tasks.workunit.client.1.smithi019.stdout:4/23: write f2 [3322543,42084] 0 2024-03-16T09:01:07.227 INFO:tasks.workunit.client.1.smithi019.stdout:9/26: rmdir d1/d7/db 0 2024-03-16T09:01:07.227 INFO:tasks.workunit.client.1.smithi019.stdout:9/27: chown d1/c5 28 1 2024-03-16T09:01:07.228 INFO:tasks.workunit.client.1.smithi019.stdout:0/27: dread f2 [0,4194304] 0 2024-03-16T09:01:07.228 INFO:tasks.workunit.client.1.smithi019.stdout:0/28: dread - f4 zero size 2024-03-16T09:01:07.228 INFO:tasks.workunit.client.1.smithi019.stdout:0/29: dread - f3 zero size 2024-03-16T09:01:07.228 INFO:tasks.workunit.client.1.smithi019.stdout:0/30: dread - f3 zero size 2024-03-16T09:01:07.228 INFO:tasks.workunit.client.1.smithi019.stdout:0/31: write f3 [365280,126416] 0 2024-03-16T09:01:07.228 INFO:tasks.workunit.client.1.smithi019.stdout:0/32: read f2 [743384,64952] 0 2024-03-16T09:01:07.229 INFO:tasks.workunit.client.1.smithi019.stdout:0/33: read f3 [35880,125702] 0 2024-03-16T09:01:07.229 INFO:tasks.workunit.client.1.smithi019.stdout:4/24: fdatasync f2 0 2024-03-16T09:01:07.235 INFO:tasks.workunit.client.1.smithi019.stdout:8/27: dwrite f4 [0,4194304] 0 2024-03-16T09:01:07.237 INFO:tasks.workunit.client.1.smithi019.stdout:8/28: symlink d6/lc 0 2024-03-16T09:01:07.237 INFO:tasks.workunit.client.1.smithi019.stdout:8/29: read - d6/f7 zero size 2024-03-16T09:01:07.254 INFO:tasks.workunit.client.1.smithi019.stdout:8/30: dwrite d6/f7 [0,4194304] 0 2024-03-16T09:01:07.254 INFO:tasks.workunit.client.1.smithi019.stdout:8/31: write f5 [563243,64786] 0 2024-03-16T09:01:07.256 INFO:tasks.workunit.client.1.smithi019.stdout:8/32: mknod d6/cd 0 2024-03-16T09:01:07.256 INFO:tasks.workunit.client.1.smithi019.stdout:8/33: write f5 [556316,120607] 0 2024-03-16T09:01:07.257 INFO:tasks.workunit.client.1.smithi019.stdout:8/34: mkdir d6/de 0 2024-03-16T09:01:07.258 INFO:tasks.workunit.client.1.smithi019.stdout:8/35: creat d6/de/ff x:0 0 0 2024-03-16T09:01:07.286 INFO:tasks.workunit.client.1.smithi019.stdout:3/16: fsync f0 0 2024-03-16T09:01:07.287 INFO:tasks.workunit.client.1.smithi019.stdout:3/17: unlink l1 0 2024-03-16T09:01:07.288 INFO:tasks.workunit.client.1.smithi019.stdout:3/18: mkdir d3 0 2024-03-16T09:01:07.289 INFO:tasks.workunit.client.1.smithi019.stdout:3/19: mknod d3/c4 0 2024-03-16T09:01:07.289 INFO:tasks.workunit.client.1.smithi019.stdout:3/20: fdatasync f2 0 2024-03-16T09:01:07.290 INFO:tasks.workunit.client.1.smithi019.stdout:3/21: creat d3/f5 x:0 0 0 2024-03-16T09:01:07.290 INFO:tasks.workunit.client.1.smithi019.stdout:3/22: readlink - no filename 2024-03-16T09:01:07.290 INFO:tasks.workunit.client.1.smithi019.stdout:3/23: truncate d3/f5 194757 0 2024-03-16T09:01:07.290 INFO:tasks.workunit.client.1.smithi019.stdout:3/24: truncate f2 4367823 0 2024-03-16T09:01:07.290 INFO:tasks.workunit.client.1.smithi019.stdout:3/25: dread d3/f5 [0,4194304] 0 2024-03-16T09:01:07.294 INFO:tasks.workunit.client.1.smithi019.stdout:3/26: write f2 [1430894,22205] 0 2024-03-16T09:01:07.394 INFO:tasks.workunit.client.1.smithi019.stdout:9/28: fsync d1/f3 0 2024-03-16T09:01:07.394 INFO:tasks.workunit.client.1.smithi019.stdout:9/29: write d1/f3 [477180,41879] 0 2024-03-16T09:01:07.394 INFO:tasks.workunit.client.1.smithi019.stdout:9/30: chown d1/l8 2416 1 2024-03-16T09:01:07.394 INFO:tasks.workunit.client.1.smithi019.stdout:9/31: write d1/f3 [163276,37265] 0 2024-03-16T09:01:07.397 INFO:tasks.workunit.client.1.smithi019.stdout:6/25: fsync fb 0 2024-03-16T09:01:07.400 INFO:tasks.workunit.client.1.smithi019.stdout:0/34: rmdir d7 39 2024-03-16T09:01:07.401 INFO:tasks.workunit.client.1.smithi019.stdout:0/35: read - f4 zero size 2024-03-16T09:01:07.402 INFO:tasks.workunit.client.1.smithi019.stdout:6/26: rename f9 to fc 0 2024-03-16T09:01:07.406 INFO:tasks.workunit.client.1.smithi019.stdout:4/25: dwrite f2 [4194304,4194304] 0 2024-03-16T09:01:07.407 INFO:tasks.workunit.client.1.smithi019.stdout:0/36: mknod d7/c9 0 2024-03-16T09:01:07.409 INFO:tasks.workunit.client.1.smithi019.stdout:0/37: symlink d7/la 0 2024-03-16T09:01:07.410 INFO:tasks.workunit.client.1.smithi019.stdout:4/26: link d4/l5 d4/d6/l9 0 2024-03-16T09:01:07.410 INFO:tasks.workunit.client.1.smithi019.stdout:4/27: rename d4/d6/d7 to d4/d6/d7/da 22 2024-03-16T09:01:07.410 INFO:tasks.workunit.client.1.smithi019.stdout:7/29: fsync f6 0 2024-03-16T09:01:07.410 INFO:tasks.workunit.client.1.smithi019.stdout:7/30: write f6 [749149,44143] 0 2024-03-16T09:01:07.411 INFO:tasks.workunit.client.1.smithi019.stdout:6/27: write f3 [1079677,48376] 0 2024-03-16T09:01:07.413 INFO:tasks.workunit.client.1.smithi019.stdout:6/28: symlink ld 0 2024-03-16T09:01:07.413 INFO:tasks.workunit.client.1.smithi019.stdout:6/29: dread - fb zero size 2024-03-16T09:01:07.413 INFO:tasks.workunit.client.1.smithi019.stdout:6/30: truncate fb 792747 0 2024-03-16T09:01:07.414 INFO:tasks.workunit.client.1.smithi019.stdout:6/31: mkdir de 0 2024-03-16T09:01:07.415 INFO:tasks.workunit.client.1.smithi019.stdout:6/32: symlink de/lf 0 2024-03-16T09:01:07.418 INFO:tasks.workunit.client.1.smithi019.stdout:6/33: mkdir de/d10 0 2024-03-16T09:01:07.418 INFO:tasks.workunit.client.1.smithi019.stdout:6/34: stat f7 0 2024-03-16T09:01:07.422 INFO:tasks.workunit.client.1.smithi019.stdout:8/36: rmdir d6 39 2024-03-16T09:01:07.425 INFO:tasks.workunit.client.1.smithi019.stdout:8/37: mknod d6/c10 0 2024-03-16T09:01:07.425 INFO:tasks.workunit.client.1.smithi019.stdout:8/38: truncate d6/f9 1009822 0 2024-03-16T09:01:07.425 INFO:tasks.workunit.client.1.smithi019.stdout:8/39: readlink d6/lc 0 2024-03-16T09:01:07.426 INFO:tasks.workunit.client.1.smithi019.stdout:3/27: fsync d3/f5 0 2024-03-16T09:01:07.426 INFO:tasks.workunit.client.1.smithi019.stdout:3/28: write d3/f5 [769569,42248] 0 2024-03-16T09:01:07.426 INFO:tasks.workunit.client.1.smithi019.stdout:3/29: stat d3/f5 0 2024-03-16T09:01:07.427 INFO:tasks.workunit.client.1.smithi019.stdout:3/30: readlink - no filename 2024-03-16T09:01:07.427 INFO:tasks.workunit.client.1.smithi019.stdout:7/31: dwrite f5 [0,4194304] 0 2024-03-16T09:01:07.427 INFO:tasks.workunit.client.1.smithi019.stdout:7/32: stat f5 0 2024-03-16T09:01:07.427 INFO:tasks.workunit.client.1.smithi019.stdout:7/33: chown f6 146999024 1 2024-03-16T09:01:07.434 INFO:tasks.workunit.client.1.smithi019.stdout:7/34: rename f5 to f7 0 2024-03-16T09:01:07.436 INFO:tasks.workunit.client.1.smithi019.stdout:3/31: truncate f2 2417216 0 2024-03-16T09:01:07.440 INFO:tasks.workunit.client.1.smithi019.stdout:0/38: dwrite f3 [0,4194304] 0 2024-03-16T09:01:07.444 INFO:tasks.workunit.client.1.smithi019.stdout:0/39: rename d7/c9 to d7/cb 0 2024-03-16T09:01:07.445 INFO:tasks.workunit.client.1.smithi019.stdout:0/40: creat d7/d8/fc x:0 0 0 2024-03-16T09:01:07.446 INFO:tasks.workunit.client.1.smithi019.stdout:0/41: link f4 d7/fd 0 2024-03-16T09:01:07.448 INFO:tasks.workunit.client.1.smithi019.stdout:0/42: symlink d7/le 0 2024-03-16T09:01:07.448 INFO:tasks.workunit.client.1.smithi019.stdout:0/43: chown d7/la 1 1 2024-03-16T09:01:07.448 INFO:tasks.workunit.client.1.smithi019.stdout:0/44: read - d7/fd zero size 2024-03-16T09:01:07.448 INFO:tasks.workunit.client.1.smithi019.stdout:0/45: readlink d7/la 0 2024-03-16T09:01:07.449 INFO:tasks.workunit.client.1.smithi019.stdout:0/46: read - d7/fd zero size 2024-03-16T09:01:07.449 INFO:tasks.workunit.client.1.smithi019.stdout:6/35: dwrite f7 [0,4194304] 0 2024-03-16T09:01:07.449 INFO:tasks.workunit.client.1.smithi019.stdout:8/40: dwrite f5 [0,4194304] 0 2024-03-16T09:01:07.450 INFO:tasks.workunit.client.1.smithi019.stdout:6/36: rmdir de/d10 0 2024-03-16T09:01:07.451 INFO:tasks.workunit.client.1.smithi019.stdout:4/28: dwrite f2 [4194304,4194304] 0 2024-03-16T09:01:07.455 INFO:tasks.workunit.client.1.smithi019.stdout:8/41: mknod d6/de/c11 0 2024-03-16T09:01:07.455 INFO:tasks.workunit.client.1.smithi019.stdout:8/42: stat d6/c10 0 2024-03-16T09:01:07.460 INFO:tasks.workunit.client.1.smithi019.stdout:0/47: read f3 [3074993,2440] 0 2024-03-16T09:01:07.460 INFO:tasks.workunit.client.1.smithi019.stdout:8/43: mknod d6/de/c12 0 2024-03-16T09:01:07.462 INFO:tasks.workunit.client.1.smithi019.stdout:0/48: unlink f3 0 2024-03-16T09:01:07.462 INFO:tasks.workunit.client.1.smithi019.stdout:0/49: chown l5 10917692 1 2024-03-16T09:01:07.464 INFO:tasks.workunit.client.1.smithi019.stdout:8/44: mknod d6/de/c13 0 2024-03-16T09:01:07.464 INFO:tasks.workunit.client.1.smithi019.stdout:8/45: truncate f0 773647 0 2024-03-16T09:01:07.464 INFO:tasks.workunit.client.1.smithi019.stdout:8/46: readlink l1 0 2024-03-16T09:01:07.465 INFO:tasks.workunit.client.1.smithi019.stdout:0/50: mknod d7/cf 0 2024-03-16T09:01:07.465 INFO:tasks.workunit.client.1.smithi019.stdout:3/32: rename f0 to d3/f6 0 2024-03-16T09:01:07.466 INFO:tasks.workunit.client.1.smithi019.stdout:3/33: fdatasync d3/f5 0 2024-03-16T09:01:07.466 INFO:tasks.workunit.client.1.smithi019.stdout:4/29: dread f2 [4194304,4194304] 0 2024-03-16T09:01:07.466 INFO:tasks.workunit.client.1.smithi019.stdout:8/47: mkdir d6/de/d14 0 2024-03-16T09:01:07.467 INFO:tasks.workunit.client.1.smithi019.stdout:6/37: dwrite fc [0,4194304] 0 2024-03-16T09:01:07.467 INFO:tasks.workunit.client.1.smithi019.stdout:0/51: mknod d7/d8/c10 0 2024-03-16T09:01:07.468 INFO:tasks.workunit.client.1.smithi019.stdout:3/34: mkdir d3/d7 0 2024-03-16T09:01:07.469 INFO:tasks.workunit.client.1.smithi019.stdout:4/30: getdents d4/d6 0 2024-03-16T09:01:07.470 INFO:tasks.workunit.client.1.smithi019.stdout:8/48: creat d6/f15 x:0 0 0 2024-03-16T09:01:07.473 INFO:tasks.workunit.client.1.smithi019.stdout:0/52: rename d7/cb to d7/c11 0 2024-03-16T09:01:07.475 INFO:tasks.workunit.client.1.smithi019.stdout:4/31: symlink d4/lb 0 2024-03-16T09:01:07.478 INFO:tasks.workunit.client.1.smithi019.stdout:4/32: rmdir d4/d6 39 2024-03-16T09:01:07.479 INFO:tasks.workunit.client.1.smithi019.stdout:4/33: fsync f2 0 2024-03-16T09:01:07.479 INFO:tasks.workunit.client.1.smithi019.stdout:4/34: fsync f2 0 2024-03-16T09:01:07.479 INFO:tasks.workunit.client.1.smithi019.stdout:4/35: chown d4 403 1 2024-03-16T09:01:07.480 INFO:tasks.workunit.client.1.smithi019.stdout:4/36: rmdir d4/d6 39 2024-03-16T09:01:07.483 INFO:tasks.workunit.client.1.smithi019.stdout:4/37: dread f2 [4194304,4194304] 0 2024-03-16T09:01:07.483 INFO:tasks.workunit.client.1.smithi019.stdout:4/38: fsync f2 0 2024-03-16T09:01:07.484 INFO:tasks.workunit.client.1.smithi019.stdout:4/39: symlink d4/d6/lc 0 2024-03-16T09:01:07.484 INFO:tasks.workunit.client.1.smithi019.stdout:4/40: fsync f2 0 2024-03-16T09:01:07.484 INFO:tasks.workunit.client.1.smithi019.stdout:4/41: mknod d4/d6/d7/cd 0 2024-03-16T09:01:07.485 INFO:tasks.workunit.client.1.smithi019.stdout:4/42: mknod d4/ce 0 2024-03-16T09:01:07.485 INFO:tasks.workunit.client.1.smithi019.stdout:4/43: truncate f2 2198217 0 2024-03-16T09:01:07.486 INFO:tasks.workunit.client.1.smithi019.stdout:4/44: mknod d4/d6/d7/cf 0 2024-03-16T09:01:07.486 INFO:tasks.workunit.client.1.smithi019.stdout:6/38: dwrite f7 [0,4194304] 0 2024-03-16T09:01:07.487 INFO:tasks.workunit.client.1.smithi019.stdout:8/49: dwrite f4 [4194304,4194304] 0 2024-03-16T09:01:07.490 INFO:tasks.workunit.client.1.smithi019.stdout:0/53: getdents d7/d8 0 2024-03-16T09:01:07.491 INFO:tasks.workunit.client.1.smithi019.stdout:0/54: creat d7/d8/f12 x:0 0 0 2024-03-16T09:01:07.491 INFO:tasks.workunit.client.1.smithi019.stdout:3/35: getdents d3 0 2024-03-16T09:01:07.492 INFO:tasks.workunit.client.1.smithi019.stdout:0/55: mkdir d7/d13 0 2024-03-16T09:01:07.492 INFO:tasks.workunit.client.1.smithi019.stdout:0/56: fsync d7/fd 0 2024-03-16T09:01:07.492 INFO:tasks.workunit.client.1.smithi019.stdout:0/57: chown d7/d13 3543807 1 2024-03-16T09:01:07.493 INFO:tasks.workunit.client.1.smithi019.stdout:3/36: rmdir d3/d7 0 2024-03-16T09:01:07.493 INFO:tasks.workunit.client.1.smithi019.stdout:3/37: stat f2 0 2024-03-16T09:01:07.494 INFO:tasks.workunit.client.1.smithi019.stdout:3/38: chown d3/c4 681570 1 2024-03-16T09:01:07.496 INFO:tasks.workunit.client.1.smithi019.stdout:3/39: dread d3/f5 [0,4194304] 0 2024-03-16T09:01:07.497 INFO:tasks.workunit.client.1.smithi019.stdout:6/39: read f7 [3328949,91561] 0 2024-03-16T09:01:07.500 INFO:tasks.workunit.client.1.smithi019.stdout:6/40: rename c2 to de/c11 0 2024-03-16T09:01:07.500 INFO:tasks.workunit.client.1.smithi019.stdout:8/50: dread f5 [0,4194304] 0 2024-03-16T09:01:07.501 INFO:tasks.workunit.client.1.smithi019.stdout:3/40: unlink d3/f6 0 2024-03-16T09:01:07.501 INFO:tasks.workunit.client.1.smithi019.stdout:6/41: mknod de/c12 0 2024-03-16T09:01:07.502 INFO:tasks.workunit.client.1.smithi019.stdout:8/51: mkdir d6/de/d16 0 2024-03-16T09:01:07.508 INFO:tasks.workunit.client.1.smithi019.stdout:6/42: dread f3 [0,4194304] 0 2024-03-16T09:01:07.508 INFO:tasks.workunit.client.1.smithi019.stdout:6/43: chown c1 18 1 2024-03-16T09:01:07.508 INFO:tasks.workunit.client.1.smithi019.stdout:6/44: fdatasync fc 0 2024-03-16T09:01:07.508 INFO:tasks.workunit.client.1.smithi019.stdout:6/45: fsync fb 0 2024-03-16T09:01:07.511 INFO:tasks.workunit.client.1.smithi019.stdout:6/46: rename c8 to de/c13 0 2024-03-16T09:01:07.511 INFO:tasks.workunit.client.1.smithi019.stdout:6/47: write fb [1031514,54556] 0 2024-03-16T09:01:07.511 INFO:tasks.workunit.client.1.smithi019.stdout:6/48: readlink l0 0 2024-03-16T09:01:07.511 INFO:tasks.workunit.client.1.smithi019.stdout:6/49: fdatasync f3 0 2024-03-16T09:01:07.511 INFO:tasks.workunit.client.1.smithi019.stdout:6/50: write f7 [2875628,28064] 0 2024-03-16T09:01:07.511 INFO:tasks.workunit.client.1.smithi019.stdout:4/45: dwrite f2 [0,4194304] 0 2024-03-16T09:01:07.512 INFO:tasks.workunit.client.1.smithi019.stdout:8/52: write f4 [2990067,24423] 0 2024-03-16T09:01:07.520 INFO:tasks.workunit.client.1.smithi019.stdout:4/46: truncate f2 3651382 0 2024-03-16T09:01:07.522 INFO:tasks.workunit.client.1.smithi019.stdout:4/47: mknod d4/d6/d7/c10 0 2024-03-16T09:01:07.522 INFO:tasks.workunit.client.1.smithi019.stdout:8/53: dread d6/f7 [0,4194304] 0 2024-03-16T09:01:07.533 INFO:tasks.workunit.client.1.smithi019.stdout:6/51: dwrite f3 [0,4194304] 0 2024-03-16T09:01:07.541 INFO:tasks.workunit.client.1.smithi019.stdout:8/54: dwrite f5 [0,4194304] 0 2024-03-16T09:01:07.541 INFO:tasks.workunit.client.1.smithi019.stdout:8/55: truncate d6/de/ff 948577 0 2024-03-16T09:01:07.541 INFO:tasks.workunit.client.1.smithi019.stdout:8/56: chown d6/cb 21173 1 2024-03-16T09:01:07.542 INFO:tasks.workunit.client.1.smithi019.stdout:8/57: dread f0 [0,4194304] 0 2024-03-16T09:01:07.543 INFO:tasks.workunit.client.1.smithi019.stdout:8/58: symlink d6/l17 0 2024-03-16T09:01:07.544 INFO:tasks.workunit.client.1.smithi019.stdout:8/59: link f0 d6/de/d14/f18 0 2024-03-16T09:01:07.544 INFO:tasks.workunit.client.1.smithi019.stdout:6/52: dwrite fb [0,4194304] 0 2024-03-16T09:01:07.546 INFO:tasks.workunit.client.1.smithi019.stdout:8/60: unlink d6/f9 0 2024-03-16T09:01:07.546 INFO:tasks.workunit.client.1.smithi019.stdout:8/61: getdents d6/de/d16 0 2024-03-16T09:01:07.547 INFO:tasks.workunit.client.1.smithi019.stdout:6/53: getdents de 0 2024-03-16T09:01:07.549 INFO:tasks.workunit.client.1.smithi019.stdout:6/54: creat de/f14 x:0 0 0 2024-03-16T09:01:07.550 INFO:tasks.workunit.client.1.smithi019.stdout:6/55: creat de/f15 x:0 0 0 2024-03-16T09:01:07.552 INFO:tasks.workunit.client.1.smithi019.stdout:6/56: rename l0 to de/l16 0 2024-03-16T09:01:07.560 INFO:tasks.workunit.client.1.smithi019.stdout:8/62: dwrite d6/de/d14/f18 [0,4194304] 0 2024-03-16T09:01:07.561 INFO:tasks.workunit.client.1.smithi019.stdout:8/63: write f2 [159061,21949] 0 2024-03-16T09:01:07.562 INFO:tasks.workunit.client.1.smithi019.stdout:6/57: rename de/l16 to de/l17 0 2024-03-16T09:01:07.565 INFO:tasks.workunit.client.1.smithi019.stdout:8/64: rename d6/de/d14 to d6/de/d16/d19 0 2024-03-16T09:01:07.566 INFO:tasks.workunit.client.1.smithi019.stdout:8/65: link f5 d6/de/d16/d19/f1a 0 2024-03-16T09:01:07.569 INFO:tasks.workunit.client.1.smithi019.stdout:8/66: rename f0 to d6/de/f1b 0 2024-03-16T09:01:07.570 INFO:tasks.workunit.client.1.smithi019.stdout:8/67: creat d6/de/d16/d19/f1c x:0 0 0 2024-03-16T09:01:07.571 INFO:tasks.workunit.client.1.smithi019.stdout:8/68: stat d6/de/f1b 0 2024-03-16T09:01:07.578 INFO:tasks.workunit.client.1.smithi019.stdout:6/58: dwrite fc [0,4194304] 0 2024-03-16T09:01:07.578 INFO:tasks.workunit.client.1.smithi019.stdout:8/69: dread d6/de/d16/d19/f1a [0,4194304] 0 2024-03-16T09:01:07.579 INFO:tasks.workunit.client.1.smithi019.stdout:6/59: creat de/f18 x:0 0 0 2024-03-16T09:01:07.582 INFO:tasks.workunit.client.1.smithi019.stdout:8/70: truncate d6/de/d16/d19/f18 1541905 0 2024-03-16T09:01:07.585 INFO:tasks.workunit.client.1.smithi019.stdout:8/71: creat d6/de/f1d x:0 0 0 2024-03-16T09:01:07.585 INFO:tasks.workunit.client.1.smithi019.stdout:8/72: truncate d6/f7 5115831 0 2024-03-16T09:01:07.594 INFO:tasks.workunit.client.1.smithi019.stdout:6/60: dwrite fb [4194304,4194304] 0 2024-03-16T09:01:07.602 INFO:tasks.workunit.client.1.smithi019.stdout:6/61: dread fc [0,4194304] 0 2024-03-16T09:01:07.617 INFO:tasks.workunit.client.1.smithi019.stdout:6/62: dwrite fb [4194304,4194304] 0 2024-03-16T09:01:07.618 INFO:tasks.workunit.client.1.smithi019.stdout:6/63: mknod de/c19 0 2024-03-16T09:01:07.636 INFO:tasks.workunit.client.1.smithi019.stdout:6/64: dwrite f7 [0,4194304] 0 2024-03-16T09:01:07.656 INFO:tasks.workunit.client.1.smithi019.stdout:6/65: dwrite de/f14 [0,4194304] 0 2024-03-16T09:01:07.657 INFO:tasks.workunit.client.1.smithi019.stdout:6/66: write f7 [5127969,74300] 0 2024-03-16T09:01:07.658 INFO:tasks.workunit.client.0.smithi007.stdout:7/181: sync 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:9/226: sync 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:5/150: sync 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:6/217: sync 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:1/152: sync 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:4/140: sync 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:1/153: truncate d9/dd/d1b/f2c 658849 0 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:8/289: sync 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:1/154: write d9/dd/d1b/f2c [1664997,73586] 0 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:1/155: chown f0 1187 1 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:8/290: write d0/d2/f67 [23162,129113] 0 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:8/291: readlink d0/dc/dd/l47 0 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:3/258: sync 2024-03-16T09:01:07.659 INFO:tasks.workunit.client.0.smithi007.stdout:2/308: sync 2024-03-16T09:01:07.660 INFO:tasks.workunit.client.0.smithi007.stdout:0/165: sync 2024-03-16T09:01:07.661 INFO:tasks.workunit.client.0.smithi007.stdout:4/141: creat d4/d14/d1c/f2c x:0 0 0 2024-03-16T09:01:07.661 INFO:tasks.workunit.client.0.smithi007.stdout:5/151: creat d1/d7/d16/d1e/f31 x:0 0 0 2024-03-16T09:01:07.662 INFO:tasks.workunit.client.1.smithi019.stdout:6/67: rmdir de 39 2024-03-16T09:01:07.666 INFO:tasks.workunit.client.0.smithi007.stdout:6/218: mkdir d2/d13/d1f/d49 0 2024-03-16T09:01:07.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/292: symlink d0/d2/d18/l68 0 2024-03-16T09:01:07.667 INFO:tasks.workunit.client.1.smithi019.stdout:6/68: rmdir de 39 2024-03-16T09:01:07.667 INFO:tasks.workunit.client.0.smithi007.stdout:0/166: rmdir d3/d9/da/d18/d21/d25/d33 0 2024-03-16T09:01:07.669 INFO:tasks.workunit.client.0.smithi007.stdout:3/259: dread d15/f16 [0,4194304] 0 2024-03-16T09:01:07.669 INFO:tasks.workunit.client.0.smithi007.stdout:3/260: truncate d15/f1e 1011321 0 2024-03-16T09:01:07.669 INFO:tasks.workunit.client.0.smithi007.stdout:5/152: mknod d1/d7/d1c/c32 0 2024-03-16T09:01:07.669 INFO:tasks.workunit.client.0.smithi007.stdout:5/153: write d1/fa [3153525,90582] 0 2024-03-16T09:01:07.669 INFO:tasks.workunit.client.0.smithi007.stdout:5/154: chown f0 261021220 1 2024-03-16T09:01:07.671 INFO:tasks.workunit.client.0.smithi007.stdout:8/293: dread d0/dc/f3a [0,4194304] 0 2024-03-16T09:01:07.671 INFO:tasks.workunit.client.0.smithi007.stdout:8/294: write d0/d26/f2f [3058355,18977] 0 2024-03-16T09:01:07.671 INFO:tasks.workunit.client.0.smithi007.stdout:4/142: rename d4/f12 to d4/d14/f2d 0 2024-03-16T09:01:07.671 INFO:tasks.workunit.client.1.smithi019.stdout:6/69: truncate de/f14 3868979 0 2024-03-16T09:01:07.672 INFO:tasks.workunit.client.0.smithi007.stdout:0/167: symlink d3/d9/da/d18/d21/d25/l37 0 2024-03-16T09:01:07.672 INFO:tasks.workunit.client.0.smithi007.stdout:0/168: write d3/f4 [562458,57665] 0 2024-03-16T09:01:07.674 INFO:tasks.workunit.client.1.smithi019.stdout:6/70: unlink de/l17 0 2024-03-16T09:01:07.674 INFO:tasks.workunit.client.0.smithi007.stdout:6/219: dread d2/d10/d12/f35 [0,4194304] 0 2024-03-16T09:01:07.675 INFO:tasks.workunit.client.0.smithi007.stdout:5/155: rename d1/d7/d8/de/f1a to d1/d7/d16/f33 0 2024-03-16T09:01:07.676 INFO:tasks.workunit.client.0.smithi007.stdout:5/156: fdatasync d1/f10 0 2024-03-16T09:01:07.676 INFO:tasks.workunit.client.0.smithi007.stdout:0/169: creat d3/d9/da/d18/d21/f38 x:0 0 0 2024-03-16T09:01:07.677 INFO:tasks.workunit.client.1.smithi019.stdout:6/71: creat de/f1a x:0 0 0 2024-03-16T09:01:07.678 INFO:tasks.workunit.client.0.smithi007.stdout:6/220: creat d2/d13/d1f/d34/f4a x:0 0 0 2024-03-16T09:01:07.681 INFO:tasks.workunit.client.1.smithi019.stdout:6/72: mknod de/c1b 0 2024-03-16T09:01:07.682 INFO:tasks.workunit.client.0.smithi007.stdout:6/221: mkdir d2/d13/d4b 0 2024-03-16T09:01:07.683 INFO:tasks.workunit.client.1.smithi019.stdout:6/73: unlink de/f18 0 2024-03-16T09:01:07.697 INFO:tasks.workunit.client.1.smithi019.stdout:6/74: dread f3 [0,4194304] 0 2024-03-16T09:01:07.698 INFO:tasks.workunit.client.0.smithi007.stdout:8/295: dread d0/dc/f23 [0,4194304] 0 2024-03-16T09:01:07.699 INFO:tasks.workunit.client.0.smithi007.stdout:2/309: dwrite dc/df/d24/d28/d4c/d5e/f5b [0,4194304] 0 2024-03-16T09:01:07.728 INFO:tasks.workunit.client.0.smithi007.stdout:8/296: write d0/dc/f12 [3813983,107236] 0 2024-03-16T09:01:07.731 INFO:tasks.workunit.client.0.smithi007.stdout:1/156: dwrite d9/dd/d1b/f25 [0,4194304] 0 2024-03-16T09:01:07.732 INFO:tasks.workunit.client.0.smithi007.stdout:1/157: chown d9/c27 12571 1 2024-03-16T09:01:07.735 INFO:tasks.workunit.client.0.smithi007.stdout:1/158: mknod d9/dd/d1b/d2d/c2f 0 2024-03-16T09:01:07.737 INFO:tasks.workunit.client.0.smithi007.stdout:1/159: mkdir d9/dd/d1b/d2d/d30 0 2024-03-16T09:01:07.737 INFO:tasks.workunit.client.0.smithi007.stdout:9/227: dwrite f17 [0,4194304] 0 2024-03-16T09:01:07.740 INFO:tasks.workunit.client.0.smithi007.stdout:7/182: dwrite d0/f3c [0,4194304] 0 2024-03-16T09:01:07.740 INFO:tasks.workunit.client.0.smithi007.stdout:4/143: dwrite d4/f28 [0,4194304] 0 2024-03-16T09:01:07.741 INFO:tasks.workunit.client.0.smithi007.stdout:6/222: dwrite d2/d10/d12/d15/f28 [0,4194304] 0 2024-03-16T09:01:07.741 INFO:tasks.workunit.client.0.smithi007.stdout:3/261: dwrite d15/d18/f35 [0,4194304] 0 2024-03-16T09:01:07.741 INFO:tasks.workunit.client.0.smithi007.stdout:3/262: chown d15/d18/d1f/d3b/f44 26024735 1 2024-03-16T09:01:07.742 INFO:tasks.workunit.client.0.smithi007.stdout:9/228: truncate ff 3325946 0 2024-03-16T09:01:07.743 INFO:tasks.workunit.client.0.smithi007.stdout:0/170: dwrite d3/d9/da/d10/d1a/f2d [0,4194304] 0 2024-03-16T09:01:07.743 INFO:tasks.workunit.client.0.smithi007.stdout:0/171: dread - d3/d9/da/d10/d1a/f1b zero size 2024-03-16T09:01:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:7/183: truncate d0/dc/fd 3816778 0 2024-03-16T09:01:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:7/184: read - d0/dc/d24/f3b zero size 2024-03-16T09:01:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:7/185: chown d0/dc/d24 4153 1 2024-03-16T09:01:07.747 INFO:tasks.workunit.client.0.smithi007.stdout:1/160: mknod d9/dd/c31 0 2024-03-16T09:01:07.752 INFO:tasks.workunit.client.0.smithi007.stdout:4/144: creat d4/f2e x:0 0 0 2024-03-16T09:01:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:6/223: unlink d2/fb 0 2024-03-16T09:01:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:6/224: read d2/d11/f21 [372934,117656] 0 2024-03-16T09:01:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:2/310: dwrite dc/df/d24/d28/d4c/f67 [0,4194304] 0 2024-03-16T09:01:07.761 INFO:tasks.workunit.client.0.smithi007.stdout:3/263: rename d15/d18/l24 to d15/d18/d1f/d29/d3e/l53 0 2024-03-16T09:01:07.765 INFO:tasks.workunit.client.0.smithi007.stdout:0/172: mknod d3/c39 0 2024-03-16T09:01:07.765 INFO:tasks.workunit.client.0.smithi007.stdout:7/186: readlink d0/l2 0 2024-03-16T09:01:07.765 INFO:tasks.workunit.client.0.smithi007.stdout:7/187: creat d0/dc/f3d x:0 0 0 2024-03-16T09:01:07.765 INFO:tasks.workunit.client.0.smithi007.stdout:7/188: write d0/f3 [3267693,58911] 0 2024-03-16T09:01:07.769 INFO:tasks.workunit.client.0.smithi007.stdout:2/311: rename dc/df/d24/d28/d4c/d5e/l56 to dc/df/d24/d28/d2e/d35/l6a 0 2024-03-16T09:01:07.769 INFO:tasks.workunit.client.0.smithi007.stdout:2/312: truncate dc/df/d24/d28/d2e/d35/f40 35848 0 2024-03-16T09:01:07.769 INFO:tasks.workunit.client.0.smithi007.stdout:2/313: read dc/df/f14 [2560425,49332] 0 2024-03-16T09:01:07.769 INFO:tasks.workunit.client.0.smithi007.stdout:2/314: rename dc/df/d24/d28 to dc/df/d24/d28/d2e/d35/d6b 22 2024-03-16T09:01:07.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/157: dwrite d1/fa [0,4194304] 0 2024-03-16T09:01:07.769 INFO:tasks.workunit.client.0.smithi007.stdout:5/158: write d1/d7/d16/f2b [116978,41196] 0 2024-03-16T09:01:07.770 INFO:tasks.workunit.client.1.smithi019.stdout:6/75: read de/f14 [2477169,3777] 0 2024-03-16T09:01:07.770 INFO:tasks.workunit.client.0.smithi007.stdout:1/161: dwrite d9/f19 [4194304,4194304] 0 2024-03-16T09:01:07.770 INFO:tasks.workunit.client.0.smithi007.stdout:3/264: link d15/d18/d1f/f39 d15/d18/d1f/f54 0 2024-03-16T09:01:07.770 INFO:tasks.workunit.client.0.smithi007.stdout:9/229: truncate d24/f2d 2257179 0 2024-03-16T09:01:07.771 INFO:tasks.workunit.client.0.smithi007.stdout:1/162: dread d9/dd/f14 [0,4194304] 0 2024-03-16T09:01:07.772 INFO:tasks.workunit.client.0.smithi007.stdout:0/173: mkdir d3/d9/da/d18/d27/d34/d3a 0 2024-03-16T09:01:07.772 INFO:tasks.workunit.client.0.smithi007.stdout:0/174: readlink d3/d9/da/d10/l2e 0 2024-03-16T09:01:07.772 INFO:tasks.workunit.client.0.smithi007.stdout:8/297: dwrite d0/d26/f30 [0,4194304] 0 2024-03-16T09:01:07.772 INFO:tasks.workunit.client.0.smithi007.stdout:6/225: dread d2/d10/f36 [0,4194304] 0 2024-03-16T09:01:07.773 INFO:tasks.workunit.client.1.smithi019.stdout:6/76: mkdir de/d1c 0 2024-03-16T09:01:07.773 INFO:tasks.workunit.client.1.smithi019.stdout:6/77: stat de/d1c 0 2024-03-16T09:01:07.773 INFO:tasks.workunit.client.1.smithi019.stdout:6/78: readlink ld 0 2024-03-16T09:01:07.777 INFO:tasks.workunit.client.0.smithi007.stdout:4/145: link d4/f28 d4/f2f 0 2024-03-16T09:01:07.778 INFO:tasks.workunit.client.1.smithi019.stdout:6/79: rmdir de 39 2024-03-16T09:01:07.778 INFO:tasks.workunit.client.0.smithi007.stdout:2/315: mkdir dc/df/d6c 0 2024-03-16T09:01:07.780 INFO:tasks.workunit.client.1.smithi019.stdout:6/80: mkdir de/d1c/d1d 0 2024-03-16T09:01:07.781 INFO:tasks.workunit.client.0.smithi007.stdout:8/298: dread d0/d2/f67 [0,4194304] 0 2024-03-16T09:01:07.782 INFO:tasks.workunit.client.1.smithi019.stdout:6/81: rename de/c13 to de/d1c/d1d/c1e 0 2024-03-16T09:01:07.783 INFO:tasks.workunit.client.1.smithi019.stdout:6/82: mknod de/d1c/c1f 0 2024-03-16T09:01:07.783 INFO:tasks.workunit.client.0.smithi007.stdout:3/265: truncate fa 1166542 0 2024-03-16T09:01:07.786 INFO:tasks.workunit.client.0.smithi007.stdout:1/163: link d9/f18 d9/dd/d1b/d28/f32 0 2024-03-16T09:01:07.786 INFO:tasks.workunit.client.0.smithi007.stdout:9/230: dwrite d24/f2a [0,4194304] 0 2024-03-16T09:01:07.786 INFO:tasks.workunit.client.0.smithi007.stdout:0/175: truncate d3/d9/da/d18/d21/f12 1316422 0 2024-03-16T09:01:07.788 INFO:tasks.workunit.client.1.smithi019.stdout:6/83: getdents de 0 2024-03-16T09:01:07.788 INFO:tasks.workunit.client.1.smithi019.stdout:6/84: readlink ld 0 2024-03-16T09:01:07.789 INFO:tasks.workunit.client.1.smithi019.stdout:6/85: getdents de 0 2024-03-16T09:01:07.789 INFO:tasks.workunit.client.1.smithi019.stdout:6/86: chown de/c11 1006435 1 2024-03-16T09:01:07.791 INFO:tasks.workunit.client.1.smithi019.stdout:6/87: creat de/d1c/f20 x:0 0 0 2024-03-16T09:01:07.798 INFO:tasks.workunit.client.0.smithi007.stdout:7/189: link d0/c16 d0/dc/d24/d25/c3e 0 2024-03-16T09:01:07.798 INFO:tasks.workunit.client.0.smithi007.stdout:2/316: symlink dc/d27/l6d 0 2024-03-16T09:01:07.800 INFO:tasks.workunit.client.0.smithi007.stdout:5/159: rename d1/d7/d8 to d1/d7/d1c/d19/d34 0 2024-03-16T09:01:07.802 INFO:tasks.workunit.client.0.smithi007.stdout:9/231: creat d24/f44 x:0 0 0 2024-03-16T09:01:07.805 INFO:tasks.workunit.client.0.smithi007.stdout:0/176: symlink d3/d9/d31/l3b 0 2024-03-16T09:01:07.805 INFO:tasks.workunit.client.0.smithi007.stdout:3/266: dwrite d15/d18/d1f/d3b/f45 [0,4194304] 0 2024-03-16T09:01:07.805 INFO:tasks.workunit.client.0.smithi007.stdout:0/177: creat d3/d9/db/d1c/f3c x:0 0 0 2024-03-16T09:01:07.805 INFO:tasks.workunit.client.0.smithi007.stdout:0/178: creat d3/d9/da/d18/d23/f3d x:0 0 0 2024-03-16T09:01:07.805 INFO:tasks.workunit.client.0.smithi007.stdout:8/299: dwrite d0/f16 [0,4194304] 0 2024-03-16T09:01:07.805 INFO:tasks.workunit.client.0.smithi007.stdout:8/300: dread - d0/d2/f66 zero size 2024-03-16T09:01:07.805 INFO:tasks.workunit.client.1.smithi019.stdout:6/88: dread fb [4194304,4194304] 0 2024-03-16T09:01:07.806 INFO:tasks.workunit.client.1.smithi019.stdout:6/89: rename de/d1c/d1d/c1e to de/d1c/c21 0 2024-03-16T09:01:07.806 INFO:tasks.workunit.client.1.smithi019.stdout:6/90: chown de/d1c 368286 1 2024-03-16T09:01:07.806 INFO:tasks.workunit.client.0.smithi007.stdout:6/226: link d2/d11/f25 d2/d13/d1f/d34/f4c 0 2024-03-16T09:01:07.807 INFO:tasks.workunit.client.1.smithi019.stdout:6/91: mknod de/d1c/c22 0 2024-03-16T09:01:07.808 INFO:tasks.workunit.client.0.smithi007.stdout:4/146: rmdir d4/d14 39 2024-03-16T09:01:07.810 INFO:tasks.workunit.client.0.smithi007.stdout:2/317: mkdir dc/df/d24/d28/d2e/d6e 0 2024-03-16T09:01:07.812 INFO:tasks.workunit.client.0.smithi007.stdout:5/160: symlink d1/d7/d1c/d19/d34/de/l35 0 2024-03-16T09:01:07.812 INFO:tasks.workunit.client.0.smithi007.stdout:5/161: write d1/f10 [660479,106557] 0 2024-03-16T09:01:07.818 INFO:tasks.workunit.client.0.smithi007.stdout:7/190: dwrite d0/d19/f27 [4194304,4194304] 0 2024-03-16T09:01:07.820 INFO:tasks.workunit.client.0.smithi007.stdout:5/162: dread d1/d7/d1c/f17 [0,4194304] 0 2024-03-16T09:01:07.820 INFO:tasks.workunit.client.0.smithi007.stdout:5/163: fsync d1/d7/d1c/f15 0 2024-03-16T09:01:07.821 INFO:tasks.workunit.client.0.smithi007.stdout:8/301: link d0/d2/d18/l24 d0/d26/d2d/d43/d4c/l69 0 2024-03-16T09:01:07.821 INFO:tasks.workunit.client.0.smithi007.stdout:8/302: dread - d0/dc/f35 zero size 2024-03-16T09:01:07.821 INFO:tasks.workunit.client.1.smithi019.stdout:6/92: getdents de/d1c 0 2024-03-16T09:01:07.822 INFO:tasks.workunit.client.0.smithi007.stdout:6/227: rmdir d2/d13 39 2024-03-16T09:01:07.822 INFO:tasks.workunit.client.0.smithi007.stdout:6/228: fsync d2/d10/f3a 0 2024-03-16T09:01:07.823 INFO:tasks.workunit.client.1.smithi019.stdout:6/93: stat l5 0 2024-03-16T09:01:07.823 INFO:tasks.workunit.client.0.smithi007.stdout:4/147: creat d4/d14/d1c/f30 x:0 0 0 2024-03-16T09:01:07.825 INFO:tasks.workunit.client.0.smithi007.stdout:1/164: rename d9/f18 to d9/dd/f33 0 2024-03-16T09:01:07.827 INFO:tasks.workunit.client.0.smithi007.stdout:7/191: symlink d0/d19/d1f/l3f 0 2024-03-16T09:01:07.827 INFO:tasks.workunit.client.0.smithi007.stdout:7/192: chown d0/dc/f3d 311 1 2024-03-16T09:01:07.829 INFO:tasks.workunit.client.1.smithi019.stdout:6/94: dread de/f14 [0,4194304] 0 2024-03-16T09:01:07.829 INFO:tasks.workunit.client.1.smithi019.stdout:6/95: stat de/lf 0 2024-03-16T09:01:07.830 INFO:tasks.workunit.client.0.smithi007.stdout:5/164: mkdir d1/d7/d1c/d19/d34/d1d/d36 0 2024-03-16T09:01:07.830 INFO:tasks.workunit.client.1.smithi019.stdout:6/96: chown f3 710328799 1 2024-03-16T09:01:07.830 INFO:tasks.workunit.client.1.smithi019.stdout:6/97: mknod de/d1c/d1d/c23 0 2024-03-16T09:01:07.831 INFO:tasks.workunit.client.1.smithi019.stdout:6/98: creat de/d1c/d1d/f24 x:0 0 0 2024-03-16T09:01:07.832 INFO:tasks.workunit.client.0.smithi007.stdout:8/303: mknod d0/d2/d5f/c6a 0 2024-03-16T09:01:07.832 INFO:tasks.workunit.client.0.smithi007.stdout:2/318: dwrite dc/f10 [0,4194304] 0 2024-03-16T09:01:07.835 INFO:tasks.workunit.client.0.smithi007.stdout:9/232: getdents d24 0 2024-03-16T09:01:07.835 INFO:tasks.workunit.client.0.smithi007.stdout:9/233: fdatasync f7 0 2024-03-16T09:01:07.835 INFO:tasks.workunit.client.0.smithi007.stdout:9/234: fsync f1b 0 2024-03-16T09:01:07.835 INFO:tasks.workunit.client.0.smithi007.stdout:9/235: write d24/f43 [222997,50527] 0 2024-03-16T09:01:07.836 INFO:tasks.workunit.client.0.smithi007.stdout:3/267: rename d15/d18/d1f/d3b/f45 to d15/d18/d1f/d29/d43/f55 0 2024-03-16T09:01:07.836 INFO:tasks.workunit.client.0.smithi007.stdout:6/229: dread d2/d11/f25 [0,4194304] 0 2024-03-16T09:01:07.837 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:07 smithi007 ceph-mon[29470]: pgmap v17: 65 pgs: 65 active+clean; 591 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 49 MiB/s rd, 68 MiB/s wr, 1.07k op/s 2024-03-16T09:01:07.837 INFO:tasks.workunit.client.1.smithi019.stdout:6/99: dread f3 [0,4194304] 0 2024-03-16T09:01:07.837 INFO:tasks.workunit.client.1.smithi019.stdout:6/100: chown de/d1c/d1d/c23 48 1 2024-03-16T09:01:07.838 INFO:tasks.workunit.client.0.smithi007.stdout:1/165: rmdir d9/dd/d1b/d2d 39 2024-03-16T09:01:07.839 INFO:tasks.workunit.client.1.smithi019.stdout:6/101: creat de/f25 x:0 0 0 2024-03-16T09:01:07.841 INFO:tasks.workunit.client.1.smithi019.stdout:6/102: mkdir de/d26 0 2024-03-16T09:01:07.841 INFO:tasks.workunit.client.1.smithi019.stdout:6/103: chown de/lf 9336 1 2024-03-16T09:01:07.841 INFO:tasks.workunit.client.0.smithi007.stdout:5/165: symlink d1/d7/d1c/d19/d34/d1d/l37 0 2024-03-16T09:01:07.841 INFO:tasks.workunit.client.0.smithi007.stdout:7/193: symlink d0/dc/d24/d25/l40 0 2024-03-16T09:01:07.842 INFO:tasks.workunit.client.0.smithi007.stdout:2/319: dread f6 [0,4194304] 0 2024-03-16T09:01:07.842 INFO:tasks.workunit.client.0.smithi007.stdout:8/304: creat d0/dc/d50/f6b x:0 0 0 2024-03-16T09:01:07.842 INFO:tasks.workunit.client.0.smithi007.stdout:8/305: dread - d0/d26/d2d/f61 zero size 2024-03-16T09:01:07.842 INFO:tasks.workunit.client.0.smithi007.stdout:8/306: chown d0/l37 32 1 2024-03-16T09:01:07.842 INFO:tasks.workunit.client.0.smithi007.stdout:8/307: dread - d0/d2/f5d zero size 2024-03-16T09:01:07.843 INFO:tasks.workunit.client.1.smithi019.stdout:6/104: creat de/d1c/f27 x:0 0 0 2024-03-16T09:01:07.843 INFO:tasks.workunit.client.1.smithi019.stdout:6/105: write de/f14 [1369907,122618] 0 2024-03-16T09:01:07.843 INFO:tasks.workunit.client.0.smithi007.stdout:4/148: symlink d4/l31 0 2024-03-16T09:01:07.843 INFO:tasks.workunit.client.0.smithi007.stdout:4/149: fdatasync d4/d7/f1d 0 2024-03-16T09:01:07.843 INFO:tasks.workunit.client.0.smithi007.stdout:4/150: readlink d4/l2a 0 2024-03-16T09:01:07.843 INFO:tasks.workunit.client.0.smithi007.stdout:4/151: chown d4/d14 163830 1 2024-03-16T09:01:07.844 INFO:tasks.workunit.client.0.smithi007.stdout:9/236: mknod d24/c45 0 2024-03-16T09:01:07.844 INFO:tasks.workunit.client.0.smithi007.stdout:9/237: dread - d24/f34 zero size 2024-03-16T09:01:07.845 INFO:tasks.workunit.client.0.smithi007.stdout:9/238: stat d24/l35 0 2024-03-16T09:01:07.847 INFO:tasks.workunit.client.0.smithi007.stdout:0/179: rename d3/d9/cd to d3/d9/da/c3e 0 2024-03-16T09:01:07.848 INFO:tasks.workunit.client.0.smithi007.stdout:5/166: write d1/d7/f12 [365484,101624] 0 2024-03-16T09:01:07.849 INFO:tasks.workunit.client.0.smithi007.stdout:3/268: mkdir d15/d18/d1f/d56 0 2024-03-16T09:01:07.850 INFO:tasks.workunit.client.0.smithi007.stdout:6/230: creat d2/d13/d1f/f4d x:0 0 0 2024-03-16T09:01:07.852 INFO:tasks.workunit.client.0.smithi007.stdout:8/308: getdents d0/d26/d2d/d43/d4c 0 2024-03-16T09:01:07.853 INFO:tasks.workunit.client.0.smithi007.stdout:8/309: creat d0/d26/d2d/f6c x:0 0 0 2024-03-16T09:01:07.853 INFO:tasks.workunit.client.1.smithi019.stdout:6/106: write f7 [2680646,113026] 0 2024-03-16T09:01:07.853 INFO:tasks.workunit.client.1.smithi019.stdout:6/107: creat de/d1c/f28 x:0 0 0 2024-03-16T09:01:07.853 INFO:tasks.workunit.client.1.smithi019.stdout:6/108: write de/f1a [654696,9498] 0 2024-03-16T09:01:07.854 INFO:tasks.workunit.client.0.smithi007.stdout:4/152: creat d4/d14/d1c/f32 x:0 0 0 2024-03-16T09:01:07.854 INFO:tasks.workunit.client.0.smithi007.stdout:4/153: stat d4/l24 0 2024-03-16T09:01:07.854 INFO:tasks.workunit.client.0.smithi007.stdout:8/310: dread d0/f31 [0,4194304] 0 2024-03-16T09:01:07.856 INFO:tasks.workunit.client.1.smithi019.stdout:6/109: chown c1 12600778 1 2024-03-16T09:01:07.856 INFO:tasks.workunit.client.1.smithi019.stdout:6/110: fdatasync fb 0 2024-03-16T09:01:07.856 INFO:tasks.workunit.client.1.smithi019.stdout:6/111: rename de/d1c to de/d1c/d29 22 2024-03-16T09:01:07.856 INFO:tasks.workunit.client.1.smithi019.stdout:6/112: write de/d1c/f28 [519724,31278] 0 2024-03-16T09:01:07.856 INFO:tasks.workunit.client.1.smithi019.stdout:6/113: dread - de/d1c/d1d/f24 zero size 2024-03-16T09:01:07.856 INFO:tasks.workunit.client.1.smithi019.stdout:6/114: chown de/d1c 1165950901 1 2024-03-16T09:01:07.856 INFO:tasks.workunit.client.0.smithi007.stdout:9/239: unlink d24/f2c 0 2024-03-16T09:01:07.857 INFO:tasks.workunit.client.0.smithi007.stdout:9/240: chown d24/f3f 1 1 2024-03-16T09:01:07.857 INFO:tasks.workunit.client.0.smithi007.stdout:9/241: dread - d24/f26 zero size 2024-03-16T09:01:07.860 INFO:tasks.workunit.client.0.smithi007.stdout:0/180: mkdir d3/d9/db/d3f 0 2024-03-16T09:01:07.860 INFO:tasks.workunit.client.0.smithi007.stdout:1/166: dwrite d9/dd/d1b/f25 [4194304,4194304] 0 2024-03-16T09:01:07.866 INFO:tasks.workunit.client.0.smithi007.stdout:9/242: read f16 [1350439,74545] 0 2024-03-16T09:01:07.867 INFO:tasks.workunit.client.0.smithi007.stdout:2/320: dwrite f7 [0,4194304] 0 2024-03-16T09:01:07.867 INFO:tasks.workunit.client.0.smithi007.stdout:2/321: truncate dc/f3a 493075 0 2024-03-16T09:01:07.867 INFO:tasks.workunit.client.0.smithi007.stdout:2/322: fsync dc/df/f29 0 2024-03-16T09:01:07.867 INFO:tasks.workunit.client.0.smithi007.stdout:5/167: creat d1/d7/f38 x:0 0 0 2024-03-16T09:01:07.867 INFO:tasks.workunit.client.0.smithi007.stdout:5/168: chown d1/c2f 84396792 1 2024-03-16T09:01:07.868 INFO:tasks.workunit.client.1.smithi019.stdout:6/115: dread f3 [0,4194304] 0 2024-03-16T09:01:07.868 INFO:tasks.workunit.client.0.smithi007.stdout:3/269: dread - d15/d18/f33 zero size 2024-03-16T09:01:07.870 INFO:tasks.workunit.client.1.smithi019.stdout:6/116: creat de/d26/f2a x:0 0 0 2024-03-16T09:01:07.871 INFO:tasks.workunit.client.0.smithi007.stdout:1/167: write d9/f24 [1216903,109750] 0 2024-03-16T09:01:07.871 INFO:tasks.workunit.client.0.smithi007.stdout:0/181: symlink d3/d9/da/d18/d27/d34/d3a/l40 0 2024-03-16T09:01:07.871 INFO:tasks.workunit.client.0.smithi007.stdout:0/182: dread - d3/d9/da/d18/d23/f32 zero size 2024-03-16T09:01:07.873 INFO:tasks.workunit.client.0.smithi007.stdout:1/168: symlink d9/dd/d1b/d2d/d30/l34 0 2024-03-16T09:01:07.875 INFO:tasks.workunit.client.0.smithi007.stdout:0/183: link d3/c39 d3/d9/da/d18/d27/d34/d3a/c41 0 2024-03-16T09:01:07.877 INFO:tasks.workunit.client.0.smithi007.stdout:1/169: unlink d9/dd/l16 0 2024-03-16T09:01:07.879 INFO:tasks.workunit.client.0.smithi007.stdout:9/243: read d24/f2a [1570114,54944] 0 2024-03-16T09:01:07.879 INFO:tasks.workunit.client.0.smithi007.stdout:0/184: unlink d3/d9/da/d18/d27/d34/d3a/l40 0 2024-03-16T09:01:07.879 INFO:tasks.workunit.client.0.smithi007.stdout:0/185: write d3/f8 [1710742,35300] 0 2024-03-16T09:01:07.880 INFO:tasks.workunit.client.1.smithi019.stdout:6/117: write fb [4979084,75342] 0 2024-03-16T09:01:07.882 INFO:tasks.workunit.client.0.smithi007.stdout:9/244: creat d24/f46 x:0 0 0 2024-03-16T09:01:07.884 INFO:tasks.workunit.client.0.smithi007.stdout:0/186: rmdir d3/d9/db/d3f 0 2024-03-16T09:01:07.884 INFO:tasks.workunit.client.0.smithi007.stdout:0/187: write d3/d9/da/d10/d1a/f20 [746163,40115] 0 2024-03-16T09:01:07.884 INFO:tasks.workunit.client.0.smithi007.stdout:2/323: write dc/df/f18 [1639096,108246] 0 2024-03-16T09:01:07.884 INFO:tasks.workunit.client.0.smithi007.stdout:6/231: dwrite d2/f46 [0,4194304] 0 2024-03-16T09:01:07.884 INFO:tasks.workunit.client.0.smithi007.stdout:3/270: write d15/d18/f22 [1855671,104235] 0 2024-03-16T09:01:07.885 INFO:tasks.workunit.client.0.smithi007.stdout:9/245: mknod d24/c47 0 2024-03-16T09:01:07.885 INFO:tasks.workunit.client.0.smithi007.stdout:9/246: stat d24/l3a 0 2024-03-16T09:01:07.885 INFO:tasks.workunit.client.0.smithi007.stdout:9/247: creat d24/f48 x:0 0 0 2024-03-16T09:01:07.885 INFO:tasks.workunit.client.0.smithi007.stdout:9/248: fdatasync d24/f39 0 2024-03-16T09:01:07.886 INFO:tasks.workunit.client.0.smithi007.stdout:0/188: symlink d3/d9/db/d2c/l42 0 2024-03-16T09:01:07.886 INFO:tasks.workunit.client.0.smithi007.stdout:0/189: write d3/d9/f11 [1305282,67916] 0 2024-03-16T09:01:07.887 INFO:tasks.workunit.client.0.smithi007.stdout:4/154: dwrite d4/d14/d1c/f26 [0,4194304] 0 2024-03-16T09:01:07.891 INFO:tasks.workunit.client.0.smithi007.stdout:5/169: dwrite d1/f2 [0,4194304] 0 2024-03-16T09:01:07.891 INFO:tasks.workunit.client.0.smithi007.stdout:8/311: dwrite d0/f3c [0,4194304] 0 2024-03-16T09:01:07.898 INFO:tasks.workunit.client.0.smithi007.stdout:1/170: dwrite d9/f22 [0,4194304] 0 2024-03-16T09:01:07.905 INFO:tasks.workunit.client.0.smithi007.stdout:3/271: getdents d15/d18/d1f/d3b 0 2024-03-16T09:01:07.906 INFO:tasks.workunit.client.0.smithi007.stdout:3/272: chown d15/f16 7033906 1 2024-03-16T09:01:07.906 INFO:tasks.workunit.client.0.smithi007.stdout:3/273: dread - d15/d18/d1f/f41 zero size 2024-03-16T09:01:07.907 INFO:tasks.workunit.client.1.smithi019.stdout:6/118: dread fb [4194304,4194304] 0 2024-03-16T09:01:07.907 INFO:tasks.workunit.client.1.smithi019.stdout:6/119: chown de/d1c/f20 5 1 2024-03-16T09:01:07.909 INFO:tasks.workunit.client.1.smithi019.stdout:6/120: link de/d1c/c22 de/c2b 0 2024-03-16T09:01:07.909 INFO:tasks.workunit.client.1.smithi019.stdout:6/121: write de/f1a [1132419,99272] 0 2024-03-16T09:01:07.910 INFO:tasks.workunit.client.0.smithi007.stdout:9/249: dwrite d24/f48 [0,4194304] 0 2024-03-16T09:01:07.910 INFO:tasks.workunit.client.0.smithi007.stdout:6/232: dwrite d2/f7 [0,4194304] 0 2024-03-16T09:01:07.911 INFO:tasks.workunit.client.0.smithi007.stdout:9/250: truncate d24/f44 763787 0 2024-03-16T09:01:07.911 INFO:tasks.workunit.client.0.smithi007.stdout:9/251: write d24/f31 [136324,76245] 0 2024-03-16T09:01:07.911 INFO:tasks.workunit.client.0.smithi007.stdout:2/324: dwrite dc/f3a [0,4194304] 0 2024-03-16T09:01:07.911 INFO:tasks.workunit.client.0.smithi007.stdout:2/325: readlink dc/df/d24/d28/d4c/d5e/l55 0 2024-03-16T09:01:07.911 INFO:tasks.workunit.client.0.smithi007.stdout:3/274: dread d15/d18/d1f/d29/f25 [0,4194304] 0 2024-03-16T09:01:07.911 INFO:tasks.workunit.client.0.smithi007.stdout:3/275: write d15/d18/f35 [5114270,63186] 0 2024-03-16T09:01:07.914 INFO:tasks.workunit.client.0.smithi007.stdout:5/170: mknod d1/d7/d16/d1e/c39 0 2024-03-16T09:01:07.914 INFO:tasks.workunit.client.0.smithi007.stdout:5/171: fdatasync d1/d7/d16/d1e/f31 0 2024-03-16T09:01:07.917 INFO:tasks.workunit.client.0.smithi007.stdout:2/326: dread dc/f68 [0,4194304] 0 2024-03-16T09:01:07.918 INFO:tasks.workunit.client.0.smithi007.stdout:9/252: write f1b [215160,74547] 0 2024-03-16T09:01:07.920 INFO:tasks.workunit.client.0.smithi007.stdout:1/171: dwrite f7 [0,4194304] 0 2024-03-16T09:01:07.924 INFO:tasks.workunit.client.1.smithi019.stdout:6/122: dread de/f14 [0,4194304] 0 2024-03-16T09:01:07.930 INFO:tasks.workunit.client.0.smithi007.stdout:6/233: dwrite d2/d11/f25 [0,4194304] 0 2024-03-16T09:01:07.932 INFO:tasks.workunit.client.0.smithi007.stdout:3/276: creat d15/d18/d1f/d3b/f57 x:0 0 0 2024-03-16T09:01:07.932 INFO:tasks.workunit.client.0.smithi007.stdout:3/277: write d15/d18/d1f/f39 [960526,69415] 0 2024-03-16T09:01:07.933 INFO:tasks.workunit.client.0.smithi007.stdout:8/312: link d0/d26/d2d/c36 d0/d2/c6d 0 2024-03-16T09:01:07.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/313: fdatasync d0/d2/f67 0 2024-03-16T09:01:07.935 INFO:tasks.workunit.client.0.smithi007.stdout:8/314: write d0/d2/d18/f48 [281656,25293] 0 2024-03-16T09:01:07.935 INFO:tasks.workunit.client.0.smithi007.stdout:8/315: stat d0/d26/l63 0 2024-03-16T09:01:07.935 INFO:tasks.workunit.client.0.smithi007.stdout:8/316: write d0/f31 [999918,76155] 0 2024-03-16T09:01:07.936 INFO:tasks.workunit.client.0.smithi007.stdout:5/172: creat d1/d7/d1c/d19/f3a x:0 0 0 2024-03-16T09:01:07.936 INFO:tasks.workunit.client.0.smithi007.stdout:5/173: readlink d1/d7/d1c/d19/d34/de/l24 0 2024-03-16T09:01:07.937 INFO:tasks.workunit.client.0.smithi007.stdout:2/327: creat dc/df/d24/d28/d2e/d6e/f6f x:0 0 0 2024-03-16T09:01:07.937 INFO:tasks.workunit.client.0.smithi007.stdout:2/328: dread - dc/df/d24/d28/d2e/d6e/f6f zero size 2024-03-16T09:01:07.941 INFO:tasks.workunit.client.1.smithi019.stdout:6/123: dwrite de/f25 [0,4194304] 0 2024-03-16T09:01:07.941 INFO:tasks.workunit.client.0.smithi007.stdout:1/172: link f5 d9/f35 0 2024-03-16T09:01:07.943 INFO:tasks.workunit.client.0.smithi007.stdout:0/190: rename d3/d9/da/d10 to d3/d9/da/d18/d21/d25/d43 0 2024-03-16T09:01:07.950 INFO:tasks.workunit.client.0.smithi007.stdout:8/317: symlink d0/d2/d18/l6e 0 2024-03-16T09:01:07.951 INFO:tasks.workunit.client.0.smithi007.stdout:2/329: mknod dc/df/d24/d28/d4c/d5e/c70 0 2024-03-16T09:01:07.951 INFO:tasks.workunit.client.0.smithi007.stdout:1/173: dread d9/f35 [0,4194304] 0 2024-03-16T09:01:07.953 INFO:tasks.workunit.client.0.smithi007.stdout:3/278: rename d15/f4d to d15/d18/d1f/f58 0 2024-03-16T09:01:07.954 INFO:tasks.workunit.client.0.smithi007.stdout:3/279: creat d15/f59 x:0 0 0 2024-03-16T09:01:07.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/330: mkdir dc/df/d24/d28/d71 0 2024-03-16T09:01:07.955 INFO:tasks.workunit.client.0.smithi007.stdout:1/174: creat d9/dd/d1b/d28/f36 x:0 0 0 2024-03-16T09:01:07.957 INFO:tasks.workunit.client.0.smithi007.stdout:0/191: rename d3/f28 to d3/d9/da/d18/d21/f44 0 2024-03-16T09:01:07.957 INFO:tasks.workunit.client.0.smithi007.stdout:0/192: creat d3/d9/da/d18/d23/f45 x:0 0 0 2024-03-16T09:01:07.957 INFO:tasks.workunit.client.0.smithi007.stdout:0/193: dread - d3/d9/db/d1c/f3c zero size 2024-03-16T09:01:07.957 INFO:tasks.workunit.client.0.smithi007.stdout:0/194: fdatasync d3/d9/da/d18/d21/d25/d43/d1a/f2d 0 2024-03-16T09:01:07.957 INFO:tasks.workunit.client.0.smithi007.stdout:6/234: dwrite d2/d11/f47 [0,4194304] 0 2024-03-16T09:01:07.958 INFO:tasks.workunit.client.0.smithi007.stdout:5/174: dwrite d1/d7/d1c/d19/f26 [0,4194304] 0 2024-03-16T09:01:07.959 INFO:tasks.workunit.client.0.smithi007.stdout:3/280: symlink d15/d18/d1f/d29/d3e/l5a 0 2024-03-16T09:01:07.960 INFO:tasks.workunit.client.0.smithi007.stdout:6/235: read d2/d10/f3a [482397,70126] 0 2024-03-16T09:01:07.960 INFO:tasks.workunit.client.0.smithi007.stdout:9/253: getdents d24 0 2024-03-16T09:01:07.962 INFO:tasks.workunit.client.0.smithi007.stdout:2/331: symlink dc/df/d24/d28/d2e/l72 0 2024-03-16T09:01:07.963 INFO:tasks.workunit.client.0.smithi007.stdout:2/332: dread dc/fd [0,4194304] 0 2024-03-16T09:01:07.963 INFO:tasks.workunit.client.0.smithi007.stdout:1/175: symlink d9/dd/d1b/d2d/l37 0 2024-03-16T09:01:07.965 INFO:tasks.workunit.client.0.smithi007.stdout:0/195: creat d3/d9/da/d18/d23/d36/f46 x:0 0 0 2024-03-16T09:01:07.965 INFO:tasks.workunit.client.0.smithi007.stdout:0/196: creat d3/d9/da/d18/d21/f47 x:0 0 0 2024-03-16T09:01:07.966 INFO:tasks.workunit.client.0.smithi007.stdout:8/318: write d0/d26/f30 [1623424,62541] 0 2024-03-16T09:01:07.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/281: truncate d15/d18/d1f/f2e 478098 0 2024-03-16T09:01:07.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/282: chown ld 16302273 1 2024-03-16T09:01:07.971 INFO:tasks.workunit.client.0.smithi007.stdout:6/236: dread d2/d11/f25 [0,4194304] 0 2024-03-16T09:01:07.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/283: write d15/d18/f33 [21262,68254] 0 2024-03-16T09:01:07.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/284: chown lf 580518254 1 2024-03-16T09:01:07.971 INFO:tasks.workunit.client.0.smithi007.stdout:9/254: creat d24/f49 x:0 0 0 2024-03-16T09:01:07.974 INFO:tasks.workunit.client.0.smithi007.stdout:1/176: mknod d9/c38 0 2024-03-16T09:01:07.975 INFO:tasks.workunit.client.0.smithi007.stdout:1/177: fdatasync f0 0 2024-03-16T09:01:07.975 INFO:tasks.workunit.client.0.smithi007.stdout:1/178: readlink d9/l20 0 2024-03-16T09:01:07.975 INFO:tasks.workunit.client.0.smithi007.stdout:1/179: write d9/dd/d1b/f2c [1931352,123813] 0 2024-03-16T09:01:07.976 INFO:tasks.workunit.client.0.smithi007.stdout:3/285: dread fa [0,4194304] 0 2024-03-16T09:01:07.980 INFO:tasks.workunit.client.0.smithi007.stdout:1/180: getdents d9/dd/d1b/d2d/d30 0 2024-03-16T09:01:07.982 INFO:tasks.workunit.client.0.smithi007.stdout:3/286: creat d15/d18/d1f/d29/f5b x:0 0 0 2024-03-16T09:01:07.983 INFO:tasks.workunit.client.0.smithi007.stdout:9/255: dread d24/f31 [0,4194304] 0 2024-03-16T09:01:07.983 INFO:tasks.workunit.client.1.smithi019.stdout:6/124: dwrite de/d1c/f28 [0,4194304] 0 2024-03-16T09:01:07.983 INFO:tasks.workunit.client.1.smithi019.stdout:6/125: readlink ld 0 2024-03-16T09:01:07.985 INFO:tasks.workunit.client.1.smithi019.stdout:6/126: mkdir de/d1c/d1d/d2c 0 2024-03-16T09:01:07.987 INFO:tasks.workunit.client.1.smithi019.stdout:6/127: symlink de/d1c/l2d 0 2024-03-16T09:01:07.987 INFO:tasks.workunit.client.1.smithi019.stdout:6/128: fsync de/d1c/f28 0 2024-03-16T09:01:07.987 INFO:tasks.workunit.client.1.smithi019.stdout:6/129: creat de/d1c/f2e x:0 0 0 2024-03-16T09:01:07.989 INFO:tasks.workunit.client.0.smithi007.stdout:1/181: dread d9/dd/d1b/d28/f2b [0,4194304] 0 2024-03-16T09:01:07.989 INFO:tasks.workunit.client.0.smithi007.stdout:1/182: creat d9/f39 x:0 0 0 2024-03-16T09:01:07.989 INFO:tasks.workunit.client.0.smithi007.stdout:1/183: fdatasync d9/f12 0 2024-03-16T09:01:07.989 INFO:tasks.workunit.client.0.smithi007.stdout:1/184: write d9/f24 [2493010,62531] 0 2024-03-16T09:01:07.992 INFO:tasks.workunit.client.0.smithi007.stdout:2/333: dwrite dc/f11 [0,4194304] 0 2024-03-16T09:01:07.997 INFO:tasks.workunit.client.0.smithi007.stdout:0/197: rename d3/d9/da/d18/d21 to d3/d9/da/d48 0 2024-03-16T09:01:08.002 INFO:tasks.workunit.client.1.smithi019.stdout:6/130: write fc [2874035,29077] 0 2024-03-16T09:01:08.002 INFO:tasks.workunit.client.1.smithi019.stdout:6/131: mkdir de/d1c/d1d/d2c/d2f 0 2024-03-16T09:01:08.003 INFO:tasks.workunit.client.0.smithi007.stdout:0/198: getdents d3/d9/da/d18/d27/d34 0 2024-03-16T09:01:08.003 INFO:tasks.workunit.client.0.smithi007.stdout:3/287: rename d15/d18/c2b to d15/d18/d1f/d56/c5c 0 2024-03-16T09:01:08.003 INFO:tasks.workunit.client.0.smithi007.stdout:0/199: rename d3/d9/db/d1c/f3c to d3/f49 0 2024-03-16T09:01:08.003 INFO:tasks.workunit.client.0.smithi007.stdout:0/200: chown d3/d9/da/d48/d25/d43/d1a 951637033 1 2024-03-16T09:01:08.003 INFO:tasks.workunit.client.0.smithi007.stdout:1/185: dread d9/dd/d1b/f25 [4194304,4194304] 0 2024-03-16T09:01:08.003 INFO:tasks.workunit.client.0.smithi007.stdout:6/237: dwrite d2/d11/f26 [0,4194304] 0 2024-03-16T09:01:08.003 INFO:tasks.workunit.client.0.smithi007.stdout:9/256: dwrite f1b [0,4194304] 0 2024-03-16T09:01:08.004 INFO:tasks.workunit.client.1.smithi019.stdout:6/132: unlink f3 0 2024-03-16T09:01:08.004 INFO:tasks.workunit.client.1.smithi019.stdout:6/133: write de/d1c/f20 [604494,59888] 0 2024-03-16T09:01:08.004 INFO:tasks.workunit.client.1.smithi019.stdout:6/134: creat de/d1c/d1d/f30 x:0 0 0 2024-03-16T09:01:08.004 INFO:tasks.workunit.client.0.smithi007.stdout:3/288: mknod d15/d18/c5d 0 2024-03-16T09:01:08.005 INFO:tasks.workunit.client.0.smithi007.stdout:3/289: read f5 [316191,114515] 0 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:1/186: rename d9 to d9/dd/d3a 22 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:2/334: dread dc/df/d24/d28/d2e/f52 [0,4194304] 0 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:2/335: fdatasync dc/df/d24/d28/d2e/f52 0 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:2/336: dread - dc/df/d24/d28/d4c/f57 zero size 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:2/337: write dc/df/d24/d28/d4c/f59 [1920140,105431] 0 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:0/201: symlink d3/d9/db/l4a 0 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:0/202: fsync d3/d9/db/fc 0 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:0/203: chown d3/d9/da/d18/d23/f32 585 1 2024-03-16T09:01:08.006 INFO:tasks.workunit.client.0.smithi007.stdout:0/204: write d3/d9/da/d48/d25/d43/d1a/f20 [1068357,57464] 0 2024-03-16T09:01:08.007 INFO:tasks.workunit.client.1.smithi019.stdout:6/135: unlink fc 0 2024-03-16T09:01:08.008 INFO:tasks.workunit.client.1.smithi019.stdout:6/136: write de/d1c/d1d/f30 [398460,108457] 0 2024-03-16T09:01:08.008 INFO:tasks.workunit.client.1.smithi019.stdout:6/137: readlink de/d1c/l2d 0 2024-03-16T09:01:08.008 INFO:tasks.workunit.client.0.smithi007.stdout:6/238: mkdir d2/d13/d1f/d4e 0 2024-03-16T09:01:08.008 INFO:tasks.workunit.client.0.smithi007.stdout:9/257: fdatasync d24/f39 0 2024-03-16T09:01:08.009 INFO:tasks.workunit.client.0.smithi007.stdout:3/290: symlink d15/d49/l5e 0 2024-03-16T09:01:08.013 INFO:tasks.workunit.client.0.smithi007.stdout:9/258: readlink d24/l30 0 2024-03-16T09:01:08.015 INFO:tasks.workunit.client.0.smithi007.stdout:3/291: mknod d15/d18/d1f/d3b/c5f 0 2024-03-16T09:01:08.015 INFO:tasks.workunit.client.0.smithi007.stdout:3/292: truncate d15/d18/f19 4880780 0 2024-03-16T09:01:08.016 INFO:tasks.workunit.client.0.smithi007.stdout:9/259: chown d24/l37 24839592 1 2024-03-16T09:01:08.018 INFO:tasks.workunit.client.0.smithi007.stdout:6/239: dread d2/fa [0,4194304] 0 2024-03-16T09:01:08.018 INFO:tasks.workunit.client.0.smithi007.stdout:9/260: getdents d24 0 2024-03-16T09:01:08.019 INFO:tasks.workunit.client.0.smithi007.stdout:9/261: getdents d24 0 2024-03-16T09:01:08.020 INFO:tasks.workunit.client.0.smithi007.stdout:0/205: read d3/d9/f11 [415105,22455] 0 2024-03-16T09:01:08.020 INFO:tasks.workunit.client.0.smithi007.stdout:9/262: mkdir d24/d4a 0 2024-03-16T09:01:08.021 INFO:tasks.workunit.client.0.smithi007.stdout:0/206: mkdir d3/d9/da/d48/d4b 0 2024-03-16T09:01:08.022 INFO:tasks.workunit.client.0.smithi007.stdout:9/263: symlink d24/l4b 0 2024-03-16T09:01:08.022 INFO:tasks.workunit.client.0.smithi007.stdout:2/338: dwrite dc/f3a [0,4194304] 0 2024-03-16T09:01:08.023 INFO:tasks.workunit.client.0.smithi007.stdout:0/207: unlink d3/d9/da/d18/d23/f3d 0 2024-03-16T09:01:08.026 INFO:tasks.workunit.client.0.smithi007.stdout:3/293: dread d15/d18/f22 [0,4194304] 0 2024-03-16T09:01:08.028 INFO:tasks.workunit.client.0.smithi007.stdout:3/294: link d15/d18/d1f/c34 d15/d18/d1f/d29/d3e/c60 0 2024-03-16T09:01:08.028 INFO:tasks.workunit.client.0.smithi007.stdout:3/295: mknod d15/d18/d1f/d29/d3e/c61 0 2024-03-16T09:01:08.029 INFO:tasks.workunit.client.1.smithi019.stdout:6/138: dwrite de/d1c/d1d/f24 [0,4194304] 0 2024-03-16T09:01:08.030 INFO:tasks.workunit.client.0.smithi007.stdout:0/208: dread d3/d9/da/d48/f12 [0,4194304] 0 2024-03-16T09:01:08.030 INFO:tasks.workunit.client.0.smithi007.stdout:3/296: mknod d15/d18/d1f/d29/d43/c62 0 2024-03-16T09:01:08.030 INFO:tasks.workunit.client.0.smithi007.stdout:0/209: creat d3/d9/da/f4c x:0 0 0 2024-03-16T09:01:08.030 INFO:tasks.workunit.client.0.smithi007.stdout:0/210: creat d3/d9/da/d18/d23/f4d x:0 0 0 2024-03-16T09:01:08.031 INFO:tasks.workunit.client.0.smithi007.stdout:0/211: write d3/d9/f26 [1239938,67902] 0 2024-03-16T09:01:08.031 INFO:tasks.workunit.client.0.smithi007.stdout:1/187: dwrite d9/f39 [0,4194304] 0 2024-03-16T09:01:08.031 INFO:tasks.workunit.client.0.smithi007.stdout:9/264: write f1b [2498317,69989] 0 2024-03-16T09:01:08.032 INFO:tasks.workunit.client.1.smithi019.stdout:6/139: rename de/lf to de/l31 0 2024-03-16T09:01:08.033 INFO:tasks.workunit.client.0.smithi007.stdout:3/297: mkdir d15/d18/d1f/d3b/d63 0 2024-03-16T09:01:08.035 INFO:tasks.workunit.client.0.smithi007.stdout:2/339: dread f7 [0,4194304] 0 2024-03-16T09:01:08.035 INFO:tasks.workunit.client.0.smithi007.stdout:2/340: readlink dc/d27/l6d 0 2024-03-16T09:01:08.036 INFO:tasks.workunit.client.0.smithi007.stdout:2/341: chown dc/df/d24/d28/d4c/f67 439 1 2024-03-16T09:01:08.036 INFO:tasks.workunit.client.0.smithi007.stdout:2/342: stat dc/df/d24/d28/d71 0 2024-03-16T09:01:08.036 INFO:tasks.workunit.client.0.smithi007.stdout:1/188: dread d9/dd/d1b/f2c [0,4194304] 0 2024-03-16T09:01:08.037 INFO:tasks.workunit.client.1.smithi019.stdout:6/140: rmdir de/d1c 39 2024-03-16T09:01:08.038 INFO:tasks.workunit.client.1.smithi019.stdout:6/141: rmdir de/d1c/d1d/d2c 39 2024-03-16T09:01:08.038 INFO:tasks.workunit.client.1.smithi019.stdout:6/142: truncate de/f25 5239841 0 2024-03-16T09:01:08.039 INFO:tasks.workunit.client.0.smithi007.stdout:9/265: mkdir d24/d4c 0 2024-03-16T09:01:08.040 INFO:tasks.workunit.client.1.smithi019.stdout:6/143: mkdir de/d26/d32 0 2024-03-16T09:01:08.041 INFO:tasks.workunit.client.1.smithi019.stdout:6/144: mknod de/d1c/d1d/d2c/c33 0 2024-03-16T09:01:08.041 INFO:tasks.workunit.client.1.smithi019.stdout:6/145: write fb [7562488,124901] 0 2024-03-16T09:01:08.041 INFO:tasks.workunit.client.1.smithi019.stdout:6/146: chown de/c1b 23985249 1 2024-03-16T09:01:08.041 INFO:tasks.workunit.client.1.smithi019.stdout:6/147: readlink de/d1c/l2d 0 2024-03-16T09:01:08.042 INFO:tasks.workunit.client.0.smithi007.stdout:2/343: creat dc/df/d24/d28/d4c/f73 x:0 0 0 2024-03-16T09:01:08.043 INFO:tasks.workunit.client.1.smithi019.stdout:6/148: symlink de/d26/d32/l34 0 2024-03-16T09:01:08.045 INFO:tasks.workunit.client.0.smithi007.stdout:1/189: dread d9/f22 [0,4194304] 0 2024-03-16T09:01:08.049 INFO:tasks.workunit.client.0.smithi007.stdout:9/266: creat d24/f4d x:0 0 0 2024-03-16T09:01:08.053 INFO:tasks.workunit.client.0.smithi007.stdout:3/298: rmdir d15/d18 39 2024-03-16T09:01:08.055 INFO:tasks.workunit.client.0.smithi007.stdout:1/190: truncate f5 846953 0 2024-03-16T09:01:08.056 INFO:tasks.workunit.client.0.smithi007.stdout:9/267: creat d24/d4c/f4e x:0 0 0 2024-03-16T09:01:08.057 INFO:tasks.workunit.client.0.smithi007.stdout:3/299: getdents d15/d18/d1f/d3b 0 2024-03-16T09:01:08.057 INFO:tasks.workunit.client.0.smithi007.stdout:2/344: rmdir dc/df/d24/d28/d4c 39 2024-03-16T09:01:08.058 INFO:tasks.workunit.client.0.smithi007.stdout:7/194: sync 2024-03-16T09:01:08.059 INFO:tasks.workunit.client.0.smithi007.stdout:3/300: dread d15/d18/d1f/f2e [0,4194304] 0 2024-03-16T09:01:08.059 INFO:tasks.workunit.client.0.smithi007.stdout:3/301: truncate f13 1599828 0 2024-03-16T09:01:08.062 INFO:tasks.workunit.client.0.smithi007.stdout:3/302: link d15/d18/f22 d15/d18/d1f/d29/d3e/d4c/f64 0 2024-03-16T09:01:08.062 INFO:tasks.workunit.client.0.smithi007.stdout:3/303: truncate d15/d18/d1f/f2e 848400 0 2024-03-16T09:01:08.064 INFO:tasks.workunit.client.0.smithi007.stdout:3/304: creat d15/d18/d1f/d56/f65 x:0 0 0 2024-03-16T09:01:08.064 INFO:tasks.workunit.client.0.smithi007.stdout:2/345: write dc/df/f29 [2694633,120159] 0 2024-03-16T09:01:08.067 INFO:tasks.workunit.client.0.smithi007.stdout:2/346: truncate dc/f15 5176574 0 2024-03-16T09:01:08.067 INFO:tasks.workunit.client.0.smithi007.stdout:2/347: write dc/df/f18 [63938,1942] 0 2024-03-16T09:01:08.067 INFO:tasks.workunit.client.0.smithi007.stdout:2/348: write dc/df/d24/d28/d2e/f5c [754140,61639] 0 2024-03-16T09:01:08.081 INFO:tasks.workunit.client.0.smithi007.stdout:9/268: dwrite f1b [0,4194304] 0 2024-03-16T09:01:08.081 INFO:tasks.workunit.client.0.smithi007.stdout:1/191: dwrite d9/dd/d1b/d28/d29/f2a [4194304,4194304] 0 2024-03-16T09:01:08.081 INFO:tasks.workunit.client.0.smithi007.stdout:3/305: dwrite d15/d18/d1f/d56/f65 [0,4194304] 0 2024-03-16T09:01:08.082 INFO:tasks.workunit.client.0.smithi007.stdout:2/349: write dc/f11 [3398180,113471] 0 2024-03-16T09:01:08.082 INFO:tasks.workunit.client.0.smithi007.stdout:9/269: unlink d24/c45 0 2024-03-16T09:01:08.084 INFO:tasks.workunit.client.0.smithi007.stdout:3/306: truncate d15/d18/d1f/d29/f31 1791328 0 2024-03-16T09:01:08.084 INFO:tasks.workunit.client.0.smithi007.stdout:3/307: fsync d15/d18/d1f/d3b/f44 0 2024-03-16T09:01:08.086 INFO:tasks.workunit.client.0.smithi007.stdout:2/350: link dc/df/l30 dc/df/d24/d28/d2e/l74 0 2024-03-16T09:01:08.086 INFO:tasks.workunit.client.0.smithi007.stdout:2/351: readlink dc/df/l30 0 2024-03-16T09:01:08.088 INFO:tasks.workunit.client.0.smithi007.stdout:1/192: getdents d9/dd/d1b 0 2024-03-16T09:01:08.088 INFO:tasks.workunit.client.0.smithi007.stdout:1/193: chown d9/lb 11 1 2024-03-16T09:01:08.090 INFO:tasks.workunit.client.0.smithi007.stdout:2/352: mknod dc/df/d24/d28/d4c/c75 0 2024-03-16T09:01:08.097 INFO:tasks.workunit.client.0.smithi007.stdout:1/194: mknod d9/dd/c3b 0 2024-03-16T09:01:08.104 INFO:tasks.workunit.client.0.smithi007.stdout:1/195: dread d9/f12 [0,4194304] 0 2024-03-16T09:01:08.105 INFO:tasks.workunit.client.0.smithi007.stdout:1/196: truncate d9/f21 263178 0 2024-03-16T09:01:08.111 INFO:tasks.workunit.client.0.smithi007.stdout:9/270: dwrite d24/f3c [0,4194304] 0 2024-03-16T09:01:08.111 INFO:tasks.workunit.client.0.smithi007.stdout:2/353: dwrite dc/df/d24/d28/d2e/f5c [0,4194304] 0 2024-03-16T09:01:08.111 INFO:tasks.workunit.client.0.smithi007.stdout:2/354: truncate dc/df/d24/d28/d2e/f45 543622 0 2024-03-16T09:01:08.115 INFO:tasks.workunit.client.0.smithi007.stdout:2/355: unlink dc/df/d24/d28/d2e/l74 0 2024-03-16T09:01:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:1/197: write f7 [1043711,50084] 0 2024-03-16T09:01:08.121 INFO:tasks.workunit.client.0.smithi007.stdout:1/198: mknod d9/dd/d1b/d2d/d30/c3c 0 2024-03-16T09:01:08.121 INFO:tasks.workunit.client.0.smithi007.stdout:1/199: chown d9/dd/d1b/d28/f32 133395 1 2024-03-16T09:01:08.122 INFO:tasks.workunit.client.0.smithi007.stdout:9/271: dwrite d24/f43 [0,4194304] 0 2024-03-16T09:01:08.122 INFO:tasks.workunit.client.0.smithi007.stdout:1/200: link d9/c1a d9/dd/d1b/c3d 0 2024-03-16T09:01:08.124 INFO:tasks.workunit.client.0.smithi007.stdout:9/272: link d24/f3c d24/f4f 0 2024-03-16T09:01:08.124 INFO:tasks.workunit.client.0.smithi007.stdout:9/273: creat d24/f50 x:0 0 0 2024-03-16T09:01:08.125 INFO:tasks.workunit.client.0.smithi007.stdout:1/201: rename d9/dd/f33 to d9/dd/d1b/d28/f3e 0 2024-03-16T09:01:08.127 INFO:tasks.workunit.client.0.smithi007.stdout:9/274: symlink d24/d4c/l51 0 2024-03-16T09:01:08.127 INFO:tasks.workunit.client.0.smithi007.stdout:9/275: dread - d24/d4c/f4e zero size 2024-03-16T09:01:08.128 INFO:tasks.workunit.client.0.smithi007.stdout:1/202: creat d9/dd/d1b/d2d/f3f x:0 0 0 2024-03-16T09:01:08.128 INFO:tasks.workunit.client.0.smithi007.stdout:2/356: dwrite dc/df/d24/d28/d2e/d35/f40 [0,4194304] 0 2024-03-16T09:01:08.129 INFO:tasks.workunit.client.0.smithi007.stdout:2/357: fsync dc/f15 0 2024-03-16T09:01:08.129 INFO:tasks.workunit.client.0.smithi007.stdout:9/276: mknod d24/c52 0 2024-03-16T09:01:08.130 INFO:tasks.workunit.client.0.smithi007.stdout:9/277: dread d24/f25 [0,4194304] 0 2024-03-16T09:01:08.132 INFO:tasks.workunit.client.0.smithi007.stdout:1/203: unlink d9/l1d 0 2024-03-16T09:01:08.132 INFO:tasks.workunit.client.0.smithi007.stdout:1/204: chown d9/dd/f14 1821539 1 2024-03-16T09:01:08.137 INFO:tasks.workunit.client.0.smithi007.stdout:9/278: link d24/f4d d24/d4a/f53 0 2024-03-16T09:01:08.137 INFO:tasks.workunit.client.0.smithi007.stdout:9/279: creat d24/f54 x:0 0 0 2024-03-16T09:01:08.137 INFO:tasks.workunit.client.0.smithi007.stdout:9/280: truncate d24/f3d 970372 0 2024-03-16T09:01:08.138 INFO:tasks.workunit.client.0.smithi007.stdout:2/358: getdents dc/d27 0 2024-03-16T09:01:08.138 INFO:tasks.workunit.client.0.smithi007.stdout:2/359: dread - dc/df/d24/d28/d2e/d6e/f6f zero size 2024-03-16T09:01:08.138 INFO:tasks.workunit.client.0.smithi007.stdout:2/360: creat dc/df/d24/d28/d2e/d6e/f76 x:0 0 0 2024-03-16T09:01:08.139 INFO:tasks.workunit.client.0.smithi007.stdout:1/205: write d9/dd/d1b/f25 [3670109,11377] 0 2024-03-16T09:01:08.142 INFO:tasks.workunit.client.0.smithi007.stdout:2/361: chown dc/c16 803 1 2024-03-16T09:01:08.144 INFO:tasks.workunit.client.0.smithi007.stdout:2/362: truncate dc/df/d24/d28/d4c/d5e/f5b 715458 0 2024-03-16T09:01:08.147 INFO:tasks.workunit.client.0.smithi007.stdout:1/206: dread d9/dd/d1b/d28/f2b [0,4194304] 0 2024-03-16T09:01:08.154 INFO:tasks.workunit.client.0.smithi007.stdout:9/281: dwrite d24/f3c [0,4194304] 0 2024-03-16T09:01:08.156 INFO:tasks.workunit.client.0.smithi007.stdout:9/282: getdents d24 0 2024-03-16T09:01:08.164 INFO:tasks.workunit.client.0.smithi007.stdout:1/207: dwrite d9/dd/d1b/f2c [0,4194304] 0 2024-03-16T09:01:08.164 INFO:tasks.workunit.client.0.smithi007.stdout:1/208: write d9/dd/d1b/d28/f32 [1449972,81568] 0 2024-03-16T09:01:08.174 INFO:tasks.workunit.client.0.smithi007.stdout:4/155: sync 2024-03-16T09:01:08.174 INFO:tasks.workunit.client.0.smithi007.stdout:4/156: truncate d4/ff 1301633 0 2024-03-16T09:01:08.177 INFO:tasks.workunit.client.0.smithi007.stdout:4/157: rename d4/d14/d1c/d1e/f25 to d4/d7/f33 0 2024-03-16T09:01:08.178 INFO:tasks.workunit.client.0.smithi007.stdout:4/158: mkdir d4/d14/d34 0 2024-03-16T09:01:08.180 INFO:tasks.workunit.client.0.smithi007.stdout:4/159: link d4/d14/d1c/f29 d4/d14/f35 0 2024-03-16T09:01:08.180 INFO:tasks.workunit.client.0.smithi007.stdout:4/160: mknod d4/d14/d34/c36 0 2024-03-16T09:01:08.181 INFO:tasks.workunit.client.0.smithi007.stdout:4/161: creat d4/d14/d1c/f37 x:0 0 0 2024-03-16T09:01:08.181 INFO:tasks.workunit.client.0.smithi007.stdout:9/283: dwrite d24/f27 [0,4194304] 0 2024-03-16T09:01:08.182 INFO:tasks.workunit.client.0.smithi007.stdout:9/284: mkdir d24/d4a/d55 0 2024-03-16T09:01:08.182 INFO:tasks.workunit.client.0.smithi007.stdout:9/285: chown d24/c2f 30 1 2024-03-16T09:01:08.190 INFO:tasks.workunit.client.0.smithi007.stdout:9/286: dread d24/f43 [0,4194304] 0 2024-03-16T09:01:08.195 INFO:tasks.workunit.client.0.smithi007.stdout:4/162: dwrite d4/f2f [0,4194304] 0 2024-03-16T09:01:08.195 INFO:tasks.workunit.client.0.smithi007.stdout:4/163: stat d4/l2a 0 2024-03-16T09:01:08.196 INFO:tasks.workunit.client.0.smithi007.stdout:4/164: mkdir d4/d38 0 2024-03-16T09:01:08.197 INFO:tasks.workunit.client.0.smithi007.stdout:4/165: write d4/ff [944402,105273] 0 2024-03-16T09:01:08.199 INFO:tasks.workunit.client.0.smithi007.stdout:4/166: symlink d4/d14/d34/l39 0 2024-03-16T09:01:08.199 INFO:tasks.workunit.client.0.smithi007.stdout:4/167: fdatasync d4/f22 0 2024-03-16T09:01:08.199 INFO:tasks.workunit.client.0.smithi007.stdout:4/168: truncate d4/d14/d1c/f26 3877203 0 2024-03-16T09:01:08.200 INFO:tasks.workunit.client.0.smithi007.stdout:4/169: creat d4/d14/f3a x:0 0 0 2024-03-16T09:01:08.202 INFO:tasks.workunit.client.0.smithi007.stdout:4/170: rename d4/d14/f3a to d4/d14/d1c/f3b 0 2024-03-16T09:01:08.206 INFO:tasks.workunit.client.0.smithi007.stdout:4/171: unlink d4/l31 0 2024-03-16T09:01:08.206 INFO:tasks.workunit.client.0.smithi007.stdout:4/172: write d4/d7/fe [1374328,73353] 0 2024-03-16T09:01:08.206 INFO:tasks.workunit.client.0.smithi007.stdout:4/173: chown d4/d14/d1c/f32 2194657 1 2024-03-16T09:01:08.206 INFO:tasks.workunit.client.0.smithi007.stdout:9/287: dwrite d24/f34 [0,4194304] 0 2024-03-16T09:01:08.210 INFO:tasks.workunit.client.0.smithi007.stdout:4/174: write d4/f28 [4157484,130384] 0 2024-03-16T09:01:08.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:07 smithi019 ceph-mon[36407]: pgmap v17: 65 pgs: 65 active+clean; 591 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 49 MiB/s rd, 68 MiB/s wr, 1.07k op/s 2024-03-16T09:01:08.213 INFO:tasks.workunit.client.0.smithi007.stdout:9/288: dread d24/f48 [0,4194304] 0 2024-03-16T09:01:08.213 INFO:tasks.workunit.client.0.smithi007.stdout:9/289: read f16 [1362009,84662] 0 2024-03-16T09:01:08.215 INFO:tasks.workunit.client.0.smithi007.stdout:4/175: dread d4/f10 [0,4194304] 0 2024-03-16T09:01:08.219 INFO:tasks.workunit.client.0.smithi007.stdout:4/176: rename d4/d7/l2b to d4/l3c 0 2024-03-16T09:01:08.227 INFO:tasks.workunit.client.0.smithi007.stdout:9/290: dwrite d24/f3c [4194304,4194304] 0 2024-03-16T09:01:08.247 INFO:tasks.workunit.client.0.smithi007.stdout:9/291: dwrite d24/f29 [0,4194304] 0 2024-03-16T09:01:08.247 INFO:tasks.workunit.client.0.smithi007.stdout:9/292: truncate d24/f50 1023763 0 2024-03-16T09:01:08.266 INFO:tasks.workunit.client.0.smithi007.stdout:9/293: dwrite d24/f29 [0,4194304] 0 2024-03-16T09:01:08.267 INFO:tasks.workunit.client.0.smithi007.stdout:9/294: rmdir d24 39 2024-03-16T09:01:08.268 INFO:tasks.workunit.client.0.smithi007.stdout:9/295: mkdir d24/d4a/d55/d56 0 2024-03-16T09:01:08.269 INFO:tasks.workunit.client.0.smithi007.stdout:9/296: rename d24/l3e to d24/d4c/l57 0 2024-03-16T09:01:08.274 INFO:tasks.workunit.client.0.smithi007.stdout:9/297: write d24/f43 [2040622,50062] 0 2024-03-16T09:01:08.274 INFO:tasks.workunit.client.0.smithi007.stdout:9/298: write f1d [1260266,76214] 0 2024-03-16T09:01:08.274 INFO:tasks.workunit.client.0.smithi007.stdout:9/299: stat d24/d4c/f4e 0 2024-03-16T09:01:08.274 INFO:tasks.workunit.client.0.smithi007.stdout:9/300: fsync d24/f3f 0 2024-03-16T09:01:08.279 INFO:tasks.workunit.client.0.smithi007.stdout:9/301: dread f8 [4194304,4194304] 0 2024-03-16T09:01:08.279 INFO:tasks.workunit.client.0.smithi007.stdout:9/302: fsync d24/f4f 0 2024-03-16T09:01:08.282 INFO:tasks.workunit.client.0.smithi007.stdout:9/303: getdents d24/d4c 0 2024-03-16T09:01:08.282 INFO:tasks.workunit.client.0.smithi007.stdout:9/304: stat f1c 0 2024-03-16T09:01:08.291 INFO:tasks.workunit.client.0.smithi007.stdout:9/305: write d24/f3f [2101372,129535] 0 2024-03-16T09:01:08.293 INFO:tasks.workunit.client.0.smithi007.stdout:9/306: dread d24/f3d [0,4194304] 0 2024-03-16T09:01:08.294 INFO:tasks.workunit.client.0.smithi007.stdout:9/307: symlink d24/d4c/l58 0 2024-03-16T09:01:08.295 INFO:tasks.workunit.client.0.smithi007.stdout:9/308: unlink d24/l3a 0 2024-03-16T09:01:08.295 INFO:tasks.workunit.client.0.smithi007.stdout:9/309: dread - d24/f54 zero size 2024-03-16T09:01:08.296 INFO:tasks.workunit.client.0.smithi007.stdout:9/310: mknod d24/d4a/c59 0 2024-03-16T09:01:08.296 INFO:tasks.workunit.client.0.smithi007.stdout:9/311: truncate d24/f31 808063 0 2024-03-16T09:01:08.296 INFO:tasks.workunit.client.0.smithi007.stdout:9/312: dread - d24/f38 zero size 2024-03-16T09:01:08.297 INFO:tasks.workunit.client.0.smithi007.stdout:9/313: creat d24/d4a/d55/d56/f5a x:0 0 0 2024-03-16T09:01:08.303 INFO:tasks.workunit.client.0.smithi007.stdout:9/314: write f1e [7156857,1561] 0 2024-03-16T09:01:08.313 INFO:tasks.workunit.client.0.smithi007.stdout:9/315: dwrite d24/f39 [0,4194304] 0 2024-03-16T09:01:08.320 INFO:tasks.workunit.client.0.smithi007.stdout:9/316: dread d24/f29 [0,4194304] 0 2024-03-16T09:01:08.334 INFO:tasks.workunit.client.0.smithi007.stdout:9/317: dwrite f1e [4194304,4194304] 0 2024-03-16T09:01:08.340 INFO:tasks.workunit.client.0.smithi007.stdout:9/318: dread d24/f4f [4194304,4194304] 0 2024-03-16T09:01:08.341 INFO:tasks.workunit.client.0.smithi007.stdout:9/319: dread d24/f3d [0,4194304] 0 2024-03-16T09:01:08.342 INFO:tasks.workunit.client.0.smithi007.stdout:9/320: creat d24/d4a/d55/f5b x:0 0 0 2024-03-16T09:01:08.343 INFO:tasks.workunit.client.0.smithi007.stdout:9/321: creat d24/f5c x:0 0 0 2024-03-16T09:01:08.344 INFO:tasks.workunit.client.0.smithi007.stdout:9/322: creat d24/d4a/d55/f5d x:0 0 0 2024-03-16T09:01:08.345 INFO:tasks.workunit.client.0.smithi007.stdout:9/323: mknod d24/d4a/d55/c5e 0 2024-03-16T09:01:08.353 INFO:tasks.workunit.client.0.smithi007.stdout:9/324: dread d24/f3c [0,4194304] 0 2024-03-16T09:01:08.354 INFO:tasks.workunit.client.0.smithi007.stdout:9/325: truncate d24/f44 149752 0 2024-03-16T09:01:08.356 INFO:tasks.workunit.client.0.smithi007.stdout:5/175: sync 2024-03-16T09:01:08.358 INFO:tasks.workunit.client.0.smithi007.stdout:5/176: rename d1/d7/d1c/c30 to d1/d7/d1c/d19/d34/d28/c3b 0 2024-03-16T09:01:08.359 INFO:tasks.workunit.client.0.smithi007.stdout:5/177: mkdir d1/d7/d16/d1b/d3c 0 2024-03-16T09:01:08.359 INFO:tasks.workunit.client.0.smithi007.stdout:5/178: creat d1/d7/d16/d1e/f3d x:0 0 0 2024-03-16T09:01:08.361 INFO:tasks.workunit.client.0.smithi007.stdout:5/179: unlink d1/d7/d16/f2b 0 2024-03-16T09:01:08.363 INFO:tasks.workunit.client.0.smithi007.stdout:5/180: mkdir d1/d7/d1c/d19/d34/d3e 0 2024-03-16T09:01:08.367 INFO:tasks.workunit.client.0.smithi007.stdout:9/326: dwrite d24/f50 [0,4194304] 0 2024-03-16T09:01:08.369 INFO:tasks.workunit.client.0.smithi007.stdout:5/181: write d1/d7/d1c/d19/f26 [1577250,52742] 0 2024-03-16T09:01:08.370 INFO:tasks.workunit.client.0.smithi007.stdout:9/327: rename d24/l37 to d24/d4a/d55/l5f 0 2024-03-16T09:01:08.370 INFO:tasks.workunit.client.0.smithi007.stdout:9/328: write d24/f27 [4749566,77334] 0 2024-03-16T09:01:08.374 INFO:tasks.workunit.client.0.smithi007.stdout:9/329: getdents d24 0 2024-03-16T09:01:08.374 INFO:tasks.workunit.client.0.smithi007.stdout:9/330: creat d24/d4a/d55/d56/f60 x:0 0 0 2024-03-16T09:01:08.377 INFO:tasks.workunit.client.0.smithi007.stdout:9/331: mkdir d24/d4a/d61 0 2024-03-16T09:01:08.377 INFO:tasks.workunit.client.0.smithi007.stdout:9/332: readlink le 0 2024-03-16T09:01:08.389 INFO:tasks.workunit.client.0.smithi007.stdout:9/333: write f17 [1985233,60707] 0 2024-03-16T09:01:08.389 INFO:tasks.workunit.client.0.smithi007.stdout:5/182: dwrite d1/fa [0,4194304] 0 2024-03-16T09:01:08.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/334: fsync d24/f3c 0 2024-03-16T09:01:08.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/335: readlink d24/d4c/l51 0 2024-03-16T09:01:08.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/336: stat d24/d4a/d55/d56/f5a 0 2024-03-16T09:01:08.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/337: write f7 [1898453,47134] 0 2024-03-16T09:01:08.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/338: chown d24/l35 42 1 2024-03-16T09:01:08.396 INFO:tasks.workunit.client.0.smithi007.stdout:5/183: dread d1/d7/d1c/d19/f26 [0,4194304] 0 2024-03-16T09:01:08.397 INFO:tasks.workunit.client.0.smithi007.stdout:5/184: symlink d1/d7/d16/d1b/l3f 0 2024-03-16T09:01:08.402 INFO:tasks.workunit.client.0.smithi007.stdout:9/339: dwrite d24/f44 [0,4194304] 0 2024-03-16T09:01:08.402 INFO:tasks.workunit.client.0.smithi007.stdout:9/340: chown f16 22 1 2024-03-16T09:01:08.404 INFO:tasks.workunit.client.0.smithi007.stdout:9/341: creat d24/d4a/d61/f62 x:0 0 0 2024-03-16T09:01:08.404 INFO:tasks.workunit.client.0.smithi007.stdout:9/342: write d24/f5c [327141,75042] 0 2024-03-16T09:01:08.405 INFO:tasks.workunit.client.0.smithi007.stdout:9/343: rename d24/f38 to d24/d4c/f63 0 2024-03-16T09:01:08.406 INFO:tasks.workunit.client.0.smithi007.stdout:9/344: stat f7 0 2024-03-16T09:01:08.406 INFO:tasks.workunit.client.0.smithi007.stdout:9/345: chown d24/f43 142328 1 2024-03-16T09:01:08.406 INFO:tasks.workunit.client.0.smithi007.stdout:9/346: chown d24/d4a/d61 10578735 1 2024-03-16T09:01:08.410 INFO:tasks.workunit.client.0.smithi007.stdout:5/185: dwrite d1/d7/d16/d1e/f3d [0,4194304] 0 2024-03-16T09:01:08.411 INFO:tasks.workunit.client.0.smithi007.stdout:5/186: dread - d1/d7/d1c/d19/f3a zero size 2024-03-16T09:01:08.416 INFO:tasks.workunit.client.0.smithi007.stdout:9/347: getdents d24/d4a/d61 0 2024-03-16T09:01:08.416 INFO:tasks.workunit.client.0.smithi007.stdout:9/348: read - d24/d4a/d55/f5b zero size 2024-03-16T09:01:08.417 INFO:tasks.workunit.client.0.smithi007.stdout:9/349: write d24/d4a/d55/d56/f60 [342467,97321] 0 2024-03-16T09:01:08.418 INFO:tasks.workunit.client.0.smithi007.stdout:9/350: getdents d24/d4c 0 2024-03-16T09:01:08.419 INFO:tasks.workunit.client.0.smithi007.stdout:9/351: creat d24/d4c/f64 x:0 0 0 2024-03-16T09:01:08.424 INFO:tasks.workunit.client.0.smithi007.stdout:7/195: sync 2024-03-16T09:01:08.424 INFO:tasks.workunit.client.0.smithi007.stdout:6/240: sync 2024-03-16T09:01:08.424 INFO:tasks.workunit.client.0.smithi007.stdout:4/177: sync 2024-03-16T09:01:08.424 INFO:tasks.workunit.client.0.smithi007.stdout:8/319: sync 2024-03-16T09:01:08.424 INFO:tasks.workunit.client.0.smithi007.stdout:2/363: sync 2024-03-16T09:01:08.424 INFO:tasks.workunit.client.0.smithi007.stdout:2/364: fdatasync dc/df/d24/d28/f66 0 2024-03-16T09:01:08.424 INFO:tasks.workunit.client.0.smithi007.stdout:3/308: sync 2024-03-16T09:01:08.425 INFO:tasks.workunit.client.0.smithi007.stdout:1/209: sync 2024-03-16T09:01:08.425 INFO:tasks.workunit.client.0.smithi007.stdout:0/212: sync 2024-03-16T09:01:08.425 INFO:tasks.workunit.client.0.smithi007.stdout:0/213: readlink d3/d9/db/d2c/l42 0 2024-03-16T09:01:08.425 INFO:tasks.workunit.client.0.smithi007.stdout:1/210: read d9/dd/d1b/d28/f3e [1327290,25288] 0 2024-03-16T09:01:08.425 INFO:tasks.workunit.client.0.smithi007.stdout:3/309: write d15/d49/f50 [270440,42594] 0 2024-03-16T09:01:08.425 INFO:tasks.workunit.client.0.smithi007.stdout:2/365: read f6 [1686269,126052] 0 2024-03-16T09:01:08.427 INFO:tasks.workunit.client.0.smithi007.stdout:4/178: rmdir d4/d38 0 2024-03-16T09:01:08.427 INFO:tasks.workunit.client.0.smithi007.stdout:7/196: truncate d0/dc/f15 1331006 0 2024-03-16T09:01:08.427 INFO:tasks.workunit.client.0.smithi007.stdout:7/197: creat d0/d19/f41 x:0 0 0 2024-03-16T09:01:08.428 INFO:tasks.workunit.client.0.smithi007.stdout:8/320: mkdir d0/d26/d2d/d43/d6f 0 2024-03-16T09:01:08.428 INFO:tasks.workunit.client.0.smithi007.stdout:0/214: write d3/f8 [1001983,18307] 0 2024-03-16T09:01:08.428 INFO:tasks.workunit.client.0.smithi007.stdout:0/215: write d3/d9/da/d48/d25/d43/d1a/f20 [2085931,34267] 0 2024-03-16T09:01:08.429 INFO:tasks.workunit.client.0.smithi007.stdout:1/211: creat d9/dd/d1b/d2d/d30/f40 x:0 0 0 2024-03-16T09:01:08.431 INFO:tasks.workunit.client.0.smithi007.stdout:4/179: link d4/l24 d4/d14/d1c/d1e/l3d 0 2024-03-16T09:01:08.431 INFO:tasks.workunit.client.0.smithi007.stdout:4/180: chown d4/d14/d34/l39 15679 1 2024-03-16T09:01:08.431 INFO:tasks.workunit.client.0.smithi007.stdout:4/181: dread - d4/d14/d1c/f2c zero size 2024-03-16T09:01:08.433 INFO:tasks.workunit.client.0.smithi007.stdout:7/198: chown d0/c8 0 1 2024-03-16T09:01:08.433 INFO:tasks.workunit.client.0.smithi007.stdout:7/199: write d0/dc/d24/f3b [993834,127351] 0 2024-03-16T09:01:08.441 INFO:tasks.workunit.client.0.smithi007.stdout:4/182: creat d4/d14/d1c/f3e x:0 0 0 2024-03-16T09:01:08.442 INFO:tasks.workunit.client.0.smithi007.stdout:4/183: stat d4/f2e 0 2024-03-16T09:01:08.443 INFO:tasks.workunit.client.0.smithi007.stdout:4/184: creat d4/d14/d1c/d1e/f3f x:0 0 0 2024-03-16T09:01:08.444 INFO:tasks.workunit.client.0.smithi007.stdout:0/216: read d3/d9/da/d48/d25/d43/d1a/f2d [3545038,58397] 0 2024-03-16T09:01:08.444 INFO:tasks.workunit.client.0.smithi007.stdout:0/217: chown d3/d9/db/d2c/f2f 1333 1 2024-03-16T09:01:08.444 INFO:tasks.workunit.client.0.smithi007.stdout:9/352: dwrite f1c [0,4194304] 0 2024-03-16T09:01:08.444 INFO:tasks.workunit.client.0.smithi007.stdout:9/353: chown d24/l41 389 1 2024-03-16T09:01:08.446 INFO:tasks.workunit.client.0.smithi007.stdout:4/185: creat d4/f40 x:0 0 0 2024-03-16T09:01:08.446 INFO:tasks.workunit.client.0.smithi007.stdout:4/186: write d4/d14/d1c/d1e/f3f [993805,14046] 0 2024-03-16T09:01:08.446 INFO:tasks.workunit.client.0.smithi007.stdout:3/310: dwrite f5 [0,4194304] 0 2024-03-16T09:01:08.448 INFO:tasks.workunit.client.0.smithi007.stdout:7/200: dread d0/d19/f2d [0,4194304] 0 2024-03-16T09:01:08.450 INFO:tasks.workunit.client.0.smithi007.stdout:9/354: unlink d24/f49 0 2024-03-16T09:01:08.452 INFO:tasks.workunit.client.0.smithi007.stdout:8/321: rmdir d0/d26/d2d 39 2024-03-16T09:01:08.452 INFO:tasks.workunit.client.0.smithi007.stdout:8/322: stat d0/fe 0 2024-03-16T09:01:08.454 INFO:tasks.workunit.client.0.smithi007.stdout:3/311: creat d15/d18/d1f/d3b/d63/f66 x:0 0 0 2024-03-16T09:01:08.454 INFO:tasks.workunit.client.0.smithi007.stdout:3/312: stat d15/d18/d1f/d29/d3e/d4c/f64 0 2024-03-16T09:01:08.454 INFO:tasks.workunit.client.0.smithi007.stdout:3/313: fsync d15/d18/d1f/d29/f31 0 2024-03-16T09:01:08.455 INFO:tasks.workunit.client.0.smithi007.stdout:2/366: dwrite dc/d27/f65 [0,4194304] 0 2024-03-16T09:01:08.457 INFO:tasks.workunit.client.0.smithi007.stdout:3/314: symlink d15/d18/d1f/d3b/d63/l67 0 2024-03-16T09:01:08.457 INFO:tasks.workunit.client.0.smithi007.stdout:9/355: read d24/f29 [201860,66087] 0 2024-03-16T09:01:08.457 INFO:tasks.workunit.client.0.smithi007.stdout:9/356: creat d24/d4a/d55/f65 x:0 0 0 2024-03-16T09:01:08.458 INFO:tasks.workunit.client.0.smithi007.stdout:2/367: dread dc/f23 [0,4194304] 0 2024-03-16T09:01:08.462 INFO:tasks.workunit.client.0.smithi007.stdout:3/315: rename d15/c2f to d15/d49/c68 0 2024-03-16T09:01:08.465 INFO:tasks.workunit.client.0.smithi007.stdout:9/357: creat d24/d4a/f66 x:0 0 0 2024-03-16T09:01:08.466 INFO:tasks.workunit.client.0.smithi007.stdout:2/368: symlink dc/df/d6c/l77 0 2024-03-16T09:01:08.466 INFO:tasks.workunit.client.0.smithi007.stdout:2/369: chown dc/df/d24/d28/d71 2390 1 2024-03-16T09:01:08.466 INFO:tasks.workunit.client.0.smithi007.stdout:2/370: creat dc/df/d24/d28/d2e/f78 x:0 0 0 2024-03-16T09:01:08.466 INFO:tasks.workunit.client.0.smithi007.stdout:2/371: write dc/df/d24/d28/d2e/f45 [746548,45128] 0 2024-03-16T09:01:08.467 INFO:tasks.workunit.client.0.smithi007.stdout:2/372: stat dc/df/d24/d28/d2e/d6e/f6f 0 2024-03-16T09:01:08.468 INFO:tasks.workunit.client.0.smithi007.stdout:3/316: creat d15/d18/d1f/d29/d43/f69 x:0 0 0 2024-03-16T09:01:08.470 INFO:tasks.workunit.client.0.smithi007.stdout:3/317: mkdir d15/d18/d6a 0 2024-03-16T09:01:08.470 INFO:tasks.workunit.client.0.smithi007.stdout:3/318: symlink d15/d18/d1f/l6b 0 2024-03-16T09:01:08.471 INFO:tasks.workunit.client.0.smithi007.stdout:3/319: mknod d15/d18/d1f/c6c 0 2024-03-16T09:01:08.473 INFO:tasks.workunit.client.0.smithi007.stdout:3/320: unlink d15/d18/d1f/d3b/f3f 0 2024-03-16T09:01:08.473 INFO:tasks.workunit.client.0.smithi007.stdout:9/358: dread d24/f50 [0,4194304] 0 2024-03-16T09:01:08.483 INFO:tasks.workunit.client.0.smithi007.stdout:6/241: dwrite d2/d11/d1e/f2f [0,4194304] 0 2024-03-16T09:01:08.483 INFO:tasks.workunit.client.0.smithi007.stdout:1/212: dwrite d9/dd/d1b/d28/f32 [0,4194304] 0 2024-03-16T09:01:08.485 INFO:tasks.workunit.client.0.smithi007.stdout:0/218: dwrite d3/d9/da/d18/f2b [0,4194304] 0 2024-03-16T09:01:08.485 INFO:tasks.workunit.client.0.smithi007.stdout:8/323: dwrite d0/d2/f4b [0,4194304] 0 2024-03-16T09:01:08.485 INFO:tasks.workunit.client.0.smithi007.stdout:0/219: dread - d3/d9/da/d48/d25/d43/d1a/f30 zero size 2024-03-16T09:01:08.485 INFO:tasks.workunit.client.0.smithi007.stdout:7/201: dwrite d0/d19/f41 [0,4194304] 0 2024-03-16T09:01:08.485 INFO:tasks.workunit.client.0.smithi007.stdout:4/187: rmdir d4/d14/d1c/d1e 39 2024-03-16T09:01:08.487 INFO:tasks.workunit.client.0.smithi007.stdout:1/213: symlink d9/dd/d1b/d28/d29/l41 0 2024-03-16T09:01:08.492 INFO:tasks.workunit.client.0.smithi007.stdout:8/324: mkdir d0/d26/d2d/d43/d70 0 2024-03-16T09:01:08.494 INFO:tasks.workunit.client.0.smithi007.stdout:0/220: creat d3/d9/da/d48/f4e x:0 0 0 2024-03-16T09:01:08.497 INFO:tasks.workunit.client.0.smithi007.stdout:3/321: unlink d15/d49/c68 0 2024-03-16T09:01:08.500 INFO:tasks.workunit.client.0.smithi007.stdout:7/202: mkdir d0/d19/d1f/d42 0 2024-03-16T09:01:08.500 INFO:tasks.workunit.client.0.smithi007.stdout:7/203: chown d0/dc/d24/d25/l40 2651319 1 2024-03-16T09:01:08.503 INFO:tasks.workunit.client.0.smithi007.stdout:7/204: dread d0/d19/f26 [0,4194304] 0 2024-03-16T09:01:08.504 INFO:tasks.workunit.client.0.smithi007.stdout:1/214: rmdir d9/dd/d1b 39 2024-03-16T09:01:08.504 INFO:tasks.workunit.client.0.smithi007.stdout:1/215: fsync f0 0 2024-03-16T09:01:08.504 INFO:tasks.workunit.client.0.smithi007.stdout:6/242: dwrite d2/d11/f21 [0,4194304] 0 2024-03-16T09:01:08.504 INFO:tasks.workunit.client.0.smithi007.stdout:6/243: dread - d2/d11/d1e/f40 zero size 2024-03-16T09:01:08.504 INFO:tasks.workunit.client.0.smithi007.stdout:6/244: creat d2/d13/d1f/f4f x:0 0 0 2024-03-16T09:01:08.505 INFO:tasks.workunit.client.0.smithi007.stdout:8/325: mknod d0/d26/d2d/d43/d70/c71 0 2024-03-16T09:01:08.505 INFO:tasks.workunit.client.0.smithi007.stdout:8/326: write d0/d2/f5d [38590,127012] 0 2024-03-16T09:01:08.510 INFO:tasks.workunit.client.0.smithi007.stdout:0/221: unlink d3/d9/da/d18/f2b 0 2024-03-16T09:01:08.510 INFO:tasks.workunit.client.0.smithi007.stdout:6/245: dread d2/d11/d1e/f2f [0,4194304] 0 2024-03-16T09:01:08.511 INFO:tasks.workunit.client.0.smithi007.stdout:6/246: chown d2/d13/d24/c33 2 1 2024-03-16T09:01:08.511 INFO:tasks.workunit.client.0.smithi007.stdout:3/322: mknod d15/d49/c6d 0 2024-03-16T09:01:08.511 INFO:tasks.workunit.client.0.smithi007.stdout:4/188: rmdir d4/d14 39 2024-03-16T09:01:08.512 INFO:tasks.workunit.client.0.smithi007.stdout:7/205: mkdir d0/dc/d24/d43 0 2024-03-16T09:01:08.513 INFO:tasks.workunit.client.0.smithi007.stdout:1/216: creat d9/dd/d1b/d28/d29/f42 x:0 0 0 2024-03-16T09:01:08.513 INFO:tasks.workunit.client.0.smithi007.stdout:1/217: rename d9/dd/d1b to d9/dd/d1b/d28/d29/d43 22 2024-03-16T09:01:08.513 INFO:tasks.workunit.client.0.smithi007.stdout:1/218: read - d9/dd/d1b/d28/f36 zero size 2024-03-16T09:01:08.515 INFO:tasks.workunit.client.0.smithi007.stdout:8/327: mkdir d0/d26/d2d/d43/d72 0 2024-03-16T09:01:08.516 INFO:tasks.workunit.client.0.smithi007.stdout:0/222: unlink d3/d9/da/d18/d23/d36/f46 0 2024-03-16T09:01:08.516 INFO:tasks.workunit.client.0.smithi007.stdout:0/223: truncate d3/d9/da/f4c 751594 0 2024-03-16T09:01:08.517 INFO:tasks.workunit.client.0.smithi007.stdout:0/224: read d3/f6 [1095450,98400] 0 2024-03-16T09:01:08.517 INFO:tasks.workunit.client.0.smithi007.stdout:3/323: write f5 [4071284,106859] 0 2024-03-16T09:01:08.518 INFO:tasks.workunit.client.0.smithi007.stdout:6/247: mkdir d2/d13/d3f/d50 0 2024-03-16T09:01:08.518 INFO:tasks.workunit.client.0.smithi007.stdout:6/248: truncate d2/d13/d1f/d34/f4a 605021 0 2024-03-16T09:01:08.518 INFO:tasks.workunit.client.0.smithi007.stdout:4/189: getdents d4/d14 0 2024-03-16T09:01:08.520 INFO:tasks.workunit.client.0.smithi007.stdout:1/219: link d9/f19 d9/dd/d1b/d2d/f44 0 2024-03-16T09:01:08.520 INFO:tasks.workunit.client.0.smithi007.stdout:1/220: write d9/dd/d1b/d28/f36 [286827,20526] 0 2024-03-16T09:01:08.521 INFO:tasks.workunit.client.0.smithi007.stdout:7/206: write d0/d19/f41 [331991,31221] 0 2024-03-16T09:01:08.523 INFO:tasks.workunit.client.0.smithi007.stdout:3/324: link d15/d18/d1f/d56/c5c d15/d18/d1f/d3b/c6e 0 2024-03-16T09:01:08.523 INFO:tasks.workunit.client.0.smithi007.stdout:3/325: read d15/d18/f33 [59851,67906] 0 2024-03-16T09:01:08.523 INFO:tasks.workunit.client.0.smithi007.stdout:3/326: write d15/f16 [233898,19848] 0 2024-03-16T09:01:08.523 INFO:tasks.workunit.client.0.smithi007.stdout:8/328: read d0/d2/d18/f3b [4566092,91357] 0 2024-03-16T09:01:08.523 INFO:tasks.workunit.client.0.smithi007.stdout:8/329: chown d0/d2/fa 1486878066 1 2024-03-16T09:01:08.523 INFO:tasks.workunit.client.0.smithi007.stdout:8/330: dread - d0/dc/d50/f6b zero size 2024-03-16T09:01:08.527 INFO:tasks.workunit.client.0.smithi007.stdout:3/327: mknod d15/d18/d1f/c6f 0 2024-03-16T09:01:08.528 INFO:tasks.workunit.client.0.smithi007.stdout:8/331: unlink d0/d26/d2d/d43/l53 0 2024-03-16T09:01:08.528 INFO:tasks.workunit.client.0.smithi007.stdout:8/332: read - d0/d2/d18/f52 zero size 2024-03-16T09:01:08.529 INFO:tasks.workunit.client.0.smithi007.stdout:6/249: getdents d2/d11/d1e 0 2024-03-16T09:01:08.532 INFO:tasks.workunit.client.0.smithi007.stdout:6/250: mknod d2/d13/d24/c51 0 2024-03-16T09:01:08.534 INFO:tasks.workunit.client.0.smithi007.stdout:4/190: write d4/d14/f19 [3209465,63597] 0 2024-03-16T09:01:08.534 INFO:tasks.workunit.client.0.smithi007.stdout:0/225: dwrite d3/d9/db/d2c/f2f [0,4194304] 0 2024-03-16T09:01:08.534 INFO:tasks.workunit.client.0.smithi007.stdout:0/226: fsync d3/d9/db/fc 0 2024-03-16T09:01:08.535 INFO:tasks.workunit.client.0.smithi007.stdout:0/227: readlink d3/d9/da/d48/d25/d43/l2e 0 2024-03-16T09:01:08.536 INFO:tasks.workunit.client.0.smithi007.stdout:6/251: creat d2/d13/d1f/d4e/f52 x:0 0 0 2024-03-16T09:01:08.537 INFO:tasks.workunit.client.0.smithi007.stdout:1/221: write d9/dd/d1b/d28/d29/f2a [4788863,61791] 0 2024-03-16T09:01:08.547 INFO:tasks.workunit.client.0.smithi007.stdout:1/222: dread d9/dd/d1b/f2c [0,4194304] 0 2024-03-16T09:01:08.547 INFO:tasks.workunit.client.0.smithi007.stdout:1/223: readlink d9/l20 0 2024-03-16T09:01:08.547 INFO:tasks.workunit.client.0.smithi007.stdout:1/224: truncate f7 4783552 0 2024-03-16T09:01:08.551 INFO:tasks.workunit.client.0.smithi007.stdout:8/333: dwrite d0/d26/f2f [8388608,4194304] 0 2024-03-16T09:01:08.556 INFO:tasks.workunit.client.0.smithi007.stdout:3/328: dwrite f10 [0,4194304] 0 2024-03-16T09:01:08.559 INFO:tasks.workunit.client.0.smithi007.stdout:1/225: mkdir d9/dd/d1b/d2d/d30/d45 0 2024-03-16T09:01:08.560 INFO:tasks.workunit.client.0.smithi007.stdout:3/329: symlink d15/d18/d1f/d29/d3e/d4c/l70 0 2024-03-16T09:01:08.561 INFO:tasks.workunit.client.0.smithi007.stdout:1/226: link d9/dd/d1b/d28/f36 d9/dd/d1b/d28/f46 0 2024-03-16T09:01:08.561 INFO:tasks.workunit.client.0.smithi007.stdout:1/227: creat d9/dd/d1b/d28/d29/f47 x:0 0 0 2024-03-16T09:01:08.563 INFO:tasks.workunit.client.0.smithi007.stdout:3/330: symlink d15/d18/d1f/d3b/l71 0 2024-03-16T09:01:08.566 INFO:tasks.workunit.client.0.smithi007.stdout:1/228: dread d9/f13 [0,4194304] 0 2024-03-16T09:01:08.567 INFO:tasks.workunit.client.0.smithi007.stdout:3/331: rename d15/d49 to d15/d18/d1f/d29/d43/d72 0 2024-03-16T09:01:08.568 INFO:tasks.workunit.client.0.smithi007.stdout:3/332: getdents d15/d18 0 2024-03-16T09:01:08.569 INFO:tasks.workunit.client.0.smithi007.stdout:3/333: mknod d15/d18/d1f/d56/c73 0 2024-03-16T09:01:08.569 INFO:tasks.workunit.client.0.smithi007.stdout:3/334: truncate d15/d18/f35 5801982 0 2024-03-16T09:01:08.569 INFO:tasks.workunit.client.0.smithi007.stdout:3/335: chown d15/d18/d1f/d29/l3c 501 1 2024-03-16T09:01:08.569 INFO:tasks.workunit.client.0.smithi007.stdout:3/336: readlink d15/d18/l27 0 2024-03-16T09:01:08.574 INFO:tasks.workunit.client.0.smithi007.stdout:4/191: dwrite d4/f2e [0,4194304] 0 2024-03-16T09:01:08.575 INFO:tasks.workunit.client.0.smithi007.stdout:4/192: fsync d4/ff 0 2024-03-16T09:01:08.575 INFO:tasks.workunit.client.0.smithi007.stdout:8/334: dwrite d0/d2/f67 [0,4194304] 0 2024-03-16T09:01:08.575 INFO:tasks.workunit.client.0.smithi007.stdout:0/228: dwrite d3/f6 [0,4194304] 0 2024-03-16T09:01:08.580 INFO:tasks.workunit.client.0.smithi007.stdout:8/335: symlink d0/l73 0 2024-03-16T09:01:08.580 INFO:tasks.workunit.client.0.smithi007.stdout:8/336: write d0/ff [5304104,5473] 0 2024-03-16T09:01:08.581 INFO:tasks.workunit.client.0.smithi007.stdout:8/337: write d0/d2/d18/f3f [921578,121042] 0 2024-03-16T09:01:08.581 INFO:tasks.workunit.client.0.smithi007.stdout:8/338: chown d0/dc/f23 91196867 1 2024-03-16T09:01:08.581 INFO:tasks.workunit.client.0.smithi007.stdout:0/229: getdents d3/d9/db 0 2024-03-16T09:01:08.581 INFO:tasks.workunit.client.0.smithi007.stdout:1/229: read d9/f19 [6850147,43357] 0 2024-03-16T09:01:08.581 INFO:tasks.workunit.client.0.smithi007.stdout:1/230: stat d9/dd/d1b 0 2024-03-16T09:01:08.583 INFO:tasks.workunit.client.0.smithi007.stdout:0/230: creat d3/d9/da/d18/d27/d34/d3a/f4f x:0 0 0 2024-03-16T09:01:08.585 INFO:tasks.workunit.client.0.smithi007.stdout:8/339: mkdir d0/d2/d74 0 2024-03-16T09:01:08.586 INFO:tasks.workunit.client.0.smithi007.stdout:8/340: mkdir d0/dc/d75 0 2024-03-16T09:01:08.588 INFO:tasks.workunit.client.0.smithi007.stdout:8/341: link d0/dc/l60 d0/dc/d50/l76 0 2024-03-16T09:01:08.589 INFO:tasks.workunit.client.0.smithi007.stdout:8/342: creat d0/d26/d2d/d43/d4c/f77 x:0 0 0 2024-03-16T09:01:08.589 INFO:tasks.workunit.client.0.smithi007.stdout:8/343: fdatasync d0/d2/d18/f4e 0 2024-03-16T09:01:08.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/231: write d9/dd/d1b/d28/f32 [1574319,8190] 0 2024-03-16T09:01:08.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/232: chown f7 1039 1 2024-03-16T09:01:08.593 INFO:tasks.workunit.client.0.smithi007.stdout:4/193: dwrite d4/d14/d1c/f3b [0,4194304] 0 2024-03-16T09:01:08.600 INFO:tasks.workunit.client.0.smithi007.stdout:8/344: dread d0/f16 [0,4194304] 0 2024-03-16T09:01:08.600 INFO:tasks.workunit.client.0.smithi007.stdout:4/194: mknod d4/d7/c41 0 2024-03-16T09:01:08.600 INFO:tasks.workunit.client.0.smithi007.stdout:8/345: fdatasync d0/d2/d18/f4e 0 2024-03-16T09:01:08.600 INFO:tasks.workunit.client.0.smithi007.stdout:8/346: creat d0/d26/d2d/d43/d4c/f78 x:0 0 0 2024-03-16T09:01:08.602 INFO:tasks.workunit.client.0.smithi007.stdout:4/195: link d4/d14/d1c/f29 d4/d14/d1c/d1e/f42 0 2024-03-16T09:01:08.603 INFO:tasks.workunit.client.0.smithi007.stdout:8/347: creat d0/d26/f79 x:0 0 0 2024-03-16T09:01:08.603 INFO:tasks.workunit.client.0.smithi007.stdout:8/348: chown d0/d26/d2d/d43 2 1 2024-03-16T09:01:08.603 INFO:tasks.workunit.client.0.smithi007.stdout:4/196: rmdir d4/d14 39 2024-03-16T09:01:08.604 INFO:tasks.workunit.client.0.smithi007.stdout:4/197: rename d4/d14 to d4/d14/d1c/d1e/d43 22 2024-03-16T09:01:08.604 INFO:tasks.workunit.client.0.smithi007.stdout:4/198: chown d4/f40 25580316 1 2024-03-16T09:01:08.606 INFO:tasks.workunit.client.0.smithi007.stdout:0/231: dwrite d3/d9/da/f4c [0,4194304] 0 2024-03-16T09:01:08.606 INFO:tasks.workunit.client.0.smithi007.stdout:0/232: fdatasync d3/d9/da/d48/d25/d43/d1a/f2d 0 2024-03-16T09:01:08.606 INFO:tasks.workunit.client.0.smithi007.stdout:8/349: link d0/dc/l2e d0/d26/d2d/d43/d72/l7a 0 2024-03-16T09:01:08.606 INFO:tasks.workunit.client.0.smithi007.stdout:8/350: fdatasync d0/d2/fa 0 2024-03-16T09:01:08.606 INFO:tasks.workunit.client.0.smithi007.stdout:8/351: fsync d0/d26/f30 0 2024-03-16T09:01:08.607 INFO:tasks.workunit.client.0.smithi007.stdout:4/199: mkdir d4/d14/d1c/d1e/d44 0 2024-03-16T09:01:08.611 INFO:tasks.workunit.client.0.smithi007.stdout:0/233: symlink d3/d9/db/d1c/l50 0 2024-03-16T09:01:08.611 INFO:tasks.workunit.client.0.smithi007.stdout:0/234: dread - d3/d9/da/d18/d23/f32 zero size 2024-03-16T09:01:08.611 INFO:tasks.workunit.client.0.smithi007.stdout:0/235: truncate d3/d9/da/f1e 580452 0 2024-03-16T09:01:08.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/236: creat d3/d9/da/d48/d4b/f51 x:0 0 0 2024-03-16T09:01:08.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/237: readlink d3/d9/db/d1c/l50 0 2024-03-16T09:01:08.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/238: stat d3/d9/f26 0 2024-03-16T09:01:08.614 INFO:tasks.workunit.client.0.smithi007.stdout:8/352: getdents d0/dc 0 2024-03-16T09:01:08.614 INFO:tasks.workunit.client.0.smithi007.stdout:8/353: chown d0/d2/d74 26710563 1 2024-03-16T09:01:08.615 INFO:tasks.workunit.client.0.smithi007.stdout:1/233: dwrite d9/dd/d1b/d28/f3e [0,4194304] 0 2024-03-16T09:01:08.615 INFO:tasks.workunit.client.0.smithi007.stdout:8/354: creat d0/f7b x:0 0 0 2024-03-16T09:01:08.618 INFO:tasks.workunit.client.0.smithi007.stdout:1/234: unlink d9/f24 0 2024-03-16T09:01:08.620 INFO:tasks.workunit.client.0.smithi007.stdout:0/239: dread d3/f6 [0,4194304] 0 2024-03-16T09:01:08.620 INFO:tasks.workunit.client.0.smithi007.stdout:0/240: stat d3/d9/da/d48/d25 0 2024-03-16T09:01:08.628 INFO:tasks.workunit.client.0.smithi007.stdout:1/235: link d9/f13 d9/f48 0 2024-03-16T09:01:08.629 INFO:tasks.workunit.client.0.smithi007.stdout:1/236: getdents d9/dd/d1b 0 2024-03-16T09:01:08.630 INFO:tasks.workunit.client.0.smithi007.stdout:1/237: creat d9/dd/d1b/f49 x:0 0 0 2024-03-16T09:01:08.631 INFO:tasks.workunit.client.0.smithi007.stdout:0/241: dwrite d3/d9/da/d18/d23/f32 [0,4194304] 0 2024-03-16T09:01:08.631 INFO:tasks.workunit.client.0.smithi007.stdout:0/242: creat d3/d9/da/d48/d4b/f52 x:0 0 0 2024-03-16T09:01:08.640 INFO:tasks.workunit.client.0.smithi007.stdout:1/238: dread d9/dd/d1b/d28/f2b [0,4194304] 0 2024-03-16T09:01:08.641 INFO:tasks.workunit.client.0.smithi007.stdout:0/243: dread d3/d9/db/fc [0,4194304] 0 2024-03-16T09:01:08.642 INFO:tasks.workunit.client.0.smithi007.stdout:0/244: dread - d3/d9/da/d48/d25/d43/d1a/f30 zero size 2024-03-16T09:01:08.642 INFO:tasks.workunit.client.0.smithi007.stdout:1/239: creat d9/dd/d1b/d2d/d30/d45/f4a x:0 0 0 2024-03-16T09:01:08.643 INFO:tasks.workunit.client.0.smithi007.stdout:1/240: unlink c6 0 2024-03-16T09:01:08.655 INFO:tasks.workunit.client.0.smithi007.stdout:0/245: dwrite d3/d9/da/d48/d25/d43/d1a/f30 [0,4194304] 0 2024-03-16T09:01:08.655 INFO:tasks.workunit.client.0.smithi007.stdout:0/246: write d3/f8 [4522597,53790] 0 2024-03-16T09:01:08.667 INFO:tasks.workunit.client.0.smithi007.stdout:1/241: dwrite d9/dd/d1b/d28/f2b [0,4194304] 0 2024-03-16T09:01:08.678 INFO:tasks.workunit.client.0.smithi007.stdout:0/247: dwrite d3/d9/da/f4c [0,4194304] 0 2024-03-16T09:01:08.679 INFO:tasks.workunit.client.0.smithi007.stdout:0/248: mknod d3/d9/db/d2c/c53 0 2024-03-16T09:01:08.680 INFO:tasks.workunit.client.0.smithi007.stdout:0/249: stat d3/d9/da/c3e 0 2024-03-16T09:01:08.680 INFO:tasks.workunit.client.0.smithi007.stdout:0/250: fdatasync d3/d9/da/d18/d27/d34/d3a/f4f 0 2024-03-16T09:01:08.691 INFO:tasks.workunit.client.0.smithi007.stdout:0/251: dread d3/f6 [0,4194304] 0 2024-03-16T09:01:08.691 INFO:tasks.workunit.client.0.smithi007.stdout:0/252: chown d3/d9/db/fc 926707 1 2024-03-16T09:01:08.691 INFO:tasks.workunit.client.0.smithi007.stdout:1/242: dwrite d9/dd/d1b/d28/d29/f47 [0,4194304] 0 2024-03-16T09:01:08.692 INFO:tasks.workunit.client.0.smithi007.stdout:0/253: mknod d3/d9/da/d48/d4b/c54 0 2024-03-16T09:01:08.694 INFO:tasks.workunit.client.0.smithi007.stdout:0/254: mkdir d3/d9/da/d18/d27/d34/d55 0 2024-03-16T09:01:08.695 INFO:tasks.workunit.client.0.smithi007.stdout:0/255: rename d3/d9/db/d2c/l42 to d3/d9/db/l56 0 2024-03-16T09:01:08.695 INFO:tasks.workunit.client.0.smithi007.stdout:0/256: dread - d3/d9/da/d48/d4b/f52 zero size 2024-03-16T09:01:08.696 INFO:tasks.workunit.client.0.smithi007.stdout:0/257: chown d3/d9/db/d2c 9101603 1 2024-03-16T09:01:08.703 INFO:tasks.workunit.client.0.smithi007.stdout:1/243: dread d9/dd/d1b/d28/d29/f47 [0,4194304] 0 2024-03-16T09:01:08.705 INFO:tasks.workunit.client.0.smithi007.stdout:1/244: symlink d9/dd/d1b/d2d/d30/l4b 0 2024-03-16T09:01:08.709 INFO:tasks.workunit.client.0.smithi007.stdout:1/245: dread d9/dd/d1b/d28/d29/f2a [0,4194304] 0 2024-03-16T09:01:08.713 INFO:tasks.workunit.client.0.smithi007.stdout:0/258: dwrite d3/f8 [0,4194304] 0 2024-03-16T09:01:08.724 INFO:tasks.workunit.client.0.smithi007.stdout:1/246: dwrite d9/f48 [0,4194304] 0 2024-03-16T09:01:08.727 INFO:tasks.workunit.client.0.smithi007.stdout:1/247: unlink d9/dd/d1b/d2d/d30/f40 0 2024-03-16T09:01:08.727 INFO:tasks.workunit.client.0.smithi007.stdout:5/187: sync 2024-03-16T09:01:08.728 INFO:tasks.workunit.client.0.smithi007.stdout:1/248: truncate d9/dd/d1b/d2d/f44 1500330 0 2024-03-16T09:01:08.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/249: mkdir d9/dd/d1b/d28/d4c 0 2024-03-16T09:01:08.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/250: dread - d9/dd/d1b/d2d/f3f zero size 2024-03-16T09:01:08.731 INFO:tasks.workunit.client.0.smithi007.stdout:0/259: dwrite d3/f8 [0,4194304] 0 2024-03-16T09:01:08.731 INFO:tasks.workunit.client.0.smithi007.stdout:0/260: write d3/d9/da/d48/d25/d43/d1a/f1b [104836,113015] 0 2024-03-16T09:01:08.733 INFO:tasks.workunit.client.0.smithi007.stdout:0/261: rmdir d3/d9/da/d18/d27/d34 39 2024-03-16T09:01:08.734 INFO:tasks.workunit.client.0.smithi007.stdout:1/251: dread d9/dd/d1b/f25 [0,4194304] 0 2024-03-16T09:01:08.734 INFO:tasks.workunit.client.0.smithi007.stdout:1/252: fsync d9/dd/d1b/d28/d29/f2a 0 2024-03-16T09:01:08.734 INFO:tasks.workunit.client.0.smithi007.stdout:1/253: chown d9/lb 11332 1 2024-03-16T09:01:08.735 INFO:tasks.workunit.client.0.smithi007.stdout:0/262: creat d3/d9/da/d18/d27/d34/f57 x:0 0 0 2024-03-16T09:01:08.735 INFO:tasks.workunit.client.0.smithi007.stdout:0/263: truncate d3/d9/da/d48/f12 1550156 0 2024-03-16T09:01:08.736 INFO:tasks.workunit.client.0.smithi007.stdout:0/264: mkdir d3/d9/da/d48/d25/d43/d1a/d58 0 2024-03-16T09:01:08.739 INFO:tasks.workunit.client.0.smithi007.stdout:0/265: getdents d3/d9/da 0 2024-03-16T09:01:08.837 INFO:tasks.workunit.client.0.smithi007.stdout:2/373: sync 2024-03-16T09:01:08.837 INFO:tasks.workunit.client.0.smithi007.stdout:9/359: sync 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:7/207: sync 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:4/200: sync 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:8/355: sync 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:7/208: chown d0/dc/d24/d25/l40 1286739823 1 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:9/360: write f17 [5016918,67436] 0 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:3/337: sync 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:6/252: sync 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:3/338: fsync d15/d18/d1f/d3b/f44 0 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:3/339: chown d15/f1e 52558 1 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:3/340: truncate d15/d18/d1f/d3b/f47 5560244 0 2024-03-16T09:01:08.838 INFO:tasks.workunit.client.0.smithi007.stdout:2/374: truncate f7 1209031 0 2024-03-16T09:01:08.839 INFO:tasks.workunit.client.0.smithi007.stdout:4/201: getdents d4/d7 0 2024-03-16T09:01:08.841 INFO:tasks.workunit.client.0.smithi007.stdout:8/356: link d0/d2/l65 d0/d26/d2d/d43/d4c/l7c 0 2024-03-16T09:01:08.843 INFO:tasks.workunit.client.0.smithi007.stdout:9/361: rename l21 to d24/l67 0 2024-03-16T09:01:08.844 INFO:tasks.workunit.client.0.smithi007.stdout:2/375: mknod dc/df/d24/d28/d4c/d5e/c79 0 2024-03-16T09:01:08.844 INFO:tasks.workunit.client.0.smithi007.stdout:2/376: write dc/df/d24/d28/d4c/d5e/f5b [1101100,113197] 0 2024-03-16T09:01:08.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/202: creat d4/d14/d34/f45 x:0 0 0 2024-03-16T09:01:08.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/203: dread - d4/d14/d1c/d1e/f42 zero size 2024-03-16T09:01:08.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/204: fdatasync d4/d14/d1c/d1e/f42 0 2024-03-16T09:01:08.847 INFO:tasks.workunit.client.0.smithi007.stdout:7/209: getdents d0 0 2024-03-16T09:01:08.848 INFO:tasks.workunit.client.0.smithi007.stdout:9/362: creat d24/d4a/d61/f68 x:0 0 0 2024-03-16T09:01:08.850 INFO:tasks.workunit.client.0.smithi007.stdout:3/341: rename d15/d18/d1f/c34 to d15/d18/d6a/c74 0 2024-03-16T09:01:08.855 INFO:tasks.workunit.client.0.smithi007.stdout:9/363: dread d24/f27 [0,4194304] 0 2024-03-16T09:01:08.857 INFO:tasks.workunit.client.0.smithi007.stdout:2/377: link dc/f15 dc/df/d24/d28/d2e/d4a/f7a 0 2024-03-16T09:01:08.857 INFO:tasks.workunit.client.0.smithi007.stdout:4/205: write d4/d14/d1c/f3b [197257,70866] 0 2024-03-16T09:01:08.857 INFO:tasks.workunit.client.0.smithi007.stdout:2/378: stat dc/df/d24/d28/d2e/d35/c46 0 2024-03-16T09:01:08.857 INFO:tasks.workunit.client.0.smithi007.stdout:4/206: chown d4/d14/d1c/d1e/l3d 421 1 2024-03-16T09:01:08.858 INFO:tasks.workunit.client.0.smithi007.stdout:9/364: mknod d24/d4a/d55/c69 0 2024-03-16T09:01:08.858 INFO:tasks.workunit.client.0.smithi007.stdout:9/365: stat d24/d4a/d55/f5b 0 2024-03-16T09:01:08.859 INFO:tasks.workunit.client.0.smithi007.stdout:7/210: getdents d0/dc 0 2024-03-16T09:01:08.860 INFO:tasks.workunit.client.0.smithi007.stdout:9/366: mknod d24/d4a/d61/c6a 0 2024-03-16T09:01:08.861 INFO:tasks.workunit.client.0.smithi007.stdout:7/211: creat d0/dc/d24/d25/d3a/f44 x:0 0 0 2024-03-16T09:01:08.861 INFO:tasks.workunit.client.0.smithi007.stdout:7/212: fsync d0/d19/f27 0 2024-03-16T09:01:08.862 INFO:tasks.workunit.client.0.smithi007.stdout:4/207: rename d4/c13 to d4/d7/c46 0 2024-03-16T09:01:08.863 INFO:tasks.workunit.client.0.smithi007.stdout:9/367: mknod d24/d4a/c6b 0 2024-03-16T09:01:08.864 INFO:tasks.workunit.client.0.smithi007.stdout:7/213: creat d0/d19/d1f/d42/f45 x:0 0 0 2024-03-16T09:01:08.864 INFO:tasks.workunit.client.0.smithi007.stdout:7/214: chown d0/d19/f27 2 1 2024-03-16T09:01:08.865 INFO:tasks.workunit.client.0.smithi007.stdout:9/368: rename d24/d4a/c59 to d24/c6c 0 2024-03-16T09:01:08.865 INFO:tasks.workunit.client.0.smithi007.stdout:9/369: fsync d24/f2d 0 2024-03-16T09:01:08.865 INFO:tasks.workunit.client.0.smithi007.stdout:9/370: dread - d24/d4a/d55/f5b zero size 2024-03-16T09:01:08.865 INFO:tasks.workunit.client.0.smithi007.stdout:9/371: write d24/f46 [109249,29489] 0 2024-03-16T09:01:08.873 INFO:tasks.workunit.client.0.smithi007.stdout:3/342: dwrite d15/d18/f33 [0,4194304] 0 2024-03-16T09:01:08.873 INFO:tasks.workunit.client.0.smithi007.stdout:8/357: dwrite d0/dc/dd/f2c [0,4194304] 0 2024-03-16T09:01:08.874 INFO:tasks.workunit.client.0.smithi007.stdout:2/379: dread dc/f11 [0,4194304] 0 2024-03-16T09:01:08.874 INFO:tasks.workunit.client.0.smithi007.stdout:6/253: dwrite d2/d10/d12/d15/f28 [0,4194304] 0 2024-03-16T09:01:08.875 INFO:tasks.workunit.client.0.smithi007.stdout:6/254: getdents d2/d11/d1e 0 2024-03-16T09:01:08.879 INFO:tasks.workunit.client.0.smithi007.stdout:8/358: creat d0/d26/d2d/d43/d6f/f7d x:0 0 0 2024-03-16T09:01:08.881 INFO:tasks.workunit.client.0.smithi007.stdout:6/255: rmdir d2/d11 39 2024-03-16T09:01:08.881 INFO:tasks.workunit.client.0.smithi007.stdout:6/256: stat d2/d13/d1f/d4e/f52 0 2024-03-16T09:01:08.882 INFO:tasks.workunit.client.0.smithi007.stdout:6/257: readlink d2/d10/d12/l31 0 2024-03-16T09:01:08.883 INFO:tasks.workunit.client.0.smithi007.stdout:8/359: mkdir d0/d2/d7e 0 2024-03-16T09:01:08.885 INFO:tasks.workunit.client.0.smithi007.stdout:8/360: creat d0/d26/d2d/d64/f7f x:0 0 0 2024-03-16T09:01:08.888 INFO:tasks.workunit.client.0.smithi007.stdout:8/361: mkdir d0/dc/d80 0 2024-03-16T09:01:08.888 INFO:tasks.workunit.client.0.smithi007.stdout:8/362: write d0/f22 [3590830,61103] 0 2024-03-16T09:01:08.889 INFO:tasks.workunit.client.0.smithi007.stdout:8/363: link d0/d2/d18/l6e d0/d26/d2d/d43/d70/l81 0 2024-03-16T09:01:08.889 INFO:tasks.workunit.client.0.smithi007.stdout:2/380: dread dc/f10 [0,4194304] 0 2024-03-16T09:01:08.891 INFO:tasks.workunit.client.0.smithi007.stdout:2/381: rename dc/df/d24/d28/d4c/d5e/l55 to dc/df/d24/l7b 0 2024-03-16T09:01:08.893 INFO:tasks.workunit.client.0.smithi007.stdout:2/382: link dc/c25 dc/df/d24/d28/d4c/c7c 0 2024-03-16T09:01:08.897 INFO:tasks.workunit.client.0.smithi007.stdout:4/208: dwrite d4/f5 [4194304,4194304] 0 2024-03-16T09:01:08.897 INFO:tasks.workunit.client.0.smithi007.stdout:4/209: write d4/f22 [854940,84969] 0 2024-03-16T09:01:08.904 INFO:tasks.workunit.client.0.smithi007.stdout:2/383: dread dc/df/d24/d28/f66 [0,4194304] 0 2024-03-16T09:01:08.904 INFO:tasks.workunit.client.0.smithi007.stdout:2/384: chown dc/df/d24/d28/d2e/d6e 110815 1 2024-03-16T09:01:08.905 INFO:tasks.workunit.client.0.smithi007.stdout:2/385: creat dc/df/d24/d28/d4c/f7d x:0 0 0 2024-03-16T09:01:08.906 INFO:tasks.workunit.client.0.smithi007.stdout:2/386: creat dc/f7e x:0 0 0 2024-03-16T09:01:08.906 INFO:tasks.workunit.client.0.smithi007.stdout:2/387: chown dc/df/d24/d28/d4c/f73 0 1 2024-03-16T09:01:08.909 INFO:tasks.workunit.client.0.smithi007.stdout:7/215: dwrite d0/d19/f26 [0,4194304] 0 2024-03-16T09:01:08.913 INFO:tasks.workunit.client.0.smithi007.stdout:9/372: dwrite fd [0,4194304] 0 2024-03-16T09:01:08.913 INFO:tasks.workunit.client.0.smithi007.stdout:3/343: dwrite d15/d18/d1f/d3b/d63/f66 [0,4194304] 0 2024-03-16T09:01:08.913 INFO:tasks.workunit.client.0.smithi007.stdout:8/364: dwrite d0/d26/d2d/d43/d4c/f78 [0,4194304] 0 2024-03-16T09:01:08.913 INFO:tasks.workunit.client.0.smithi007.stdout:8/365: truncate d0/dc/f35 409896 0 2024-03-16T09:01:08.914 INFO:tasks.workunit.client.0.smithi007.stdout:4/210: dwrite d4/d14/d1c/f3e [0,4194304] 0 2024-03-16T09:01:08.914 INFO:tasks.workunit.client.0.smithi007.stdout:4/211: dread - d4/d7/f1d zero size 2024-03-16T09:01:08.914 INFO:tasks.workunit.client.0.smithi007.stdout:4/212: chown d4/d7/fe 939 1 2024-03-16T09:01:08.917 INFO:tasks.workunit.client.0.smithi007.stdout:2/388: dread dc/df/f29 [0,4194304] 0 2024-03-16T09:01:08.922 INFO:tasks.workunit.client.0.smithi007.stdout:8/366: creat d0/dc/f82 x:0 0 0 2024-03-16T09:01:08.923 INFO:tasks.workunit.client.0.smithi007.stdout:2/389: write dc/f23 [1132511,22783] 0 2024-03-16T09:01:08.923 INFO:tasks.workunit.client.0.smithi007.stdout:9/373: write f1c [1985985,111563] 0 2024-03-16T09:01:08.924 INFO:tasks.workunit.client.0.smithi007.stdout:4/213: symlink d4/d14/l47 0 2024-03-16T09:01:08.926 INFO:tasks.workunit.client.0.smithi007.stdout:2/390: creat dc/d27/f7f x:0 0 0 2024-03-16T09:01:08.926 INFO:tasks.workunit.client.0.smithi007.stdout:2/391: chown dc/df/d24/d28/d2e/d6e/f6f 0 1 2024-03-16T09:01:08.927 INFO:tasks.workunit.client.0.smithi007.stdout:9/374: truncate d24/f2d 3082213 0 2024-03-16T09:01:08.928 INFO:tasks.workunit.client.0.smithi007.stdout:4/214: symlink d4/d14/l48 0 2024-03-16T09:01:08.933 INFO:tasks.workunit.client.0.smithi007.stdout:8/367: read d0/d26/d2d/f32 [549331,39148] 0 2024-03-16T09:01:08.933 INFO:tasks.workunit.client.0.smithi007.stdout:8/368: stat d0/d26/f39 0 2024-03-16T09:01:08.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/369: symlink d0/d26/d2d/d43/d4c/l83 0 2024-03-16T09:01:08.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/370: chown d0/d2/d18/f52 3122713 1 2024-03-16T09:01:08.936 INFO:tasks.workunit.client.0.smithi007.stdout:8/371: symlink d0/d26/d2d/d43/d72/l84 0 2024-03-16T09:01:08.936 INFO:tasks.workunit.client.0.smithi007.stdout:8/372: write d0/f31 [2001641,21179] 0 2024-03-16T09:01:08.944 INFO:tasks.workunit.client.0.smithi007.stdout:4/215: dwrite d4/d7/f1d [0,4194304] 0 2024-03-16T09:01:08.944 INFO:tasks.workunit.client.0.smithi007.stdout:4/216: truncate d4/f17 2112512 0 2024-03-16T09:01:08.946 INFO:tasks.workunit.client.0.smithi007.stdout:4/217: mkdir d4/d14/d49 0 2024-03-16T09:01:08.947 INFO:tasks.workunit.client.0.smithi007.stdout:4/218: mknod d4/c4a 0 2024-03-16T09:01:08.948 INFO:tasks.workunit.client.0.smithi007.stdout:4/219: creat d4/d14/d1c/d1e/f4b x:0 0 0 2024-03-16T09:01:08.948 INFO:tasks.workunit.client.0.smithi007.stdout:4/220: write d4/d14/d1c/d1e/f4b [55874,25113] 0 2024-03-16T09:01:08.949 INFO:tasks.workunit.client.0.smithi007.stdout:4/221: rmdir d4/d14/d1c/d1e/d44 0 2024-03-16T09:01:08.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/392: rmdir dc/df/d24 39 2024-03-16T09:01:08.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/393: chown dc/df/f34 1 1 2024-03-16T09:01:08.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/394: readlink dc/df/l1b 0 2024-03-16T09:01:08.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/395: fdatasync dc/df/d24/d28/d4c/d5e/f5b 0 2024-03-16T09:01:08.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/396: read dc/df/f14 [2766545,127537] 0 2024-03-16T09:01:08.954 INFO:tasks.workunit.client.0.smithi007.stdout:2/397: chown dc/df/l33 414 1 2024-03-16T09:01:08.956 INFO:tasks.workunit.client.0.smithi007.stdout:2/398: creat dc/df/d24/f80 x:0 0 0 2024-03-16T09:01:08.956 INFO:tasks.workunit.client.0.smithi007.stdout:2/399: dread - dc/f7e zero size 2024-03-16T09:01:08.958 INFO:tasks.workunit.client.0.smithi007.stdout:2/400: rmdir dc/df/d24/d28/d4c/d5e 39 2024-03-16T09:01:08.958 INFO:tasks.workunit.client.0.smithi007.stdout:2/401: dread - dc/df/f5a zero size 2024-03-16T09:01:08.962 INFO:tasks.workunit.client.0.smithi007.stdout:3/344: write d15/d18/f23 [3444463,18355] 0 2024-03-16T09:01:08.964 INFO:tasks.workunit.client.0.smithi007.stdout:7/216: dwrite d0/f18 [0,4194304] 0 2024-03-16T09:01:08.970 INFO:tasks.workunit.client.0.smithi007.stdout:4/222: dread d4/f22 [0,4194304] 0 2024-03-16T09:01:08.970 INFO:tasks.workunit.client.0.smithi007.stdout:4/223: truncate d4/d14/d1c/f37 131133 0 2024-03-16T09:01:08.970 INFO:tasks.workunit.client.0.smithi007.stdout:4/224: creat d4/d14/d1c/d1e/f4c x:0 0 0 2024-03-16T09:01:08.974 INFO:tasks.workunit.client.0.smithi007.stdout:4/225: rmdir d4/d14/d49 0 2024-03-16T09:01:08.974 INFO:tasks.workunit.client.0.smithi007.stdout:4/226: fdatasync d4/d7/fe 0 2024-03-16T09:01:08.974 INFO:tasks.workunit.client.0.smithi007.stdout:4/227: dread - d4/d14/d1c/f2c zero size 2024-03-16T09:01:08.974 INFO:tasks.workunit.client.0.smithi007.stdout:8/373: dwrite d0/f3c [4194304,4194304] 0 2024-03-16T09:01:08.974 INFO:tasks.workunit.client.0.smithi007.stdout:7/217: write d0/d19/f26 [3754636,79300] 0 2024-03-16T09:01:08.975 INFO:tasks.workunit.client.0.smithi007.stdout:7/218: fdatasync d0/dc/fd 0 2024-03-16T09:01:08.975 INFO:tasks.workunit.client.0.smithi007.stdout:9/375: dwrite d24/d4a/d55/d56/f5a [0,4194304] 0 2024-03-16T09:01:08.977 INFO:tasks.workunit.client.0.smithi007.stdout:8/374: unlink d0/dc/f35 0 2024-03-16T09:01:08.979 INFO:tasks.workunit.client.0.smithi007.stdout:7/219: creat d0/dc/d24/f46 x:0 0 0 2024-03-16T09:01:08.982 INFO:tasks.workunit.client.0.smithi007.stdout:3/345: dwrite d15/d18/d1f/d29/d43/f55 [0,4194304] 0 2024-03-16T09:01:08.983 INFO:tasks.workunit.client.0.smithi007.stdout:9/376: link lc d24/l6d 0 2024-03-16T09:01:08.983 INFO:tasks.workunit.client.0.smithi007.stdout:9/377: readlink d24/l41 0 2024-03-16T09:01:08.984 INFO:tasks.workunit.client.0.smithi007.stdout:2/402: dwrite dc/df/f18 [0,4194304] 0 2024-03-16T09:01:08.985 INFO:tasks.workunit.client.0.smithi007.stdout:9/378: symlink d24/d4a/d55/d56/l6e 0 2024-03-16T09:01:08.985 INFO:tasks.workunit.client.0.smithi007.stdout:9/379: readlink lc 0 2024-03-16T09:01:08.986 INFO:tasks.workunit.client.0.smithi007.stdout:2/403: symlink dc/df/d24/d28/d2e/l81 0 2024-03-16T09:01:08.987 INFO:tasks.workunit.client.0.smithi007.stdout:8/375: dread d0/f31 [0,4194304] 0 2024-03-16T09:01:08.988 INFO:tasks.workunit.client.0.smithi007.stdout:8/376: chown d0/d2/d18/f48 10705528 1 2024-03-16T09:01:08.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/346: dread d15/d18/d1f/d29/d3e/d4c/f64 [0,4194304] 0 2024-03-16T09:01:08.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/347: chown d15/d18/f19 10765 1 2024-03-16T09:01:08.992 INFO:tasks.workunit.client.0.smithi007.stdout:9/380: dread d24/f3c [4194304,4194304] 0 2024-03-16T09:01:08.992 INFO:tasks.workunit.client.0.smithi007.stdout:9/381: truncate d24/f31 1829034 0 2024-03-16T09:01:08.992 INFO:tasks.workunit.client.0.smithi007.stdout:4/228: dwrite d4/d7/f20 [0,4194304] 0 2024-03-16T09:01:08.993 INFO:tasks.workunit.client.0.smithi007.stdout:7/220: read d0/fa [1185399,56878] 0 2024-03-16T09:01:08.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:08 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:01:09.000 INFO:tasks.workunit.client.0.smithi007.stdout:3/348: fdatasync d15/d18/f23 0 2024-03-16T09:01:09.000 INFO:tasks.workunit.client.0.smithi007.stdout:8/377: dread d0/f22 [0,4194304] 0 2024-03-16T09:01:09.002 INFO:tasks.workunit.client.0.smithi007.stdout:4/229: link d4/d7/f1d d4/f4d 0 2024-03-16T09:01:09.006 INFO:tasks.workunit.client.0.smithi007.stdout:3/349: symlink d15/d18/d1f/d29/d43/d72/l75 0 2024-03-16T09:01:09.006 INFO:tasks.workunit.client.0.smithi007.stdout:4/230: dread d4/ff [0,4194304] 0 2024-03-16T09:01:09.007 INFO:tasks.workunit.client.0.smithi007.stdout:8/378: symlink d0/d26/l85 0 2024-03-16T09:01:09.008 INFO:tasks.workunit.client.0.smithi007.stdout:4/231: rmdir d4/d7 39 2024-03-16T09:01:09.009 INFO:tasks.workunit.client.0.smithi007.stdout:8/379: link d0/dc/c33 d0/d26/d2d/d43/d70/c86 0 2024-03-16T09:01:09.009 INFO:tasks.workunit.client.0.smithi007.stdout:8/380: write d0/d26/f5b [460044,80852] 0 2024-03-16T09:01:09.010 INFO:tasks.workunit.client.0.smithi007.stdout:4/232: rename d4/d7/l8 to d4/d7/l4e 0 2024-03-16T09:01:09.011 INFO:tasks.workunit.client.0.smithi007.stdout:8/381: mkdir d0/d26/d2d/d43/d4c/d87 0 2024-03-16T09:01:09.012 INFO:tasks.workunit.client.0.smithi007.stdout:8/382: symlink d0/d26/d2d/l88 0 2024-03-16T09:01:09.012 INFO:tasks.workunit.client.0.smithi007.stdout:8/383: truncate d0/f16 8676591 0 2024-03-16T09:01:09.014 INFO:tasks.workunit.client.0.smithi007.stdout:2/404: dwrite dc/df/d24/d28/d4c/d5e/f5b [0,4194304] 0 2024-03-16T09:01:09.014 INFO:tasks.workunit.client.0.smithi007.stdout:7/221: dwrite d0/f18 [0,4194304] 0 2024-03-16T09:01:09.022 INFO:tasks.workunit.client.0.smithi007.stdout:7/222: read d0/dc/f1a [1533267,97883] 0 2024-03-16T09:01:09.023 INFO:tasks.workunit.client.0.smithi007.stdout:8/384: truncate d0/f16 8413961 0 2024-03-16T09:01:09.023 INFO:tasks.workunit.client.0.smithi007.stdout:8/385: stat d0/d26/d2d/d43/d4c/f78 0 2024-03-16T09:01:09.023 INFO:tasks.workunit.client.0.smithi007.stdout:8/386: truncate d0/dc/d50/f6b 565505 0 2024-03-16T09:01:09.024 INFO:tasks.workunit.client.0.smithi007.stdout:7/223: symlink d0/dc/d24/d25/d3a/l47 0 2024-03-16T09:01:09.027 INFO:tasks.workunit.client.0.smithi007.stdout:2/405: dwrite dc/df/d24/d28/d2e/d35/f40 [0,4194304] 0 2024-03-16T09:01:09.030 INFO:tasks.workunit.client.0.smithi007.stdout:2/406: creat dc/df/d24/d28/f82 x:0 0 0 2024-03-16T09:01:09.030 INFO:tasks.workunit.client.0.smithi007.stdout:2/407: write dc/df/d24/f80 [341679,105930] 0 2024-03-16T09:01:09.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/408: mknod dc/df/d6c/c83 0 2024-03-16T09:01:09.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/409: fdatasync dc/df/d24/f63 0 2024-03-16T09:01:09.044 INFO:tasks.workunit.client.0.smithi007.stdout:7/224: dwrite d0/fa [0,4194304] 0 2024-03-16T09:01:09.045 INFO:tasks.workunit.client.0.smithi007.stdout:8/387: dwrite d0/d26/d2d/d43/d4c/f78 [0,4194304] 0 2024-03-16T09:01:09.045 INFO:tasks.workunit.client.0.smithi007.stdout:2/410: dread dc/df/f18 [0,4194304] 0 2024-03-16T09:01:09.045 INFO:tasks.workunit.client.0.smithi007.stdout:8/388: creat d0/dc/f89 x:0 0 0 2024-03-16T09:01:09.046 INFO:tasks.workunit.client.0.smithi007.stdout:7/225: rename d0/d19/f20 to d0/d19/f48 0 2024-03-16T09:01:09.047 INFO:tasks.workunit.client.0.smithi007.stdout:2/411: mknod dc/df/d6c/c84 0 2024-03-16T09:01:09.050 INFO:tasks.workunit.client.0.smithi007.stdout:8/389: mkdir d0/dc/d8a 0 2024-03-16T09:01:09.052 INFO:tasks.workunit.client.0.smithi007.stdout:8/390: rename d0/d26/d2d/d43/d4c/d87 to d0/dc/dd/d8b 0 2024-03-16T09:01:09.053 INFO:tasks.workunit.client.0.smithi007.stdout:8/391: truncate d0/dc/f12 4318701 0 2024-03-16T09:01:09.053 INFO:tasks.workunit.client.0.smithi007.stdout:2/412: getdents dc/d27 0 2024-03-16T09:01:09.053 INFO:tasks.workunit.client.0.smithi007.stdout:2/413: chown dc/df/d24/d28/d2e/d6e/f76 8991603 1 2024-03-16T09:01:09.054 INFO:tasks.workunit.client.0.smithi007.stdout:8/392: symlink d0/d26/d2d/d64/l8c 0 2024-03-16T09:01:09.062 INFO:tasks.workunit.client.0.smithi007.stdout:8/393: rmdir d0/dc/dd 39 2024-03-16T09:01:09.063 INFO:tasks.workunit.client.0.smithi007.stdout:8/394: write d0/d26/d2d/f32 [5168076,124729] 0 2024-03-16T09:01:09.072 INFO:tasks.workunit.client.0.smithi007.stdout:7/226: dwrite d0/dc/f1a [0,4194304] 0 2024-03-16T09:01:09.072 INFO:tasks.workunit.client.0.smithi007.stdout:7/227: getdents d0/dc/d38 0 2024-03-16T09:01:09.072 INFO:tasks.workunit.client.0.smithi007.stdout:2/414: dwrite dc/df/d24/d28/d2e/f52 [0,4194304] 0 2024-03-16T09:01:09.074 INFO:tasks.workunit.client.0.smithi007.stdout:2/415: creat dc/df/d24/d28/d2e/f85 x:0 0 0 2024-03-16T09:01:09.079 INFO:tasks.workunit.client.0.smithi007.stdout:8/395: dread d0/d2/f67 [0,4194304] 0 2024-03-16T09:01:09.079 INFO:tasks.workunit.client.0.smithi007.stdout:2/416: creat dc/df/d6c/f86 x:0 0 0 2024-03-16T09:01:09.080 INFO:tasks.workunit.client.0.smithi007.stdout:7/228: dread d0/d19/f27 [0,4194304] 0 2024-03-16T09:01:09.081 INFO:tasks.workunit.client.0.smithi007.stdout:2/417: symlink dc/d27/l87 0 2024-03-16T09:01:09.082 INFO:tasks.workunit.client.0.smithi007.stdout:2/418: mknod dc/df/d24/d28/d2e/d4a/c88 0 2024-03-16T09:01:09.082 INFO:tasks.workunit.client.0.smithi007.stdout:2/419: dread - dc/df/f5a zero size 2024-03-16T09:01:09.082 INFO:tasks.workunit.client.0.smithi007.stdout:2/420: dread - dc/df/d24/d28/d2e/f85 zero size 2024-03-16T09:01:09.088 INFO:tasks.workunit.client.0.smithi007.stdout:8/396: dread d0/dc/dd/f2c [0,4194304] 0 2024-03-16T09:01:09.088 INFO:tasks.workunit.client.0.smithi007.stdout:8/397: creat d0/d2/d18/f8d x:0 0 0 2024-03-16T09:01:09.090 INFO:tasks.workunit.client.0.smithi007.stdout:8/398: mkdir d0/dc/d50/d8e 0 2024-03-16T09:01:09.090 INFO:tasks.workunit.client.0.smithi007.stdout:8/399: readlink d0/dc/l2e 0 2024-03-16T09:01:09.090 INFO:tasks.workunit.client.0.smithi007.stdout:8/400: fsync d0/f7b 0 2024-03-16T09:01:09.090 INFO:tasks.workunit.client.0.smithi007.stdout:8/401: dread - d0/d26/d2d/f6c zero size 2024-03-16T09:01:09.091 INFO:tasks.workunit.client.0.smithi007.stdout:8/402: write d0/f31 [1116322,25508] 0 2024-03-16T09:01:09.091 INFO:tasks.workunit.client.0.smithi007.stdout:8/403: dread - d0/d26/d2d/d43/d6f/f7d zero size 2024-03-16T09:01:09.091 INFO:tasks.workunit.client.0.smithi007.stdout:8/404: chown d0/d2/f4b 1230 1 2024-03-16T09:01:09.096 INFO:tasks.workunit.client.0.smithi007.stdout:8/405: dread d0/d2/d18/f3b [4194304,4194304] 0 2024-03-16T09:01:09.097 INFO:tasks.workunit.client.0.smithi007.stdout:8/406: fsync d0/d2/fa 0 2024-03-16T09:01:09.097 INFO:tasks.workunit.client.0.smithi007.stdout:8/407: write d0/d26/f39 [401022,31868] 0 2024-03-16T09:01:09.097 INFO:tasks.workunit.client.0.smithi007.stdout:8/408: creat d0/d26/d2d/d43/d6f/f8f x:0 0 0 2024-03-16T09:01:09.097 INFO:tasks.workunit.client.0.smithi007.stdout:8/409: readlink d0/d2/l1c 0 2024-03-16T09:01:09.097 INFO:tasks.workunit.client.0.smithi007.stdout:8/410: chown d0/f22 13 1 2024-03-16T09:01:09.098 INFO:tasks.workunit.client.0.smithi007.stdout:8/411: symlink d0/d2/l90 0 2024-03-16T09:01:09.099 INFO:tasks.workunit.client.0.smithi007.stdout:8/412: creat d0/dc/d8a/f91 x:0 0 0 2024-03-16T09:01:09.099 INFO:tasks.workunit.client.0.smithi007.stdout:8/413: mknod d0/d26/d2d/d43/d70/c92 0 2024-03-16T09:01:09.100 INFO:tasks.workunit.client.0.smithi007.stdout:8/414: chown d0/d2/d18/f3f 3 1 2024-03-16T09:01:09.101 INFO:tasks.workunit.client.0.smithi007.stdout:8/415: fsync d0/d2/d18/f8d 0 2024-03-16T09:01:09.102 INFO:tasks.workunit.client.0.smithi007.stdout:8/416: mkdir d0/d2/d5f/d93 0 2024-03-16T09:01:09.102 INFO:tasks.workunit.client.0.smithi007.stdout:8/417: write d0/d2/f4b [4661561,41149] 0 2024-03-16T09:01:09.105 INFO:tasks.workunit.client.0.smithi007.stdout:8/418: creat d0/d26/d2d/d43/d72/f94 x:0 0 0 2024-03-16T09:01:09.105 INFO:tasks.workunit.client.0.smithi007.stdout:8/419: fsync d0/dc/f23 0 2024-03-16T09:01:09.106 INFO:tasks.workunit.client.0.smithi007.stdout:8/420: mkdir d0/d26/d95 0 2024-03-16T09:01:09.107 INFO:tasks.workunit.client.0.smithi007.stdout:8/421: creat d0/d2/d74/f96 x:0 0 0 2024-03-16T09:01:09.108 INFO:tasks.workunit.client.0.smithi007.stdout:8/422: mknod d0/c97 0 2024-03-16T09:01:09.108 INFO:tasks.workunit.client.0.smithi007.stdout:8/423: stat d0/dc/d75 0 2024-03-16T09:01:09.109 INFO:tasks.workunit.client.0.smithi007.stdout:8/424: creat d0/dc/d54/f98 x:0 0 0 2024-03-16T09:01:09.118 INFO:tasks.workunit.client.0.smithi007.stdout:8/425: write d0/d26/d2d/d43/d4c/f78 [2216783,117034] 0 2024-03-16T09:01:09.122 INFO:tasks.workunit.client.0.smithi007.stdout:5/188: sync 2024-03-16T09:01:09.122 INFO:tasks.workunit.client.0.smithi007.stdout:1/254: sync 2024-03-16T09:01:09.122 INFO:tasks.workunit.client.0.smithi007.stdout:5/189: fsync d1/d7/d16/f33 0 2024-03-16T09:01:09.125 INFO:tasks.workunit.client.0.smithi007.stdout:8/426: dread d0/d2/fa [0,4194304] 0 2024-03-16T09:01:09.125 INFO:tasks.workunit.client.0.smithi007.stdout:8/427: creat d0/dc/d8a/f99 x:0 0 0 2024-03-16T09:01:09.125 INFO:tasks.workunit.client.0.smithi007.stdout:8/428: stat d0/d2/d74/f96 0 2024-03-16T09:01:09.127 INFO:tasks.workunit.client.0.smithi007.stdout:8/429: dread d0/d2/d18/f3f [0,4194304] 0 2024-03-16T09:01:09.127 INFO:tasks.workunit.client.0.smithi007.stdout:8/430: readlink d0/dc/l3d 0 2024-03-16T09:01:09.130 INFO:tasks.workunit.client.0.smithi007.stdout:8/431: fsync d0/d26/d2d/d43/d4c/f78 0 2024-03-16T09:01:09.130 INFO:tasks.workunit.client.0.smithi007.stdout:8/432: truncate d0/d2/d74/f96 876444 0 2024-03-16T09:01:09.134 INFO:tasks.workunit.client.0.smithi007.stdout:8/433: creat d0/dc/d75/f9a x:0 0 0 2024-03-16T09:01:09.134 INFO:tasks.workunit.client.0.smithi007.stdout:1/255: write d9/f13 [227412,57693] 0 2024-03-16T09:01:09.134 INFO:tasks.workunit.client.0.smithi007.stdout:1/256: dread - d9/dd/d1b/f49 zero size 2024-03-16T09:01:09.135 INFO:tasks.workunit.client.0.smithi007.stdout:8/434: creat d0/d26/d95/f9b x:0 0 0 2024-03-16T09:01:09.137 INFO:tasks.workunit.client.0.smithi007.stdout:1/257: creat d9/dd/d1b/f4d x:0 0 0 2024-03-16T09:01:09.137 INFO:tasks.workunit.client.0.smithi007.stdout:5/190: dread d1/fa [0,4194304] 0 2024-03-16T09:01:09.137 INFO:tasks.workunit.client.0.smithi007.stdout:1/258: write d9/dd/d1b/d28/f32 [4691593,113557] 0 2024-03-16T09:01:09.138 INFO:tasks.workunit.client.0.smithi007.stdout:8/435: mknod d0/d2/d7e/c9c 0 2024-03-16T09:01:09.139 INFO:tasks.workunit.client.0.smithi007.stdout:5/191: mknod d1/d7/d1c/d19/d34/d28/c40 0 2024-03-16T09:01:09.140 INFO:tasks.workunit.client.0.smithi007.stdout:1/259: mknod d9/c4e 0 2024-03-16T09:01:09.140 INFO:tasks.workunit.client.0.smithi007.stdout:1/260: creat d9/dd/d1b/d2d/d30/f4f x:0 0 0 2024-03-16T09:01:09.141 INFO:tasks.workunit.client.0.smithi007.stdout:8/436: rename d0/d2/d7e/c9c to d0/dc/dd/c9d 0 2024-03-16T09:01:09.141 INFO:tasks.workunit.client.0.smithi007.stdout:8/437: write d0/d2/d18/f48 [558861,3217] 0 2024-03-16T09:01:09.142 INFO:tasks.workunit.client.0.smithi007.stdout:5/192: unlink d1/d7/d1c/d19/d34/d28/c3b 0 2024-03-16T09:01:09.144 INFO:tasks.workunit.client.0.smithi007.stdout:8/438: rename d0/d2/d5f/d93 to d0/dc/dd/d8b/d9e 0 2024-03-16T09:01:09.144 INFO:tasks.workunit.client.0.smithi007.stdout:5/193: mkdir d1/d7/d1c/d19/d34/d3e/d41 0 2024-03-16T09:01:09.146 INFO:tasks.workunit.client.0.smithi007.stdout:8/439: link d0/l1b d0/d26/d2d/d43/d70/l9f 0 2024-03-16T09:01:09.146 INFO:tasks.workunit.client.0.smithi007.stdout:8/440: write d0/f7b [943188,25860] 0 2024-03-16T09:01:09.146 INFO:tasks.workunit.client.0.smithi007.stdout:8/441: chown d0/d26 1694223122 1 2024-03-16T09:01:09.146 INFO:tasks.workunit.client.0.smithi007.stdout:5/194: symlink d1/d7/l42 0 2024-03-16T09:01:09.147 INFO:tasks.workunit.client.0.smithi007.stdout:5/195: creat d1/d7/d1c/d19/d34/d3e/f43 x:0 0 0 2024-03-16T09:01:09.149 INFO:tasks.workunit.client.0.smithi007.stdout:5/196: rename d1/cd to d1/d7/d1c/d19/d34/c44 0 2024-03-16T09:01:09.154 INFO:tasks.workunit.client.0.smithi007.stdout:1/261: dwrite d9/dd/d1b/d28/d29/f42 [0,4194304] 0 2024-03-16T09:01:09.154 INFO:tasks.workunit.client.0.smithi007.stdout:1/262: chown d9/c1a 1835 1 2024-03-16T09:01:09.176 INFO:tasks.workunit.client.0.smithi007.stdout:8/442: dwrite d0/d26/f79 [0,4194304] 0 2024-03-16T09:01:09.177 INFO:tasks.workunit.client.0.smithi007.stdout:5/197: dwrite d1/d7/d1c/f23 [0,4194304] 0 2024-03-16T09:01:09.177 INFO:tasks.workunit.client.0.smithi007.stdout:1/263: dwrite d9/dd/d1b/d2d/f44 [0,4194304] 0 2024-03-16T09:01:09.177 INFO:tasks.workunit.client.0.smithi007.stdout:8/443: creat d0/dc/dd/d8b/d9e/fa0 x:0 0 0 2024-03-16T09:01:09.178 INFO:tasks.workunit.client.0.smithi007.stdout:8/444: chown d0/d26/f30 20224610 1 2024-03-16T09:01:09.179 INFO:tasks.workunit.client.0.smithi007.stdout:5/198: creat d1/d7/d1c/d19/d34/d1d/d36/f45 x:0 0 0 2024-03-16T09:01:09.182 INFO:tasks.workunit.client.0.smithi007.stdout:8/445: rename d0/f31 to d0/d26/fa1 0 2024-03-16T09:01:09.183 INFO:tasks.workunit.client.0.smithi007.stdout:8/446: write d0/d26/d2d/f61 [481712,39647] 0 2024-03-16T09:01:09.188 INFO:tasks.workunit.client.0.smithi007.stdout:8/447: dread d0/d26/fa1 [0,4194304] 0 2024-03-16T09:01:09.188 INFO:tasks.workunit.client.0.smithi007.stdout:8/448: chown d0/dc/d8a 5472933 1 2024-03-16T09:01:09.188 INFO:tasks.workunit.client.0.smithi007.stdout:5/199: getdents d1/d7/d1c/d19/d34/d1d/d36 0 2024-03-16T09:01:09.188 INFO:tasks.workunit.client.0.smithi007.stdout:5/200: chown d1/d7/d1c/d19/d34/d1d/d36/f45 1 1 2024-03-16T09:01:09.189 INFO:tasks.workunit.client.0.smithi007.stdout:8/449: unlink d0/d2/f4b 0 2024-03-16T09:01:09.189 INFO:tasks.workunit.client.0.smithi007.stdout:8/450: fsync d0/d2/d18/f3f 0 2024-03-16T09:01:09.190 INFO:tasks.workunit.client.0.smithi007.stdout:5/201: symlink d1/d7/l46 0 2024-03-16T09:01:09.198 INFO:tasks.workunit.client.0.smithi007.stdout:1/264: dwrite d9/f22 [0,4194304] 0 2024-03-16T09:01:09.198 INFO:tasks.workunit.client.0.smithi007.stdout:1/265: chown d9/dd/d1b/d2d/d30 241032 1 2024-03-16T09:01:09.198 INFO:tasks.workunit.client.0.smithi007.stdout:1/266: creat d9/dd/d1b/d28/f50 x:0 0 0 2024-03-16T09:01:09.198 INFO:tasks.workunit.client.0.smithi007.stdout:1/267: stat d9/lb 0 2024-03-16T09:01:09.199 INFO:tasks.workunit.client.0.smithi007.stdout:5/202: read d1/f2 [2666084,87567] 0 2024-03-16T09:01:09.201 INFO:tasks.workunit.client.0.smithi007.stdout:5/203: link d1/d7/d16/d1e/f3d d1/d7/d16/d1e/f47 0 2024-03-16T09:01:09.201 INFO:tasks.workunit.client.0.smithi007.stdout:5/204: dread - d1/d7/d1c/d19/d34/d1d/d36/f45 zero size 2024-03-16T09:01:09.201 INFO:tasks.workunit.client.0.smithi007.stdout:5/205: chown d1/d7/d16/d1e/l21 10 1 2024-03-16T09:01:09.201 INFO:tasks.workunit.client.0.smithi007.stdout:5/206: mkdir d1/d7/d1c/d19/d34/d3e/d41/d48 0 2024-03-16T09:01:09.202 INFO:tasks.workunit.client.0.smithi007.stdout:8/451: dwrite d0/d26/fa1 [0,4194304] 0 2024-03-16T09:01:09.204 INFO:tasks.workunit.client.0.smithi007.stdout:8/452: rename d0/dc/d75/f9a to d0/d26/fa2 0 2024-03-16T09:01:09.204 INFO:tasks.workunit.client.0.smithi007.stdout:8/453: chown d0/d26/l85 0 1 2024-03-16T09:01:09.205 INFO:tasks.workunit.client.0.smithi007.stdout:8/454: symlink d0/dc/d54/la3 0 2024-03-16T09:01:09.205 INFO:tasks.workunit.client.0.smithi007.stdout:8/455: dread - d0/d2/f66 zero size 2024-03-16T09:01:09.206 INFO:tasks.workunit.client.0.smithi007.stdout:8/456: mkdir d0/dc/d54/da4 0 2024-03-16T09:01:09.208 INFO:tasks.workunit.client.0.smithi007.stdout:8/457: unlink d0/d2/c6d 0 2024-03-16T09:01:09.208 INFO:tasks.workunit.client.0.smithi007.stdout:8/458: truncate d0/dc/d8a/f91 309944 0 2024-03-16T09:01:09.209 INFO:tasks.workunit.client.0.smithi007.stdout:8/459: mkdir d0/d26/d2d/d43/d6f/da5 0 2024-03-16T09:01:09.209 INFO:tasks.workunit.client.0.smithi007.stdout:8/460: readlink d0/dc/l13 0 2024-03-16T09:01:09.211 INFO:tasks.workunit.client.0.smithi007.stdout:8/461: mkdir d0/d26/d2d/d64/da6 0 2024-03-16T09:01:09.211 INFO:tasks.workunit.client.0.smithi007.stdout:8/462: mkdir d0/d2/d18/da7 0 2024-03-16T09:01:09.212 INFO:tasks.workunit.client.0.smithi007.stdout:8/463: creat d0/dc/d50/d8e/fa8 x:0 0 0 2024-03-16T09:01:09.212 INFO:tasks.workunit.client.0.smithi007.stdout:8/464: truncate d0/dc/f82 813786 0 2024-03-16T09:01:09.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:08 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:01:09.213 INFO:tasks.workunit.client.0.smithi007.stdout:8/465: mknod d0/dc/dd/ca9 0 2024-03-16T09:01:09.214 INFO:tasks.workunit.client.0.smithi007.stdout:8/466: creat d0/d2/faa x:0 0 0 2024-03-16T09:01:09.215 INFO:tasks.workunit.client.0.smithi007.stdout:8/467: unlink d0/d26/d2d/f61 0 2024-03-16T09:01:09.217 INFO:tasks.workunit.client.0.smithi007.stdout:8/468: creat d0/d26/d2d/fab x:0 0 0 2024-03-16T09:01:09.220 INFO:tasks.workunit.client.0.smithi007.stdout:8/469: rename d0/dc/dd/l47 to d0/d26/lac 0 2024-03-16T09:01:09.221 INFO:tasks.workunit.client.0.smithi007.stdout:8/470: mkdir d0/d26/d2d/d43/d72/dad 0 2024-03-16T09:01:09.221 INFO:tasks.workunit.client.0.smithi007.stdout:8/471: readlink d0/l9 0 2024-03-16T09:01:09.222 INFO:tasks.workunit.client.0.smithi007.stdout:8/472: mknod d0/d26/d95/cae 0 2024-03-16T09:01:09.228 INFO:tasks.workunit.client.0.smithi007.stdout:1/268: dwrite d9/dd/d1b/d28/f3e [0,4194304] 0 2024-03-16T09:01:09.228 INFO:tasks.workunit.client.0.smithi007.stdout:5/207: dwrite d1/d7/d1c/d19/f26 [0,4194304] 0 2024-03-16T09:01:09.228 INFO:tasks.workunit.client.0.smithi007.stdout:5/208: read - d1/d7/d1c/d19/f3a zero size 2024-03-16T09:01:09.229 INFO:tasks.workunit.client.0.smithi007.stdout:5/209: creat d1/d7/d1c/d19/d34/de/f49 x:0 0 0 2024-03-16T09:01:09.234 INFO:tasks.workunit.client.0.smithi007.stdout:8/473: rmdir d0/d26/d2d 39 2024-03-16T09:01:09.234 INFO:tasks.workunit.client.0.smithi007.stdout:8/474: chown d0/dc/dd/d8b/d9e 5 1 2024-03-16T09:01:09.235 INFO:tasks.workunit.client.0.smithi007.stdout:8/475: getdents d0/d26/d2d/d43 0 2024-03-16T09:01:09.235 INFO:tasks.workunit.client.0.smithi007.stdout:8/476: fsync d0/d2/d18/f8d 0 2024-03-16T09:01:09.237 INFO:tasks.workunit.client.0.smithi007.stdout:8/477: truncate d0/f3c 3201050 0 2024-03-16T09:01:09.237 INFO:tasks.workunit.client.0.smithi007.stdout:8/478: dread - d0/dc/d8a/f99 zero size 2024-03-16T09:01:09.237 INFO:tasks.workunit.client.0.smithi007.stdout:8/479: chown d0/d2/d18/f4e 1 1 2024-03-16T09:01:09.246 INFO:tasks.workunit.client.0.smithi007.stdout:5/210: dwrite d1/d7/d1c/d19/d34/d3e/f43 [0,4194304] 0 2024-03-16T09:01:09.247 INFO:tasks.workunit.client.0.smithi007.stdout:8/480: dread d0/d26/f2f [0,4194304] 0 2024-03-16T09:01:09.248 INFO:tasks.workunit.client.0.smithi007.stdout:1/269: dwrite d9/dd/d1b/d28/f46 [0,4194304] 0 2024-03-16T09:01:09.248 INFO:tasks.workunit.client.0.smithi007.stdout:5/211: symlink d1/d7/d16/d1e/d2a/l4a 0 2024-03-16T09:01:09.249 INFO:tasks.workunit.client.0.smithi007.stdout:8/481: mknod d0/dc/dd/d8b/d9e/caf 0 2024-03-16T09:01:09.249 INFO:tasks.workunit.client.0.smithi007.stdout:8/482: write d0/d2/f67 [4495133,84225] 0 2024-03-16T09:01:09.250 INFO:tasks.workunit.client.0.smithi007.stdout:8/483: chown d0/d26/d2d/fab 358050521 1 2024-03-16T09:01:09.252 INFO:tasks.workunit.client.0.smithi007.stdout:5/212: symlink d1/d7/d16/d1e/d2a/l4b 0 2024-03-16T09:01:09.260 INFO:tasks.workunit.client.0.smithi007.stdout:5/213: write d1/f10 [1785607,86038] 0 2024-03-16T09:01:09.262 INFO:tasks.workunit.client.0.smithi007.stdout:8/484: dwrite d0/dc/f23 [4194304,4194304] 0 2024-03-16T09:01:09.263 INFO:tasks.workunit.client.0.smithi007.stdout:8/485: stat d0/d26/d95/f9b 0 2024-03-16T09:01:09.263 INFO:tasks.workunit.client.0.smithi007.stdout:5/214: link d1/d7/d16/d1e/f3d d1/d7/d1c/d19/d34/d3e/d41/f4c 0 2024-03-16T09:01:09.265 INFO:tasks.workunit.client.0.smithi007.stdout:5/215: mkdir d1/d7/d1c/d19/d34/d4d 0 2024-03-16T09:01:09.267 INFO:tasks.workunit.client.0.smithi007.stdout:5/216: unlink d1/d7/d1c/d19/d34/d28/l29 0 2024-03-16T09:01:09.268 INFO:tasks.workunit.client.0.smithi007.stdout:5/217: mkdir d1/d7/d1c/d19/d34/d1d/d36/d4e 0 2024-03-16T09:01:09.270 INFO:tasks.workunit.client.0.smithi007.stdout:5/218: creat d1/d7/d1c/d19/d34/d1d/f4f x:0 0 0 2024-03-16T09:01:09.276 INFO:tasks.workunit.client.0.smithi007.stdout:5/219: dread d1/d7/d1c/d19/f26 [0,4194304] 0 2024-03-16T09:01:09.277 INFO:tasks.workunit.client.0.smithi007.stdout:5/220: symlink d1/d7/d1c/d19/d34/de/l50 0 2024-03-16T09:01:09.277 INFO:tasks.workunit.client.0.smithi007.stdout:5/221: fsync d1/d7/d1c/d19/d34/d1d/f1f 0 2024-03-16T09:01:09.282 INFO:tasks.workunit.client.0.smithi007.stdout:5/222: dread d1/d7/d1c/f15 [0,4194304] 0 2024-03-16T09:01:09.283 INFO:tasks.workunit.client.0.smithi007.stdout:5/223: creat d1/d7/d1c/d19/d34/d1d/d36/d4e/f51 x:0 0 0 2024-03-16T09:01:09.283 INFO:tasks.workunit.client.0.smithi007.stdout:5/224: chown d1/d7/d1c/d19/d34/de/l50 459655 1 2024-03-16T09:01:09.284 INFO:tasks.workunit.client.0.smithi007.stdout:5/225: mknod d1/d7/d1c/d19/d34/de/c52 0 2024-03-16T09:01:09.285 INFO:tasks.workunit.client.0.smithi007.stdout:5/226: rename d1/c2f to d1/d7/c53 0 2024-03-16T09:01:09.294 INFO:tasks.workunit.client.0.smithi007.stdout:5/227: dread d1/d7/f12 [0,4194304] 0 2024-03-16T09:01:09.296 INFO:tasks.workunit.client.0.smithi007.stdout:8/486: dwrite d0/d2/d18/f4e [0,4194304] 0 2024-03-16T09:01:09.296 INFO:tasks.workunit.client.0.smithi007.stdout:5/228: unlink d1/d7/d16/d1e/d2a/l4a 0 2024-03-16T09:01:09.296 INFO:tasks.workunit.client.0.smithi007.stdout:5/229: stat d1/d7/d1c/d19/d34/d1d/l37 0 2024-03-16T09:01:09.297 INFO:tasks.workunit.client.0.smithi007.stdout:8/487: creat d0/d2/fb0 x:0 0 0 2024-03-16T09:01:09.297 INFO:tasks.workunit.client.0.smithi007.stdout:8/488: write d0/d26/d2d/f6c [100415,117026] 0 2024-03-16T09:01:09.298 INFO:tasks.workunit.client.0.smithi007.stdout:8/489: truncate d0/d2/f67 4086538 0 2024-03-16T09:01:09.298 INFO:tasks.workunit.client.0.smithi007.stdout:8/490: chown d0/dc/d75 1640 1 2024-03-16T09:01:09.298 INFO:tasks.workunit.client.0.smithi007.stdout:8/491: creat d0/dc/d75/fb1 x:0 0 0 2024-03-16T09:01:09.298 INFO:tasks.workunit.client.0.smithi007.stdout:8/492: write d0/dc/d75/fb1 [1002670,9217] 0 2024-03-16T09:01:09.299 INFO:tasks.workunit.client.0.smithi007.stdout:8/493: unlink d0/d26/d2d/l88 0 2024-03-16T09:01:09.301 INFO:tasks.workunit.client.0.smithi007.stdout:8/494: rename d0/fe to d0/dc/dd/d8b/d9e/fb2 0 2024-03-16T09:01:09.303 INFO:tasks.workunit.client.0.smithi007.stdout:5/230: read d1/d7/d16/d1e/f3d [1772988,109781] 0 2024-03-16T09:01:09.303 INFO:tasks.workunit.client.0.smithi007.stdout:5/231: creat d1/d7/d1c/d19/d34/d1d/d36/d4e/f54 x:0 0 0 2024-03-16T09:01:09.306 INFO:tasks.workunit.client.0.smithi007.stdout:5/232: getdents d1/d7/d16 0 2024-03-16T09:01:09.306 INFO:tasks.workunit.client.0.smithi007.stdout:5/233: creat d1/d7/d1c/d19/d34/d1d/d36/d4e/f55 x:0 0 0 2024-03-16T09:01:09.309 INFO:tasks.workunit.client.0.smithi007.stdout:8/495: dread d0/f16 [4194304,4194304] 0 2024-03-16T09:01:09.310 INFO:tasks.workunit.client.0.smithi007.stdout:8/496: mknod d0/d26/d2d/d64/cb3 0 2024-03-16T09:01:09.319 INFO:tasks.workunit.client.0.smithi007.stdout:5/234: dread d1/fc [0,4194304] 0 2024-03-16T09:01:09.321 INFO:tasks.workunit.client.0.smithi007.stdout:5/235: rename d1/d7/f38 to d1/d7/d1c/d19/d34/d28/f56 0 2024-03-16T09:01:09.321 INFO:tasks.workunit.client.0.smithi007.stdout:8/497: write d0/d2/d18/f4e [894972,55294] 0 2024-03-16T09:01:09.321 INFO:tasks.workunit.client.0.smithi007.stdout:5/236: creat d1/d7/d16/d1e/d2a/f57 x:0 0 0 2024-03-16T09:01:09.325 INFO:tasks.workunit.client.0.smithi007.stdout:8/498: dread d0/d2/f67 [0,4194304] 0 2024-03-16T09:01:09.327 INFO:tasks.workunit.client.0.smithi007.stdout:5/237: dread d1/d7/d16/d1e/f3d [0,4194304] 0 2024-03-16T09:01:09.327 INFO:tasks.workunit.client.0.smithi007.stdout:5/238: read d1/d7/f12 [1601540,58396] 0 2024-03-16T09:01:09.327 INFO:tasks.workunit.client.0.smithi007.stdout:5/239: chown d1/d7/d16/d1e/c39 0 1 2024-03-16T09:01:09.327 INFO:tasks.workunit.client.0.smithi007.stdout:5/240: creat d1/d7/d16/d1e/d2a/f58 x:0 0 0 2024-03-16T09:01:09.327 INFO:tasks.workunit.client.0.smithi007.stdout:5/241: readlink d1/d7/d16/d1e/d2a/l4b 0 2024-03-16T09:01:09.327 INFO:tasks.workunit.client.0.smithi007.stdout:8/499: rename d0/dc/l13 to d0/d2/d18/da7/lb4 0 2024-03-16T09:01:09.329 INFO:tasks.workunit.client.0.smithi007.stdout:8/500: symlink d0/d2/d18/da7/lb5 0 2024-03-16T09:01:09.329 INFO:tasks.workunit.client.0.smithi007.stdout:8/501: dread - d0/dc/d8a/f99 zero size 2024-03-16T09:01:09.329 INFO:tasks.workunit.client.0.smithi007.stdout:8/502: creat d0/dc/d8a/fb6 x:0 0 0 2024-03-16T09:01:09.330 INFO:tasks.workunit.client.0.smithi007.stdout:8/503: symlink d0/d26/d2d/d43/d6f/da5/lb7 0 2024-03-16T09:01:09.330 INFO:tasks.workunit.client.0.smithi007.stdout:8/504: write d0/d26/fa2 [719434,38397] 0 2024-03-16T09:01:09.330 INFO:tasks.workunit.client.0.smithi007.stdout:8/505: chown d0/d26/d2d/d43/d6f 16521769 1 2024-03-16T09:01:09.343 INFO:tasks.workunit.client.0.smithi007.stdout:5/242: dwrite d1/d7/d1c/d19/f26 [0,4194304] 0 2024-03-16T09:01:09.345 INFO:tasks.workunit.client.0.smithi007.stdout:5/243: mknod d1/c59 0 2024-03-16T09:01:09.352 INFO:tasks.workunit.client.0.smithi007.stdout:5/244: write d1/d7/d1c/d19/d34/d3e/f43 [65730,32205] 0 2024-03-16T09:01:09.353 INFO:tasks.workunit.client.0.smithi007.stdout:8/506: dwrite d0/d26/f2f [8388608,4194304] 0 2024-03-16T09:01:09.354 INFO:tasks.workunit.client.0.smithi007.stdout:5/245: rename d1/d7/d1c/d19/f3a to d1/d7/d1c/d19/d34/d1d/f5a 0 2024-03-16T09:01:09.355 INFO:tasks.workunit.client.0.smithi007.stdout:8/507: mkdir d0/dc/d50/d8e/db8 0 2024-03-16T09:01:09.356 INFO:tasks.workunit.client.0.smithi007.stdout:8/508: fdatasync d0/d26/fa1 0 2024-03-16T09:01:09.356 INFO:tasks.workunit.client.0.smithi007.stdout:8/509: chown d0/d2/l1c 110 1 2024-03-16T09:01:09.360 INFO:tasks.workunit.client.0.smithi007.stdout:5/246: unlink d1/d7/d16/d1e/d2a/f58 0 2024-03-16T09:01:09.360 INFO:tasks.workunit.client.0.smithi007.stdout:5/247: stat d1/d7/d1c/d19/d34/de 0 2024-03-16T09:01:09.361 INFO:tasks.workunit.client.0.smithi007.stdout:5/248: mknod d1/d7/d1c/d19/d34/d28/c5b 0 2024-03-16T09:01:09.363 INFO:tasks.workunit.client.0.smithi007.stdout:5/249: symlink d1/d7/d1c/d19/d34/d28/l5c 0 2024-03-16T09:01:09.379 INFO:tasks.workunit.client.0.smithi007.stdout:8/510: dwrite d0/d26/fa2 [0,4194304] 0 2024-03-16T09:01:09.379 INFO:tasks.workunit.client.0.smithi007.stdout:8/511: chown d0/l1b 1489632 1 2024-03-16T09:01:09.379 INFO:tasks.workunit.client.0.smithi007.stdout:8/512: creat d0/d26/d95/fb9 x:0 0 0 2024-03-16T09:01:09.379 INFO:tasks.workunit.client.0.smithi007.stdout:8/513: write d0/d26/d2d/f32 [2426593,116325] 0 2024-03-16T09:01:09.379 INFO:tasks.workunit.client.0.smithi007.stdout:5/250: dwrite d1/f10 [0,4194304] 0 2024-03-16T09:01:09.380 INFO:tasks.workunit.client.0.smithi007.stdout:8/514: dread d0/dc/d8a/f91 [0,4194304] 0 2024-03-16T09:01:09.382 INFO:tasks.workunit.client.0.smithi007.stdout:5/251: truncate d1/d7/d1c/d19/d34/d3e/f43 475660 0 2024-03-16T09:01:09.383 INFO:tasks.workunit.client.0.smithi007.stdout:8/515: mkdir d0/d26/d2d/d43/d4c/dba 0 2024-03-16T09:01:09.383 INFO:tasks.workunit.client.0.smithi007.stdout:8/516: truncate d0/dc/f3a 740671 0 2024-03-16T09:01:09.383 INFO:tasks.workunit.client.0.smithi007.stdout:8/517: dread - d0/dc/d8a/fb6 zero size 2024-03-16T09:01:09.384 INFO:tasks.workunit.client.0.smithi007.stdout:5/252: unlink d1/d7/d1c/f23 0 2024-03-16T09:01:09.384 INFO:tasks.workunit.client.0.smithi007.stdout:5/253: stat d1/fa 0 2024-03-16T09:01:09.385 INFO:tasks.workunit.client.0.smithi007.stdout:5/254: symlink d1/d7/d1c/d19/d34/l5d 0 2024-03-16T09:01:09.401 INFO:tasks.workunit.client.0.smithi007.stdout:8/518: dwrite d0/dc/d50/f6b [0,4194304] 0 2024-03-16T09:01:09.401 INFO:tasks.workunit.client.0.smithi007.stdout:8/519: write d0/d26/d2d/d64/f7f [1002525,95412] 0 2024-03-16T09:01:09.401 INFO:tasks.workunit.client.0.smithi007.stdout:5/255: dwrite d1/d7/d1c/d19/d34/d1d/d36/d4e/f54 [0,4194304] 0 2024-03-16T09:01:09.401 INFO:tasks.workunit.client.0.smithi007.stdout:5/256: write d1/d7/d16/d1e/f31 [270115,99693] 0 2024-03-16T09:01:09.401 INFO:tasks.workunit.client.0.smithi007.stdout:5/257: stat d1/d7/d1c/d19/d34/d1d/d36/d4e/f51 0 2024-03-16T09:01:09.401 INFO:tasks.workunit.client.0.smithi007.stdout:5/258: truncate d1/d7/d1c/d19/d34/d1d/d36/f45 1043248 0 2024-03-16T09:01:09.409 INFO:tasks.workunit.client.0.smithi007.stdout:5/259: dread d1/d7/d1c/f17 [0,4194304] 0 2024-03-16T09:01:09.409 INFO:tasks.workunit.client.0.smithi007.stdout:5/260: getdents d1/d7/d16/d1b/d3c 0 2024-03-16T09:01:09.413 INFO:tasks.workunit.client.0.smithi007.stdout:5/261: rename d1/d7/d1c/d19/d34/d28 to d1/d7/d1c/d19/d34/d5e 0 2024-03-16T09:01:09.413 INFO:tasks.workunit.client.0.smithi007.stdout:5/262: fsync d1/d7/d16/d1e/f3d 0 2024-03-16T09:01:09.414 INFO:tasks.workunit.client.0.smithi007.stdout:5/263: mknod d1/d7/d1c/d19/d34/d5e/c5f 0 2024-03-16T09:01:09.414 INFO:tasks.workunit.client.0.smithi007.stdout:5/264: write d1/d7/d16/d1e/f31 [456853,124570] 0 2024-03-16T09:01:09.416 INFO:tasks.workunit.client.0.smithi007.stdout:5/265: creat d1/d7/d16/d1b/f60 x:0 0 0 2024-03-16T09:01:09.416 INFO:tasks.workunit.client.0.smithi007.stdout:5/266: creat d1/d7/d16/d1e/d2a/f61 x:0 0 0 2024-03-16T09:01:09.423 INFO:tasks.workunit.client.0.smithi007.stdout:8/520: dwrite d0/d26/fa2 [0,4194304] 0 2024-03-16T09:01:09.427 INFO:tasks.workunit.client.0.smithi007.stdout:8/521: mknod d0/cbb 0 2024-03-16T09:01:09.428 INFO:tasks.workunit.client.0.smithi007.stdout:8/522: unlink d0/d26/d2d/fab 0 2024-03-16T09:01:09.428 INFO:tasks.workunit.client.0.smithi007.stdout:8/523: write d0/f22 [3662541,87929] 0 2024-03-16T09:01:09.435 INFO:tasks.workunit.client.0.smithi007.stdout:5/267: dwrite d1/fa [0,4194304] 0 2024-03-16T09:01:09.437 INFO:tasks.workunit.client.0.smithi007.stdout:5/268: mkdir d1/d7/d1c/d19/d34/d3e/d62 0 2024-03-16T09:01:09.437 INFO:tasks.workunit.client.0.smithi007.stdout:5/269: write d1/d7/d16/d1e/f47 [2018952,30856] 0 2024-03-16T09:01:09.437 INFO:tasks.workunit.client.0.smithi007.stdout:5/270: readlink d1/d7/d16/d1e/d2a/l4b 0 2024-03-16T09:01:09.443 INFO:tasks.workunit.client.0.smithi007.stdout:5/271: link d1/d7/d1c/d19/d34/de/c11 d1/d7/d1c/d19/d34/d3e/c63 0 2024-03-16T09:01:09.443 INFO:tasks.workunit.client.0.smithi007.stdout:5/272: dread - d1/d7/d1c/d19/d34/de/f49 zero size 2024-03-16T09:01:09.444 INFO:tasks.workunit.client.0.smithi007.stdout:8/524: dwrite d0/dc/dd/d8b/d9e/fa0 [0,4194304] 0 2024-03-16T09:01:09.444 INFO:tasks.workunit.client.0.smithi007.stdout:8/525: symlink d0/dc/d50/d8e/db8/lbc 0 2024-03-16T09:01:09.445 INFO:tasks.workunit.client.0.smithi007.stdout:8/526: mkdir d0/dbd 0 2024-03-16T09:01:09.445 INFO:tasks.workunit.client.0.smithi007.stdout:8/527: write d0/dc/d54/f98 [211190,122307] 0 2024-03-16T09:01:09.447 INFO:tasks.workunit.client.0.smithi007.stdout:8/528: truncate d0/dc/dd/f2c 114740 0 2024-03-16T09:01:09.456 INFO:tasks.workunit.client.0.smithi007.stdout:5/273: dwrite d1/d7/d16/d1e/d2a/f57 [0,4194304] 0 2024-03-16T09:01:09.456 INFO:tasks.workunit.client.0.smithi007.stdout:5/274: write d1/d7/d1c/d19/d34/d5e/f56 [751986,105342] 0 2024-03-16T09:01:09.463 INFO:tasks.workunit.client.0.smithi007.stdout:5/275: mknod d1/d7/d1c/d19/d34/d3e/d62/c64 0 2024-03-16T09:01:09.463 INFO:tasks.workunit.client.0.smithi007.stdout:5/276: symlink d1/d7/d16/d1e/l65 0 2024-03-16T09:01:09.463 INFO:tasks.workunit.client.0.smithi007.stdout:5/277: write d1/d7/d1c/d19/d34/d1d/d36/f45 [1112342,37542] 0 2024-03-16T09:01:09.467 INFO:tasks.workunit.client.0.smithi007.stdout:8/529: dwrite d0/d26/d95/f9b [0,4194304] 0 2024-03-16T09:01:09.470 INFO:tasks.workunit.client.0.smithi007.stdout:8/530: truncate d0/dc/dd/d8b/d9e/fa0 1149134 0 2024-03-16T09:01:09.474 INFO:tasks.workunit.client.0.smithi007.stdout:8/531: truncate d0/d26/f79 3031267 0 2024-03-16T09:01:09.474 INFO:tasks.workunit.client.0.smithi007.stdout:8/532: chown d0/dc/dd/f2c 67 1 2024-03-16T09:01:09.474 INFO:tasks.workunit.client.0.smithi007.stdout:5/278: dread d1/fa [0,4194304] 0 2024-03-16T09:01:09.475 INFO:tasks.workunit.client.0.smithi007.stdout:8/533: symlink d0/d2/d18/da7/lbe 0 2024-03-16T09:01:09.475 INFO:tasks.workunit.client.0.smithi007.stdout:8/534: write d0/d26/d95/fb9 [318590,47920] 0 2024-03-16T09:01:09.476 INFO:tasks.workunit.client.0.smithi007.stdout:5/279: link d1/d7/d16/d1e/f3d d1/d7/d1c/d19/d34/f66 0 2024-03-16T09:01:09.476 INFO:tasks.workunit.client.0.smithi007.stdout:5/280: dread - d1/d7/d1c/d19/d34/de/f49 zero size 2024-03-16T09:01:09.476 INFO:tasks.workunit.client.0.smithi007.stdout:5/281: creat d1/d7/d16/d1b/f67 x:0 0 0 2024-03-16T09:01:09.482 INFO:tasks.workunit.client.0.smithi007.stdout:5/282: dread d1/d7/d1c/f15 [0,4194304] 0 2024-03-16T09:01:09.484 INFO:tasks.workunit.client.0.smithi007.stdout:5/283: truncate d1/d7/d1c/f17 3727600 0 2024-03-16T09:01:09.484 INFO:tasks.workunit.client.0.smithi007.stdout:5/284: write d1/d7/d1c/d19/d34/d1d/f5a [265968,130853] 0 2024-03-16T09:01:09.484 INFO:tasks.workunit.client.0.smithi007.stdout:5/285: readlink d1/d7/d16/d1e/l65 0 2024-03-16T09:01:09.484 INFO:tasks.workunit.client.0.smithi007.stdout:5/286: chown d1/d7/d1c/d19/d34/l5d 495615 1 2024-03-16T09:01:09.486 INFO:tasks.workunit.client.0.smithi007.stdout:5/287: mknod d1/c68 0 2024-03-16T09:01:09.489 INFO:tasks.workunit.client.0.smithi007.stdout:8/535: dwrite d0/d2/f5d [0,4194304] 0 2024-03-16T09:01:09.494 INFO:tasks.workunit.client.0.smithi007.stdout:5/288: dread d1/f2 [0,4194304] 0 2024-03-16T09:01:09.495 INFO:tasks.workunit.client.0.smithi007.stdout:8/536: write d0/d2/f67 [303130,8551] 0 2024-03-16T09:01:09.496 INFO:tasks.workunit.client.0.smithi007.stdout:8/537: symlink d0/d2/d7e/lbf 0 2024-03-16T09:01:09.499 INFO:tasks.workunit.client.0.smithi007.stdout:1/270: sync 2024-03-16T09:01:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:2/421: sync 2024-03-16T09:01:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:3/350: sync 2024-03-16T09:01:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:7/229: sync 2024-03-16T09:01:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:4/233: sync 2024-03-16T09:01:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:1/271: truncate d9/dd/d1b/d2d/d30/d45/f4a 932090 0 2024-03-16T09:01:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:1/272: readlink d9/dd/d1b/d28/d29/l41 0 2024-03-16T09:01:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:7/230: write d0/d19/f41 [2121350,23070] 0 2024-03-16T09:01:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:9/382: sync 2024-03-16T09:01:09.501 INFO:tasks.workunit.client.0.smithi007.stdout:9/383: readlink l12 0 2024-03-16T09:01:09.501 INFO:tasks.workunit.client.0.smithi007.stdout:0/266: sync 2024-03-16T09:01:09.501 INFO:tasks.workunit.client.0.smithi007.stdout:0/267: creat d3/d9/da/d18/f59 x:0 0 0 2024-03-16T09:01:09.501 INFO:tasks.workunit.client.0.smithi007.stdout:2/422: mknod dc/df/c89 0 2024-03-16T09:01:09.501 INFO:tasks.workunit.client.0.smithi007.stdout:6/258: sync 2024-03-16T09:01:09.505 INFO:tasks.workunit.client.0.smithi007.stdout:5/289: dread d1/d7/d16/d1e/d2a/f57 [0,4194304] 0 2024-03-16T09:01:09.506 INFO:tasks.workunit.client.0.smithi007.stdout:5/290: creat d1/d7/d1c/d19/d34/d3e/f69 x:0 0 0 2024-03-16T09:01:09.507 INFO:tasks.workunit.client.0.smithi007.stdout:9/384: truncate d24/d4c/f4e 716844 0 2024-03-16T09:01:09.508 INFO:tasks.workunit.client.0.smithi007.stdout:8/538: read d0/d2/f5d [2907966,13007] 0 2024-03-16T09:01:09.509 INFO:tasks.workunit.client.0.smithi007.stdout:6/259: link d2/fd d2/d13/d3f/d50/f53 0 2024-03-16T09:01:09.512 INFO:tasks.workunit.client.0.smithi007.stdout:0/268: write d3/d9/da/d48/d25/d43/d1a/f30 [3924309,42970] 0 2024-03-16T09:01:09.514 INFO:tasks.workunit.client.0.smithi007.stdout:5/291: rename d1/d7/d1c/d19/d34/f66 to d1/d7/d1c/d19/d34/d3e/f6a 0 2024-03-16T09:01:09.514 INFO:tasks.workunit.client.0.smithi007.stdout:5/292: chown d1/d7/d1c/d19/d34/d3e/f69 10 1 2024-03-16T09:01:09.517 INFO:tasks.workunit.client.0.smithi007.stdout:8/539: link d0/dc/c4d d0/d26/d2d/d43/d4c/dba/cc0 0 2024-03-16T09:01:09.518 INFO:tasks.workunit.client.0.smithi007.stdout:6/260: mkdir d2/d11/d1e/d54 0 2024-03-16T09:01:09.521 INFO:tasks.workunit.client.0.smithi007.stdout:0/269: symlink d3/d9/da/d18/d27/d34/d3a/l5a 0 2024-03-16T09:01:09.522 INFO:tasks.workunit.client.0.smithi007.stdout:5/293: symlink d1/d7/d1c/d19/d34/l6b 0 2024-03-16T09:01:09.524 INFO:tasks.workunit.client.0.smithi007.stdout:8/540: mknod d0/d26/d2d/d43/d4c/cc1 0 2024-03-16T09:01:09.524 INFO:tasks.workunit.client.0.smithi007.stdout:8/541: truncate d0/d2/d18/f4e 4815300 0 2024-03-16T09:01:09.529 INFO:tasks.workunit.client.0.smithi007.stdout:0/270: dread d3/f6 [0,4194304] 0 2024-03-16T09:01:09.529 INFO:tasks.workunit.client.0.smithi007.stdout:0/271: stat d3/d9/da/d18/d23 0 2024-03-16T09:01:09.529 INFO:tasks.workunit.client.0.smithi007.stdout:0/272: chown d3/d9/da/d48/d4b/f52 337852823 1 2024-03-16T09:01:09.529 INFO:tasks.workunit.client.0.smithi007.stdout:5/294: rename d1/d7/d1c/f17 to d1/d7/d1c/d19/d34/d1d/d36/d4e/f6c 0 2024-03-16T09:01:09.530 INFO:tasks.workunit.client.0.smithi007.stdout:2/423: dread dc/df/f14 [0,4194304] 0 2024-03-16T09:01:09.531 INFO:tasks.workunit.client.1.smithi019.stdout:4/48: sync 2024-03-16T09:01:09.532 INFO:tasks.workunit.client.1.smithi019.stdout:0/58: sync 2024-03-16T09:01:09.532 INFO:tasks.workunit.client.1.smithi019.stdout:4/49: readlink d4/d6/l9 0 2024-03-16T09:01:09.532 INFO:tasks.workunit.client.1.smithi019.stdout:1/17: sync 2024-03-16T09:01:09.532 INFO:tasks.workunit.client.1.smithi019.stdout:8/73: sync 2024-03-16T09:01:09.535 INFO:tasks.workunit.client.1.smithi019.stdout:4/50: symlink d4/l11 0 2024-03-16T09:01:09.536 INFO:tasks.workunit.client.1.smithi019.stdout:1/18: mknod c2 0 2024-03-16T09:01:09.536 INFO:tasks.workunit.client.1.smithi019.stdout:1/19: dread - no filename 2024-03-16T09:01:09.536 INFO:tasks.workunit.client.1.smithi019.stdout:1/20: write - no filename 2024-03-16T09:01:09.536 INFO:tasks.workunit.client.1.smithi019.stdout:1/21: write - no filename 2024-03-16T09:01:09.537 INFO:tasks.workunit.client.1.smithi019.stdout:1/22: fdatasync - no filename 2024-03-16T09:01:09.537 INFO:tasks.workunit.client.1.smithi019.stdout:1/23: fsync - no filename 2024-03-16T09:01:09.537 INFO:tasks.workunit.client.0.smithi007.stdout:2/424: write dc/df/d24/d28/d4c/f67 [300709,31480] 0 2024-03-16T09:01:09.538 INFO:tasks.workunit.client.1.smithi019.stdout:0/59: getdents d7/d8 0 2024-03-16T09:01:09.538 INFO:tasks.workunit.client.1.smithi019.stdout:0/60: write d7/d8/f12 [392635,52733] 0 2024-03-16T09:01:09.539 INFO:tasks.workunit.client.1.smithi019.stdout:4/51: creat d4/f12 x:0 0 0 2024-03-16T09:01:09.539 INFO:tasks.workunit.client.1.smithi019.stdout:4/52: read - d4/f12 zero size 2024-03-16T09:01:09.539 INFO:tasks.workunit.client.1.smithi019.stdout:4/53: read - d4/f12 zero size 2024-03-16T09:01:09.540 INFO:tasks.workunit.client.1.smithi019.stdout:1/24: mknod c3 0 2024-03-16T09:01:09.540 INFO:tasks.workunit.client.1.smithi019.stdout:1/25: fdatasync - no filename 2024-03-16T09:01:09.544 INFO:tasks.workunit.client.1.smithi019.stdout:6/149: sync 2024-03-16T09:01:09.544 INFO:tasks.workunit.client.1.smithi019.stdout:7/35: sync 2024-03-16T09:01:09.544 INFO:tasks.workunit.client.1.smithi019.stdout:2/7: sync 2024-03-16T09:01:09.544 INFO:tasks.workunit.client.1.smithi019.stdout:5/41: sync 2024-03-16T09:01:09.545 INFO:tasks.workunit.client.1.smithi019.stdout:9/32: sync 2024-03-16T09:01:09.545 INFO:tasks.workunit.client.1.smithi019.stdout:3/41: sync 2024-03-16T09:01:09.545 INFO:tasks.workunit.client.1.smithi019.stdout:7/36: creat f8 x:0 0 0 2024-03-16T09:01:09.545 INFO:tasks.workunit.client.1.smithi019.stdout:7/37: dread - f8 zero size 2024-03-16T09:01:09.547 INFO:tasks.workunit.client.1.smithi019.stdout:2/8: unlink c0 0 2024-03-16T09:01:09.547 INFO:tasks.workunit.client.1.smithi019.stdout:2/9: creat f3 x:0 0 0 2024-03-16T09:01:09.547 INFO:tasks.workunit.client.1.smithi019.stdout:2/10: write f2 [766998,95420] 0 2024-03-16T09:01:09.547 INFO:tasks.workunit.client.1.smithi019.stdout:2/11: creat f4 x:0 0 0 2024-03-16T09:01:09.549 INFO:tasks.workunit.client.1.smithi019.stdout:5/42: creat d6/f8 x:0 0 0 2024-03-16T09:01:09.549 INFO:tasks.workunit.client.1.smithi019.stdout:5/43: creat d6/f9 x:0 0 0 2024-03-16T09:01:09.550 INFO:tasks.workunit.client.1.smithi019.stdout:9/33: creat d1/d7/fc x:0 0 0 2024-03-16T09:01:09.551 INFO:tasks.workunit.client.1.smithi019.stdout:3/42: write f2 [1695936,69422] 0 2024-03-16T09:01:09.553 INFO:tasks.workunit.client.1.smithi019.stdout:6/150: link ca de/d26/d32/c35 0 2024-03-16T09:01:09.555 INFO:tasks.workunit.client.1.smithi019.stdout:2/12: mkdir d5 0 2024-03-16T09:01:09.556 INFO:tasks.workunit.client.1.smithi019.stdout:5/44: mknod d6/ca 0 2024-03-16T09:01:09.556 INFO:tasks.workunit.client.1.smithi019.stdout:5/45: fdatasync d6/f7 0 2024-03-16T09:01:09.556 INFO:tasks.workunit.client.1.smithi019.stdout:5/46: chown d6 14719516 1 2024-03-16T09:01:09.557 INFO:tasks.workunit.client.0.smithi007.stdout:4/234: dwrite d4/d14/d1c/f30 [0,4194304] 0 2024-03-16T09:01:09.558 INFO:tasks.workunit.client.1.smithi019.stdout:6/151: creat de/d1c/d1d/f36 x:0 0 0 2024-03-16T09:01:09.559 INFO:tasks.workunit.client.0.smithi007.stdout:4/235: symlink d4/d14/l4f 0 2024-03-16T09:01:09.559 INFO:tasks.workunit.client.0.smithi007.stdout:4/236: chown d4/d14/f35 1338 1 2024-03-16T09:01:09.559 INFO:tasks.workunit.client.0.smithi007.stdout:4/237: truncate d4/d14/f1b 243195 0 2024-03-16T09:01:09.559 INFO:tasks.workunit.client.1.smithi019.stdout:2/13: mkdir d5/d6 0 2024-03-16T09:01:09.560 INFO:tasks.workunit.client.1.smithi019.stdout:2/14: write f2 [1473019,13580] 0 2024-03-16T09:01:09.560 INFO:tasks.workunit.client.1.smithi019.stdout:9/34: getdents d1 0 2024-03-16T09:01:09.561 INFO:tasks.workunit.client.1.smithi019.stdout:2/15: creat d5/f7 x:0 0 0 2024-03-16T09:01:09.561 INFO:tasks.workunit.client.1.smithi019.stdout:2/16: chown f4 1769 1 2024-03-16T09:01:09.562 INFO:tasks.workunit.client.1.smithi019.stdout:7/38: write f7 [1705041,86576] 0 2024-03-16T09:01:09.563 INFO:tasks.workunit.client.1.smithi019.stdout:9/35: link f0 d1/d7/fd 0 2024-03-16T09:01:09.564 INFO:tasks.workunit.client.1.smithi019.stdout:3/43: mknod d3/c8 0 2024-03-16T09:01:09.566 INFO:tasks.workunit.client.1.smithi019.stdout:2/17: link f1 d5/d6/f8 0 2024-03-16T09:01:09.566 INFO:tasks.workunit.client.1.smithi019.stdout:2/18: write f2 [2297495,41217] 0 2024-03-16T09:01:09.567 INFO:tasks.workunit.client.1.smithi019.stdout:5/47: dread f3 [0,4194304] 0 2024-03-16T09:01:09.568 INFO:tasks.workunit.client.1.smithi019.stdout:7/39: mkdir d9 0 2024-03-16T09:01:09.568 INFO:tasks.workunit.client.1.smithi019.stdout:7/40: write f7 [477003,127414] 0 2024-03-16T09:01:09.568 INFO:tasks.workunit.client.1.smithi019.stdout:7/41: chown f7 10 1 2024-03-16T09:01:09.569 INFO:tasks.workunit.client.0.smithi007.stdout:3/351: dwrite d15/d18/d1f/f58 [0,4194304] 0 2024-03-16T09:01:09.569 INFO:tasks.workunit.client.0.smithi007.stdout:9/385: dwrite d24/f2a [0,4194304] 0 2024-03-16T09:01:09.570 INFO:tasks.workunit.client.1.smithi019.stdout:9/36: unlink d1/l8 0 2024-03-16T09:01:09.573 INFO:tasks.workunit.client.0.smithi007.stdout:3/352: mknod d15/c76 0 2024-03-16T09:01:09.573 INFO:tasks.workunit.client.0.smithi007.stdout:3/353: chown d15/d18/f4f 48179 1 2024-03-16T09:01:09.573 INFO:tasks.workunit.client.1.smithi019.stdout:2/19: mkdir d5/d6/d9 0 2024-03-16T09:01:09.573 INFO:tasks.workunit.client.1.smithi019.stdout:2/20: write d5/f7 [344322,114097] 0 2024-03-16T09:01:09.573 INFO:tasks.workunit.client.1.smithi019.stdout:2/21: truncate f1 4434066 0 2024-03-16T09:01:09.574 INFO:tasks.workunit.client.1.smithi019.stdout:5/48: link d6/f9 d6/fb 0 2024-03-16T09:01:09.575 INFO:tasks.workunit.client.1.smithi019.stdout:5/49: readlink l5 0 2024-03-16T09:01:09.575 INFO:tasks.workunit.client.0.smithi007.stdout:1/273: dwrite d9/f48 [0,4194304] 0 2024-03-16T09:01:09.575 INFO:tasks.workunit.client.0.smithi007.stdout:1/274: dread - d9/dd/d1b/d2d/f3f zero size 2024-03-16T09:01:09.575 INFO:tasks.workunit.client.0.smithi007.stdout:1/275: chown d9/dd/d1b/d2d/d30/d45 6833 1 2024-03-16T09:01:09.575 INFO:tasks.workunit.client.0.smithi007.stdout:1/276: readlink d9/dd/d1b/d2d/d30/l4b 0 2024-03-16T09:01:09.577 INFO:tasks.workunit.client.1.smithi019.stdout:3/44: symlink d3/l9 0 2024-03-16T09:01:09.578 INFO:tasks.workunit.client.1.smithi019.stdout:3/45: chown d3/f5 166225972 1 2024-03-16T09:01:09.578 INFO:tasks.workunit.client.1.smithi019.stdout:2/22: mknod d5/ca 0 2024-03-16T09:01:09.578 INFO:tasks.workunit.client.1.smithi019.stdout:2/23: stat d5/d6/d9 0 2024-03-16T09:01:09.578 INFO:tasks.workunit.client.1.smithi019.stdout:6/152: dread de/f25 [0,4194304] 0 2024-03-16T09:01:09.578 INFO:tasks.workunit.client.0.smithi007.stdout:8/542: dwrite d0/d2/fa [0,4194304] 0 2024-03-16T09:01:09.579 INFO:tasks.workunit.client.0.smithi007.stdout:0/273: dwrite d3/d9/da/d48/d25/d43/d1a/f2d [0,4194304] 0 2024-03-16T09:01:09.579 INFO:tasks.workunit.client.1.smithi019.stdout:9/37: dread f0 [0,4194304] 0 2024-03-16T09:01:09.580 INFO:tasks.workunit.client.0.smithi007.stdout:8/543: creat d0/d2/d5f/fc2 x:0 0 0 2024-03-16T09:01:09.581 INFO:tasks.workunit.client.1.smithi019.stdout:8/74: dwrite f5 [0,4194304] 0 2024-03-16T09:01:09.581 INFO:tasks.workunit.client.1.smithi019.stdout:8/75: readlink d6/lc 0 2024-03-16T09:01:09.582 INFO:tasks.workunit.client.1.smithi019.stdout:8/76: chown d6/de/f1b 3 1 2024-03-16T09:01:09.585 INFO:tasks.workunit.client.0.smithi007.stdout:4/238: read d4/d7/f1d [154128,3616] 0 2024-03-16T09:01:09.585 INFO:tasks.workunit.client.0.smithi007.stdout:4/239: chown d4/d7/f20 21209 1 2024-03-16T09:01:09.585 INFO:tasks.workunit.client.0.smithi007.stdout:0/274: unlink d3/d9/f11 0 2024-03-16T09:01:09.585 INFO:tasks.workunit.client.0.smithi007.stdout:5/295: dwrite d1/d7/d16/d1e/f31 [0,4194304] 0 2024-03-16T09:01:09.586 INFO:tasks.workunit.client.0.smithi007.stdout:8/544: dread d0/dc/d54/f98 [0,4194304] 0 2024-03-16T09:01:09.586 INFO:tasks.workunit.client.0.smithi007.stdout:8/545: dread - d0/d2/d18/f52 zero size 2024-03-16T09:01:09.586 INFO:tasks.workunit.client.1.smithi019.stdout:2/24: unlink f3 0 2024-03-16T09:01:09.586 INFO:tasks.workunit.client.1.smithi019.stdout:2/25: readlink - no filename 2024-03-16T09:01:09.586 INFO:tasks.workunit.client.1.smithi019.stdout:2/26: write f2 [3014431,10029] 0 2024-03-16T09:01:09.586 INFO:tasks.workunit.client.1.smithi019.stdout:2/27: creat d5/fb x:0 0 0 2024-03-16T09:01:09.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/277: dread d9/dd/d1b/d28/f3e [0,4194304] 0 2024-03-16T09:01:09.590 INFO:tasks.workunit.client.0.smithi007.stdout:1/278: creat d9/dd/d1b/d2d/d30/d45/f51 x:0 0 0 2024-03-16T09:01:09.590 INFO:tasks.workunit.client.1.smithi019.stdout:6/153: creat de/d26/d32/f37 x:0 0 0 2024-03-16T09:01:09.590 INFO:tasks.workunit.client.1.smithi019.stdout:6/154: chown de/c19 1022901 1 2024-03-16T09:01:09.590 INFO:tasks.workunit.client.1.smithi019.stdout:6/155: creat de/d1c/d1d/f38 x:0 0 0 2024-03-16T09:01:09.591 INFO:tasks.workunit.client.0.smithi007.stdout:1/279: read d9/dd/d1b/d2d/d30/d45/f4a [451824,93216] 0 2024-03-16T09:01:09.591 INFO:tasks.workunit.client.0.smithi007.stdout:1/280: dread - d9/dd/d1b/d2d/f3f zero size 2024-03-16T09:01:09.591 INFO:tasks.workunit.client.0.smithi007.stdout:1/281: stat d9/dd/d1b/f49 0 2024-03-16T09:01:09.592 INFO:tasks.workunit.client.0.smithi007.stdout:0/275: mkdir d3/d9/db/d2c/d5b 0 2024-03-16T09:01:09.593 INFO:tasks.workunit.client.0.smithi007.stdout:0/276: readlink d3/d9/db/d1c/l50 0 2024-03-16T09:01:09.593 INFO:tasks.workunit.client.0.smithi007.stdout:0/277: dread - d3/d9/da/d18/d27/d34/d3a/f4f zero size 2024-03-16T09:01:09.593 INFO:tasks.workunit.client.1.smithi019.stdout:3/46: rename d3/c4 to d3/ca 0 2024-03-16T09:01:09.593 INFO:tasks.workunit.client.1.smithi019.stdout:3/47: write f2 [1316835,107980] 0 2024-03-16T09:01:09.595 INFO:tasks.workunit.client.1.smithi019.stdout:9/38: stat d1/l9 0 2024-03-16T09:01:09.595 INFO:tasks.workunit.client.0.smithi007.stdout:8/546: mkdir d0/dc/d75/dc3 0 2024-03-16T09:01:09.595 INFO:tasks.workunit.client.0.smithi007.stdout:5/296: mknod d1/d7/d1c/d19/d34/d3e/d62/c6d 0 2024-03-16T09:01:09.595 INFO:tasks.workunit.client.0.smithi007.stdout:8/547: write d0/d26/d2d/d43/d4c/f78 [4270151,82386] 0 2024-03-16T09:01:09.597 INFO:tasks.workunit.client.1.smithi019.stdout:4/54: fsync d4/f12 0 2024-03-16T09:01:09.598 INFO:tasks.workunit.client.1.smithi019.stdout:4/55: dread - d4/f12 zero size 2024-03-16T09:01:09.598 INFO:tasks.workunit.client.1.smithi019.stdout:4/56: stat d4/ce 0 2024-03-16T09:01:09.598 INFO:tasks.workunit.client.1.smithi019.stdout:1/26: getdents . 0 2024-03-16T09:01:09.598 INFO:tasks.workunit.client.1.smithi019.stdout:1/27: chown c2 30 1 2024-03-16T09:01:09.598 INFO:tasks.workunit.client.1.smithi019.stdout:1/28: truncate - no filename 2024-03-16T09:01:09.598 INFO:tasks.workunit.client.1.smithi019.stdout:1/29: dread - no filename 2024-03-16T09:01:09.599 INFO:tasks.workunit.client.0.smithi007.stdout:4/240: getdents d4/d14 0 2024-03-16T09:01:09.599 INFO:tasks.workunit.client.0.smithi007.stdout:4/241: write d4/f28 [5112437,25030] 0 2024-03-16T09:01:09.600 INFO:tasks.workunit.client.1.smithi019.stdout:2/28: rmdir d5/d6 39 2024-03-16T09:01:09.600 INFO:tasks.workunit.client.1.smithi019.stdout:2/29: write d5/fb [694081,88419] 0 2024-03-16T09:01:09.600 INFO:tasks.workunit.client.1.smithi019.stdout:2/30: getdents d5/d6/d9 0 2024-03-16T09:01:09.601 INFO:tasks.workunit.client.0.smithi007.stdout:9/386: dwrite d24/d4a/f66 [0,4194304] 0 2024-03-16T09:01:09.601 INFO:tasks.workunit.client.0.smithi007.stdout:8/548: read d0/d2/d74/f96 [851400,109015] 0 2024-03-16T09:01:09.603 INFO:tasks.workunit.client.0.smithi007.stdout:3/354: dwrite d15/d18/d1f/d29/d43/f55 [0,4194304] 0 2024-03-16T09:01:09.603 INFO:tasks.workunit.client.0.smithi007.stdout:3/355: fsync d15/d18/d1f/d29/f31 0 2024-03-16T09:01:09.606 INFO:tasks.workunit.client.0.smithi007.stdout:3/356: mkdir d15/d18/d1f/d77 0 2024-03-16T09:01:09.606 INFO:tasks.workunit.client.0.smithi007.stdout:3/357: readlink d15/d18/d1f/l6b 0 2024-03-16T09:01:09.606 INFO:tasks.workunit.client.0.smithi007.stdout:3/358: write d15/d18/f23 [1054688,87475] 0 2024-03-16T09:01:09.608 INFO:tasks.workunit.client.1.smithi019.stdout:9/39: unlink f0 0 2024-03-16T09:01:09.608 INFO:tasks.workunit.client.1.smithi019.stdout:7/42: dwrite f8 [0,4194304] 0 2024-03-16T09:01:09.608 INFO:tasks.workunit.client.1.smithi019.stdout:9/40: write d1/d7/fc [891525,43738] 0 2024-03-16T09:01:09.608 INFO:tasks.workunit.client.1.smithi019.stdout:7/43: write f6 [1518521,124845] 0 2024-03-16T09:01:09.608 INFO:tasks.workunit.client.1.smithi019.stdout:8/77: dwrite d6/de/d16/d19/f1c [0,4194304] 0 2024-03-16T09:01:09.610 INFO:tasks.workunit.client.0.smithi007.stdout:3/359: mkdir d15/d18/d6a/d78 0 2024-03-16T09:01:09.610 INFO:tasks.workunit.client.0.smithi007.stdout:5/297: dread d1/d7/d1c/d19/d34/d3e/d41/f4c [0,4194304] 0 2024-03-16T09:01:09.612 INFO:tasks.workunit.client.1.smithi019.stdout:2/31: fsync f4 0 2024-03-16T09:01:09.612 INFO:tasks.workunit.client.1.smithi019.stdout:2/32: chown d5/f7 153370 1 2024-03-16T09:01:09.612 INFO:tasks.workunit.client.1.smithi019.stdout:2/33: write f4 [190114,82743] 0 2024-03-16T09:01:09.612 INFO:tasks.workunit.client.1.smithi019.stdout:2/34: readlink - no filename 2024-03-16T09:01:09.612 INFO:tasks.workunit.client.1.smithi019.stdout:6/156: rename de/d1c/d1d/f38 to de/d26/f39 0 2024-03-16T09:01:09.612 INFO:tasks.workunit.client.1.smithi019.stdout:4/57: rename d4 to d4/d6/d7/d13 22 2024-03-16T09:01:09.612 INFO:tasks.workunit.client.1.smithi019.stdout:1/30: mknod c4 0 2024-03-16T09:01:09.614 INFO:tasks.workunit.client.1.smithi019.stdout:9/41: dread d1/f3 [0,4194304] 0 2024-03-16T09:01:09.614 INFO:tasks.workunit.client.1.smithi019.stdout:9/42: chown d1/d7/la 862 1 2024-03-16T09:01:09.614 INFO:tasks.workunit.client.1.smithi019.stdout:9/43: chown d1/d7/fc 642666866 1 2024-03-16T09:01:09.614 INFO:tasks.workunit.client.1.smithi019.stdout:3/48: dread f2 [0,4194304] 0 2024-03-16T09:01:09.616 INFO:tasks.workunit.client.1.smithi019.stdout:7/44: mkdir d9/da 0 2024-03-16T09:01:09.616 INFO:tasks.workunit.client.1.smithi019.stdout:9/44: dread d1/f3 [0,4194304] 0 2024-03-16T09:01:09.619 INFO:tasks.workunit.client.0.smithi007.stdout:8/549: dread d0/d26/d95/f9b [0,4194304] 0 2024-03-16T09:01:09.619 INFO:tasks.workunit.client.1.smithi019.stdout:8/78: mknod d6/c1e 0 2024-03-16T09:01:09.620 INFO:tasks.workunit.client.1.smithi019.stdout:5/50: dwrite f3 [4194304,4194304] 0 2024-03-16T09:01:09.621 INFO:tasks.workunit.client.1.smithi019.stdout:2/35: creat d5/d6/d9/fc x:0 0 0 2024-03-16T09:01:09.621 INFO:tasks.workunit.client.1.smithi019.stdout:2/36: chown f2 934010 1 2024-03-16T09:01:09.621 INFO:tasks.workunit.client.1.smithi019.stdout:2/37: creat d5/d6/d9/fd x:0 0 0 2024-03-16T09:01:09.621 INFO:tasks.workunit.client.1.smithi019.stdout:2/38: stat d5 0 2024-03-16T09:01:09.622 INFO:tasks.workunit.client.0.smithi007.stdout:8/550: link d0/d2/d18/da7/lbe d0/dc/dd/d8b/lc4 0 2024-03-16T09:01:09.622 INFO:tasks.workunit.client.0.smithi007.stdout:8/551: write d0/f7b [947292,25518] 0 2024-03-16T09:01:09.622 INFO:tasks.workunit.client.0.smithi007.stdout:8/552: stat d0/dc/d50/d8e/fa8 0 2024-03-16T09:01:09.623 INFO:tasks.workunit.client.1.smithi019.stdout:6/157: rename de/d1c/d1d/f24 to de/d1c/d1d/d2c/f3a 0 2024-03-16T09:01:09.623 INFO:tasks.workunit.client.0.smithi007.stdout:8/553: rmdir d0/dc/dd/d8b/d9e 39 2024-03-16T09:01:09.625 INFO:tasks.workunit.client.0.smithi007.stdout:8/554: mkdir d0/d2/d18/da7/dc5 0 2024-03-16T09:01:09.625 INFO:tasks.workunit.client.1.smithi019.stdout:4/58: symlink d4/d6/l14 0 2024-03-16T09:01:09.626 INFO:tasks.workunit.client.0.smithi007.stdout:8/555: rmdir d0/d26/d2d/d43/d72 39 2024-03-16T09:01:09.628 INFO:tasks.workunit.client.1.smithi019.stdout:1/31: chown c2 171 1 2024-03-16T09:01:09.628 INFO:tasks.workunit.client.1.smithi019.stdout:1/32: dwrite - no filename 2024-03-16T09:01:09.628 INFO:tasks.workunit.client.1.smithi019.stdout:1/33: stat c4 0 2024-03-16T09:01:09.631 INFO:tasks.workunit.client.1.smithi019.stdout:7/45: mkdir d9/db 0 2024-03-16T09:01:09.633 INFO:tasks.workunit.client.1.smithi019.stdout:9/45: mkdir d1/d7/de 0 2024-03-16T09:01:09.633 INFO:tasks.workunit.client.1.smithi019.stdout:6/158: dread de/d1c/f28 [0,4194304] 0 2024-03-16T09:01:09.635 INFO:tasks.workunit.client.1.smithi019.stdout:8/79: unlink d6/cd 0 2024-03-16T09:01:09.635 INFO:tasks.workunit.client.1.smithi019.stdout:8/80: dread - d6/f15 zero size 2024-03-16T09:01:09.636 INFO:tasks.workunit.client.0.smithi007.stdout:0/278: dwrite d3/d9/da/d48/d25/d43/d1a/f1b [0,4194304] 0 2024-03-16T09:01:09.638 INFO:tasks.workunit.client.0.smithi007.stdout:0/279: creat d3/d9/d31/f5c x:0 0 0 2024-03-16T09:01:09.638 INFO:tasks.workunit.client.0.smithi007.stdout:0/280: dread - d3/d9/da/d48/d4b/f51 zero size 2024-03-16T09:01:09.638 INFO:tasks.workunit.client.1.smithi019.stdout:5/51: symlink d6/lc 0 2024-03-16T09:01:09.639 INFO:tasks.workunit.client.0.smithi007.stdout:0/281: mknod d3/d9/d31/c5d 0 2024-03-16T09:01:09.639 INFO:tasks.workunit.client.0.smithi007.stdout:0/282: creat d3/d9/db/d2c/f5e x:0 0 0 2024-03-16T09:01:09.640 INFO:tasks.workunit.client.0.smithi007.stdout:0/283: symlink d3/d9/da/d18/d27/d34/d3a/l5f 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:1/282: dwrite f7 [0,4194304] 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:1/283: write d9/dd/d1b/d2d/d30/d45/f51 [449005,113449] 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:1/284: fdatasync d9/f22 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:1/285: creat d9/dd/d1b/d2d/d30/f52 x:0 0 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:1/286: creat d9/dd/d1b/d2d/f53 x:0 0 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:8/556: dwrite d0/d26/fa1 [0,4194304] 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:8/557: fsync d0/d2/f5d 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:9/387: dwrite d24/f50 [0,4194304] 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:1/287: dread d9/f35 [0,4194304] 0 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:1/288: dread - d9/dd/d1b/f49 zero size 2024-03-16T09:01:09.647 INFO:tasks.workunit.client.0.smithi007.stdout:3/360: dwrite d15/d18/f35 [0,4194304] 0 2024-03-16T09:01:09.648 INFO:tasks.workunit.client.0.smithi007.stdout:3/361: dread - d15/d18/d1f/d3b/f44 zero size 2024-03-16T09:01:09.648 INFO:tasks.workunit.client.0.smithi007.stdout:3/362: write fa [1859410,23046] 0 2024-03-16T09:01:09.648 INFO:tasks.workunit.client.0.smithi007.stdout:8/558: mkdir d0/d26/d2d/d43/dc6 0 2024-03-16T09:01:09.649 INFO:tasks.workunit.client.1.smithi019.stdout:3/49: getdents d3 0 2024-03-16T09:01:09.649 INFO:tasks.workunit.client.1.smithi019.stdout:4/59: dwrite f2 [0,4194304] 0 2024-03-16T09:01:09.649 INFO:tasks.workunit.client.0.smithi007.stdout:1/289: truncate d9/dd/d1b/d28/f32 2158405 0 2024-03-16T09:01:09.650 INFO:tasks.workunit.client.1.smithi019.stdout:6/159: rmdir de/d1c/d1d/d2c/d2f 0 2024-03-16T09:01:09.650 INFO:tasks.workunit.client.1.smithi019.stdout:6/160: fsync fb 0 2024-03-16T09:01:09.650 INFO:tasks.workunit.client.1.smithi019.stdout:6/161: write de/f25 [6110459,30899] 0 2024-03-16T09:01:09.650 INFO:tasks.workunit.client.1.smithi019.stdout:6/162: truncate de/d1c/f2e 923031 0 2024-03-16T09:01:09.652 INFO:tasks.workunit.client.1.smithi019.stdout:8/81: mknod d6/de/d16/d19/c1f 0 2024-03-16T09:01:09.652 INFO:tasks.workunit.client.0.smithi007.stdout:9/388: dread d24/f29 [0,4194304] 0 2024-03-16T09:01:09.655 INFO:tasks.workunit.client.0.smithi007.stdout:3/363: rename d15/d18/d1f/d29/l3c to d15/l79 0 2024-03-16T09:01:09.656 INFO:tasks.workunit.client.1.smithi019.stdout:5/52: mkdir d6/dd 0 2024-03-16T09:01:09.659 INFO:tasks.workunit.client.0.smithi007.stdout:9/389: dread f16 [0,4194304] 0 2024-03-16T09:01:09.662 INFO:tasks.workunit.client.0.smithi007.stdout:0/284: truncate d3/d9/da/d48/d25/d43/d1a/f2d 4156515 0 2024-03-16T09:01:09.663 INFO:tasks.workunit.client.0.smithi007.stdout:0/285: mkdir d3/d9/db/d60 0 2024-03-16T09:01:09.663 INFO:tasks.workunit.client.0.smithi007.stdout:0/286: chown d3/d9/da/d48/d25/d43/c14 74 1 2024-03-16T09:01:09.664 INFO:tasks.workunit.client.0.smithi007.stdout:0/287: mkdir d3/d9/da/d48/d25/d43/d1a/d58/d61 0 2024-03-16T09:01:09.666 INFO:tasks.workunit.client.0.smithi007.stdout:3/364: dread d15/d18/d1f/d3b/d63/f66 [0,4194304] 0 2024-03-16T09:01:09.668 INFO:tasks.workunit.client.0.smithi007.stdout:0/288: rename d3/d9/d31 to d3/d9/db/d1c/d62 0 2024-03-16T09:01:09.668 INFO:tasks.workunit.client.0.smithi007.stdout:0/289: stat d3/d9/da/d18/d23 0 2024-03-16T09:01:09.668 INFO:tasks.workunit.client.0.smithi007.stdout:0/290: fsync d3/d9/db/d2c/f2f 0 2024-03-16T09:01:09.668 INFO:tasks.workunit.client.0.smithi007.stdout:0/291: chown d3/d9/f26 271 1 2024-03-16T09:01:09.668 INFO:tasks.workunit.client.0.smithi007.stdout:0/292: creat d3/d9/da/d18/d23/f63 x:0 0 0 2024-03-16T09:01:09.670 INFO:tasks.workunit.client.0.smithi007.stdout:0/293: mknod d3/d9/da/d18/d23/c64 0 2024-03-16T09:01:09.670 INFO:tasks.workunit.client.0.smithi007.stdout:0/294: dread - d3/d9/da/d48/f47 zero size 2024-03-16T09:01:09.670 INFO:tasks.workunit.client.0.smithi007.stdout:0/295: read - d3/f22 zero size 2024-03-16T09:01:09.671 INFO:tasks.workunit.client.0.smithi007.stdout:0/296: mknod d3/d9/da/d48/d25/d43/d1a/d58/d61/c65 0 2024-03-16T09:01:09.672 INFO:tasks.workunit.client.0.smithi007.stdout:0/297: truncate d3/d9/da/d48/d25/d43/d1a/f1b 1340925 0 2024-03-16T09:01:09.673 INFO:tasks.workunit.client.0.smithi007.stdout:0/298: read d3/d9/da/d48/d25/d43/d1a/f20 [1045633,112342] 0 2024-03-16T09:01:09.674 INFO:tasks.workunit.client.0.smithi007.stdout:0/299: readlink d3/d9/da/d48/d25/d43/l2e 0 2024-03-16T09:01:09.674 INFO:tasks.workunit.client.0.smithi007.stdout:0/300: truncate d3/d9/da/d18/d23/f4d 399845 0 2024-03-16T09:01:09.674 INFO:tasks.workunit.client.0.smithi007.stdout:0/301: chown d3/d9/db/d1c/d62/l3b 1338750 1 2024-03-16T09:01:09.674 INFO:tasks.workunit.client.0.smithi007.stdout:0/302: chown d3/d9/db/d2c/f5e 13235 1 2024-03-16T09:01:09.674 INFO:tasks.workunit.client.0.smithi007.stdout:0/303: mknod d3/d9/da/d18/c66 0 2024-03-16T09:01:09.675 INFO:tasks.workunit.client.1.smithi019.stdout:4/60: getdents d4/d6/d7 0 2024-03-16T09:01:09.675 INFO:tasks.workunit.client.1.smithi019.stdout:6/163: mknod de/c3b 0 2024-03-16T09:01:09.675 INFO:tasks.workunit.client.1.smithi019.stdout:6/164: creat de/f3c x:0 0 0 2024-03-16T09:01:09.675 INFO:tasks.workunit.client.0.smithi007.stdout:0/304: creat d3/d9/da/d18/d27/d34/d55/f67 x:0 0 0 2024-03-16T09:01:09.676 INFO:tasks.workunit.client.1.smithi019.stdout:8/82: symlink d6/l20 0 2024-03-16T09:01:09.684 INFO:tasks.workunit.client.0.smithi007.stdout:3/365: read f10 [726506,2411] 0 2024-03-16T09:01:09.687 INFO:tasks.workunit.client.0.smithi007.stdout:1/290: dwrite d9/f12 [0,4194304] 0 2024-03-16T09:01:09.687 INFO:tasks.workunit.client.1.smithi019.stdout:3/50: dwrite f2 [0,4194304] 0 2024-03-16T09:01:09.687 INFO:tasks.workunit.client.1.smithi019.stdout:9/46: dwrite d1/f3 [0,4194304] 0 2024-03-16T09:01:09.688 INFO:tasks.workunit.client.1.smithi019.stdout:4/61: dread f2 [0,4194304] 0 2024-03-16T09:01:09.689 INFO:tasks.workunit.client.1.smithi019.stdout:4/62: chown d4/d6/d7/cf 1460 1 2024-03-16T09:01:09.689 INFO:tasks.workunit.client.0.smithi007.stdout:8/559: dwrite d0/d2/f67 [0,4194304] 0 2024-03-16T09:01:09.689 INFO:tasks.workunit.client.1.smithi019.stdout:3/51: read d3/f5 [655906,100819] 0 2024-03-16T09:01:09.690 INFO:tasks.workunit.client.0.smithi007.stdout:1/291: link d9/l20 d9/dd/d1b/d2d/d30/l54 0 2024-03-16T09:01:09.694 INFO:tasks.workunit.client.1.smithi019.stdout:4/63: write f2 [347637,8324] 0 2024-03-16T09:01:09.694 INFO:tasks.workunit.client.0.smithi007.stdout:8/560: write d0/f3c [2294836,74879] 0 2024-03-16T09:01:09.695 INFO:tasks.workunit.client.0.smithi007.stdout:1/292: creat d9/dd/d1b/d28/d4c/f55 x:0 0 0 2024-03-16T09:01:09.695 INFO:tasks.workunit.client.0.smithi007.stdout:1/293: write d9/dd/f14 [763390,5286] 0 2024-03-16T09:01:09.702 INFO:tasks.workunit.client.0.smithi007.stdout:1/294: mknod d9/dd/d1b/d28/d4c/c56 0 2024-03-16T09:01:09.702 INFO:tasks.workunit.client.0.smithi007.stdout:1/295: stat d9/dd/d1b/d28/f36 0 2024-03-16T09:01:09.702 INFO:tasks.workunit.client.0.smithi007.stdout:1/296: creat d9/dd/d1b/d2d/f57 x:0 0 0 2024-03-16T09:01:09.707 INFO:tasks.workunit.client.0.smithi007.stdout:1/297: mkdir d9/dd/d1b/d58 0 2024-03-16T09:01:09.708 INFO:tasks.workunit.client.0.smithi007.stdout:8/561: dread d0/d26/d95/f9b [0,4194304] 0 2024-03-16T09:01:09.708 INFO:tasks.workunit.client.0.smithi007.stdout:8/562: chown d0/l73 0 1 2024-03-16T09:01:09.708 INFO:tasks.workunit.client.0.smithi007.stdout:1/298: creat d9/dd/f59 x:0 0 0 2024-03-16T09:01:09.709 INFO:tasks.workunit.client.0.smithi007.stdout:8/563: creat d0/dc/d54/da4/fc7 x:0 0 0 2024-03-16T09:01:09.709 INFO:tasks.workunit.client.0.smithi007.stdout:1/299: symlink d9/dd/d1b/d2d/l5a 0 2024-03-16T09:01:09.710 INFO:tasks.workunit.client.0.smithi007.stdout:1/300: creat d9/dd/d1b/d2d/d30/d45/f5b x:0 0 0 2024-03-16T09:01:09.711 INFO:tasks.workunit.client.0.smithi007.stdout:8/564: creat d0/d26/d2d/d43/d6f/da5/fc8 x:0 0 0 2024-03-16T09:01:09.721 INFO:tasks.workunit.client.0.smithi007.stdout:0/305: dwrite d3/d9/da/f4c [0,4194304] 0 2024-03-16T09:01:09.722 INFO:tasks.workunit.client.1.smithi019.stdout:6/165: dwrite de/f1a [0,4194304] 0 2024-03-16T09:01:09.722 INFO:tasks.workunit.client.1.smithi019.stdout:6/166: fsync de/d1c/f27 0 2024-03-16T09:01:09.722 INFO:tasks.workunit.client.1.smithi019.stdout:6/167: fsync de/f1a 0 2024-03-16T09:01:09.722 INFO:tasks.workunit.client.1.smithi019.stdout:6/168: readlink de/d1c/l2d 0 2024-03-16T09:01:09.722 INFO:tasks.workunit.client.1.smithi019.stdout:6/169: dread - de/d1c/f27 zero size 2024-03-16T09:01:09.723 INFO:tasks.workunit.client.0.smithi007.stdout:8/565: write d0/d2/fa [2054577,30069] 0 2024-03-16T09:01:09.723 INFO:tasks.workunit.client.0.smithi007.stdout:0/306: rename d3/d9/f26 to d3/d9/da/d48/d25/f68 0 2024-03-16T09:01:09.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/366: dwrite d15/d18/d1f/d29/d43/d72/f50 [0,4194304] 0 2024-03-16T09:01:09.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/367: write fa [954674,87651] 0 2024-03-16T09:01:09.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/368: creat d15/d18/d1f/d29/d43/f7a x:0 0 0 2024-03-16T09:01:09.725 INFO:tasks.workunit.client.0.smithi007.stdout:1/301: dwrite d9/dd/d1b/f25 [0,4194304] 0 2024-03-16T09:01:09.726 INFO:tasks.workunit.client.0.smithi007.stdout:1/302: creat d9/dd/d1b/d2d/d30/d45/f5c x:0 0 0 2024-03-16T09:01:09.726 INFO:tasks.workunit.client.0.smithi007.stdout:1/303: creat d9/dd/d1b/d28/d29/f5d x:0 0 0 2024-03-16T09:01:09.727 INFO:tasks.workunit.client.0.smithi007.stdout:8/566: dread d0/d2/d18/f3f [0,4194304] 0 2024-03-16T09:01:09.727 INFO:tasks.workunit.client.0.smithi007.stdout:8/567: truncate d0/d2/fb0 659471 0 2024-03-16T09:01:09.730 INFO:tasks.workunit.client.0.smithi007.stdout:3/369: truncate d15/d18/d1f/d3b/f47 1738112 0 2024-03-16T09:01:09.732 INFO:tasks.workunit.client.0.smithi007.stdout:8/568: mkdir d0/dc/dd/d8b/dc9 0 2024-03-16T09:01:09.733 INFO:tasks.workunit.client.0.smithi007.stdout:8/569: dread - d0/dc/d50/d8e/fa8 zero size 2024-03-16T09:01:09.734 INFO:tasks.workunit.client.0.smithi007.stdout:3/370: creat d15/d18/d1f/d29/f7b x:0 0 0 2024-03-16T09:01:09.734 INFO:tasks.workunit.client.0.smithi007.stdout:3/371: stat d15/d18/d1f 0 2024-03-16T09:01:09.735 INFO:tasks.workunit.client.0.smithi007.stdout:3/372: unlink d15/d18/d1f/d29/f7b 0 2024-03-16T09:01:09.736 INFO:tasks.workunit.client.0.smithi007.stdout:3/373: mkdir d15/d18/d1f/d29/d7c 0 2024-03-16T09:01:09.754 INFO:tasks.workunit.client.0.smithi007.stdout:3/374: dwrite d15/f1e [0,4194304] 0 2024-03-16T09:01:09.756 INFO:tasks.workunit.client.0.smithi007.stdout:3/375: creat d15/d18/d6a/f7d x:0 0 0 2024-03-16T09:01:09.756 INFO:tasks.workunit.client.0.smithi007.stdout:3/376: readlink l4 0 2024-03-16T09:01:09.756 INFO:tasks.workunit.client.0.smithi007.stdout:3/377: readlink d15/l40 0 2024-03-16T09:01:09.756 INFO:tasks.workunit.client.0.smithi007.stdout:3/378: readlink d15/d18/d1f/d29/d3e/l5a 0 2024-03-16T09:01:09.757 INFO:tasks.workunit.client.0.smithi007.stdout:3/379: write d15/d18/d1f/f41 [539381,38386] 0 2024-03-16T09:01:09.757 INFO:tasks.workunit.client.0.smithi007.stdout:3/380: creat d15/d18/d6a/f7e x:0 0 0 2024-03-16T09:01:09.758 INFO:tasks.workunit.client.0.smithi007.stdout:3/381: symlink d15/d18/d1f/d29/d43/l7f 0 2024-03-16T09:01:09.759 INFO:tasks.workunit.client.0.smithi007.stdout:3/382: truncate d15/d18/f33 971465 0 2024-03-16T09:01:09.761 INFO:tasks.workunit.client.0.smithi007.stdout:3/383: getdents d15 0 2024-03-16T09:01:09.762 INFO:tasks.workunit.client.0.smithi007.stdout:3/384: getdents d15/d18/d1f/d29/d43/d72 0 2024-03-16T09:01:09.762 INFO:tasks.workunit.client.0.smithi007.stdout:3/385: stat d15/d18/d1f/l6b 0 2024-03-16T09:01:09.762 INFO:tasks.workunit.client.0.smithi007.stdout:3/386: fdatasync d15/d18/d1f/d29/d3e/d4c/f52 0 2024-03-16T09:01:09.767 INFO:tasks.workunit.client.1.smithi019.stdout:8/83: link d6/de/c12 d6/de/d16/d19/c21 0 2024-03-16T09:01:09.769 INFO:tasks.workunit.client.1.smithi019.stdout:5/53: rmdir d6 39 2024-03-16T09:01:09.771 INFO:tasks.workunit.client.1.smithi019.stdout:7/46: truncate f7 2520357 0 2024-03-16T09:01:09.774 INFO:tasks.workunit.client.1.smithi019.stdout:3/52: unlink d3/ca 0 2024-03-16T09:01:09.782 INFO:tasks.workunit.client.0.smithi007.stdout:3/387: read d15/d18/f35 [5729456,109718] 0 2024-03-16T09:01:09.784 INFO:tasks.workunit.client.0.smithi007.stdout:3/388: mknod d15/d18/d6a/c80 0 2024-03-16T09:01:09.784 INFO:tasks.workunit.client.0.smithi007.stdout:3/389: chown d15/f2c 54 1 2024-03-16T09:01:09.785 INFO:tasks.workunit.client.1.smithi019.stdout:2/39: rename d5/d6/d9 to d5/de 0 2024-03-16T09:01:09.785 INFO:tasks.workunit.client.1.smithi019.stdout:2/40: creat d5/ff x:0 0 0 2024-03-16T09:01:09.785 INFO:tasks.workunit.client.1.smithi019.stdout:2/41: readlink - no filename 2024-03-16T09:01:09.786 INFO:tasks.workunit.client.0.smithi007.stdout:3/390: getdents d15/d18/d1f/d29/d3e 0 2024-03-16T09:01:09.786 INFO:tasks.workunit.client.1.smithi019.stdout:7/47: creat d9/fc x:0 0 0 2024-03-16T09:01:09.787 INFO:tasks.workunit.client.0.smithi007.stdout:3/391: truncate d15/d18/d1f/d56/f65 786157 0 2024-03-16T09:01:09.791 INFO:tasks.workunit.client.1.smithi019.stdout:3/53: rename d3/l9 to d3/lb 0 2024-03-16T09:01:09.791 INFO:tasks.workunit.client.1.smithi019.stdout:9/47: dread d1/d7/fc [0,4194304] 0 2024-03-16T09:01:09.796 INFO:tasks.workunit.client.1.smithi019.stdout:8/84: rename d6/c10 to d6/de/d16/c22 0 2024-03-16T09:01:09.797 INFO:tasks.workunit.client.1.smithi019.stdout:7/48: mknod d9/db/cd 0 2024-03-16T09:01:09.799 INFO:tasks.workunit.client.1.smithi019.stdout:3/54: link d3/f5 d3/fc 0 2024-03-16T09:01:09.799 INFO:tasks.workunit.client.1.smithi019.stdout:3/55: stat d3/f5 0 2024-03-16T09:01:09.801 INFO:tasks.workunit.client.1.smithi019.stdout:5/54: rename d6/ca to d6/dd/ce 0 2024-03-16T09:01:09.801 INFO:tasks.workunit.client.1.smithi019.stdout:8/85: read d6/de/d16/d19/f18 [1156869,116224] 0 2024-03-16T09:01:09.802 INFO:tasks.workunit.client.1.smithi019.stdout:3/56: creat d3/fd x:0 0 0 2024-03-16T09:01:09.803 INFO:tasks.workunit.client.1.smithi019.stdout:9/48: creat d1/d7/de/ff x:0 0 0 2024-03-16T09:01:09.803 INFO:tasks.workunit.client.1.smithi019.stdout:5/55: symlink d6/dd/lf 0 2024-03-16T09:01:09.803 INFO:tasks.workunit.client.1.smithi019.stdout:5/56: write d6/fb [307822,101376] 0 2024-03-16T09:01:09.804 INFO:tasks.workunit.client.1.smithi019.stdout:3/57: mknod d3/ce 0 2024-03-16T09:01:09.804 INFO:tasks.workunit.client.1.smithi019.stdout:3/58: stat d3/c8 0 2024-03-16T09:01:09.804 INFO:tasks.workunit.client.1.smithi019.stdout:3/59: creat d3/ff x:0 0 0 2024-03-16T09:01:09.805 INFO:tasks.workunit.client.1.smithi019.stdout:3/60: creat d3/f10 x:0 0 0 2024-03-16T09:01:09.805 INFO:tasks.workunit.client.1.smithi019.stdout:3/61: fdatasync d3/ff 0 2024-03-16T09:01:09.805 INFO:tasks.workunit.client.1.smithi019.stdout:3/62: write d3/fc [1591241,20178] 0 2024-03-16T09:01:09.805 INFO:tasks.workunit.client.1.smithi019.stdout:3/63: chown d3/ce 2 1 2024-03-16T09:01:09.805 INFO:tasks.workunit.client.1.smithi019.stdout:5/57: mknod d6/c10 0 2024-03-16T09:01:09.807 INFO:tasks.workunit.client.1.smithi019.stdout:9/49: dread d1/d7/fd [0,4194304] 0 2024-03-16T09:01:09.807 INFO:tasks.workunit.client.1.smithi019.stdout:9/50: creat d1/d7/de/f10 x:0 0 0 2024-03-16T09:01:09.807 INFO:tasks.workunit.client.1.smithi019.stdout:9/51: dread - d1/d7/de/f10 zero size 2024-03-16T09:01:09.807 INFO:tasks.workunit.client.1.smithi019.stdout:9/52: fdatasync d1/d7/de/ff 0 2024-03-16T09:01:09.807 INFO:tasks.workunit.client.1.smithi019.stdout:9/53: truncate d1/d7/de/ff 493117 0 2024-03-16T09:01:09.808 INFO:tasks.workunit.client.1.smithi019.stdout:5/58: symlink d6/l11 0 2024-03-16T09:01:09.810 INFO:tasks.workunit.client.1.smithi019.stdout:5/59: symlink d6/dd/l12 0 2024-03-16T09:01:09.810 INFO:tasks.workunit.client.1.smithi019.stdout:5/60: chown d6/f8 0 1 2024-03-16T09:01:09.810 INFO:tasks.workunit.client.1.smithi019.stdout:5/61: stat l5 0 2024-03-16T09:01:09.810 INFO:tasks.workunit.client.1.smithi019.stdout:5/62: write d6/fb [345322,54070] 0 2024-03-16T09:01:09.810 INFO:tasks.workunit.client.1.smithi019.stdout:5/63: stat d6/f7 0 2024-03-16T09:01:09.811 INFO:tasks.workunit.client.1.smithi019.stdout:5/64: dread f1 [0,4194304] 0 2024-03-16T09:01:09.811 INFO:tasks.workunit.client.1.smithi019.stdout:5/65: mknod d6/c13 0 2024-03-16T09:01:09.811 INFO:tasks.workunit.client.1.smithi019.stdout:5/66: rmdir d6/dd 39 2024-03-16T09:01:09.826 INFO:tasks.workunit.client.0.smithi007.stdout:8/570: dwrite d0/dc/f89 [0,4194304] 0 2024-03-16T09:01:09.826 INFO:tasks.workunit.client.0.smithi007.stdout:3/392: dwrite d15/d18/d1f/f39 [0,4194304] 0 2024-03-16T09:01:09.826 INFO:tasks.workunit.client.0.smithi007.stdout:3/393: fdatasync f5 0 2024-03-16T09:01:09.826 INFO:tasks.workunit.client.0.smithi007.stdout:3/394: stat fa 0 2024-03-16T09:01:09.826 INFO:tasks.workunit.client.0.smithi007.stdout:3/395: dread - d15/d18/d1f/d29/d43/f69 zero size 2024-03-16T09:01:09.827 INFO:tasks.workunit.client.0.smithi007.stdout:1/304: dwrite d9/dd/f14 [0,4194304] 0 2024-03-16T09:01:09.827 INFO:tasks.workunit.client.1.smithi019.stdout:2/42: dwrite d5/fb [0,4194304] 0 2024-03-16T09:01:09.828 INFO:tasks.workunit.client.0.smithi007.stdout:1/305: read f0 [5344774,68824] 0 2024-03-16T09:01:09.828 INFO:tasks.workunit.client.0.smithi007.stdout:1/306: fsync d9/dd/d1b/d28/d29/f42 0 2024-03-16T09:01:09.830 INFO:tasks.workunit.client.0.smithi007.stdout:3/396: write d15/d18/f33 [308223,43000] 0 2024-03-16T09:01:09.830 INFO:tasks.workunit.client.0.smithi007.stdout:3/397: readlink d15/d18/d1f/l4a 0 2024-03-16T09:01:09.831 INFO:tasks.workunit.client.0.smithi007.stdout:3/398: dread - d15/f59 zero size 2024-03-16T09:01:09.831 INFO:tasks.workunit.client.0.smithi007.stdout:3/399: chown d15/d18/d1f/d3b/f47 1276046 1 2024-03-16T09:01:09.831 INFO:tasks.workunit.client.0.smithi007.stdout:3/400: creat d15/d18/d1f/d29/d43/f81 x:0 0 0 2024-03-16T09:01:09.831 INFO:tasks.workunit.client.0.smithi007.stdout:3/401: write f10 [922763,70810] 0 2024-03-16T09:01:09.831 INFO:tasks.workunit.client.0.smithi007.stdout:8/571: symlink d0/d2/lca 0 2024-03-16T09:01:09.831 INFO:tasks.workunit.client.0.smithi007.stdout:8/572: stat d0/d26/d2d/f6c 0 2024-03-16T09:01:09.832 INFO:tasks.workunit.client.0.smithi007.stdout:3/402: creat d15/d18/d6a/d78/f82 x:0 0 0 2024-03-16T09:01:09.833 INFO:tasks.workunit.client.0.smithi007.stdout:8/573: unlink d0/d26/d2d/d43/d72/f94 0 2024-03-16T09:01:09.834 INFO:tasks.workunit.client.0.smithi007.stdout:3/403: rename d15/d18/d1f/d29/l3a to d15/d18/l83 0 2024-03-16T09:01:09.836 INFO:tasks.workunit.client.1.smithi019.stdout:2/43: write d5/fb [3948652,33134] 0 2024-03-16T09:01:09.836 INFO:tasks.workunit.client.0.smithi007.stdout:8/574: rename d0/l3e to d0/d26/d2d/d43/d4c/dba/lcb 0 2024-03-16T09:01:09.836 INFO:tasks.workunit.client.0.smithi007.stdout:8/575: write d0/d26/d2d/d64/f7f [1210534,7519] 0 2024-03-16T09:01:09.836 INFO:tasks.workunit.client.0.smithi007.stdout:3/404: symlink d15/d18/d1f/l84 0 2024-03-16T09:01:09.837 INFO:tasks.workunit.client.1.smithi019.stdout:2/44: mkdir d5/de/d10 0 2024-03-16T09:01:09.839 INFO:tasks.workunit.client.1.smithi019.stdout:2/45: dread f4 [0,4194304] 0 2024-03-16T09:01:09.841 INFO:tasks.workunit.client.1.smithi019.stdout:2/46: creat d5/d6/f11 x:0 0 0 2024-03-16T09:01:09.842 INFO:tasks.workunit.client.1.smithi019.stdout:2/47: mknod d5/d6/c12 0 2024-03-16T09:01:09.842 INFO:tasks.workunit.client.1.smithi019.stdout:2/48: chown d5/de/d10 0 1 2024-03-16T09:01:09.843 INFO:tasks.workunit.client.1.smithi019.stdout:2/49: rename d5/de/fc to d5/f13 0 2024-03-16T09:01:09.843 INFO:tasks.workunit.client.1.smithi019.stdout:2/50: truncate d5/ff 638475 0 2024-03-16T09:01:09.850 INFO:tasks.workunit.client.0.smithi007.stdout:8/576: write d0/dc/f89 [2358920,7199] 0 2024-03-16T09:01:09.851 INFO:tasks.workunit.client.0.smithi007.stdout:8/577: read - d0/d2/d18/f8d zero size 2024-03-16T09:01:09.851 INFO:tasks.workunit.client.0.smithi007.stdout:8/578: chown d0/d26/d95 5 1 2024-03-16T09:01:09.851 INFO:tasks.workunit.client.0.smithi007.stdout:8/579: write d0/dc/d8a/fb6 [450759,294] 0 2024-03-16T09:01:09.851 INFO:tasks.workunit.client.0.smithi007.stdout:8/580: creat d0/dc/d80/fcc x:0 0 0 2024-03-16T09:01:09.851 INFO:tasks.workunit.client.0.smithi007.stdout:8/581: creat d0/d26/d2d/d43/d70/fcd x:0 0 0 2024-03-16T09:01:09.851 INFO:tasks.workunit.client.0.smithi007.stdout:8/582: mkdir d0/dc/dd/d8b/dc9/dce 0 2024-03-16T09:01:09.852 INFO:tasks.workunit.client.0.smithi007.stdout:8/583: fsync d0/d26/d2d/d64/f7f 0 2024-03-16T09:01:09.852 INFO:tasks.workunit.client.0.smithi007.stdout:8/584: read d0/d2/d18/f3b [2029822,100570] 0 2024-03-16T09:01:09.854 INFO:tasks.workunit.client.1.smithi019.stdout:2/51: dread d5/fb [0,4194304] 0 2024-03-16T09:01:09.854 INFO:tasks.workunit.client.1.smithi019.stdout:2/52: creat d5/d6/f14 x:0 0 0 2024-03-16T09:01:09.855 INFO:tasks.workunit.client.1.smithi019.stdout:2/53: chown f2 25 1 2024-03-16T09:01:09.855 INFO:tasks.workunit.client.1.smithi019.stdout:2/54: chown f1 24 1 2024-03-16T09:01:09.857 INFO:tasks.workunit.client.1.smithi019.stdout:2/55: symlink d5/de/l15 0 2024-03-16T09:01:09.857 INFO:tasks.workunit.client.1.smithi019.stdout:2/56: chown f1 6 1 2024-03-16T09:01:09.866 INFO:tasks.workunit.client.0.smithi007.stdout:1/307: write d9/dd/d1b/d28/f2b [859102,84573] 0 2024-03-16T09:01:09.867 INFO:tasks.workunit.client.1.smithi019.stdout:2/57: dread d5/f7 [0,4194304] 0 2024-03-16T09:01:09.869 INFO:tasks.workunit.client.0.smithi007.stdout:0/307: dwrite d3/d9/da/d48/f38 [0,4194304] 0 2024-03-16T09:01:09.870 INFO:tasks.workunit.client.1.smithi019.stdout:9/54: dwrite d1/d7/fc [0,4194304] 0 2024-03-16T09:01:09.871 INFO:tasks.workunit.client.0.smithi007.stdout:3/405: dwrite d15/d18/d1f/d3b/f57 [0,4194304] 0 2024-03-16T09:01:09.871 INFO:tasks.workunit.client.1.smithi019.stdout:7/49: dwrite d9/fc [0,4194304] 0 2024-03-16T09:01:09.871 INFO:tasks.workunit.client.1.smithi019.stdout:7/50: fdatasync f4 0 2024-03-16T09:01:09.871 INFO:tasks.workunit.client.1.smithi019.stdout:4/64: truncate f2 3147661 0 2024-03-16T09:01:09.871 INFO:tasks.workunit.client.1.smithi019.stdout:4/65: fdatasync f2 0 2024-03-16T09:01:09.874 INFO:tasks.workunit.client.1.smithi019.stdout:6/170: truncate de/f25 5685538 0 2024-03-16T09:01:09.886 INFO:tasks.workunit.client.1.smithi019.stdout:2/58: dwrite d5/de/fd [0,4194304] 0 2024-03-16T09:01:09.888 INFO:tasks.workunit.client.1.smithi019.stdout:8/86: getdents d6 0 2024-03-16T09:01:09.889 INFO:tasks.workunit.client.0.smithi007.stdout:0/308: symlink d3/d9/l69 0 2024-03-16T09:01:09.889 INFO:tasks.workunit.client.0.smithi007.stdout:0/309: fsync d3/d9/da/d48/d4b/f52 0 2024-03-16T09:01:09.889 INFO:tasks.workunit.client.0.smithi007.stdout:0/310: creat d3/d9/da/d18/f6a x:0 0 0 2024-03-16T09:01:09.892 INFO:tasks.workunit.client.0.smithi007.stdout:3/406: mkdir d15/d18/d1f/d29/d3e/d85 0 2024-03-16T09:01:09.892 INFO:tasks.workunit.client.0.smithi007.stdout:3/407: chown d15/d18/c21 92280244 1 2024-03-16T09:01:09.892 INFO:tasks.workunit.client.0.smithi007.stdout:3/408: chown d15/d18/d1f/d29/d43/d72/l75 298 1 2024-03-16T09:01:09.893 INFO:tasks.workunit.client.0.smithi007.stdout:1/308: mkdir d9/dd/d5e 0 2024-03-16T09:01:09.893 INFO:tasks.workunit.client.0.smithi007.stdout:1/309: read - d9/dd/d1b/d2d/d30/d45/f5b zero size 2024-03-16T09:01:09.893 INFO:tasks.workunit.client.0.smithi007.stdout:1/310: chown d9/dd/d1b/d2d/d30/d45/f51 45160174 1 2024-03-16T09:01:09.893 INFO:tasks.workunit.client.0.smithi007.stdout:1/311: stat d9/dd/d5e 0 2024-03-16T09:01:09.894 INFO:tasks.workunit.client.0.smithi007.stdout:0/311: symlink d3/d9/da/d18/d27/l6b 0 2024-03-16T09:01:09.895 INFO:tasks.workunit.client.0.smithi007.stdout:3/409: symlink d15/d18/d6a/d78/l86 0 2024-03-16T09:01:09.895 INFO:tasks.workunit.client.0.smithi007.stdout:3/410: fdatasync d15/d18/d1f/d29/d43/f81 0 2024-03-16T09:01:09.896 INFO:tasks.workunit.client.0.smithi007.stdout:1/312: link d9/dd/ce d9/dd/d1b/d2d/c5f 0 2024-03-16T09:01:09.896 INFO:tasks.workunit.client.0.smithi007.stdout:1/313: fdatasync d9/dd/d1b/d2d/d30/f52 0 2024-03-16T09:01:09.896 INFO:tasks.workunit.client.0.smithi007.stdout:0/312: mkdir d3/d6c 0 2024-03-16T09:01:09.897 INFO:tasks.workunit.client.0.smithi007.stdout:3/411: mkdir d15/d18/d1f/d3b/d87 0 2024-03-16T09:01:09.897 INFO:tasks.workunit.client.0.smithi007.stdout:3/412: creat d15/d18/d1f/d56/f88 x:0 0 0 2024-03-16T09:01:09.899 INFO:tasks.workunit.client.0.smithi007.stdout:1/314: getdents d9/dd/d1b/d28/d29 0 2024-03-16T09:01:09.906 INFO:tasks.workunit.client.1.smithi019.stdout:4/66: mkdir d4/d6/d15 0 2024-03-16T09:01:09.908 INFO:tasks.workunit.client.1.smithi019.stdout:6/171: symlink de/d1c/d1d/l3d 0 2024-03-16T09:01:09.908 INFO:tasks.workunit.client.1.smithi019.stdout:6/172: dread - de/d26/f2a zero size 2024-03-16T09:01:09.908 INFO:tasks.workunit.client.1.smithi019.stdout:6/173: readlink l5 0 2024-03-16T09:01:09.909 INFO:tasks.workunit.client.1.smithi019.stdout:5/67: getdents d6 0 2024-03-16T09:01:09.910 INFO:tasks.workunit.client.0.smithi007.stdout:0/313: dwrite d3/d9/da/d48/d25/d43/d1a/f30 [4194304,4194304] 0 2024-03-16T09:01:09.935 INFO:tasks.workunit.client.0.smithi007.stdout:3/413: truncate d15/d18/d1f/d3b/f57 3352860 0 2024-03-16T09:01:09.941 INFO:tasks.workunit.client.1.smithi019.stdout:3/64: dwrite d3/f5 [0,4194304] 0 2024-03-16T09:01:09.941 INFO:tasks.workunit.client.1.smithi019.stdout:3/65: dread - d3/f10 zero size 2024-03-16T09:01:09.945 INFO:tasks.workunit.client.0.smithi007.stdout:1/315: dwrite d9/dd/d1b/d2d/d30/d45/f5c [0,4194304] 0 2024-03-16T09:01:09.948 INFO:tasks.workunit.client.0.smithi007.stdout:0/314: dwrite d3/d9/db/d2c/f5e [0,4194304] 0 2024-03-16T09:01:09.948 INFO:tasks.workunit.client.0.smithi007.stdout:0/315: chown d3/d9/db 174805 1 2024-03-16T09:01:09.949 INFO:tasks.workunit.client.0.smithi007.stdout:1/316: link d9/f22 d9/dd/d1b/d28/f60 0 2024-03-16T09:01:09.949 INFO:tasks.workunit.client.1.smithi019.stdout:9/55: mkdir d1/d7/d11 0 2024-03-16T09:01:09.951 INFO:tasks.workunit.client.1.smithi019.stdout:7/51: rmdir d9 39 2024-03-16T09:01:09.952 INFO:tasks.workunit.client.1.smithi019.stdout:5/68: dwrite d6/f9 [0,4194304] 0 2024-03-16T09:01:09.953 INFO:tasks.workunit.client.0.smithi007.stdout:0/316: mknod d3/d9/da/d48/c6d 0 2024-03-16T09:01:09.953 INFO:tasks.workunit.client.0.smithi007.stdout:0/317: stat d3/d9/da/f4c 0 2024-03-16T09:01:09.953 INFO:tasks.workunit.client.0.smithi007.stdout:0/318: creat d3/d9/db/d2c/f6e x:0 0 0 2024-03-16T09:01:09.954 INFO:tasks.workunit.client.1.smithi019.stdout:4/67: rename d4/d6/l14 to d4/d6/d15/l16 0 2024-03-16T09:01:09.959 INFO:tasks.workunit.client.1.smithi019.stdout:9/56: symlink d1/d7/d11/l12 0 2024-03-16T09:01:09.959 INFO:tasks.workunit.client.1.smithi019.stdout:9/57: dread d1/d7/de/ff [0,4194304] 0 2024-03-16T09:01:09.960 INFO:tasks.workunit.client.1.smithi019.stdout:7/52: mknod d9/ce 0 2024-03-16T09:01:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:0/319: read d3/d9/da/f4c [3573484,130649] 0 2024-03-16T09:01:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:0/320: creat d3/f6f x:0 0 0 2024-03-16T09:01:09.964 INFO:tasks.workunit.client.0.smithi007.stdout:3/414: dwrite d15/d18/d6a/d78/f82 [0,4194304] 0 2024-03-16T09:01:09.965 INFO:tasks.workunit.client.1.smithi019.stdout:5/69: symlink d6/l14 0 2024-03-16T09:01:09.965 INFO:tasks.workunit.client.1.smithi019.stdout:5/70: write d6/f7 [623836,7679] 0 2024-03-16T09:01:09.965 INFO:tasks.workunit.client.1.smithi019.stdout:5/71: chown d6/f7 35096447 1 2024-03-16T09:01:09.966 INFO:tasks.workunit.client.1.smithi019.stdout:6/174: dread de/f25 [0,4194304] 0 2024-03-16T09:01:09.966 INFO:tasks.workunit.client.1.smithi019.stdout:3/66: dwrite f2 [0,4194304] 0 2024-03-16T09:01:09.968 INFO:tasks.workunit.client.1.smithi019.stdout:4/68: creat d4/d6/f17 x:0 0 0 2024-03-16T09:01:09.968 INFO:tasks.workunit.client.1.smithi019.stdout:4/69: write f2 [996523,68426] 0 2024-03-16T09:01:09.969 INFO:tasks.workunit.client.1.smithi019.stdout:9/58: rmdir d1/d7/d11 39 2024-03-16T09:01:09.969 INFO:tasks.workunit.client.1.smithi019.stdout:9/59: stat d1/d7/d11 0 2024-03-16T09:01:09.969 INFO:tasks.workunit.client.1.smithi019.stdout:9/60: truncate d1/d7/de/f10 367422 0 2024-03-16T09:01:09.969 INFO:tasks.workunit.client.1.smithi019.stdout:9/61: chown d1/d7/de/f10 4 1 2024-03-16T09:01:09.969 INFO:tasks.workunit.client.0.smithi007.stdout:0/321: rename d3/d9/da/d18/d27/d34/d55 to d3/d1f/d70 0 2024-03-16T09:01:09.970 INFO:tasks.workunit.client.1.smithi019.stdout:7/53: mknod d9/da/cf 0 2024-03-16T09:01:09.970 INFO:tasks.workunit.client.0.smithi007.stdout:1/317: dwrite d9/dd/d1b/d2d/d30/f4f [0,4194304] 0 2024-03-16T09:01:09.970 INFO:tasks.workunit.client.0.smithi007.stdout:1/318: getdents d9/dd/d5e 0 2024-03-16T09:01:09.971 INFO:tasks.workunit.client.1.smithi019.stdout:5/72: rmdir d6 39 2024-03-16T09:01:09.971 INFO:tasks.workunit.client.0.smithi007.stdout:0/322: creat d3/d1f/f71 x:0 0 0 2024-03-16T09:01:09.974 INFO:tasks.workunit.client.1.smithi019.stdout:6/175: symlink de/d1c/l3e 0 2024-03-16T09:01:09.975 INFO:tasks.workunit.client.1.smithi019.stdout:3/67: mkdir d3/d11 0 2024-03-16T09:01:09.978 INFO:tasks.workunit.client.1.smithi019.stdout:4/70: mknod d4/d6/d7/c18 0 2024-03-16T09:01:09.978 INFO:tasks.workunit.client.1.smithi019.stdout:4/71: truncate f2 3952863 0 2024-03-16T09:01:09.978 INFO:tasks.workunit.client.1.smithi019.stdout:4/72: write d4/f12 [424697,63944] 0 2024-03-16T09:01:09.980 INFO:tasks.workunit.client.1.smithi019.stdout:9/62: creat d1/d7/d11/f13 x:0 0 0 2024-03-16T09:01:09.980 INFO:tasks.workunit.client.1.smithi019.stdout:9/63: truncate d1/f3 4459474 0 2024-03-16T09:01:09.980 INFO:tasks.workunit.client.1.smithi019.stdout:9/64: stat d1/d7/de/ff 0 2024-03-16T09:01:09.980 INFO:tasks.workunit.client.1.smithi019.stdout:9/65: write d1/d7/de/ff [833073,106309] 0 2024-03-16T09:01:09.981 INFO:tasks.workunit.client.0.smithi007.stdout:3/415: dread d15/d18/f19 [0,4194304] 0 2024-03-16T09:01:09.981 INFO:tasks.workunit.client.0.smithi007.stdout:0/323: mkdir d3/d9/db/d1c/d72 0 2024-03-16T09:01:09.981 INFO:tasks.workunit.client.1.smithi019.stdout:7/54: read f8 [3508813,67013] 0 2024-03-16T09:01:09.983 INFO:tasks.workunit.client.0.smithi007.stdout:3/416: symlink d15/d18/d1f/d77/l89 0 2024-03-16T09:01:09.983 INFO:tasks.workunit.client.0.smithi007.stdout:3/417: chown d15/d18/d1f/f58 268222146 1 2024-03-16T09:01:09.983 INFO:tasks.workunit.client.0.smithi007.stdout:3/418: write d15/d18/f22 [4906472,46836] 0 2024-03-16T09:01:09.985 INFO:tasks.workunit.client.0.smithi007.stdout:0/324: rename d3/d9/db/d1c/d62 to d3/d1f/d70/d73 0 2024-03-16T09:01:09.985 INFO:tasks.workunit.client.0.smithi007.stdout:0/325: truncate d3/d9/da/d48/f12 1984656 0 2024-03-16T09:01:09.987 INFO:tasks.workunit.client.0.smithi007.stdout:0/326: truncate d3/d9/db/fc 3520445 0 2024-03-16T09:01:09.987 INFO:tasks.workunit.client.0.smithi007.stdout:0/327: stat d3/d9/da/d48/f47 0 2024-03-16T09:01:09.987 INFO:tasks.workunit.client.0.smithi007.stdout:0/328: truncate d3/d9/da/d48/f4e 114683 0 2024-03-16T09:01:09.987 INFO:tasks.workunit.client.1.smithi019.stdout:6/176: symlink de/d1c/d1d/l3f 0 2024-03-16T09:01:09.988 INFO:tasks.workunit.client.1.smithi019.stdout:3/68: creat d3/d11/f12 x:0 0 0 2024-03-16T09:01:09.988 INFO:tasks.workunit.client.0.smithi007.stdout:0/329: mkdir d3/d74 0 2024-03-16T09:01:09.988 INFO:tasks.workunit.client.0.smithi007.stdout:0/330: stat d3/d9/da/d48/d25/f68 0 2024-03-16T09:01:09.988 INFO:tasks.workunit.client.0.smithi007.stdout:0/331: dread - d3/f6f zero size 2024-03-16T09:01:09.988 INFO:tasks.workunit.client.0.smithi007.stdout:0/332: stat d3/d9/da/d18/d23/f32 0 2024-03-16T09:01:09.989 INFO:tasks.workunit.client.1.smithi019.stdout:4/73: stat d4/d6/l9 0 2024-03-16T09:01:09.990 INFO:tasks.workunit.client.0.smithi007.stdout:0/333: unlink d3/d9/da/d18/d27/l6b 0 2024-03-16T09:01:09.990 INFO:tasks.workunit.client.0.smithi007.stdout:0/334: write d3/d9/da/d48/f44 [4930922,18140] 0 2024-03-16T09:01:09.990 INFO:tasks.workunit.client.0.smithi007.stdout:0/335: chown d3/d9/da/d48/d25/d43/l2e 511237080 1 2024-03-16T09:01:09.990 INFO:tasks.workunit.client.0.smithi007.stdout:0/336: truncate d3/f49 604170 0 2024-03-16T09:01:09.990 INFO:tasks.workunit.client.0.smithi007.stdout:0/337: stat d3/d9/db/fc 0 2024-03-16T09:01:09.991 INFO:tasks.workunit.client.1.smithi019.stdout:7/55: rename d9/db/cd to d9/da/c10 0 2024-03-16T09:01:09.994 INFO:tasks.workunit.client.1.smithi019.stdout:7/56: creat d9/db/f11 x:0 0 0 2024-03-16T09:01:09.997 INFO:tasks.workunit.client.1.smithi019.stdout:6/177: write de/f25 [1445054,51218] 0 2024-03-16T09:01:09.998 INFO:tasks.workunit.client.1.smithi019.stdout:6/178: creat de/d26/f40 x:0 0 0 2024-03-16T09:01:09.998 INFO:tasks.workunit.client.1.smithi019.stdout:5/73: dwrite f3 [0,4194304] 0 2024-03-16T09:01:09.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:09 smithi007 ceph-mon[29470]: pgmap v18: 65 pgs: 65 active+clean; 591 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 49 MiB/s rd, 66 MiB/s wr, 874 op/s 2024-03-16T09:01:10.001 INFO:tasks.workunit.client.0.smithi007.stdout:1/319: dwrite d9/dd/d1b/d2d/d30/f52 [0,4194304] 0 2024-03-16T09:01:10.001 INFO:tasks.workunit.client.0.smithi007.stdout:1/320: dread - d9/dd/d1b/f49 zero size 2024-03-16T09:01:10.001 INFO:tasks.workunit.client.0.smithi007.stdout:1/321: creat d9/dd/d1b/d2d/d30/f61 x:0 0 0 2024-03-16T09:01:10.001 INFO:tasks.workunit.client.0.smithi007.stdout:0/338: getdents d3/d9/db/d1c 0 2024-03-16T09:01:10.002 INFO:tasks.workunit.client.0.smithi007.stdout:1/322: creat d9/f62 x:0 0 0 2024-03-16T09:01:10.002 INFO:tasks.workunit.client.0.smithi007.stdout:1/323: truncate d9/dd/d1b/d28/f50 262797 0 2024-03-16T09:01:10.003 INFO:tasks.workunit.client.0.smithi007.stdout:3/419: write d15/d18/d1f/d29/d43/d72/f50 [920456,68125] 0 2024-03-16T09:01:10.003 INFO:tasks.workunit.client.1.smithi019.stdout:5/74: rename d6/dd/lf to d6/dd/l15 0 2024-03-16T09:01:10.004 INFO:tasks.workunit.client.1.smithi019.stdout:5/75: chown d6/l14 841463 1 2024-03-16T09:01:10.004 INFO:tasks.workunit.client.1.smithi019.stdout:5/76: rmdir d6/dd 39 2024-03-16T09:01:10.005 INFO:tasks.workunit.client.1.smithi019.stdout:5/77: rmdir d6/dd 39 2024-03-16T09:01:10.007 INFO:tasks.workunit.client.0.smithi007.stdout:3/420: dread d15/d18/f33 [0,4194304] 0 2024-03-16T09:01:10.007 INFO:tasks.workunit.client.1.smithi019.stdout:5/78: creat d6/dd/f16 x:0 0 0 2024-03-16T09:01:10.009 INFO:tasks.workunit.client.1.smithi019.stdout:5/79: creat d6/f17 x:0 0 0 2024-03-16T09:01:10.009 INFO:tasks.workunit.client.1.smithi019.stdout:5/80: creat d6/f18 x:0 0 0 2024-03-16T09:01:10.011 INFO:tasks.workunit.client.1.smithi019.stdout:7/57: truncate d9/fc 3722148 0 2024-03-16T09:01:10.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/324: write d9/f12 [2334758,2961] 0 2024-03-16T09:01:10.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/325: getdents d9/dd/d1b/d28/d29 0 2024-03-16T09:01:10.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/326: creat d9/dd/d1b/d28/d4c/f63 x:0 0 0 2024-03-16T09:01:10.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/327: read d9/f12 [4121157,3497] 0 2024-03-16T09:01:10.013 INFO:tasks.workunit.client.0.smithi007.stdout:0/339: rmdir d3/d9/da/d48 39 2024-03-16T09:01:10.013 INFO:tasks.workunit.client.0.smithi007.stdout:0/340: creat d3/d9/da/d18/d23/f75 x:0 0 0 2024-03-16T09:01:10.014 INFO:tasks.workunit.client.1.smithi019.stdout:9/66: dwrite d1/d7/de/ff [0,4194304] 0 2024-03-16T09:01:10.016 INFO:tasks.workunit.client.1.smithi019.stdout:7/58: dread f8 [0,4194304] 0 2024-03-16T09:01:10.016 INFO:tasks.workunit.client.1.smithi019.stdout:4/74: dwrite f2 [0,4194304] 0 2024-03-16T09:01:10.016 INFO:tasks.workunit.client.0.smithi007.stdout:1/328: creat d9/dd/f64 x:0 0 0 2024-03-16T09:01:10.016 INFO:tasks.workunit.client.0.smithi007.stdout:3/421: dread d15/d18/d1f/f54 [0,4194304] 0 2024-03-16T09:01:10.016 INFO:tasks.workunit.client.0.smithi007.stdout:1/329: chown d9/dd/c31 1 1 2024-03-16T09:01:10.016 INFO:tasks.workunit.client.0.smithi007.stdout:1/330: chown d9/dd/d1b/d28/f32 229222 1 2024-03-16T09:01:10.016 INFO:tasks.workunit.client.0.smithi007.stdout:1/331: chown d9/dd/f64 220910 1 2024-03-16T09:01:10.016 INFO:tasks.workunit.client.0.smithi007.stdout:3/422: creat d15/d18/d1f/d56/f8a x:0 0 0 2024-03-16T09:01:10.017 INFO:tasks.workunit.client.1.smithi019.stdout:9/67: rename d1/d7/fc to d1/d7/d11/f14 0 2024-03-16T09:01:10.017 INFO:tasks.workunit.client.1.smithi019.stdout:9/68: creat d1/d7/d11/f15 x:0 0 0 2024-03-16T09:01:10.017 INFO:tasks.workunit.client.1.smithi019.stdout:7/59: mkdir d9/db/d12 0 2024-03-16T09:01:10.017 INFO:tasks.workunit.client.1.smithi019.stdout:7/60: fdatasync f7 0 2024-03-16T09:01:10.018 INFO:tasks.workunit.client.0.smithi007.stdout:0/341: mknod d3/d9/da/d18/d27/d34/c76 0 2024-03-16T09:01:10.018 INFO:tasks.workunit.client.0.smithi007.stdout:0/342: dread - d3/d9/da/d18/d23/f45 zero size 2024-03-16T09:01:10.022 INFO:tasks.workunit.client.0.smithi007.stdout:1/332: creat d9/dd/d5e/f65 x:0 0 0 2024-03-16T09:01:10.023 INFO:tasks.workunit.client.0.smithi007.stdout:3/423: mknod d15/d18/d1f/d3b/d87/c8b 0 2024-03-16T09:01:10.023 INFO:tasks.workunit.client.1.smithi019.stdout:7/61: mkdir d9/da/d13 0 2024-03-16T09:01:10.023 INFO:tasks.workunit.client.1.smithi019.stdout:7/62: write f8 [1619421,15771] 0 2024-03-16T09:01:10.023 INFO:tasks.workunit.client.1.smithi019.stdout:7/63: readlink - no filename 2024-03-16T09:01:10.024 INFO:tasks.workunit.client.1.smithi019.stdout:5/81: dwrite d6/dd/f16 [0,4194304] 0 2024-03-16T09:01:10.025 INFO:tasks.workunit.client.1.smithi019.stdout:9/69: unlink d1/l9 0 2024-03-16T09:01:10.025 INFO:tasks.workunit.client.0.smithi007.stdout:1/333: unlink d9/dd/d1b/d2d/f53 0 2024-03-16T09:01:10.026 INFO:tasks.workunit.client.1.smithi019.stdout:7/64: symlink d9/da/d13/l14 0 2024-03-16T09:01:10.027 INFO:tasks.workunit.client.1.smithi019.stdout:7/65: truncate d9/db/f11 325424 0 2024-03-16T09:01:10.028 INFO:tasks.workunit.client.0.smithi007.stdout:3/424: creat d15/d18/d1f/d29/d7c/f8c x:0 0 0 2024-03-16T09:01:10.028 INFO:tasks.workunit.client.0.smithi007.stdout:3/425: chown d15/c38 0 1 2024-03-16T09:01:10.028 INFO:tasks.workunit.client.0.smithi007.stdout:0/343: write d3/d9/da/d48/f44 [664978,67679] 0 2024-03-16T09:01:10.028 INFO:tasks.workunit.client.0.smithi007.stdout:0/344: chown d3/d9/db/d2c/c53 555607 1 2024-03-16T09:01:10.029 INFO:tasks.workunit.client.0.smithi007.stdout:1/334: creat d9/dd/d1b/d58/f66 x:0 0 0 2024-03-16T09:01:10.033 INFO:tasks.workunit.client.0.smithi007.stdout:1/335: unlink d9/dd/d1b/d28/d4c/f55 0 2024-03-16T09:01:10.033 INFO:tasks.workunit.client.0.smithi007.stdout:1/336: fdatasync d9/dd/d1b/d2d/f3f 0 2024-03-16T09:01:10.035 INFO:tasks.workunit.client.0.smithi007.stdout:1/337: creat d9/dd/f67 x:0 0 0 2024-03-16T09:01:10.038 INFO:tasks.workunit.client.1.smithi019.stdout:9/70: symlink d1/d7/l16 0 2024-03-16T09:01:10.038 INFO:tasks.workunit.client.1.smithi019.stdout:9/71: chown d1/d7/d11 162515544 1 2024-03-16T09:01:10.038 INFO:tasks.workunit.client.1.smithi019.stdout:5/82: write d6/fb [2050971,100236] 0 2024-03-16T09:01:10.046 INFO:tasks.workunit.client.1.smithi019.stdout:9/72: write d1/d7/de/ff [3893896,64861] 0 2024-03-16T09:01:10.046 INFO:tasks.workunit.client.1.smithi019.stdout:9/73: fdatasync d1/d7/de/f10 0 2024-03-16T09:01:10.047 INFO:tasks.workunit.client.0.smithi007.stdout:9/390: sync 2024-03-16T09:01:10.047 INFO:tasks.workunit.client.0.smithi007.stdout:5/298: sync 2024-03-16T09:01:10.047 INFO:tasks.workunit.client.0.smithi007.stdout:5/299: write f0 [8532322,49876] 0 2024-03-16T09:01:10.048 INFO:tasks.workunit.client.0.smithi007.stdout:5/300: chown d1/d7/d1c/d19/d34/d1d/l2e 149162597 1 2024-03-16T09:01:10.048 INFO:tasks.workunit.client.1.smithi019.stdout:9/74: symlink d1/d7/d11/l17 0 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:9/391: write d24/d4c/f63 [656044,24926] 0 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:8/585: sync 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:6/261: sync 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:7/231: sync 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/425: sync 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/426: write dc/d27/f4f [371031,65402] 0 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/427: chown dc/df/d24/d28/d2e/d35/c50 13339 1 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/428: stat dc/df/c89 0 2024-03-16T09:01:10.049 INFO:tasks.workunit.client.0.smithi007.stdout:4/242: sync 2024-03-16T09:01:10.050 INFO:tasks.workunit.client.1.smithi019.stdout:9/75: dread d1/d7/de/f10 [0,4194304] 0 2024-03-16T09:01:10.050 INFO:tasks.workunit.client.1.smithi019.stdout:9/76: write d1/d7/d11/f14 [4673427,43365] 0 2024-03-16T09:01:10.050 INFO:tasks.workunit.client.0.smithi007.stdout:6/262: creat d2/d13/d4b/f55 x:0 0 0 2024-03-16T09:01:10.052 INFO:tasks.workunit.client.0.smithi007.stdout:7/232: rename d0/f18 to d0/f49 0 2024-03-16T09:01:10.052 INFO:tasks.workunit.client.0.smithi007.stdout:2/429: mknod dc/df/d6c/c8a 0 2024-03-16T09:01:10.053 INFO:tasks.workunit.client.1.smithi019.stdout:6/179: dwrite de/f25 [0,4194304] 0 2024-03-16T09:01:10.053 INFO:tasks.workunit.client.1.smithi019.stdout:9/77: rename d1/c5 to d1/d7/d11/c18 0 2024-03-16T09:01:10.054 INFO:tasks.workunit.client.0.smithi007.stdout:8/586: rename d0/d2/faa to d0/dc/dd/d8b/fcf 0 2024-03-16T09:01:10.054 INFO:tasks.workunit.client.1.smithi019.stdout:6/180: symlink de/d26/l41 0 2024-03-16T09:01:10.054 INFO:tasks.workunit.client.0.smithi007.stdout:4/243: chown d4/c16 12 1 2024-03-16T09:01:10.055 INFO:tasks.workunit.client.0.smithi007.stdout:5/301: write d1/d7/d16/d1e/d2a/f57 [363510,23456] 0 2024-03-16T09:01:10.055 INFO:tasks.workunit.client.0.smithi007.stdout:5/302: creat d1/d7/d16/d1e/f6e x:0 0 0 2024-03-16T09:01:10.055 INFO:tasks.workunit.client.1.smithi019.stdout:9/78: mknod d1/d7/de/c19 0 2024-03-16T09:01:10.055 INFO:tasks.workunit.client.1.smithi019.stdout:9/79: creat d1/d7/de/f1a x:0 0 0 2024-03-16T09:01:10.057 INFO:tasks.workunit.client.0.smithi007.stdout:9/392: rmdir d24 39 2024-03-16T09:01:10.058 INFO:tasks.workunit.client.1.smithi019.stdout:4/75: dwrite d4/f12 [0,4194304] 0 2024-03-16T09:01:10.058 INFO:tasks.workunit.client.1.smithi019.stdout:9/80: rename d1/d7/d11/l12 to d1/d7/de/l1b 0 2024-03-16T09:01:10.058 INFO:tasks.workunit.client.1.smithi019.stdout:7/66: dwrite f6 [0,4194304] 0 2024-03-16T09:01:10.059 INFO:tasks.workunit.client.1.smithi019.stdout:3/69: getdents d3 0 2024-03-16T09:01:10.059 INFO:tasks.workunit.client.1.smithi019.stdout:7/67: dread d9/db/f11 [0,4194304] 0 2024-03-16T09:01:10.060 INFO:tasks.workunit.client.0.smithi007.stdout:7/233: rename d0/dc/c37 to d0/d19/d1f/c4a 0 2024-03-16T09:01:10.060 INFO:tasks.workunit.client.0.smithi007.stdout:7/234: readlink d0/l2 0 2024-03-16T09:01:10.060 INFO:tasks.workunit.client.0.smithi007.stdout:7/235: chown d0/dc/d24 19343427 1 2024-03-16T09:01:10.061 INFO:tasks.workunit.client.1.smithi019.stdout:4/76: rename d4/d6/d7/c10 to d4/d6/d15/c19 0 2024-03-16T09:01:10.061 INFO:tasks.workunit.client.1.smithi019.stdout:4/77: readlink d4/l11 0 2024-03-16T09:01:10.061 INFO:tasks.workunit.client.0.smithi007.stdout:8/587: link d0/d26/fa2 d0/dc/dd/d8b/fd0 0 2024-03-16T09:01:10.062 INFO:tasks.workunit.client.1.smithi019.stdout:9/81: mknod d1/d7/de/c1c 0 2024-03-16T09:01:10.071 INFO:tasks.workunit.client.1.smithi019.stdout:1/34: sync 2024-03-16T09:01:10.071 INFO:tasks.workunit.client.1.smithi019.stdout:1/35: dwrite - no filename 2024-03-16T09:01:10.071 INFO:tasks.workunit.client.1.smithi019.stdout:1/36: dread - no filename 2024-03-16T09:01:10.071 INFO:tasks.workunit.client.1.smithi019.stdout:1/37: dwrite - no filename 2024-03-16T09:01:10.072 INFO:tasks.workunit.client.1.smithi019.stdout:1/38: dwrite - no filename 2024-03-16T09:01:10.072 INFO:tasks.workunit.client.1.smithi019.stdout:3/70: write d3/f5 [4096511,89868] 0 2024-03-16T09:01:10.073 INFO:tasks.workunit.client.0.smithi007.stdout:9/393: rename d24/d4c/f64 to d24/d4a/d61/f6f 0 2024-03-16T09:01:10.074 INFO:tasks.workunit.client.1.smithi019.stdout:4/78: rename d4/d6/d15/l16 to d4/d6/d15/l1a 0 2024-03-16T09:01:10.076 INFO:tasks.workunit.client.0.smithi007.stdout:2/430: read dc/df/d24/d28/d2e/f5c [1399020,84693] 0 2024-03-16T09:01:10.076 INFO:tasks.workunit.client.0.smithi007.stdout:2/431: getdents dc/df/d24/d28/d71 0 2024-03-16T09:01:10.076 INFO:tasks.workunit.client.0.smithi007.stdout:6/263: dwrite d2/fd [0,4194304] 0 2024-03-16T09:01:10.076 INFO:tasks.workunit.client.0.smithi007.stdout:7/236: creat d0/d19/d1f/f4b x:0 0 0 2024-03-16T09:01:10.077 INFO:tasks.workunit.client.0.smithi007.stdout:8/588: symlink d0/dc/dd/d8b/ld1 0 2024-03-16T09:01:10.078 INFO:tasks.workunit.client.0.smithi007.stdout:1/338: getdents d9/dd/d1b/d28/d4c 0 2024-03-16T09:01:10.080 INFO:tasks.workunit.client.0.smithi007.stdout:4/244: rmdir d4/d14/d1c/d1e 39 2024-03-16T09:01:10.082 INFO:tasks.workunit.client.0.smithi007.stdout:5/303: rmdir d1/d7/d16/d1e/d2a 39 2024-03-16T09:01:10.082 INFO:tasks.workunit.client.0.smithi007.stdout:5/304: write d1/d7/d1c/d19/d34/d3e/f69 [447259,128278] 0 2024-03-16T09:01:10.083 INFO:tasks.workunit.client.1.smithi019.stdout:6/181: dwrite de/d26/f2a [0,4194304] 0 2024-03-16T09:01:10.083 INFO:tasks.workunit.client.1.smithi019.stdout:3/71: symlink d3/l13 0 2024-03-16T09:01:10.083 INFO:tasks.workunit.client.1.smithi019.stdout:1/39: symlink l5 0 2024-03-16T09:01:10.083 INFO:tasks.workunit.client.1.smithi019.stdout:3/72: read - d3/d11/f12 zero size 2024-03-16T09:01:10.085 INFO:tasks.workunit.client.0.smithi007.stdout:9/394: symlink d24/l70 0 2024-03-16T09:01:10.085 INFO:tasks.workunit.client.0.smithi007.stdout:9/395: read - d24/d4a/d61/f6f zero size 2024-03-16T09:01:10.085 INFO:tasks.workunit.client.0.smithi007.stdout:9/396: fdatasync d24/f4d 0 2024-03-16T09:01:10.088 INFO:tasks.workunit.client.1.smithi019.stdout:6/182: symlink de/d1c/d1d/l42 0 2024-03-16T09:01:10.088 INFO:tasks.workunit.client.1.smithi019.stdout:4/79: write d4/f12 [1498095,102081] 0 2024-03-16T09:01:10.088 INFO:tasks.workunit.client.1.smithi019.stdout:6/183: dread - de/d26/f40 zero size 2024-03-16T09:01:10.089 INFO:tasks.workunit.client.1.smithi019.stdout:9/82: dwrite d1/d7/de/f10 [0,4194304] 0 2024-03-16T09:01:10.092 INFO:tasks.workunit.client.1.smithi019.stdout:3/73: unlink d3/fd 0 2024-03-16T09:01:10.092 INFO:tasks.workunit.client.0.smithi007.stdout:6/264: creat d2/d11/d1e/d54/f56 x:0 0 0 2024-03-16T09:01:10.094 INFO:tasks.workunit.client.1.smithi019.stdout:4/80: mkdir d4/d1b 0 2024-03-16T09:01:10.094 INFO:tasks.workunit.client.1.smithi019.stdout:5/83: rmdir d6 39 2024-03-16T09:01:10.094 INFO:tasks.workunit.client.0.smithi007.stdout:4/245: truncate d4/d14/f19 2415801 0 2024-03-16T09:01:10.096 INFO:tasks.workunit.client.1.smithi019.stdout:6/184: mknod de/d1c/d1d/c43 0 2024-03-16T09:01:10.096 INFO:tasks.workunit.client.1.smithi019.stdout:6/185: chown l5 1688492 1 2024-03-16T09:01:10.097 INFO:tasks.workunit.client.0.smithi007.stdout:1/339: symlink d9/dd/l68 0 2024-03-16T09:01:10.097 INFO:tasks.workunit.client.0.smithi007.stdout:1/340: read - d9/dd/d1b/d2d/d30/d45/f5b zero size 2024-03-16T09:01:10.098 INFO:tasks.workunit.client.0.smithi007.stdout:8/589: dwrite d0/d26/fa1 [0,4194304] 0 2024-03-16T09:01:10.099 INFO:tasks.workunit.client.1.smithi019.stdout:9/83: dread d1/f3 [0,4194304] 0 2024-03-16T09:01:10.099 INFO:tasks.workunit.client.1.smithi019.stdout:9/84: write d1/d7/d11/f15 [394747,118481] 0 2024-03-16T09:01:10.100 INFO:tasks.workunit.client.0.smithi007.stdout:1/341: creat d9/dd/f69 x:0 0 0 2024-03-16T09:01:10.101 INFO:tasks.workunit.client.1.smithi019.stdout:1/40: mkdir d6 0 2024-03-16T09:01:10.102 INFO:tasks.workunit.client.0.smithi007.stdout:5/305: rename d1/d7/d1c/d19/d34/d1d/d36/d4e to d1/d7/d1c/d19/d34/d5e/d6f 0 2024-03-16T09:01:10.102 INFO:tasks.workunit.client.0.smithi007.stdout:5/306: chown d1/d7/d16/d1e/f47 494525 1 2024-03-16T09:01:10.102 INFO:tasks.workunit.client.0.smithi007.stdout:5/307: truncate d1/d7/d16/d1b/f60 865217 0 2024-03-16T09:01:10.102 INFO:tasks.workunit.client.0.smithi007.stdout:5/308: write d1/d7/d16/d1b/f67 [484696,38798] 0 2024-03-16T09:01:10.102 INFO:tasks.workunit.client.0.smithi007.stdout:1/342: rmdir d9/dd/d1b/d28/d29 39 2024-03-16T09:01:10.104 INFO:tasks.workunit.client.1.smithi019.stdout:4/81: creat d4/d1b/f1c x:0 0 0 2024-03-16T09:01:10.104 INFO:tasks.workunit.client.1.smithi019.stdout:4/82: stat d4/d6 0 2024-03-16T09:01:10.104 INFO:tasks.workunit.client.1.smithi019.stdout:4/83: readlink d4/l11 0 2024-03-16T09:01:10.104 INFO:tasks.workunit.client.1.smithi019.stdout:4/84: chown d4/d1b/f1c 26692371 1 2024-03-16T09:01:10.106 INFO:tasks.workunit.client.0.smithi007.stdout:4/246: rename d4/f2e to d4/d7/f50 0 2024-03-16T09:01:10.107 INFO:tasks.workunit.client.1.smithi019.stdout:6/186: rename de/d1c/d1d/l3d to de/d26/d32/l44 0 2024-03-16T09:01:10.107 INFO:tasks.workunit.client.1.smithi019.stdout:6/187: chown de/d1c/c21 205518258 1 2024-03-16T09:01:10.107 INFO:tasks.workunit.client.1.smithi019.stdout:3/74: dread f2 [0,4194304] 0 2024-03-16T09:01:10.108 INFO:tasks.workunit.client.1.smithi019.stdout:3/75: write d3/ff [466664,48582] 0 2024-03-16T09:01:10.108 INFO:tasks.workunit.client.0.smithi007.stdout:5/309: symlink d1/d7/d1c/d19/d34/d3e/d41/l70 0 2024-03-16T09:01:10.109 INFO:tasks.workunit.client.0.smithi007.stdout:5/310: write d1/d7/d16/d1e/f6e [630341,49356] 0 2024-03-16T09:01:10.109 INFO:tasks.workunit.client.0.smithi007.stdout:5/311: write d1/d7/d16/d1e/f31 [4517441,68570] 0 2024-03-16T09:01:10.109 INFO:tasks.workunit.client.1.smithi019.stdout:9/85: link d1/d7/de/c19 d1/d7/d11/c1d 0 2024-03-16T09:01:10.110 INFO:tasks.workunit.client.0.smithi007.stdout:5/312: getdents d1/d7/d1c/d19/d34/d3e/d62 0 2024-03-16T09:01:10.111 INFO:tasks.workunit.client.0.smithi007.stdout:4/247: write d4/f2f [4997059,99828] 0 2024-03-16T09:01:10.111 INFO:tasks.workunit.client.0.smithi007.stdout:4/248: readlink d4/d14/l4f 0 2024-03-16T09:01:10.111 INFO:tasks.workunit.client.0.smithi007.stdout:4/249: write d4/d14/d1c/f29 [446401,79300] 0 2024-03-16T09:01:10.113 INFO:tasks.workunit.client.0.smithi007.stdout:5/313: creat d1/d7/d1c/d19/d34/d3e/d41/f71 x:0 0 0 2024-03-16T09:01:10.113 INFO:tasks.workunit.client.1.smithi019.stdout:4/85: rename d4/d6/lc to d4/l1d 0 2024-03-16T09:01:10.113 INFO:tasks.workunit.client.1.smithi019.stdout:4/86: creat d4/f1e x:0 0 0 2024-03-16T09:01:10.114 INFO:tasks.workunit.client.0.smithi007.stdout:2/432: dwrite dc/f7e [0,4194304] 0 2024-03-16T09:01:10.114 INFO:tasks.workunit.client.0.smithi007.stdout:5/314: mkdir d1/d7/d1c/d19/d34/d5e/d6f/d72 0 2024-03-16T09:01:10.115 INFO:tasks.workunit.client.1.smithi019.stdout:3/76: link d3/d11/f12 d3/d11/f14 0 2024-03-16T09:01:10.115 INFO:tasks.workunit.client.0.smithi007.stdout:4/250: rename d4/d7/f33 to d4/f51 0 2024-03-16T09:01:10.116 INFO:tasks.workunit.client.1.smithi019.stdout:9/86: symlink d1/d7/de/l1e 0 2024-03-16T09:01:10.116 INFO:tasks.workunit.client.1.smithi019.stdout:9/87: readlink d1/d7/de/l1e 0 2024-03-16T09:01:10.116 INFO:tasks.workunit.client.1.smithi019.stdout:9/88: creat d1/d7/de/f1f x:0 0 0 2024-03-16T09:01:10.118 INFO:tasks.workunit.client.1.smithi019.stdout:6/188: write de/d1c/f28 [623941,113876] 0 2024-03-16T09:01:10.118 INFO:tasks.workunit.client.1.smithi019.stdout:6/189: creat de/f45 x:0 0 0 2024-03-16T09:01:10.118 INFO:tasks.workunit.client.0.smithi007.stdout:2/433: link dc/df/d24/d28/d4c/f59 dc/df/d24/d28/d4c/d5e/f8b 0 2024-03-16T09:01:10.118 INFO:tasks.workunit.client.0.smithi007.stdout:2/434: read - dc/df/d24/d28/d4c/f58 zero size 2024-03-16T09:01:10.119 INFO:tasks.workunit.client.0.smithi007.stdout:5/315: unlink d1/d7/d1c/d19/d34/d5e/d6f/f55 0 2024-03-16T09:01:10.121 INFO:tasks.workunit.client.0.smithi007.stdout:2/435: mkdir dc/df/d24/d28/d2e/d4a/d8c 0 2024-03-16T09:01:10.123 INFO:tasks.workunit.client.0.smithi007.stdout:5/316: creat d1/d7/d1c/d19/d34/d5e/d6f/f73 x:0 0 0 2024-03-16T09:01:10.124 INFO:tasks.workunit.client.0.smithi007.stdout:5/317: creat d1/d7/d1c/d19/d34/d1d/d36/f74 x:0 0 0 2024-03-16T09:01:10.125 INFO:tasks.workunit.client.1.smithi019.stdout:0/61: sync 2024-03-16T09:01:10.125 INFO:tasks.workunit.client.1.smithi019.stdout:0/62: chown d7/le 113 1 2024-03-16T09:01:10.125 INFO:tasks.workunit.client.1.smithi019.stdout:0/63: truncate d7/d8/fc 216361 0 2024-03-16T09:01:10.125 INFO:tasks.workunit.client.1.smithi019.stdout:2/59: sync 2024-03-16T09:01:10.125 INFO:tasks.workunit.client.1.smithi019.stdout:5/84: dwrite d6/f8 [0,4194304] 0 2024-03-16T09:01:10.126 INFO:tasks.workunit.client.0.smithi007.stdout:5/318: fdatasync d1/d7/d1c/d19/d34/d3e/f69 0 2024-03-16T09:01:10.126 INFO:tasks.workunit.client.1.smithi019.stdout:0/64: mkdir d7/d14 0 2024-03-16T09:01:10.127 INFO:tasks.workunit.client.1.smithi019.stdout:2/60: mknod d5/c16 0 2024-03-16T09:01:10.128 INFO:tasks.workunit.client.1.smithi019.stdout:0/65: dread d7/d8/f12 [0,4194304] 0 2024-03-16T09:01:10.128 INFO:tasks.workunit.client.1.smithi019.stdout:0/66: write d7/fd [204940,118392] 0 2024-03-16T09:01:10.130 INFO:tasks.workunit.client.0.smithi007.stdout:2/436: dread dc/df/f29 [0,4194304] 0 2024-03-16T09:01:10.130 INFO:tasks.workunit.client.0.smithi007.stdout:2/437: fdatasync dc/df/d24/d28/d4c/f64 0 2024-03-16T09:01:10.130 INFO:tasks.workunit.client.0.smithi007.stdout:5/319: rmdir d1/d7/d1c/d19/d34/d3e/d62 39 2024-03-16T09:01:10.130 INFO:tasks.workunit.client.0.smithi007.stdout:7/237: dwrite d0/dc/f15 [0,4194304] 0 2024-03-16T09:01:10.130 INFO:tasks.workunit.client.0.smithi007.stdout:4/251: dread d4/d14/d1c/f26 [0,4194304] 0 2024-03-16T09:01:10.132 INFO:tasks.workunit.client.0.smithi007.stdout:5/320: creat d1/d7/d1c/d19/d34/d3e/d41/d48/f75 x:0 0 0 2024-03-16T09:01:10.132 INFO:tasks.workunit.client.0.smithi007.stdout:5/321: read - d1/d7/d1c/d19/d34/d5e/d6f/f73 zero size 2024-03-16T09:01:10.133 INFO:tasks.workunit.client.1.smithi019.stdout:2/61: unlink d5/d6/f14 0 2024-03-16T09:01:10.133 INFO:tasks.workunit.client.0.smithi007.stdout:7/238: symlink d0/d19/d1f/l4c 0 2024-03-16T09:01:10.134 INFO:tasks.workunit.client.0.smithi007.stdout:4/252: symlink d4/d7/l52 0 2024-03-16T09:01:10.135 INFO:tasks.workunit.client.0.smithi007.stdout:2/438: getdents dc 0 2024-03-16T09:01:10.135 INFO:tasks.workunit.client.0.smithi007.stdout:2/439: chown dc/df/d24/d28/d2e/d4a/f7a 966964781 1 2024-03-16T09:01:10.135 INFO:tasks.workunit.client.0.smithi007.stdout:2/440: write dc/df/d6c/f86 [88124,16422] 0 2024-03-16T09:01:10.136 INFO:tasks.workunit.client.1.smithi019.stdout:4/87: rename d4/d6/d7 to d4/d6/d15/d1f 0 2024-03-16T09:01:10.136 INFO:tasks.workunit.client.1.smithi019.stdout:0/67: rmdir d7 39 2024-03-16T09:01:10.136 INFO:tasks.workunit.client.1.smithi019.stdout:0/68: chown d7 29397 1 2024-03-16T09:01:10.137 INFO:tasks.workunit.client.0.smithi007.stdout:8/590: dwrite d0/d26/f5b [0,4194304] 0 2024-03-16T09:01:10.137 INFO:tasks.workunit.client.0.smithi007.stdout:8/591: write d0/d26/d95/f9b [5216911,123304] 0 2024-03-16T09:01:10.137 INFO:tasks.workunit.client.0.smithi007.stdout:2/441: symlink dc/df/d24/l8d 0 2024-03-16T09:01:10.139 INFO:tasks.workunit.client.0.smithi007.stdout:1/343: dwrite d9/dd/d1b/d28/f46 [0,4194304] 0 2024-03-16T09:01:10.140 INFO:tasks.workunit.client.0.smithi007.stdout:5/322: dread d1/f2 [0,4194304] 0 2024-03-16T09:01:10.140 INFO:tasks.workunit.client.0.smithi007.stdout:5/323: readlink d1/d7/d1c/d19/d34/de/l35 0 2024-03-16T09:01:10.140 INFO:tasks.workunit.client.1.smithi019.stdout:4/88: mknod d4/d6/d15/d1f/c20 0 2024-03-16T09:01:10.140 INFO:tasks.workunit.client.1.smithi019.stdout:4/89: fsync d4/d6/f17 0 2024-03-16T09:01:10.140 INFO:tasks.workunit.client.1.smithi019.stdout:4/90: chown d4/d6/d15/d1f/c20 14 1 2024-03-16T09:01:10.141 INFO:tasks.workunit.client.0.smithi007.stdout:8/592: rmdir d0/d26/d2d/d43/dc6 0 2024-03-16T09:01:10.141 INFO:tasks.workunit.client.1.smithi019.stdout:0/69: creat d7/f15 x:0 0 0 2024-03-16T09:01:10.141 INFO:tasks.workunit.client.1.smithi019.stdout:8/87: sync 2024-03-16T09:01:10.143 INFO:tasks.workunit.client.0.smithi007.stdout:2/442: creat dc/d27/f8e x:0 0 0 2024-03-16T09:01:10.143 INFO:tasks.workunit.client.0.smithi007.stdout:2/443: creat dc/d27/f8f x:0 0 0 2024-03-16T09:01:10.144 INFO:tasks.workunit.client.0.smithi007.stdout:4/253: dread d4/d14/f35 [0,4194304] 0 2024-03-16T09:01:10.146 INFO:tasks.workunit.client.1.smithi019.stdout:9/89: rename d1/d7/d11/f15 to d1/f20 0 2024-03-16T09:01:10.146 INFO:tasks.workunit.client.1.smithi019.stdout:9/90: truncate d1/d7/de/f1f 1028541 0 2024-03-16T09:01:10.146 INFO:tasks.workunit.client.1.smithi019.stdout:6/190: dwrite de/f15 [0,4194304] 0 2024-03-16T09:01:10.147 INFO:tasks.workunit.client.1.smithi019.stdout:6/191: dread - de/d26/f40 zero size 2024-03-16T09:01:10.147 INFO:tasks.workunit.client.1.smithi019.stdout:6/192: creat de/d1c/d1d/f46 x:0 0 0 2024-03-16T09:01:10.147 INFO:tasks.workunit.client.0.smithi007.stdout:5/324: mknod d1/d7/c76 0 2024-03-16T09:01:10.148 INFO:tasks.workunit.client.1.smithi019.stdout:7/68: truncate f6 3287565 0 2024-03-16T09:01:10.150 INFO:tasks.workunit.client.0.smithi007.stdout:2/444: symlink dc/df/d24/d28/d2e/d4a/l90 0 2024-03-16T09:01:10.153 INFO:tasks.workunit.client.1.smithi019.stdout:8/88: truncate d6/de/d16/d19/f1a 2776553 0 2024-03-16T09:01:10.153 INFO:tasks.workunit.client.1.smithi019.stdout:8/89: fdatasync d6/de/ff 0 2024-03-16T09:01:10.153 INFO:tasks.workunit.client.1.smithi019.stdout:8/90: chown l1 3 1 2024-03-16T09:01:10.154 INFO:tasks.workunit.client.0.smithi007.stdout:4/254: creat d4/d14/d1c/d1e/f53 x:0 0 0 2024-03-16T09:01:10.154 INFO:tasks.workunit.client.0.smithi007.stdout:4/255: dread - d4/d14/d1c/f32 zero size 2024-03-16T09:01:10.154 INFO:tasks.workunit.client.0.smithi007.stdout:5/325: creat d1/d7/d1c/d19/d34/d4d/f77 x:0 0 0 2024-03-16T09:01:10.160 INFO:tasks.workunit.client.1.smithi019.stdout:6/193: creat de/d1c/f47 x:0 0 0 2024-03-16T09:01:10.161 INFO:tasks.workunit.client.1.smithi019.stdout:7/69: mknod d9/da/d13/c15 0 2024-03-16T09:01:10.165 INFO:tasks.workunit.client.1.smithi019.stdout:8/91: rename d6/cb to d6/de/d16/c23 0 2024-03-16T09:01:10.166 INFO:tasks.workunit.client.1.smithi019.stdout:8/92: fsync d6/de/f1b 0 2024-03-16T09:01:10.166 INFO:tasks.workunit.client.1.smithi019.stdout:8/93: chown d6/de/d16/d19/c1f 472 1 2024-03-16T09:01:10.166 INFO:tasks.workunit.client.1.smithi019.stdout:8/94: write d6/de/ff [931430,47949] 0 2024-03-16T09:01:10.167 INFO:tasks.workunit.client.0.smithi007.stdout:5/326: read d1/d7/d16/f33 [3544,85237] 0 2024-03-16T09:01:10.167 INFO:tasks.workunit.client.0.smithi007.stdout:2/445: dread dc/df/d24/d28/d2e/d4a/f7a [0,4194304] 0 2024-03-16T09:01:10.167 INFO:tasks.workunit.client.0.smithi007.stdout:2/446: fdatasync f7 0 2024-03-16T09:01:10.167 INFO:tasks.workunit.client.0.smithi007.stdout:2/447: chown dc/f3f 31 1 2024-03-16T09:01:10.168 INFO:tasks.workunit.client.1.smithi019.stdout:9/91: mkdir d1/d7/d21 0 2024-03-16T09:01:10.168 INFO:tasks.workunit.client.1.smithi019.stdout:9/92: write d1/d7/de/ff [716212,117433] 0 2024-03-16T09:01:10.168 INFO:tasks.workunit.client.1.smithi019.stdout:9/93: creat d1/d7/d11/f22 x:0 0 0 2024-03-16T09:01:10.168 INFO:tasks.workunit.client.1.smithi019.stdout:9/94: dread d1/d7/de/f1f [0,4194304] 0 2024-03-16T09:01:10.168 INFO:tasks.workunit.client.1.smithi019.stdout:9/95: fdatasync d1/d7/d11/f13 0 2024-03-16T09:01:10.168 INFO:tasks.workunit.client.1.smithi019.stdout:6/194: mknod de/d1c/d1d/d2c/c48 0 2024-03-16T09:01:10.168 INFO:tasks.workunit.client.1.smithi019.stdout:6/195: creat de/f49 x:0 0 0 2024-03-16T09:01:10.169 INFO:tasks.workunit.client.0.smithi007.stdout:1/344: dwrite d9/f12 [0,4194304] 0 2024-03-16T09:01:10.169 INFO:tasks.workunit.client.0.smithi007.stdout:4/256: dwrite d4/d14/d1c/f2c [0,4194304] 0 2024-03-16T09:01:10.169 INFO:tasks.workunit.client.0.smithi007.stdout:7/239: dwrite d0/dc/d24/f3b [0,4194304] 0 2024-03-16T09:01:10.172 INFO:tasks.workunit.client.1.smithi019.stdout:2/62: dwrite f4 [0,4194304] 0 2024-03-16T09:01:10.172 INFO:tasks.workunit.client.1.smithi019.stdout:7/70: dread f8 [0,4194304] 0 2024-03-16T09:01:10.178 INFO:tasks.workunit.client.1.smithi019.stdout:9/96: mknod d1/d7/de/c23 0 2024-03-16T09:01:10.179 INFO:tasks.workunit.client.1.smithi019.stdout:3/77: rmdir d3/d11 39 2024-03-16T09:01:10.179 INFO:tasks.workunit.client.1.smithi019.stdout:3/78: dread - d3/d11/f14 zero size 2024-03-16T09:01:10.179 INFO:tasks.workunit.client.1.smithi019.stdout:3/79: rename d3 to d3/d15 22 2024-03-16T09:01:10.181 INFO:tasks.workunit.client.0.smithi007.stdout:1/345: dread d9/f12 [0,4194304] 0 2024-03-16T09:01:10.184 INFO:tasks.workunit.client.0.smithi007.stdout:4/257: creat d4/f54 x:0 0 0 2024-03-16T09:01:10.185 INFO:tasks.workunit.client.1.smithi019.stdout:2/63: mknod d5/de/d10/c17 0 2024-03-16T09:01:10.185 INFO:tasks.workunit.client.1.smithi019.stdout:2/64: dread - d5/d6/f11 zero size 2024-03-16T09:01:10.186 INFO:tasks.workunit.client.0.smithi007.stdout:7/240: write d0/dc/d24/f3b [2371284,24001] 0 2024-03-16T09:01:10.187 INFO:tasks.workunit.client.1.smithi019.stdout:7/71: creat d9/f16 x:0 0 0 2024-03-16T09:01:10.188 INFO:tasks.workunit.client.0.smithi007.stdout:7/241: read d0/fa [4810668,59463] 0 2024-03-16T09:01:10.189 INFO:tasks.workunit.client.1.smithi019.stdout:9/97: readlink d1/d7/la 0 2024-03-16T09:01:10.192 INFO:tasks.workunit.client.1.smithi019.stdout:7/72: link f7 d9/da/d13/f17 0 2024-03-16T09:01:10.193 INFO:tasks.workunit.client.0.smithi007.stdout:7/242: unlink d0/dc/f1a 0 2024-03-16T09:01:10.195 INFO:tasks.workunit.client.1.smithi019.stdout:9/98: mkdir d1/d7/de/d24 0 2024-03-16T09:01:10.196 INFO:tasks.workunit.client.0.smithi007.stdout:4/258: write d4/d14/d1c/f26 [2160350,4334] 0 2024-03-16T09:01:10.196 INFO:tasks.workunit.client.0.smithi007.stdout:4/259: chown d4/d7/c41 6844 1 2024-03-16T09:01:10.197 INFO:tasks.workunit.client.1.smithi019.stdout:4/91: truncate d4/f12 3234071 0 2024-03-16T09:01:10.197 INFO:tasks.workunit.client.1.smithi019.stdout:4/92: dread - d4/d6/f17 zero size 2024-03-16T09:01:10.197 INFO:tasks.workunit.client.1.smithi019.stdout:4/93: dread - d4/d6/f17 zero size 2024-03-16T09:01:10.198 INFO:tasks.workunit.client.1.smithi019.stdout:9/99: creat d1/f25 x:0 0 0 2024-03-16T09:01:10.206 INFO:tasks.workunit.client.1.smithi019.stdout:2/65: write f1 [3145773,106125] 0 2024-03-16T09:01:10.208 INFO:tasks.workunit.client.1.smithi019.stdout:2/66: creat d5/de/d10/f18 x:0 0 0 2024-03-16T09:01:10.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:09 smithi019 ceph-mon[36407]: pgmap v18: 65 pgs: 65 active+clean; 591 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 49 MiB/s rd, 66 MiB/s wr, 874 op/s 2024-03-16T09:01:10.217 INFO:tasks.workunit.client.1.smithi019.stdout:4/94: write f2 [1277383,131054] 0 2024-03-16T09:01:10.225 INFO:tasks.workunit.client.1.smithi019.stdout:6/196: dwrite de/f3c [0,4194304] 0 2024-03-16T09:01:10.225 INFO:tasks.workunit.client.0.smithi007.stdout:5/327: dwrite d1/d7/d1c/d19/d34/d5e/f56 [0,4194304] 0 2024-03-16T09:01:10.226 INFO:tasks.workunit.client.1.smithi019.stdout:6/197: write de/d1c/f27 [747912,37685] 0 2024-03-16T09:01:10.226 INFO:tasks.workunit.client.1.smithi019.stdout:6/198: chown de/d1c/d1d/c23 1 1 2024-03-16T09:01:10.226 INFO:tasks.workunit.client.0.smithi007.stdout:2/448: dwrite dc/df/f34 [0,4194304] 0 2024-03-16T09:01:10.227 INFO:tasks.workunit.client.0.smithi007.stdout:5/328: truncate d1/d7/d16/d1e/f31 750009 0 2024-03-16T09:01:10.227 INFO:tasks.workunit.client.0.smithi007.stdout:5/329: readlink d1/d7/d1c/d19/d34/de/l50 0 2024-03-16T09:01:10.228 INFO:tasks.workunit.client.1.smithi019.stdout:6/199: unlink de/c19 0 2024-03-16T09:01:10.228 INFO:tasks.workunit.client.1.smithi019.stdout:6/200: creat de/d26/d32/f4a x:0 0 0 2024-03-16T09:01:10.229 INFO:tasks.workunit.client.0.smithi007.stdout:2/449: creat dc/df/d24/d28/d2e/d35/f91 x:0 0 0 2024-03-16T09:01:10.230 INFO:tasks.workunit.client.0.smithi007.stdout:5/330: symlink d1/d7/d16/d1b/d3c/l78 0 2024-03-16T09:01:10.230 INFO:tasks.workunit.client.0.smithi007.stdout:5/331: stat d1/d7/d1c/d19/d34/d3e/d41/d48 0 2024-03-16T09:01:10.231 INFO:tasks.workunit.client.0.smithi007.stdout:5/332: mkdir d1/d7/d16/d1e/d2a/d79 0 2024-03-16T09:01:10.250 INFO:tasks.workunit.client.1.smithi019.stdout:3/80: dwrite d3/f5 [4194304,4194304] 0 2024-03-16T09:01:10.250 INFO:tasks.workunit.client.1.smithi019.stdout:3/81: rename d3 to d3/d11/d16 22 2024-03-16T09:01:10.251 INFO:tasks.workunit.client.1.smithi019.stdout:7/73: dwrite f8 [0,4194304] 0 2024-03-16T09:01:10.251 INFO:tasks.workunit.client.0.smithi007.stdout:1/346: dwrite d9/dd/d1b/d28/d29/f42 [4194304,4194304] 0 2024-03-16T09:01:10.251 INFO:tasks.workunit.client.0.smithi007.stdout:1/347: chown d9/dd/d1b/d28/d4c/f63 988 1 2024-03-16T09:01:10.251 INFO:tasks.workunit.client.0.smithi007.stdout:1/348: truncate d9/dd/d1b/d2d/f3f 976646 0 2024-03-16T09:01:10.252 INFO:tasks.workunit.client.1.smithi019.stdout:2/67: dwrite d5/de/d10/f18 [0,4194304] 0 2024-03-16T09:01:10.253 INFO:tasks.workunit.client.1.smithi019.stdout:9/100: dwrite d1/d7/d11/f22 [0,4194304] 0 2024-03-16T09:01:10.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/243: dwrite d0/d19/f2b [0,4194304] 0 2024-03-16T09:01:10.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/244: fdatasync d0/dc/d24/d25/d3a/f44 0 2024-03-16T09:01:10.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/245: creat d0/d19/f4d x:0 0 0 2024-03-16T09:01:10.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/246: chown d0/dc/d24/f46 11552506 1 2024-03-16T09:01:10.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/247: read - d0/d19/d1f/d42/f45 zero size 2024-03-16T09:01:10.254 INFO:tasks.workunit.client.0.smithi007.stdout:1/349: rename d9/dd/d1b/d2d/f57 to d9/dd/d1b/d2d/f6a 0 2024-03-16T09:01:10.254 INFO:tasks.workunit.client.0.smithi007.stdout:1/350: write f5 [829116,89937] 0 2024-03-16T09:01:10.258 INFO:tasks.workunit.client.0.smithi007.stdout:7/248: mkdir d0/dc/d38/d4e 0 2024-03-16T09:01:10.258 INFO:tasks.workunit.client.0.smithi007.stdout:7/249: truncate d0/d19/f27 1477955 0 2024-03-16T09:01:10.277 INFO:tasks.workunit.client.1.smithi019.stdout:6/201: unlink de/d26/f40 0 2024-03-16T09:01:10.280 INFO:tasks.workunit.client.1.smithi019.stdout:4/95: dwrite d4/d1b/f1c [0,4194304] 0 2024-03-16T09:01:10.281 INFO:tasks.workunit.client.1.smithi019.stdout:3/82: mkdir d3/d17 0 2024-03-16T09:01:10.284 INFO:tasks.workunit.client.1.smithi019.stdout:2/68: dwrite d5/de/fd [0,4194304] 0 2024-03-16T09:01:10.287 INFO:tasks.workunit.client.1.smithi019.stdout:9/101: mknod d1/d7/d21/c26 0 2024-03-16T09:01:10.287 INFO:tasks.workunit.client.1.smithi019.stdout:9/102: write d1/d7/de/f1a [316279,75293] 0 2024-03-16T09:01:10.291 INFO:tasks.workunit.client.0.smithi007.stdout:5/333: dwrite d1/d7/d1c/d19/d34/d5e/d6f/f54 [0,4194304] 0 2024-03-16T09:01:10.291 INFO:tasks.workunit.client.0.smithi007.stdout:4/260: dwrite d4/f2f [4194304,4194304] 0 2024-03-16T09:01:10.291 INFO:tasks.workunit.client.0.smithi007.stdout:4/261: chown d4/d14/d1c/d1e/f4b 45776 1 2024-03-16T09:01:10.291 INFO:tasks.workunit.client.0.smithi007.stdout:4/262: chown d4/l3c 2 1 2024-03-16T09:01:10.291 INFO:tasks.workunit.client.0.smithi007.stdout:4/263: write d4/ff [2777512,50062] 0 2024-03-16T09:01:10.292 INFO:tasks.workunit.client.1.smithi019.stdout:6/202: read de/f1a [3916926,101311] 0 2024-03-16T09:01:10.292 INFO:tasks.workunit.client.1.smithi019.stdout:6/203: write de/d1c/d1d/f46 [694736,17666] 0 2024-03-16T09:01:10.292 INFO:tasks.workunit.client.1.smithi019.stdout:4/96: symlink d4/d1b/l21 0 2024-03-16T09:01:10.292 INFO:tasks.workunit.client.1.smithi019.stdout:6/204: readlink de/d1c/l2d 0 2024-03-16T09:01:10.292 INFO:tasks.workunit.client.1.smithi019.stdout:6/205: fdatasync de/d1c/d1d/f30 0 2024-03-16T09:01:10.292 INFO:tasks.workunit.client.1.smithi019.stdout:4/97: dread - d4/d6/f17 zero size 2024-03-16T09:01:10.292 INFO:tasks.workunit.client.1.smithi019.stdout:4/98: write f2 [2308682,32108] 0 2024-03-16T09:01:10.293 INFO:tasks.workunit.client.0.smithi007.stdout:2/450: dwrite dc/df/d24/d28/d2e/f45 [0,4194304] 0 2024-03-16T09:01:10.293 INFO:tasks.workunit.client.1.smithi019.stdout:3/83: unlink d3/f5 0 2024-03-16T09:01:10.293 INFO:tasks.workunit.client.0.smithi007.stdout:1/351: dwrite d9/f39 [0,4194304] 0 2024-03-16T09:01:10.293 INFO:tasks.workunit.client.0.smithi007.stdout:1/352: fsync d9/dd/d1b/d2d/d30/f61 0 2024-03-16T09:01:10.297 INFO:tasks.workunit.client.1.smithi019.stdout:2/69: mknod d5/d6/c19 0 2024-03-16T09:01:10.297 INFO:tasks.workunit.client.1.smithi019.stdout:2/70: write d5/f7 [1477257,453] 0 2024-03-16T09:01:10.297 INFO:tasks.workunit.client.1.smithi019.stdout:6/206: dread de/d1c/d1d/f46 [0,4194304] 0 2024-03-16T09:01:10.299 INFO:tasks.workunit.client.0.smithi007.stdout:1/353: mkdir d9/d6b 0 2024-03-16T09:01:10.300 INFO:tasks.workunit.client.0.smithi007.stdout:5/334: dread d1/d7/d16/d1e/f47 [0,4194304] 0 2024-03-16T09:01:10.300 INFO:tasks.workunit.client.0.smithi007.stdout:5/335: creat d1/d7/d1c/d19/d34/d3e/d41/f7a x:0 0 0 2024-03-16T09:01:10.302 INFO:tasks.workunit.client.0.smithi007.stdout:1/354: unlink d9/dd/d1b/d2d/l37 0 2024-03-16T09:01:10.304 INFO:tasks.workunit.client.0.smithi007.stdout:2/451: write dc/f10 [266210,128287] 0 2024-03-16T09:01:10.304 INFO:tasks.workunit.client.0.smithi007.stdout:2/452: creat dc/d27/f92 x:0 0 0 2024-03-16T09:01:10.305 INFO:tasks.workunit.client.0.smithi007.stdout:5/336: mknod d1/d7/d1c/c7b 0 2024-03-16T09:01:10.305 INFO:tasks.workunit.client.0.smithi007.stdout:5/337: creat d1/d7/f7c x:0 0 0 2024-03-16T09:01:10.308 INFO:tasks.workunit.client.1.smithi019.stdout:2/71: dread d5/f7 [0,4194304] 0 2024-03-16T09:01:10.308 INFO:tasks.workunit.client.1.smithi019.stdout:2/72: dread - d5/d6/f11 zero size 2024-03-16T09:01:10.308 INFO:tasks.workunit.client.0.smithi007.stdout:2/453: symlink dc/df/d24/d28/d2e/d35/l93 0 2024-03-16T09:01:10.308 INFO:tasks.workunit.client.0.smithi007.stdout:2/454: write dc/df/d24/d28/d4c/f57 [963041,111569] 0 2024-03-16T09:01:10.308 INFO:tasks.workunit.client.0.smithi007.stdout:2/455: write dc/fd [11587,49239] 0 2024-03-16T09:01:10.309 INFO:tasks.workunit.client.1.smithi019.stdout:3/84: creat d3/d17/f18 x:0 0 0 2024-03-16T09:01:10.310 INFO:tasks.workunit.client.0.smithi007.stdout:5/338: mkdir d1/d7/d16/d1e/d7d 0 2024-03-16T09:01:10.310 INFO:tasks.workunit.client.0.smithi007.stdout:5/339: write d1/d7/d16/d1e/f31 [1151729,92112] 0 2024-03-16T09:01:10.311 INFO:tasks.workunit.client.1.smithi019.stdout:6/207: symlink de/d1c/l4b 0 2024-03-16T09:01:10.313 INFO:tasks.workunit.client.1.smithi019.stdout:3/85: symlink d3/d17/l19 0 2024-03-16T09:01:10.313 INFO:tasks.workunit.client.0.smithi007.stdout:4/264: dwrite d4/d14/d1c/d1e/f4c [0,4194304] 0 2024-03-16T09:01:10.313 INFO:tasks.workunit.client.0.smithi007.stdout:1/355: getdents d9 0 2024-03-16T09:01:10.313 INFO:tasks.workunit.client.0.smithi007.stdout:1/356: write d9/dd/d1b/d28/f32 [1880471,110455] 0 2024-03-16T09:01:10.314 INFO:tasks.workunit.client.1.smithi019.stdout:6/208: link de/d1c/d1d/d2c/c33 de/d26/d32/c4c 0 2024-03-16T09:01:10.314 INFO:tasks.workunit.client.1.smithi019.stdout:6/209: creat de/f4d x:0 0 0 2024-03-16T09:01:10.315 INFO:tasks.workunit.client.1.smithi019.stdout:9/103: rename d1/d7/d11 to d1/d7/d21/d27 0 2024-03-16T09:01:10.316 INFO:tasks.workunit.client.0.smithi007.stdout:5/340: rename d1/d7/d1c/d19/d34/d3e/d62/c6d to d1/d7/d1c/d19/d34/d5e/c7e 0 2024-03-16T09:01:10.316 INFO:tasks.workunit.client.1.smithi019.stdout:3/86: symlink d3/d11/l1a 0 2024-03-16T09:01:10.317 INFO:tasks.workunit.client.1.smithi019.stdout:3/87: fdatasync f2 0 2024-03-16T09:01:10.317 INFO:tasks.workunit.client.0.smithi007.stdout:2/456: getdents dc/df 0 2024-03-16T09:01:10.318 INFO:tasks.workunit.client.1.smithi019.stdout:6/210: mkdir de/d26/d32/d4e 0 2024-03-16T09:01:10.318 INFO:tasks.workunit.client.1.smithi019.stdout:6/211: readlink l5 0 2024-03-16T09:01:10.321 INFO:tasks.workunit.client.1.smithi019.stdout:3/88: unlink d3/f10 0 2024-03-16T09:01:10.321 INFO:tasks.workunit.client.1.smithi019.stdout:3/89: write d3/d17/f18 [595252,42226] 0 2024-03-16T09:01:10.323 INFO:tasks.workunit.client.0.smithi007.stdout:1/357: rename d9/dd/d1b/d28/f2b to d9/dd/d1b/d58/f6c 0 2024-03-16T09:01:10.324 INFO:tasks.workunit.client.0.smithi007.stdout:5/341: mknod d1/d7/d16/d1e/d2a/c7f 0 2024-03-16T09:01:10.325 INFO:tasks.workunit.client.0.smithi007.stdout:4/265: dread d4/f2f [4194304,4194304] 0 2024-03-16T09:01:10.326 INFO:tasks.workunit.client.1.smithi019.stdout:9/104: rename d1/d7/de/l1e to d1/l28 0 2024-03-16T09:01:10.327 INFO:tasks.workunit.client.0.smithi007.stdout:4/266: mknod d4/d14/c55 0 2024-03-16T09:01:10.327 INFO:tasks.workunit.client.0.smithi007.stdout:4/267: readlink d4/d14/l4f 0 2024-03-16T09:01:10.327 INFO:tasks.workunit.client.0.smithi007.stdout:4/268: creat d4/d14/d34/f56 x:0 0 0 2024-03-16T09:01:10.328 INFO:tasks.workunit.client.1.smithi019.stdout:3/90: mkdir d3/d1b 0 2024-03-16T09:01:10.329 INFO:tasks.workunit.client.1.smithi019.stdout:6/212: getdents de/d26 0 2024-03-16T09:01:10.331 INFO:tasks.workunit.client.0.smithi007.stdout:4/269: getdents d4 0 2024-03-16T09:01:10.332 INFO:tasks.workunit.client.1.smithi019.stdout:9/105: mknod d1/d7/de/d24/c29 0 2024-03-16T09:01:10.333 INFO:tasks.workunit.client.1.smithi019.stdout:8/95: truncate d6/de/d16/d19/f1a 1340740 0 2024-03-16T09:01:10.335 INFO:tasks.workunit.client.1.smithi019.stdout:4/99: write d4/d1b/f1c [4859032,9944] 0 2024-03-16T09:01:10.335 INFO:tasks.workunit.client.1.smithi019.stdout:4/100: stat d4 0 2024-03-16T09:01:10.335 INFO:tasks.workunit.client.1.smithi019.stdout:4/101: write f2 [1384609,4766] 0 2024-03-16T09:01:10.335 INFO:tasks.workunit.client.0.smithi007.stdout:2/457: dwrite dc/df/f29 [0,4194304] 0 2024-03-16T09:01:10.336 INFO:tasks.workunit.client.1.smithi019.stdout:3/91: unlink d3/d17/f18 0 2024-03-16T09:01:10.338 INFO:tasks.workunit.client.1.smithi019.stdout:8/96: mkdir d6/de/d24 0 2024-03-16T09:01:10.339 INFO:tasks.workunit.client.0.smithi007.stdout:2/458: mknod dc/df/c94 0 2024-03-16T09:01:10.339 INFO:tasks.workunit.client.0.smithi007.stdout:2/459: fdatasync dc/df/f14 0 2024-03-16T09:01:10.344 INFO:tasks.workunit.client.0.smithi007.stdout:4/270: dread d4/d14/d1c/f26 [0,4194304] 0 2024-03-16T09:01:10.346 INFO:tasks.workunit.client.0.smithi007.stdout:4/271: symlink d4/d14/l57 0 2024-03-16T09:01:10.346 INFO:tasks.workunit.client.0.smithi007.stdout:4/272: fsync d4/d14/d34/f45 0 2024-03-16T09:01:10.346 INFO:tasks.workunit.client.0.smithi007.stdout:4/273: write d4/f2f [3130132,31283] 0 2024-03-16T09:01:10.348 INFO:tasks.workunit.client.1.smithi019.stdout:8/97: mknod d6/de/d16/c25 0 2024-03-16T09:01:10.351 INFO:tasks.workunit.client.0.smithi007.stdout:5/342: dwrite d1/d7/f12 [0,4194304] 0 2024-03-16T09:01:10.352 INFO:tasks.workunit.client.0.smithi007.stdout:5/343: dread - d1/d7/d1c/d19/d34/d1d/d36/f74 zero size 2024-03-16T09:01:10.352 INFO:tasks.workunit.client.0.smithi007.stdout:5/344: creat d1/d7/d1c/f80 x:0 0 0 2024-03-16T09:01:10.355 INFO:tasks.workunit.client.0.smithi007.stdout:4/274: write d4/f2f [6785378,46058] 0 2024-03-16T09:01:10.355 INFO:tasks.workunit.client.1.smithi019.stdout:4/102: write d4/d1b/f1c [85073,50105] 0 2024-03-16T09:01:10.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/275: mknod d4/c58 0 2024-03-16T09:01:10.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/276: creat d4/d14/d1c/d1e/f59 x:0 0 0 2024-03-16T09:01:10.359 INFO:tasks.workunit.client.1.smithi019.stdout:4/103: rename d4/d6/d15/d1f/c20 to d4/d6/c22 0 2024-03-16T09:01:10.359 INFO:tasks.workunit.client.1.smithi019.stdout:8/98: dread d6/de/f1b [0,4194304] 0 2024-03-16T09:01:10.361 INFO:tasks.workunit.client.0.smithi007.stdout:4/277: rename d4/l24 to d4/d7/l5a 0 2024-03-16T09:01:10.362 INFO:tasks.workunit.client.1.smithi019.stdout:4/104: rename d4/d6/f17 to d4/f23 0 2024-03-16T09:01:10.362 INFO:tasks.workunit.client.1.smithi019.stdout:4/105: creat d4/d6/f24 x:0 0 0 2024-03-16T09:01:10.363 INFO:tasks.workunit.client.1.smithi019.stdout:6/213: dwrite de/d1c/d1d/f30 [0,4194304] 0 2024-03-16T09:01:10.363 INFO:tasks.workunit.client.0.smithi007.stdout:4/278: mkdir d4/d14/d1c/d5b 0 2024-03-16T09:01:10.363 INFO:tasks.workunit.client.1.smithi019.stdout:8/99: creat d6/de/d16/f26 x:0 0 0 2024-03-16T09:01:10.364 INFO:tasks.workunit.client.1.smithi019.stdout:8/100: chown d6/de/c11 66 1 2024-03-16T09:01:10.364 INFO:tasks.workunit.client.0.smithi007.stdout:4/279: rename d4/d14/l48 to d4/d14/d1c/d1e/l5c 0 2024-03-16T09:01:10.365 INFO:tasks.workunit.client.0.smithi007.stdout:1/358: dwrite d9/f39 [0,4194304] 0 2024-03-16T09:01:10.366 INFO:tasks.workunit.client.1.smithi019.stdout:4/106: symlink d4/d6/d15/l25 0 2024-03-16T09:01:10.366 INFO:tasks.workunit.client.0.smithi007.stdout:4/280: truncate d4/f5 2198383 0 2024-03-16T09:01:10.369 INFO:tasks.workunit.client.0.smithi007.stdout:4/281: truncate d4/f10 2267185 0 2024-03-16T09:01:10.376 INFO:tasks.workunit.client.0.smithi007.stdout:4/282: rename d4/f28 to d4/d14/d1c/d5b/f5d 0 2024-03-16T09:01:10.379 INFO:tasks.workunit.client.0.smithi007.stdout:4/283: creat d4/d14/d1c/d5b/f5e x:0 0 0 2024-03-16T09:01:10.382 INFO:tasks.workunit.client.0.smithi007.stdout:4/284: creat d4/f5f x:0 0 0 2024-03-16T09:01:10.387 INFO:tasks.workunit.client.0.smithi007.stdout:2/460: dwrite dc/df/d24/d28/d2e/d4a/f7a [4194304,4194304] 0 2024-03-16T09:01:10.387 INFO:tasks.workunit.client.0.smithi007.stdout:2/461: chown f7 3 1 2024-03-16T09:01:10.389 INFO:tasks.workunit.client.0.smithi007.stdout:2/462: rmdir dc/df/d24/d28/d2e/d4a/d8c 0 2024-03-16T09:01:10.390 INFO:tasks.workunit.client.0.smithi007.stdout:2/463: symlink dc/df/d24/d28/d2e/d6e/l95 0 2024-03-16T09:01:10.391 INFO:tasks.workunit.client.0.smithi007.stdout:2/464: creat dc/df/d24/d28/d4c/d5e/f96 x:0 0 0 2024-03-16T09:01:10.409 INFO:tasks.workunit.client.0.smithi007.stdout:5/345: dwrite d1/d7/d1c/d19/d34/d5e/d6f/f73 [0,4194304] 0 2024-03-16T09:01:10.410 INFO:tasks.workunit.client.1.smithi019.stdout:3/92: rmdir d3 39 2024-03-16T09:01:10.411 INFO:tasks.workunit.client.0.smithi007.stdout:5/346: mkdir d1/d7/d16/d1e/d2a/d81 0 2024-03-16T09:01:10.412 INFO:tasks.workunit.client.0.smithi007.stdout:5/347: mknod d1/d7/d1c/d19/d34/d5e/d6f/d72/c82 0 2024-03-16T09:01:10.414 INFO:tasks.workunit.client.0.smithi007.stdout:5/348: rename d1/c59 to d1/d7/d16/d1e/d2a/d79/c83 0 2024-03-16T09:01:10.414 INFO:tasks.workunit.client.0.smithi007.stdout:5/349: read - d1/d7/d1c/d19/d34/d3e/d41/d48/f75 zero size 2024-03-16T09:01:10.415 INFO:tasks.workunit.client.0.smithi007.stdout:5/350: write d1/d7/d1c/d19/d34/d3e/f6a [3192193,42237] 0 2024-03-16T09:01:10.416 INFO:tasks.workunit.client.1.smithi019.stdout:6/214: dwrite de/d1c/d1d/f46 [0,4194304] 0 2024-03-16T09:01:10.416 INFO:tasks.workunit.client.1.smithi019.stdout:9/106: dwrite d1/d7/de/f10 [0,4194304] 0 2024-03-16T09:01:10.418 INFO:tasks.workunit.client.1.smithi019.stdout:6/215: unlink de/d26/l41 0 2024-03-16T09:01:10.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/107: dwrite d4/f23 [0,4194304] 0 2024-03-16T09:01:10.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/108: creat d4/f26 x:0 0 0 2024-03-16T09:01:10.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/109: readlink d4/d6/d15/l25 0 2024-03-16T09:01:10.423 INFO:tasks.workunit.client.1.smithi019.stdout:8/101: dwrite d6/f7 [4194304,4194304] 0 2024-03-16T09:01:10.423 INFO:tasks.workunit.client.1.smithi019.stdout:8/102: stat d6/de/d16/d19 0 2024-03-16T09:01:10.424 INFO:tasks.workunit.client.1.smithi019.stdout:8/103: fdatasync d6/de/d16/d19/f18 0 2024-03-16T09:01:10.424 INFO:tasks.workunit.client.0.smithi007.stdout:1/359: dwrite d9/dd/d5e/f65 [0,4194304] 0 2024-03-16T09:01:10.425 INFO:tasks.workunit.client.0.smithi007.stdout:5/351: fsync d1/d7/d1c/d19/d34/d3e/d41/f4c 0 2024-03-16T09:01:10.425 INFO:tasks.workunit.client.0.smithi007.stdout:5/352: chown d1/d7/d16/d1b/f67 8283337 1 2024-03-16T09:01:10.425 INFO:tasks.workunit.client.0.smithi007.stdout:4/285: dwrite d4/d14/d1c/d5b/f5e [0,4194304] 0 2024-03-16T09:01:10.426 INFO:tasks.workunit.client.1.smithi019.stdout:4/110: mknod d4/d6/d15/d1f/c27 0 2024-03-16T09:01:10.426 INFO:tasks.workunit.client.1.smithi019.stdout:4/111: creat d4/d1b/f28 x:0 0 0 2024-03-16T09:01:10.429 INFO:tasks.workunit.client.1.smithi019.stdout:3/93: dwrite d3/d11/f12 [0,4194304] 0 2024-03-16T09:01:10.429 INFO:tasks.workunit.client.1.smithi019.stdout:8/104: dread d6/de/d16/d19/f1a [0,4194304] 0 2024-03-16T09:01:10.432 INFO:tasks.workunit.client.1.smithi019.stdout:4/112: rename d4/d6/d15/d1f/cf to d4/d6/d15/d1f/c29 0 2024-03-16T09:01:10.433 INFO:tasks.workunit.client.1.smithi019.stdout:3/94: dread f2 [0,4194304] 0 2024-03-16T09:01:10.444 INFO:tasks.workunit.client.1.smithi019.stdout:9/107: dwrite d1/d7/de/f1f [0,4194304] 0 2024-03-16T09:01:10.444 INFO:tasks.workunit.client.1.smithi019.stdout:9/108: chown d1/d7/de/c23 2035554491 1 2024-03-16T09:01:10.444 INFO:tasks.workunit.client.1.smithi019.stdout:9/109: creat d1/d7/de/f2a x:0 0 0 2024-03-16T09:01:10.444 INFO:tasks.workunit.client.1.smithi019.stdout:3/95: creat d3/d1b/f1c x:0 0 0 2024-03-16T09:01:10.444 INFO:tasks.workunit.client.0.smithi007.stdout:5/353: truncate d1/d7/d1c/d19/f26 1219223 0 2024-03-16T09:01:10.445 INFO:tasks.workunit.client.0.smithi007.stdout:5/354: fdatasync d1/d7/d1c/f80 0 2024-03-16T09:01:10.445 INFO:tasks.workunit.client.0.smithi007.stdout:5/355: stat d1/d7/d1c/d19/d34/d5e/d6f/f6c 0 2024-03-16T09:01:10.445 INFO:tasks.workunit.client.0.smithi007.stdout:5/356: mknod d1/d7/d16/d1b/c84 0 2024-03-16T09:01:10.445 INFO:tasks.workunit.client.0.smithi007.stdout:5/357: write d1/f2 [4819062,56032] 0 2024-03-16T09:01:10.448 INFO:tasks.workunit.client.1.smithi019.stdout:5/85: sync 2024-03-16T09:01:10.448 INFO:tasks.workunit.client.1.smithi019.stdout:7/74: sync 2024-03-16T09:01:10.449 INFO:tasks.workunit.client.1.smithi019.stdout:6/216: dwrite fb [4194304,4194304] 0 2024-03-16T09:01:10.449 INFO:tasks.workunit.client.1.smithi019.stdout:6/217: creat de/d26/d32/f4f x:0 0 0 2024-03-16T09:01:10.450 INFO:tasks.workunit.client.0.smithi007.stdout:5/358: creat d1/d7/d1c/d19/d34/d5e/d6f/d72/f85 x:0 0 0 2024-03-16T09:01:10.450 INFO:tasks.workunit.client.0.smithi007.stdout:5/359: creat d1/d7/d1c/d19/d34/d4d/f86 x:0 0 0 2024-03-16T09:01:10.450 INFO:tasks.workunit.client.1.smithi019.stdout:9/110: symlink d1/l2b 0 2024-03-16T09:01:10.450 INFO:tasks.workunit.client.1.smithi019.stdout:9/111: write d1/d7/fd [2849876,80641] 0 2024-03-16T09:01:10.451 INFO:tasks.workunit.client.1.smithi019.stdout:8/105: truncate d6/f7 7961897 0 2024-03-16T09:01:10.452 INFO:tasks.workunit.client.1.smithi019.stdout:1/41: sync 2024-03-16T09:01:10.452 INFO:tasks.workunit.client.1.smithi019.stdout:0/70: sync 2024-03-16T09:01:10.452 INFO:tasks.workunit.client.1.smithi019.stdout:2/73: sync 2024-03-16T09:01:10.453 INFO:tasks.workunit.client.0.smithi007.stdout:4/286: dwrite d4/d7/f20 [0,4194304] 0 2024-03-16T09:01:10.453 INFO:tasks.workunit.client.0.smithi007.stdout:4/287: chown d4/f5 327 1 2024-03-16T09:01:10.454 INFO:tasks.workunit.client.1.smithi019.stdout:3/96: rename d3/l13 to d3/d1b/l1d 0 2024-03-16T09:01:10.454 INFO:tasks.workunit.client.1.smithi019.stdout:3/97: fdatasync f2 0 2024-03-16T09:01:10.454 INFO:tasks.workunit.client.1.smithi019.stdout:3/98: chown d3/d1b 29546959 1 2024-03-16T09:01:10.456 INFO:tasks.workunit.client.0.smithi007.stdout:1/360: dwrite d9/dd/d1b/d58/f6c [0,4194304] 0 2024-03-16T09:01:10.456 INFO:tasks.workunit.client.0.smithi007.stdout:1/361: write f0 [2290940,42459] 0 2024-03-16T09:01:10.456 INFO:tasks.workunit.client.0.smithi007.stdout:4/288: dread d4/d14/d1c/f37 [0,4194304] 0 2024-03-16T09:01:10.457 INFO:tasks.workunit.client.0.smithi007.stdout:1/362: dread d9/dd/d1b/d2d/f3f [0,4194304] 0 2024-03-16T09:01:10.457 INFO:tasks.workunit.client.0.smithi007.stdout:1/363: creat d9/dd/d1b/d2d/d30/f6d x:0 0 0 2024-03-16T09:01:10.457 INFO:tasks.workunit.client.0.smithi007.stdout:1/364: chown d9/c1a 558564 1 2024-03-16T09:01:10.457 INFO:tasks.workunit.client.0.smithi007.stdout:4/289: stat d4/d14/d1c/d1e/l5c 0 2024-03-16T09:01:10.458 INFO:tasks.workunit.client.1.smithi019.stdout:5/86: unlink f3 0 2024-03-16T09:01:10.462 INFO:tasks.workunit.client.1.smithi019.stdout:6/218: unlink de/d1c/c1f 0 2024-03-16T09:01:10.462 INFO:tasks.workunit.client.1.smithi019.stdout:9/112: creat d1/d7/de/d24/f2c x:0 0 0 2024-03-16T09:01:10.462 INFO:tasks.workunit.client.1.smithi019.stdout:9/113: fdatasync d1/d7/de/f1f 0 2024-03-16T09:01:10.462 INFO:tasks.workunit.client.1.smithi019.stdout:9/114: stat d1/d7/d21 0 2024-03-16T09:01:10.463 INFO:tasks.workunit.client.0.smithi007.stdout:1/365: symlink d9/dd/d1b/d28/l6e 0 2024-03-16T09:01:10.463 INFO:tasks.workunit.client.1.smithi019.stdout:8/106: creat d6/f27 x:0 0 0 2024-03-16T09:01:10.463 INFO:tasks.workunit.client.0.smithi007.stdout:4/290: link d4/d14/d1c/f3b d4/d14/d1c/f60 0 2024-03-16T09:01:10.463 INFO:tasks.workunit.client.0.smithi007.stdout:4/291: creat d4/d14/d1c/d1e/f61 x:0 0 0 2024-03-16T09:01:10.465 INFO:tasks.workunit.client.1.smithi019.stdout:1/42: rmdir d6 0 2024-03-16T09:01:10.465 INFO:tasks.workunit.client.1.smithi019.stdout:1/43: fsync - no filename 2024-03-16T09:01:10.465 INFO:tasks.workunit.client.1.smithi019.stdout:1/44: dread - no filename 2024-03-16T09:01:10.466 INFO:tasks.workunit.client.1.smithi019.stdout:8/107: dread d6/de/f1b [0,4194304] 0 2024-03-16T09:01:10.466 INFO:tasks.workunit.client.1.smithi019.stdout:0/71: symlink d7/d14/l16 0 2024-03-16T09:01:10.468 INFO:tasks.workunit.client.1.smithi019.stdout:2/74: link d5/c16 d5/de/c1a 0 2024-03-16T09:01:10.469 INFO:tasks.workunit.client.0.smithi007.stdout:4/292: getdents d4 0 2024-03-16T09:01:10.469 INFO:tasks.workunit.client.0.smithi007.stdout:4/293: chown d4/d14/l47 109 1 2024-03-16T09:01:10.469 INFO:tasks.workunit.client.0.smithi007.stdout:4/294: stat d4/d14/d1c/d5b/f5d 0 2024-03-16T09:01:10.469 INFO:tasks.workunit.client.0.smithi007.stdout:4/295: creat d4/d14/d1c/d5b/f62 x:0 0 0 2024-03-16T09:01:10.470 INFO:tasks.workunit.client.1.smithi019.stdout:3/99: mknod d3/d1b/c1e 0 2024-03-16T09:01:10.470 INFO:tasks.workunit.client.1.smithi019.stdout:3/100: write d3/d1b/f1c [725864,116581] 0 2024-03-16T09:01:10.470 INFO:tasks.workunit.client.1.smithi019.stdout:6/219: stat de/d26/d32/l44 0 2024-03-16T09:01:10.472 INFO:tasks.workunit.client.1.smithi019.stdout:8/108: mkdir d6/de/d16/d19/d28 0 2024-03-16T09:01:10.472 INFO:tasks.workunit.client.1.smithi019.stdout:8/109: creat d6/de/f29 x:0 0 0 2024-03-16T09:01:10.473 INFO:tasks.workunit.client.1.smithi019.stdout:0/72: link d7/la d7/l17 0 2024-03-16T09:01:10.473 INFO:tasks.workunit.client.1.smithi019.stdout:4/113: dwrite d4/d1b/f1c [0,4194304] 0 2024-03-16T09:01:10.474 INFO:tasks.workunit.client.1.smithi019.stdout:7/75: dwrite d9/da/d13/f17 [0,4194304] 0 2024-03-16T09:01:10.474 INFO:tasks.workunit.client.1.smithi019.stdout:7/76: fsync f4 0 2024-03-16T09:01:10.474 INFO:tasks.workunit.client.0.smithi007.stdout:5/360: dwrite d1/d7/f12 [0,4194304] 0 2024-03-16T09:01:10.477 INFO:tasks.workunit.client.1.smithi019.stdout:6/220: rename c1 to de/d1c/d1d/c50 0 2024-03-16T09:01:10.477 INFO:tasks.workunit.client.1.smithi019.stdout:9/115: write d1/d7/de/f10 [3415208,60286] 0 2024-03-16T09:01:10.477 INFO:tasks.workunit.client.1.smithi019.stdout:9/116: write d1/d7/d21/d27/f13 [276453,75246] 0 2024-03-16T09:01:10.479 INFO:tasks.workunit.client.0.smithi007.stdout:5/361: mknod d1/d7/d1c/d19/d34/c87 0 2024-03-16T09:01:10.479 INFO:tasks.workunit.client.0.smithi007.stdout:5/362: chown d1/d7/d1c/d19 473 1 2024-03-16T09:01:10.480 INFO:tasks.workunit.client.0.smithi007.stdout:5/363: getdents d1 0 2024-03-16T09:01:10.481 INFO:tasks.workunit.client.1.smithi019.stdout:1/45: rename c2 to c7 0 2024-03-16T09:01:10.481 INFO:tasks.workunit.client.1.smithi019.stdout:1/46: chown c7 116370 1 2024-03-16T09:01:10.481 INFO:tasks.workunit.client.1.smithi019.stdout:1/47: truncate - no filename 2024-03-16T09:01:10.482 INFO:tasks.workunit.client.0.smithi007.stdout:5/364: mkdir d1/d7/d1c/d88 0 2024-03-16T09:01:10.482 INFO:tasks.workunit.client.0.smithi007.stdout:5/365: fsync d1/d7/d1c/d19/d34/d1d/f4f 0 2024-03-16T09:01:10.483 INFO:tasks.workunit.client.1.smithi019.stdout:8/110: mkdir d6/d2a 0 2024-03-16T09:01:10.483 INFO:tasks.workunit.client.1.smithi019.stdout:8/111: write d6/de/f1b [1921191,85784] 0 2024-03-16T09:01:10.484 INFO:tasks.workunit.client.1.smithi019.stdout:4/114: mkdir d4/d1b/d2a 0 2024-03-16T09:01:10.485 INFO:tasks.workunit.client.1.smithi019.stdout:8/112: dread d6/fa [0,4194304] 0 2024-03-16T09:01:10.485 INFO:tasks.workunit.client.1.smithi019.stdout:8/113: truncate d6/fa 1174666 0 2024-03-16T09:01:10.486 INFO:tasks.workunit.client.1.smithi019.stdout:8/114: read - d6/de/f1d zero size 2024-03-16T09:01:10.488 INFO:tasks.workunit.client.0.smithi007.stdout:1/366: dwrite d9/f35 [0,4194304] 0 2024-03-16T09:01:10.488 INFO:tasks.workunit.client.1.smithi019.stdout:7/77: unlink d9/da/d13/c15 0 2024-03-16T09:01:10.488 INFO:tasks.workunit.client.1.smithi019.stdout:6/221: mkdir de/d1c/d1d/d51 0 2024-03-16T09:01:10.489 INFO:tasks.workunit.client.1.smithi019.stdout:3/101: creat d3/d11/f1f x:0 0 0 2024-03-16T09:01:10.489 INFO:tasks.workunit.client.1.smithi019.stdout:6/222: chown de/d26/d32/l34 452197805 1 2024-03-16T09:01:10.489 INFO:tasks.workunit.client.1.smithi019.stdout:6/223: dread - de/d1c/f47 zero size 2024-03-16T09:01:10.489 INFO:tasks.workunit.client.1.smithi019.stdout:1/48: creat f8 x:0 0 0 2024-03-16T09:01:10.489 INFO:tasks.workunit.client.1.smithi019.stdout:2/75: dwrite f4 [4194304,4194304] 0 2024-03-16T09:01:10.489 INFO:tasks.workunit.client.1.smithi019.stdout:2/76: fdatasync f4 0 2024-03-16T09:01:10.490 INFO:tasks.workunit.client.1.smithi019.stdout:5/87: dwrite d6/f7 [0,4194304] 0 2024-03-16T09:01:10.493 INFO:tasks.workunit.client.1.smithi019.stdout:4/115: rename d4/d1b/l21 to d4/d6/d15/l2b 0 2024-03-16T09:01:10.493 INFO:tasks.workunit.client.1.smithi019.stdout:3/102: dread f2 [0,4194304] 0 2024-03-16T09:01:10.494 INFO:tasks.workunit.client.1.smithi019.stdout:7/78: mkdir d9/da/d13/d18 0 2024-03-16T09:01:10.495 INFO:tasks.workunit.client.1.smithi019.stdout:7/79: write d9/db/f11 [113137,53306] 0 2024-03-16T09:01:10.495 INFO:tasks.workunit.client.1.smithi019.stdout:7/80: chown d9/fc 15914 1 2024-03-16T09:01:10.495 INFO:tasks.workunit.client.1.smithi019.stdout:7/81: write d9/db/f11 [328952,28681] 0 2024-03-16T09:01:10.496 INFO:tasks.workunit.client.0.smithi007.stdout:5/366: write d1/d7/d1c/d19/d34/d1d/f1f [2453088,22032] 0 2024-03-16T09:01:10.496 INFO:tasks.workunit.client.0.smithi007.stdout:5/367: chown d1/d7/d1c/d19/d34/d5e/d6f 486701 1 2024-03-16T09:01:10.496 INFO:tasks.workunit.client.1.smithi019.stdout:6/224: symlink de/d1c/l52 0 2024-03-16T09:01:10.496 INFO:tasks.workunit.client.1.smithi019.stdout:6/225: write de/f49 [566840,115039] 0 2024-03-16T09:01:10.497 INFO:tasks.workunit.client.1.smithi019.stdout:1/49: creat f9 x:0 0 0 2024-03-16T09:01:10.497 INFO:tasks.workunit.client.1.smithi019.stdout:1/50: chown c7 157 1 2024-03-16T09:01:10.498 INFO:tasks.workunit.client.0.smithi007.stdout:5/368: mkdir d1/d7/d1c/d19/d34/d1d/d36/d89 0 2024-03-16T09:01:10.498 INFO:tasks.workunit.client.0.smithi007.stdout:5/369: chown d1/d7/c76 696 1 2024-03-16T09:01:10.500 INFO:tasks.workunit.client.1.smithi019.stdout:5/88: rmdir d6/dd 39 2024-03-16T09:01:10.500 INFO:tasks.workunit.client.0.smithi007.stdout:5/370: getdents d1/d7 0 2024-03-16T09:01:10.500 INFO:tasks.workunit.client.0.smithi007.stdout:5/371: chown d1/f10 2 1 2024-03-16T09:01:10.501 INFO:tasks.workunit.client.1.smithi019.stdout:4/116: mkdir d4/d6/d15/d1f/d2c 0 2024-03-16T09:01:10.504 INFO:tasks.workunit.client.1.smithi019.stdout:7/82: mknod d9/db/d12/c19 0 2024-03-16T09:01:10.505 INFO:tasks.workunit.client.1.smithi019.stdout:1/51: creat fa x:0 0 0 2024-03-16T09:01:10.505 INFO:tasks.workunit.client.1.smithi019.stdout:1/52: truncate fa 705626 0 2024-03-16T09:01:10.508 INFO:tasks.workunit.client.1.smithi019.stdout:6/226: read de/d26/f2a [3632455,79162] 0 2024-03-16T09:01:10.508 INFO:tasks.workunit.client.1.smithi019.stdout:6/227: readlink de/d26/d32/l44 0 2024-03-16T09:01:10.508 INFO:tasks.workunit.client.1.smithi019.stdout:3/103: rename d3/lb to d3/d11/l20 0 2024-03-16T09:01:10.509 INFO:tasks.workunit.client.1.smithi019.stdout:8/115: dwrite d6/de/d16/f26 [0,4194304] 0 2024-03-16T09:01:10.509 INFO:tasks.workunit.client.1.smithi019.stdout:8/116: creat d6/f2b x:0 0 0 2024-03-16T09:01:10.509 INFO:tasks.workunit.client.1.smithi019.stdout:8/117: truncate d6/de/f29 574728 0 2024-03-16T09:01:10.510 INFO:tasks.workunit.client.1.smithi019.stdout:7/83: mkdir d9/da/d13/d18/d1a 0 2024-03-16T09:01:10.510 INFO:tasks.workunit.client.1.smithi019.stdout:7/84: fsync d9/da/d13/f17 0 2024-03-16T09:01:10.511 INFO:tasks.workunit.client.1.smithi019.stdout:1/53: unlink l5 0 2024-03-16T09:01:10.511 INFO:tasks.workunit.client.1.smithi019.stdout:1/54: readlink - no filename 2024-03-16T09:01:10.513 INFO:tasks.workunit.client.1.smithi019.stdout:6/228: mknod de/d26/d32/d4e/c53 0 2024-03-16T09:01:10.513 INFO:tasks.workunit.client.1.smithi019.stdout:6/229: fdatasync de/d1c/f47 0 2024-03-16T09:01:10.513 INFO:tasks.workunit.client.1.smithi019.stdout:3/104: chown d3/ce 524050 1 2024-03-16T09:01:10.516 INFO:tasks.workunit.client.0.smithi007.stdout:1/367: dwrite d9/dd/f59 [0,4194304] 0 2024-03-16T09:01:10.518 INFO:tasks.workunit.client.1.smithi019.stdout:6/230: symlink de/d26/d32/d4e/l54 0 2024-03-16T09:01:10.518 INFO:tasks.workunit.client.1.smithi019.stdout:6/231: write de/f49 [985870,129190] 0 2024-03-16T09:01:10.519 INFO:tasks.workunit.client.0.smithi007.stdout:1/368: mknod d9/dd/d1b/d28/c6f 0 2024-03-16T09:01:10.521 INFO:tasks.workunit.client.1.smithi019.stdout:7/85: dread d9/fc [0,4194304] 0 2024-03-16T09:01:10.521 INFO:tasks.workunit.client.1.smithi019.stdout:7/86: write d9/db/f11 [194625,22392] 0 2024-03-16T09:01:10.523 INFO:tasks.workunit.client.1.smithi019.stdout:7/87: mkdir d9/da/d13/d18/d1a/d1b 0 2024-03-16T09:01:10.523 INFO:tasks.workunit.client.1.smithi019.stdout:7/88: fdatasync f6 0 2024-03-16T09:01:10.523 INFO:tasks.workunit.client.0.smithi007.stdout:1/369: mknod d9/dd/d1b/d2d/d30/d45/c70 0 2024-03-16T09:01:10.526 INFO:tasks.workunit.client.1.smithi019.stdout:2/77: rmdir d5 39 2024-03-16T09:01:10.526 INFO:tasks.workunit.client.1.smithi019.stdout:2/78: write d5/ff [669312,79874] 0 2024-03-16T09:01:10.529 INFO:tasks.workunit.client.1.smithi019.stdout:2/79: unlink d5/de/l15 0 2024-03-16T09:01:10.532 INFO:tasks.workunit.client.1.smithi019.stdout:2/80: creat d5/de/f1b x:0 0 0 2024-03-16T09:01:10.533 INFO:tasks.workunit.client.1.smithi019.stdout:2/81: mknod d5/de/d10/c1c 0 2024-03-16T09:01:10.534 INFO:tasks.workunit.client.1.smithi019.stdout:5/89: dwrite d6/f8 [0,4194304] 0 2024-03-16T09:01:10.534 INFO:tasks.workunit.client.1.smithi019.stdout:3/105: dwrite d3/d11/f12 [4194304,4194304] 0 2024-03-16T09:01:10.534 INFO:tasks.workunit.client.0.smithi007.stdout:5/372: dwrite d1/d7/d16/d1e/f31 [0,4194304] 0 2024-03-16T09:01:10.536 INFO:tasks.workunit.client.0.smithi007.stdout:5/373: symlink d1/d7/d1c/d19/d34/d4d/l8a 0 2024-03-16T09:01:10.536 INFO:tasks.workunit.client.0.smithi007.stdout:5/374: write d1/d7/d16/d1b/f67 [1531171,84742] 0 2024-03-16T09:01:10.538 INFO:tasks.workunit.client.1.smithi019.stdout:5/90: mkdir d6/d19 0 2024-03-16T09:01:10.542 INFO:tasks.workunit.client.1.smithi019.stdout:4/117: rmdir d4/d1b 39 2024-03-16T09:01:10.546 INFO:tasks.workunit.client.1.smithi019.stdout:3/106: write d3/d11/f14 [865928,48008] 0 2024-03-16T09:01:10.548 INFO:tasks.workunit.client.1.smithi019.stdout:6/232: dwrite de/d1c/f47 [0,4194304] 0 2024-03-16T09:01:10.550 INFO:tasks.workunit.client.1.smithi019.stdout:3/107: dread d3/d1b/f1c [0,4194304] 0 2024-03-16T09:01:10.550 INFO:tasks.workunit.client.0.smithi007.stdout:5/375: fdatasync d1/d7/d16/d1b/f67 0 2024-03-16T09:01:10.551 INFO:tasks.workunit.client.0.smithi007.stdout:5/376: dread - d1/d7/d1c/d19/d34/d5e/d6f/d72/f85 zero size 2024-03-16T09:01:10.551 INFO:tasks.workunit.client.1.smithi019.stdout:8/118: dwrite d6/de/f1b [0,4194304] 0 2024-03-16T09:01:10.552 INFO:tasks.workunit.client.1.smithi019.stdout:6/233: symlink de/d1c/d1d/d2c/l55 0 2024-03-16T09:01:10.552 INFO:tasks.workunit.client.1.smithi019.stdout:6/234: chown de/d26/d32/f37 243 1 2024-03-16T09:01:10.552 INFO:tasks.workunit.client.1.smithi019.stdout:6/235: readlink de/l31 0 2024-03-16T09:01:10.552 INFO:tasks.workunit.client.0.smithi007.stdout:5/377: link d1/d7/d1c/d19/d34/d3e/c63 d1/d7/d1c/d19/d34/d1d/d36/c8b 0 2024-03-16T09:01:10.553 INFO:tasks.workunit.client.1.smithi019.stdout:8/119: mkdir d6/de/d16/d2c 0 2024-03-16T09:01:10.553 INFO:tasks.workunit.client.1.smithi019.stdout:8/120: readlink d6/l17 0 2024-03-16T09:01:10.555 INFO:tasks.workunit.client.1.smithi019.stdout:6/236: rename de/f4d to de/d1c/f56 0 2024-03-16T09:01:10.555 INFO:tasks.workunit.client.1.smithi019.stdout:6/237: dread - de/d26/d32/f37 zero size 2024-03-16T09:01:10.555 INFO:tasks.workunit.client.1.smithi019.stdout:4/118: rmdir d4/d6 39 2024-03-16T09:01:10.555 INFO:tasks.workunit.client.1.smithi019.stdout:4/119: dread - d4/f26 zero size 2024-03-16T09:01:10.555 INFO:tasks.workunit.client.1.smithi019.stdout:4/120: chown d4/d6/d15/d1f/c29 199 1 2024-03-16T09:01:10.555 INFO:tasks.workunit.client.1.smithi019.stdout:4/121: write d4/f26 [889965,54586] 0 2024-03-16T09:01:10.556 INFO:tasks.workunit.client.1.smithi019.stdout:5/91: dwrite d6/f17 [0,4194304] 0 2024-03-16T09:01:10.556 INFO:tasks.workunit.client.1.smithi019.stdout:5/92: truncate d6/f9 4663150 0 2024-03-16T09:01:10.557 INFO:tasks.workunit.client.1.smithi019.stdout:6/238: mkdir de/d26/d32/d4e/d57 0 2024-03-16T09:01:10.561 INFO:tasks.workunit.client.1.smithi019.stdout:4/122: rmdir d4 39 2024-03-16T09:01:10.563 INFO:tasks.workunit.client.1.smithi019.stdout:6/239: link de/f25 de/d1c/f58 0 2024-03-16T09:01:10.564 INFO:tasks.workunit.client.1.smithi019.stdout:4/123: dread - d4/d6/f24 zero size 2024-03-16T09:01:10.567 INFO:tasks.workunit.client.1.smithi019.stdout:8/121: dread d6/de/d16/d19/f1c [0,4194304] 0 2024-03-16T09:01:10.570 INFO:tasks.workunit.client.1.smithi019.stdout:8/122: mknod d6/c2d 0 2024-03-16T09:01:10.571 INFO:tasks.workunit.client.1.smithi019.stdout:6/240: dread de/d1c/f28 [0,4194304] 0 2024-03-16T09:01:10.571 INFO:tasks.workunit.client.1.smithi019.stdout:6/241: readlink de/d1c/l2d 0 2024-03-16T09:01:10.572 INFO:tasks.workunit.client.1.smithi019.stdout:8/123: creat d6/de/d16/d19/f2e x:0 0 0 2024-03-16T09:01:10.573 INFO:tasks.workunit.client.1.smithi019.stdout:6/242: creat de/d26/d32/d4e/f59 x:0 0 0 2024-03-16T09:01:10.573 INFO:tasks.workunit.client.1.smithi019.stdout:6/243: creat de/d1c/d1d/f5a x:0 0 0 2024-03-16T09:01:10.575 INFO:tasks.workunit.client.1.smithi019.stdout:2/82: getdents d5/de 0 2024-03-16T09:01:10.575 INFO:tasks.workunit.client.1.smithi019.stdout:2/83: fsync d5/f7 0 2024-03-16T09:01:10.577 INFO:tasks.workunit.client.1.smithi019.stdout:2/84: getdents d5/d6 0 2024-03-16T09:01:10.578 INFO:tasks.workunit.client.1.smithi019.stdout:2/85: chown f2 28212239 1 2024-03-16T09:01:10.578 INFO:tasks.workunit.client.1.smithi019.stdout:4/124: write d4/f23 [446935,84093] 0 2024-03-16T09:01:10.578 INFO:tasks.workunit.client.1.smithi019.stdout:2/86: getdents d5 0 2024-03-16T09:01:10.580 INFO:tasks.workunit.client.1.smithi019.stdout:2/87: creat d5/d6/f1d x:0 0 0 2024-03-16T09:01:10.580 INFO:tasks.workunit.client.1.smithi019.stdout:2/88: dread - d5/d6/f11 zero size 2024-03-16T09:01:10.584 INFO:tasks.workunit.client.1.smithi019.stdout:8/124: write d6/f7 [3273462,12037] 0 2024-03-16T09:01:10.584 INFO:tasks.workunit.client.1.smithi019.stdout:8/125: write f2 [1414557,114609] 0 2024-03-16T09:01:10.588 INFO:tasks.workunit.client.1.smithi019.stdout:3/108: dwrite d3/d11/f14 [0,4194304] 0 2024-03-16T09:01:10.588 INFO:tasks.workunit.client.1.smithi019.stdout:5/93: dwrite f2 [0,4194304] 0 2024-03-16T09:01:10.588 INFO:tasks.workunit.client.1.smithi019.stdout:5/94: getdents d6/d19 0 2024-03-16T09:01:10.588 INFO:tasks.workunit.client.0.smithi007.stdout:5/378: dwrite d1/d7/d1c/d19/d34/d1d/f5a [0,4194304] 0 2024-03-16T09:01:10.592 INFO:tasks.workunit.client.0.smithi007.stdout:5/379: creat d1/d7/d1c/d19/d34/f8c x:0 0 0 2024-03-16T09:01:10.592 INFO:tasks.workunit.client.0.smithi007.stdout:5/380: creat d1/d7/d16/d1b/f8d x:0 0 0 2024-03-16T09:01:10.592 INFO:tasks.workunit.client.0.smithi007.stdout:5/381: write d1/d7/d16/d1b/f8d [732009,71505] 0 2024-03-16T09:01:10.593 INFO:tasks.workunit.client.0.smithi007.stdout:5/382: creat d1/d7/d1c/d19/d34/d5e/f8e x:0 0 0 2024-03-16T09:01:10.597 INFO:tasks.workunit.client.1.smithi019.stdout:6/244: dwrite de/f1a [0,4194304] 0 2024-03-16T09:01:10.602 INFO:tasks.workunit.client.1.smithi019.stdout:4/125: fdatasync d4/f26 0 2024-03-16T09:01:10.602 INFO:tasks.workunit.client.1.smithi019.stdout:4/126: readlink d4/d6/d15/l25 0 2024-03-16T09:01:10.607 INFO:tasks.workunit.client.1.smithi019.stdout:6/245: dread de/d1c/d1d/d2c/f3a [0,4194304] 0 2024-03-16T09:01:10.607 INFO:tasks.workunit.client.1.smithi019.stdout:6/246: chown de/d26 86 1 2024-03-16T09:01:10.614 INFO:tasks.workunit.client.1.smithi019.stdout:8/126: dwrite f5 [0,4194304] 0 2024-03-16T09:01:10.614 INFO:tasks.workunit.client.1.smithi019.stdout:8/127: chown f2 590 1 2024-03-16T09:01:10.614 INFO:tasks.workunit.client.1.smithi019.stdout:5/95: dwrite f4 [0,4194304] 0 2024-03-16T09:01:10.617 INFO:tasks.workunit.client.1.smithi019.stdout:6/247: write de/f25 [3611364,26629] 0 2024-03-16T09:01:10.618 INFO:tasks.workunit.client.1.smithi019.stdout:6/248: creat de/d26/d32/f5b x:0 0 0 2024-03-16T09:01:10.618 INFO:tasks.workunit.client.1.smithi019.stdout:6/249: read - de/d26/f39 zero size 2024-03-16T09:01:10.618 INFO:tasks.workunit.client.1.smithi019.stdout:5/96: rename d6/f7 to d6/dd/f1a 0 2024-03-16T09:01:10.620 INFO:tasks.workunit.client.1.smithi019.stdout:4/127: dwrite d4/f1e [0,4194304] 0 2024-03-16T09:01:10.620 INFO:tasks.workunit.client.1.smithi019.stdout:4/128: stat d4/d1b/f28 0 2024-03-16T09:01:10.620 INFO:tasks.workunit.client.1.smithi019.stdout:4/129: chown d4/d6/c22 140827389 1 2024-03-16T09:01:10.621 INFO:tasks.workunit.client.1.smithi019.stdout:6/250: rename de/f25 to de/d26/f5c 0 2024-03-16T09:01:10.621 INFO:tasks.workunit.client.1.smithi019.stdout:6/251: chown de/d1c/d1d/c23 792 1 2024-03-16T09:01:10.621 INFO:tasks.workunit.client.1.smithi019.stdout:6/252: creat de/d1c/f5d x:0 0 0 2024-03-16T09:01:10.622 INFO:tasks.workunit.client.1.smithi019.stdout:5/97: rmdir d6/dd 39 2024-03-16T09:01:10.624 INFO:tasks.workunit.client.1.smithi019.stdout:8/128: dread d6/de/f1b [0,4194304] 0 2024-03-16T09:01:10.625 INFO:tasks.workunit.client.1.smithi019.stdout:4/130: write d4/f26 [745966,43328] 0 2024-03-16T09:01:10.629 INFO:tasks.workunit.client.1.smithi019.stdout:8/129: dread d6/f7 [0,4194304] 0 2024-03-16T09:01:10.629 INFO:tasks.workunit.client.1.smithi019.stdout:8/130: fsync d6/de/d16/d19/f2e 0 2024-03-16T09:01:10.629 INFO:tasks.workunit.client.1.smithi019.stdout:8/131: fsync d6/de/f29 0 2024-03-16T09:01:10.629 INFO:tasks.workunit.client.1.smithi019.stdout:8/132: read - d6/de/f1d zero size 2024-03-16T09:01:10.630 INFO:tasks.workunit.client.1.smithi019.stdout:8/133: mknod d6/de/d16/d19/c2f 0 2024-03-16T09:01:10.633 INFO:tasks.workunit.client.1.smithi019.stdout:5/98: getdents d6 0 2024-03-16T09:01:10.634 INFO:tasks.workunit.client.1.smithi019.stdout:5/99: truncate d6/dd/f16 2107515 0 2024-03-16T09:01:10.634 INFO:tasks.workunit.client.1.smithi019.stdout:5/100: chown d6/dd 28868416 1 2024-03-16T09:01:10.635 INFO:tasks.workunit.client.1.smithi019.stdout:5/101: dread d6/f9 [4194304,4194304] 0 2024-03-16T09:01:10.636 INFO:tasks.workunit.client.1.smithi019.stdout:4/131: dread f2 [0,4194304] 0 2024-03-16T09:01:10.637 INFO:tasks.workunit.client.1.smithi019.stdout:4/132: mknod d4/d6/d15/c2d 0 2024-03-16T09:01:10.638 INFO:tasks.workunit.client.1.smithi019.stdout:4/133: rename d4/f23 to d4/d6/d15/f2e 0 2024-03-16T09:01:10.640 INFO:tasks.workunit.client.1.smithi019.stdout:4/134: mknod d4/d6/d15/c2f 0 2024-03-16T09:01:10.641 INFO:tasks.workunit.client.1.smithi019.stdout:4/135: creat d4/d6/d15/d1f/f30 x:0 0 0 2024-03-16T09:01:10.641 INFO:tasks.workunit.client.1.smithi019.stdout:4/136: dread - d4/d6/d15/d1f/f30 zero size 2024-03-16T09:01:10.641 INFO:tasks.workunit.client.1.smithi019.stdout:4/137: stat d4/d6/d15/l2b 0 2024-03-16T09:01:10.643 INFO:tasks.workunit.client.1.smithi019.stdout:4/138: unlink d4/d6/d15/c19 0 2024-03-16T09:01:10.643 INFO:tasks.workunit.client.1.smithi019.stdout:4/139: write d4/d1b/f28 [787752,28521] 0 2024-03-16T09:01:10.643 INFO:tasks.workunit.client.1.smithi019.stdout:4/140: write d4/f12 [3148782,115689] 0 2024-03-16T09:01:10.644 INFO:tasks.workunit.client.1.smithi019.stdout:4/141: creat d4/d6/d15/d1f/d2c/f31 x:0 0 0 2024-03-16T09:01:10.646 INFO:tasks.workunit.client.1.smithi019.stdout:6/253: dwrite de/d1c/f20 [0,4194304] 0 2024-03-16T09:01:10.650 INFO:tasks.workunit.client.1.smithi019.stdout:6/254: getdents de/d26/d32/d4e 0 2024-03-16T09:01:10.651 INFO:tasks.workunit.client.1.smithi019.stdout:6/255: mknod de/d1c/d1d/d2c/c5e 0 2024-03-16T09:01:10.652 INFO:tasks.workunit.client.1.smithi019.stdout:6/256: symlink de/d26/l5f 0 2024-03-16T09:01:10.654 INFO:tasks.workunit.client.1.smithi019.stdout:8/134: dwrite d6/f7 [0,4194304] 0 2024-03-16T09:01:10.654 INFO:tasks.workunit.client.1.smithi019.stdout:8/135: truncate d6/de/ff 1915536 0 2024-03-16T09:01:10.654 INFO:tasks.workunit.client.1.smithi019.stdout:6/257: symlink de/d26/d32/d4e/d57/l60 0 2024-03-16T09:01:10.654 INFO:tasks.workunit.client.1.smithi019.stdout:5/102: dwrite d6/f8 [0,4194304] 0 2024-03-16T09:01:10.655 INFO:tasks.workunit.client.1.smithi019.stdout:6/258: rename de/c2b to de/d26/d32/d4e/c61 0 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:7/89: sync 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:0/73: sync 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:1/55: sync 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:7/90: chown f6 0 1 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:0/74: read - d7/f15 zero size 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:7/91: stat d9/da/d13/d18/d1a/d1b 0 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:9/117: sync 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:0/75: write d7/fd [662307,103036] 0 2024-03-16T09:01:10.662 INFO:tasks.workunit.client.1.smithi019.stdout:0/76: truncate f4 1076281 0 2024-03-16T09:01:10.663 INFO:tasks.workunit.client.1.smithi019.stdout:0/77: mknod d7/d13/c18 0 2024-03-16T09:01:10.663 INFO:tasks.workunit.client.1.smithi019.stdout:0/78: creat d7/f19 x:0 0 0 2024-03-16T09:01:10.663 INFO:tasks.workunit.client.1.smithi019.stdout:0/79: truncate d7/f15 811153 0 2024-03-16T09:01:10.664 INFO:tasks.workunit.client.1.smithi019.stdout:8/136: dread d6/de/d16/d19/f1a [0,4194304] 0 2024-03-16T09:01:10.665 INFO:tasks.workunit.client.1.smithi019.stdout:8/137: mkdir d6/de/d24/d30 0 2024-03-16T09:01:10.667 INFO:tasks.workunit.client.1.smithi019.stdout:4/142: dwrite d4/f12 [0,4194304] 0 2024-03-16T09:01:10.667 INFO:tasks.workunit.client.1.smithi019.stdout:4/143: read - d4/d6/d15/d1f/d2c/f31 zero size 2024-03-16T09:01:10.668 INFO:tasks.workunit.client.1.smithi019.stdout:4/144: getdents d4/d6 0 2024-03-16T09:01:10.669 INFO:tasks.workunit.client.1.smithi019.stdout:4/145: rmdir d4 39 2024-03-16T09:01:10.670 INFO:tasks.workunit.client.1.smithi019.stdout:4/146: creat d4/d6/f32 x:0 0 0 2024-03-16T09:01:10.676 INFO:tasks.workunit.client.1.smithi019.stdout:0/80: fsync d7/f19 0 2024-03-16T09:01:10.677 INFO:tasks.workunit.client.1.smithi019.stdout:2/89: sync 2024-03-16T09:01:10.678 INFO:tasks.workunit.client.1.smithi019.stdout:2/90: write f2 [2124787,112692] 0 2024-03-16T09:01:10.680 INFO:tasks.workunit.client.1.smithi019.stdout:2/91: unlink f2 0 2024-03-16T09:01:10.680 INFO:tasks.workunit.client.1.smithi019.stdout:2/92: truncate d5/d6/f11 496642 0 2024-03-16T09:01:10.680 INFO:tasks.workunit.client.1.smithi019.stdout:4/147: dread d4/f1e [0,4194304] 0 2024-03-16T09:01:10.681 INFO:tasks.workunit.client.1.smithi019.stdout:7/92: dwrite f4 [0,4194304] 0 2024-03-16T09:01:10.682 INFO:tasks.workunit.client.1.smithi019.stdout:0/81: dread f4 [0,4194304] 0 2024-03-16T09:01:10.682 INFO:tasks.workunit.client.1.smithi019.stdout:0/82: truncate d7/d8/fc 536106 0 2024-03-16T09:01:10.683 INFO:tasks.workunit.client.1.smithi019.stdout:9/118: dwrite d1/d7/de/ff [0,4194304] 0 2024-03-16T09:01:10.683 INFO:tasks.workunit.client.1.smithi019.stdout:9/119: chown d1/f20 33450013 1 2024-03-16T09:01:10.685 INFO:tasks.workunit.client.1.smithi019.stdout:9/120: mkdir d1/d7/de/d2d 0 2024-03-16T09:01:10.685 INFO:tasks.workunit.client.1.smithi019.stdout:9/121: mkdir d1/d7/de/d24/d2e 0 2024-03-16T09:01:10.689 INFO:tasks.workunit.client.1.smithi019.stdout:2/93: dread d5/d6/f8 [0,4194304] 0 2024-03-16T09:01:10.691 INFO:tasks.workunit.client.1.smithi019.stdout:2/94: creat d5/de/d10/f1e x:0 0 0 2024-03-16T09:01:10.691 INFO:tasks.workunit.client.1.smithi019.stdout:2/95: truncate d5/ff 1169747 0 2024-03-16T09:01:10.692 INFO:tasks.workunit.client.1.smithi019.stdout:2/96: creat d5/de/f1f x:0 0 0 2024-03-16T09:01:10.693 INFO:tasks.workunit.client.1.smithi019.stdout:2/97: symlink d5/de/d10/l20 0 2024-03-16T09:01:10.695 INFO:tasks.workunit.client.1.smithi019.stdout:9/122: write d1/d7/de/f1f [3255465,114577] 0 2024-03-16T09:01:10.695 INFO:tasks.workunit.client.1.smithi019.stdout:9/123: chown d1/f20 43 1 2024-03-16T09:01:10.695 INFO:tasks.workunit.client.1.smithi019.stdout:2/98: rename d5/d6/f8 to d5/d6/f21 0 2024-03-16T09:01:10.695 INFO:tasks.workunit.client.1.smithi019.stdout:2/99: stat d5/de/d10/f1e 0 2024-03-16T09:01:10.697 INFO:tasks.workunit.client.1.smithi019.stdout:2/100: mknod d5/c22 0 2024-03-16T09:01:10.697 INFO:tasks.workunit.client.1.smithi019.stdout:2/101: fdatasync d5/ff 0 2024-03-16T09:01:10.704 INFO:tasks.workunit.client.1.smithi019.stdout:7/93: read f7 [646212,80515] 0 2024-03-16T09:01:10.706 INFO:tasks.workunit.client.1.smithi019.stdout:7/94: getdents d9/db 0 2024-03-16T09:01:10.706 INFO:tasks.workunit.client.1.smithi019.stdout:7/95: readlink d9/da/d13/l14 0 2024-03-16T09:01:10.706 INFO:tasks.workunit.client.1.smithi019.stdout:4/148: dwrite f2 [0,4194304] 0 2024-03-16T09:01:10.710 INFO:tasks.workunit.client.1.smithi019.stdout:4/149: creat d4/d6/d15/f33 x:0 0 0 2024-03-16T09:01:10.712 INFO:tasks.workunit.client.1.smithi019.stdout:4/150: link d4/d6/d15/l1a d4/d6/d15/l34 0 2024-03-16T09:01:10.712 INFO:tasks.workunit.client.1.smithi019.stdout:4/151: chown d4/d6/l9 558399 1 2024-03-16T09:01:10.712 INFO:tasks.workunit.client.1.smithi019.stdout:7/96: dread d9/fc [0,4194304] 0 2024-03-16T09:01:10.712 INFO:tasks.workunit.client.1.smithi019.stdout:4/152: unlink d4/d6/d15/l34 0 2024-03-16T09:01:10.714 INFO:tasks.workunit.client.1.smithi019.stdout:4/153: creat d4/f35 x:0 0 0 2024-03-16T09:01:10.714 INFO:tasks.workunit.client.1.smithi019.stdout:7/97: getdents d9/da/d13 0 2024-03-16T09:01:10.715 INFO:tasks.workunit.client.1.smithi019.stdout:4/154: getdents d4/d6/d15/d1f 0 2024-03-16T09:01:10.716 INFO:tasks.workunit.client.1.smithi019.stdout:4/155: symlink d4/d1b/d2a/l36 0 2024-03-16T09:01:10.717 INFO:tasks.workunit.client.1.smithi019.stdout:4/156: symlink d4/d1b/d2a/l37 0 2024-03-16T09:01:10.717 INFO:tasks.workunit.client.1.smithi019.stdout:4/157: write d4/d6/f24 [613386,89305] 0 2024-03-16T09:01:10.718 INFO:tasks.workunit.client.1.smithi019.stdout:4/158: creat d4/d6/d15/d1f/f38 x:0 0 0 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:0/345: sync 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:7/250: sync 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:9/397: sync 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:0/346: stat d3/d9/da/d18/d27/c29 0 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:3/426: sync 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:6/265: sync 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:2/465: sync 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:4/296: sync 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:2/466: stat dc/d27/f65 0 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:6/266: write d2/fa [4495057,66470] 0 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:6/267: rename d2/d11/d1e to d2/d11/d1e/d54/d57 22 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:8/593: sync 2024-03-16T09:01:10.720 INFO:tasks.workunit.client.0.smithi007.stdout:6/268: fsync d2/d13/d1f/d34/f4a 0 2024-03-16T09:01:10.721 INFO:tasks.workunit.client.1.smithi019.stdout:0/83: dwrite f4 [0,4194304] 0 2024-03-16T09:01:10.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/251: rmdir d0/dc/d38 39 2024-03-16T09:01:10.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/252: creat d0/dc/d24/d25/d3a/f4f x:0 0 0 2024-03-16T09:01:10.722 INFO:tasks.workunit.client.1.smithi019.stdout:4/159: link d4/d1b/f1c d4/d6/f39 0 2024-03-16T09:01:10.722 INFO:tasks.workunit.client.0.smithi007.stdout:9/398: creat d24/f71 x:0 0 0 2024-03-16T09:01:10.722 INFO:tasks.workunit.client.0.smithi007.stdout:9/399: write d24/f43 [2695141,122834] 0 2024-03-16T09:01:10.722 INFO:tasks.workunit.client.0.smithi007.stdout:9/400: stat d24/f48 0 2024-03-16T09:01:10.722 INFO:tasks.workunit.client.0.smithi007.stdout:9/401: fsync fd 0 2024-03-16T09:01:10.724 INFO:tasks.workunit.client.0.smithi007.stdout:0/347: getdents d3/d1f/d70/d73 0 2024-03-16T09:01:10.724 INFO:tasks.workunit.client.0.smithi007.stdout:0/348: read - d3/d1f/f71 zero size 2024-03-16T09:01:10.724 INFO:tasks.workunit.client.0.smithi007.stdout:4/297: rmdir d4/d7 39 2024-03-16T09:01:10.725 INFO:tasks.workunit.client.0.smithi007.stdout:2/467: mkdir dc/df/d24/d28/d97 0 2024-03-16T09:01:10.725 INFO:tasks.workunit.client.1.smithi019.stdout:2/102: dwrite d5/f7 [0,4194304] 0 2024-03-16T09:01:10.727 INFO:tasks.workunit.client.0.smithi007.stdout:8/594: mkdir d0/dc/d50/d8e/dd2 0 2024-03-16T09:01:10.727 INFO:tasks.workunit.client.0.smithi007.stdout:8/595: creat d0/d2/d74/fd3 x:0 0 0 2024-03-16T09:01:10.727 INFO:tasks.workunit.client.0.smithi007.stdout:8/596: chown d0/dc/c33 4114 1 2024-03-16T09:01:10.728 INFO:tasks.workunit.client.0.smithi007.stdout:6/269: mkdir d2/d13/d1f/d49/d58 0 2024-03-16T09:01:10.728 INFO:tasks.workunit.client.1.smithi019.stdout:4/160: mknod d4/d1b/d2a/c3a 0 2024-03-16T09:01:10.728 INFO:tasks.workunit.client.1.smithi019.stdout:4/161: write d4/d6/d15/d1f/f30 [378677,12077] 0 2024-03-16T09:01:10.729 INFO:tasks.workunit.client.1.smithi019.stdout:4/162: creat d4/d6/d15/d1f/d2c/f3b x:0 0 0 2024-03-16T09:01:10.733 INFO:tasks.workunit.client.1.smithi019.stdout:2/103: creat d5/f23 x:0 0 0 2024-03-16T09:01:10.737 INFO:tasks.workunit.client.1.smithi019.stdout:2/104: fsync d5/d6/f21 0 2024-03-16T09:01:10.740 INFO:tasks.workunit.client.1.smithi019.stdout:7/98: dwrite f8 [0,4194304] 0 2024-03-16T09:01:10.740 INFO:tasks.workunit.client.1.smithi019.stdout:7/99: truncate d9/da/d13/f17 4620509 0 2024-03-16T09:01:10.740 INFO:tasks.workunit.client.1.smithi019.stdout:7/100: fdatasync f6 0 2024-03-16T09:01:10.741 INFO:tasks.workunit.client.0.smithi007.stdout:0/349: dread d3/d9/da/d48/f13 [0,4194304] 0 2024-03-16T09:01:10.741 INFO:tasks.workunit.client.1.smithi019.stdout:2/105: rename d5/d6/c19 to d5/de/d10/c24 0 2024-03-16T09:01:10.741 INFO:tasks.workunit.client.1.smithi019.stdout:7/101: rename d9/da/d13/d18/d1a/d1b to d9/da/d13/d18/d1a/d1b/d1c 22 2024-03-16T09:01:10.745 INFO:tasks.workunit.client.1.smithi019.stdout:7/102: unlink d9/db/f11 0 2024-03-16T09:01:10.747 INFO:tasks.workunit.client.1.smithi019.stdout:0/84: dwrite d7/d8/f12 [0,4194304] 0 2024-03-16T09:01:10.748 INFO:tasks.workunit.client.0.smithi007.stdout:3/427: dwrite d15/d18/d1f/d29/d43/f4e [0,4194304] 0 2024-03-16T09:01:10.748 INFO:tasks.workunit.client.0.smithi007.stdout:3/428: stat d15/d18/d1f/d29/f31 0 2024-03-16T09:01:10.748 INFO:tasks.workunit.client.0.smithi007.stdout:3/429: chown d15/d18/d1f/d29 1 1 2024-03-16T09:01:10.749 INFO:tasks.workunit.client.1.smithi019.stdout:7/103: creat d9/db/f1d x:0 0 0 2024-03-16T09:01:10.750 INFO:tasks.workunit.client.1.smithi019.stdout:4/163: dread d4/d6/d15/d1f/f30 [0,4194304] 0 2024-03-16T09:01:10.751 INFO:tasks.workunit.client.0.smithi007.stdout:7/253: creat d0/dc/d38/d4e/f50 x:0 0 0 2024-03-16T09:01:10.753 INFO:tasks.workunit.client.0.smithi007.stdout:8/597: dwrite d0/d2/d74/f96 [0,4194304] 0 2024-03-16T09:01:10.753 INFO:tasks.workunit.client.0.smithi007.stdout:8/598: chown d0/d26/d95/f9b 3 1 2024-03-16T09:01:10.753 INFO:tasks.workunit.client.1.smithi019.stdout:0/85: rmdir d7/d13 39 2024-03-16T09:01:10.754 INFO:tasks.workunit.client.1.smithi019.stdout:0/86: fdatasync d7/f15 0 2024-03-16T09:01:10.755 INFO:tasks.workunit.client.0.smithi007.stdout:9/402: truncate f1b 3096114 0 2024-03-16T09:01:10.755 INFO:tasks.workunit.client.1.smithi019.stdout:3/109: sync 2024-03-16T09:01:10.757 INFO:tasks.workunit.client.1.smithi019.stdout:2/106: dwrite d5/d6/f1d [0,4194304] 0 2024-03-16T09:01:10.757 INFO:tasks.workunit.client.1.smithi019.stdout:2/107: write d5/de/f1b [778399,30320] 0 2024-03-16T09:01:10.757 INFO:tasks.workunit.client.1.smithi019.stdout:2/108: readlink d5/de/d10/l20 0 2024-03-16T09:01:10.758 INFO:tasks.workunit.client.1.smithi019.stdout:7/104: mknod d9/db/d12/c1e 0 2024-03-16T09:01:10.758 INFO:tasks.workunit.client.1.smithi019.stdout:0/87: mkdir d7/d13/d1a 0 2024-03-16T09:01:10.759 INFO:tasks.workunit.client.0.smithi007.stdout:4/298: truncate d4/d14/d1c/f30 3490606 0 2024-03-16T09:01:10.759 INFO:tasks.workunit.client.0.smithi007.stdout:4/299: creat d4/d14/d34/f63 x:0 0 0 2024-03-16T09:01:10.761 INFO:tasks.workunit.client.1.smithi019.stdout:7/105: creat d9/da/d13/d18/d1a/f1f x:0 0 0 2024-03-16T09:01:10.762 INFO:tasks.workunit.client.1.smithi019.stdout:0/88: mknod d7/d14/c1b 0 2024-03-16T09:01:10.764 INFO:tasks.workunit.client.1.smithi019.stdout:7/106: creat d9/db/d12/f20 x:0 0 0 2024-03-16T09:01:10.764 INFO:tasks.workunit.client.1.smithi019.stdout:7/107: truncate d9/db/f1d 659612 0 2024-03-16T09:01:10.765 INFO:tasks.workunit.client.1.smithi019.stdout:0/89: symlink d7/d14/l1c 0 2024-03-16T09:01:10.765 INFO:tasks.workunit.client.1.smithi019.stdout:0/90: fdatasync d7/d8/fc 0 2024-03-16T09:01:10.766 INFO:tasks.workunit.client.1.smithi019.stdout:7/108: symlink d9/da/d13/d18/d1a/l21 0 2024-03-16T09:01:10.767 INFO:tasks.workunit.client.1.smithi019.stdout:0/91: stat d7/l17 0 2024-03-16T09:01:10.767 INFO:tasks.workunit.client.1.smithi019.stdout:0/92: chown f4 472007380 1 2024-03-16T09:01:10.767 INFO:tasks.workunit.client.1.smithi019.stdout:0/93: dread - d7/f19 zero size 2024-03-16T09:01:10.768 INFO:tasks.workunit.client.1.smithi019.stdout:0/94: chown d7/d8/fc 1848584569 1 2024-03-16T09:01:10.768 INFO:tasks.workunit.client.1.smithi019.stdout:3/110: dread d3/d11/f12 [0,4194304] 0 2024-03-16T09:01:10.768 INFO:tasks.workunit.client.1.smithi019.stdout:3/111: creat d3/d11/f21 x:0 0 0 2024-03-16T09:01:10.768 INFO:tasks.workunit.client.1.smithi019.stdout:3/112: chown d3/d11/f14 28 1 2024-03-16T09:01:10.768 INFO:tasks.workunit.client.1.smithi019.stdout:4/164: dwrite f2 [0,4194304] 0 2024-03-16T09:01:10.769 INFO:tasks.workunit.client.1.smithi019.stdout:2/109: rmdir d5 39 2024-03-16T09:01:10.773 INFO:tasks.workunit.client.1.smithi019.stdout:3/113: unlink d3/d11/f12 0 2024-03-16T09:01:10.774 INFO:tasks.workunit.client.1.smithi019.stdout:4/165: getdents d4/d1b/d2a 0 2024-03-16T09:01:10.774 INFO:tasks.workunit.client.1.smithi019.stdout:3/114: read d3/d1b/f1c [704692,112792] 0 2024-03-16T09:01:10.776 INFO:tasks.workunit.client.1.smithi019.stdout:2/110: mknod d5/c25 0 2024-03-16T09:01:10.777 INFO:tasks.workunit.client.1.smithi019.stdout:7/109: truncate d9/da/d13/f17 2120919 0 2024-03-16T09:01:10.777 INFO:tasks.workunit.client.1.smithi019.stdout:4/166: rmdir d4/d1b/d2a 39 2024-03-16T09:01:10.778 INFO:tasks.workunit.client.1.smithi019.stdout:6/259: sync 2024-03-16T09:01:10.779 INFO:tasks.workunit.client.1.smithi019.stdout:9/124: sync 2024-03-16T09:01:10.779 INFO:tasks.workunit.client.1.smithi019.stdout:1/56: sync 2024-03-16T09:01:10.779 INFO:tasks.workunit.client.1.smithi019.stdout:5/103: sync 2024-03-16T09:01:10.779 INFO:tasks.workunit.client.1.smithi019.stdout:5/104: readlink d6/l11 0 2024-03-16T09:01:10.779 INFO:tasks.workunit.client.1.smithi019.stdout:6/260: creat de/d26/d32/f62 x:0 0 0 2024-03-16T09:01:10.779 INFO:tasks.workunit.client.1.smithi019.stdout:9/125: write d1/d7/d21/d27/f22 [4748815,35105] 0 2024-03-16T09:01:10.779 INFO:tasks.workunit.client.1.smithi019.stdout:6/261: dread - de/d26/f39 zero size 2024-03-16T09:01:10.779 INFO:tasks.workunit.client.1.smithi019.stdout:8/138: sync 2024-03-16T09:01:10.780 INFO:tasks.workunit.client.1.smithi019.stdout:8/139: write d6/f2b [169670,121433] 0 2024-03-16T09:01:10.780 INFO:tasks.workunit.client.1.smithi019.stdout:8/140: read d6/de/ff [1557899,101524] 0 2024-03-16T09:01:10.780 INFO:tasks.workunit.client.1.smithi019.stdout:8/141: write d6/de/f29 [37302,75147] 0 2024-03-16T09:01:10.781 INFO:tasks.workunit.client.1.smithi019.stdout:3/115: unlink d3/d11/l1a 0 2024-03-16T09:01:10.781 INFO:tasks.workunit.client.1.smithi019.stdout:3/116: dread d3/d1b/f1c [0,4194304] 0 2024-03-16T09:01:10.781 INFO:tasks.workunit.client.1.smithi019.stdout:2/111: symlink d5/d6/l26 0 2024-03-16T09:01:10.783 INFO:tasks.workunit.client.1.smithi019.stdout:0/95: rename d7/d13 to d7/d1d 0 2024-03-16T09:01:10.783 INFO:tasks.workunit.client.1.smithi019.stdout:3/117: dread d3/d1b/f1c [0,4194304] 0 2024-03-16T09:01:10.784 INFO:tasks.workunit.client.1.smithi019.stdout:7/110: rmdir d9/da/d13/d18 39 2024-03-16T09:01:10.787 INFO:tasks.workunit.client.1.smithi019.stdout:4/167: dread d4/d6/f39 [0,4194304] 0 2024-03-16T09:01:10.790 INFO:tasks.workunit.client.1.smithi019.stdout:9/126: write d1/f3 [3625578,15227] 0 2024-03-16T09:01:10.792 INFO:tasks.workunit.client.1.smithi019.stdout:5/105: creat d6/d19/f1b x:0 0 0 2024-03-16T09:01:10.792 INFO:tasks.workunit.client.1.smithi019.stdout:9/127: dread d1/d7/de/f1a [0,4194304] 0 2024-03-16T09:01:10.794 INFO:tasks.workunit.client.1.smithi019.stdout:8/142: dread d6/de/d16/d19/f18 [0,4194304] 0 2024-03-16T09:01:10.795 INFO:tasks.workunit.client.1.smithi019.stdout:2/112: symlink d5/l27 0 2024-03-16T09:01:10.796 INFO:tasks.workunit.client.1.smithi019.stdout:0/96: rename d7/d1d/c18 to d7/d1d/c1e 0 2024-03-16T09:01:10.801 INFO:tasks.workunit.client.1.smithi019.stdout:7/111: symlink d9/da/d13/d18/d1a/l22 0 2024-03-16T09:01:10.801 INFO:tasks.workunit.client.1.smithi019.stdout:7/112: creat d9/f23 x:0 0 0 2024-03-16T09:01:10.802 INFO:tasks.workunit.client.1.smithi019.stdout:1/57: rename f8 to fb 0 2024-03-16T09:01:10.803 INFO:tasks.workunit.client.1.smithi019.stdout:8/143: dread d6/de/f1b [0,4194304] 0 2024-03-16T09:01:10.803 INFO:tasks.workunit.client.1.smithi019.stdout:8/144: fsync d6/f7 0 2024-03-16T09:01:10.806 INFO:tasks.workunit.client.1.smithi019.stdout:4/168: symlink d4/d6/l3c 0 2024-03-16T09:01:10.808 INFO:tasks.workunit.client.1.smithi019.stdout:5/106: symlink d6/d19/l1c 0 2024-03-16T09:01:10.808 INFO:tasks.workunit.client.1.smithi019.stdout:9/128: rmdir d1/d7/d21/d27 39 2024-03-16T09:01:10.808 INFO:tasks.workunit.client.1.smithi019.stdout:9/129: write d1/d7/de/f1a [921430,19938] 0 2024-03-16T09:01:10.810 INFO:tasks.workunit.client.1.smithi019.stdout:6/262: rmdir de/d26/d32 39 2024-03-16T09:01:10.810 INFO:tasks.workunit.client.1.smithi019.stdout:6/263: write de/d26/d32/f37 [768572,75441] 0 2024-03-16T09:01:10.814 INFO:tasks.workunit.client.1.smithi019.stdout:3/118: mkdir d3/d11/d22 0 2024-03-16T09:01:10.814 INFO:tasks.workunit.client.1.smithi019.stdout:3/119: creat d3/d1b/f23 x:0 0 0 2024-03-16T09:01:10.814 INFO:tasks.workunit.client.1.smithi019.stdout:3/120: dread - d3/d1b/f23 zero size 2024-03-16T09:01:10.815 INFO:tasks.workunit.client.1.smithi019.stdout:2/113: rmdir d5/d6 39 2024-03-16T09:01:10.820 INFO:tasks.workunit.client.1.smithi019.stdout:5/107: dread d6/dd/f1a [0,4194304] 0 2024-03-16T09:01:10.820 INFO:tasks.workunit.client.1.smithi019.stdout:8/145: rename d6/f2b to d6/de/d16/d19/f31 0 2024-03-16T09:01:10.821 INFO:tasks.workunit.client.1.smithi019.stdout:6/264: write de/d1c/f20 [3118491,122667] 0 2024-03-16T09:01:10.821 INFO:tasks.workunit.client.1.smithi019.stdout:6/265: write de/d1c/f2e [1636686,126726] 0 2024-03-16T09:01:10.822 INFO:tasks.workunit.client.1.smithi019.stdout:9/130: unlink d1/d7/d21/d27/c1d 0 2024-03-16T09:01:10.823 INFO:tasks.workunit.client.1.smithi019.stdout:3/121: rmdir d3 39 2024-03-16T09:01:10.823 INFO:tasks.workunit.client.1.smithi019.stdout:3/122: getdents d3/d11/d22 0 2024-03-16T09:01:10.823 INFO:tasks.workunit.client.1.smithi019.stdout:3/123: stat d3/d11/d22 0 2024-03-16T09:01:10.825 INFO:tasks.workunit.client.1.smithi019.stdout:2/114: creat d5/d6/f28 x:0 0 0 2024-03-16T09:01:10.826 INFO:tasks.workunit.client.1.smithi019.stdout:1/58: rename c4 to cc 0 2024-03-16T09:01:10.827 INFO:tasks.workunit.client.1.smithi019.stdout:0/97: dwrite f4 [0,4194304] 0 2024-03-16T09:01:10.827 INFO:tasks.workunit.client.1.smithi019.stdout:7/113: dwrite f8 [0,4194304] 0 2024-03-16T09:01:10.827 INFO:tasks.workunit.client.1.smithi019.stdout:7/114: write d9/db/f1d [1695264,116804] 0 2024-03-16T09:01:10.827 INFO:tasks.workunit.client.1.smithi019.stdout:7/115: chown d9/ce 486529871 1 2024-03-16T09:01:10.829 INFO:tasks.workunit.client.1.smithi019.stdout:4/169: dwrite d4/d1b/f28 [0,4194304] 0 2024-03-16T09:01:10.831 INFO:tasks.workunit.client.1.smithi019.stdout:6/266: rename de/d26/d32/c35 to de/d26/c63 0 2024-03-16T09:01:10.831 INFO:tasks.workunit.client.1.smithi019.stdout:6/267: stat de/d1c/d1d/f30 0 2024-03-16T09:01:10.831 INFO:tasks.workunit.client.1.smithi019.stdout:3/124: mkdir d3/d11/d22/d24 0 2024-03-16T09:01:10.833 INFO:tasks.workunit.client.1.smithi019.stdout:2/115: mknod d5/d6/c29 0 2024-03-16T09:01:10.834 INFO:tasks.workunit.client.1.smithi019.stdout:1/59: creat fd x:0 0 0 2024-03-16T09:01:10.836 INFO:tasks.workunit.client.1.smithi019.stdout:0/98: symlink d7/d14/l1f 0 2024-03-16T09:01:10.838 INFO:tasks.workunit.client.1.smithi019.stdout:0/99: dread d7/f15 [0,4194304] 0 2024-03-16T09:01:10.838 INFO:tasks.workunit.client.0.smithi007.stdout:0/350: mkdir d3/d77 0 2024-03-16T09:01:10.842 INFO:tasks.workunit.client.1.smithi019.stdout:6/268: getdents de/d26/d32/d4e 0 2024-03-16T09:01:10.846 INFO:tasks.workunit.client.1.smithi019.stdout:2/116: dread d5/d6/f21 [0,4194304] 0 2024-03-16T09:01:10.847 INFO:tasks.workunit.client.1.smithi019.stdout:1/60: creat fe x:0 0 0 2024-03-16T09:01:10.847 INFO:tasks.workunit.client.1.smithi019.stdout:1/61: readlink - no filename 2024-03-16T09:01:10.847 INFO:tasks.workunit.client.1.smithi019.stdout:1/62: chown cc 26483273 1 2024-03-16T09:01:10.847 INFO:tasks.workunit.client.1.smithi019.stdout:1/63: chown fb 26240 1 2024-03-16T09:01:10.848 INFO:tasks.workunit.client.0.smithi007.stdout:3/430: getdents d15/d18/d1f/d3b/d87 0 2024-03-16T09:01:10.848 INFO:tasks.workunit.client.1.smithi019.stdout:6/269: symlink de/d1c/d1d/d51/l64 0 2024-03-16T09:01:10.848 INFO:tasks.workunit.client.1.smithi019.stdout:3/125: rename d3/d1b/f23 to d3/f25 0 2024-03-16T09:01:10.848 INFO:tasks.workunit.client.1.smithi019.stdout:3/126: write d3/d1b/f1c [883309,97913] 0 2024-03-16T09:01:10.849 INFO:tasks.workunit.client.0.smithi007.stdout:7/254: mknod d0/c51 0 2024-03-16T09:01:10.849 INFO:tasks.workunit.client.1.smithi019.stdout:2/117: symlink d5/de/d10/l2a 0 2024-03-16T09:01:10.850 INFO:tasks.workunit.client.1.smithi019.stdout:6/270: truncate f7 4836250 0 2024-03-16T09:01:10.850 INFO:tasks.workunit.client.0.smithi007.stdout:8/599: getdents d0/dc/dd/d8b/d9e 0 2024-03-16T09:01:10.851 INFO:tasks.workunit.client.1.smithi019.stdout:0/100: rename d7/cf to d7/d14/c20 0 2024-03-16T09:01:10.853 INFO:tasks.workunit.client.0.smithi007.stdout:9/403: symlink d24/d4a/d61/l72 0 2024-03-16T09:01:10.853 INFO:tasks.workunit.client.1.smithi019.stdout:1/64: rename fe to ff 0 2024-03-16T09:01:10.855 INFO:tasks.workunit.client.1.smithi019.stdout:0/101: mkdir d7/d21 0 2024-03-16T09:01:10.855 INFO:tasks.workunit.client.0.smithi007.stdout:4/300: mkdir d4/d14/d64 0 2024-03-16T09:01:10.856 INFO:tasks.workunit.client.0.smithi007.stdout:4/301: chown d4/d14/d1c/f2c 94898 1 2024-03-16T09:01:10.856 INFO:tasks.workunit.client.1.smithi019.stdout:0/102: getdents d7/d1d 0 2024-03-16T09:01:10.857 INFO:tasks.workunit.client.1.smithi019.stdout:3/127: link d3/fc d3/d11/f26 0 2024-03-16T09:01:10.858 INFO:tasks.workunit.client.0.smithi007.stdout:2/468: getdents dc/df/d24 0 2024-03-16T09:01:10.858 INFO:tasks.workunit.client.1.smithi019.stdout:0/103: unlink c0 0 2024-03-16T09:01:10.859 INFO:tasks.workunit.client.1.smithi019.stdout:0/104: write d7/f15 [552123,89990] 0 2024-03-16T09:01:10.859 INFO:tasks.workunit.client.1.smithi019.stdout:0/105: truncate d7/f19 312206 0 2024-03-16T09:01:10.860 INFO:tasks.workunit.client.1.smithi019.stdout:8/146: dwrite f2 [0,4194304] 0 2024-03-16T09:01:10.860 INFO:tasks.workunit.client.1.smithi019.stdout:8/147: fsync f5 0 2024-03-16T09:01:10.860 INFO:tasks.workunit.client.1.smithi019.stdout:8/148: creat d6/de/d16/f32 x:0 0 0 2024-03-16T09:01:10.864 INFO:tasks.workunit.client.1.smithi019.stdout:9/131: dwrite d1/d7/d21/d27/f22 [4194304,4194304] 0 2024-03-16T09:01:10.865 INFO:tasks.workunit.client.1.smithi019.stdout:4/170: dwrite d4/d6/f39 [4194304,4194304] 0 2024-03-16T09:01:10.865 INFO:tasks.workunit.client.1.smithi019.stdout:4/171: write d4/d6/d15/d1f/d2c/f31 [713151,54567] 0 2024-03-16T09:01:10.866 INFO:tasks.workunit.client.0.smithi007.stdout:6/270: rmdir d2/d13/d24 39 2024-03-16T09:01:10.867 INFO:tasks.workunit.client.0.smithi007.stdout:6/271: chown d2 3375473 1 2024-03-16T09:01:10.867 INFO:tasks.workunit.client.1.smithi019.stdout:6/271: dread de/f3c [0,4194304] 0 2024-03-16T09:01:10.867 INFO:tasks.workunit.client.1.smithi019.stdout:6/272: readlink de/l31 0 2024-03-16T09:01:10.867 INFO:tasks.workunit.client.1.smithi019.stdout:9/132: mknod d1/d7/d21/d27/c2f 0 2024-03-16T09:01:10.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/431: creat d15/d18/d1f/f8d x:0 0 0 2024-03-16T09:01:10.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/432: truncate d15/d18/d1f/d29/d3e/d4c/f52 1041985 0 2024-03-16T09:01:10.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/433: write d15/d18/d1f/f8d [1011351,1921] 0 2024-03-16T09:01:10.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/434: fdatasync d15/d18/d1f/f58 0 2024-03-16T09:01:10.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/435: chown d15/d18/d1f/d29/d3e/l53 0 1 2024-03-16T09:01:10.870 INFO:tasks.workunit.client.1.smithi019.stdout:6/273: link de/c12 de/d26/d32/c65 0 2024-03-16T09:01:10.871 INFO:tasks.workunit.client.1.smithi019.stdout:6/274: chown de/d26/d32/l34 63 1 2024-03-16T09:01:10.871 INFO:tasks.workunit.client.1.smithi019.stdout:6/275: write de/d1c/f20 [5054821,70463] 0 2024-03-16T09:01:10.871 INFO:tasks.workunit.client.0.smithi007.stdout:7/255: creat d0/dc/d24/f52 x:0 0 0 2024-03-16T09:01:10.872 INFO:tasks.workunit.client.0.smithi007.stdout:7/256: chown d0/dc/l12 209084538 1 2024-03-16T09:01:10.872 INFO:tasks.workunit.client.0.smithi007.stdout:7/257: chown d0/d19/f2d 7381493 1 2024-03-16T09:01:10.875 INFO:tasks.workunit.client.0.smithi007.stdout:9/404: rename d24/c36 to d24/c73 0 2024-03-16T09:01:10.876 INFO:tasks.workunit.client.1.smithi019.stdout:1/65: dwrite fa [0,4194304] 0 2024-03-16T09:01:10.879 INFO:tasks.workunit.client.0.smithi007.stdout:4/302: symlink d4/d14/d1c/l65 0 2024-03-16T09:01:10.879 INFO:tasks.workunit.client.0.smithi007.stdout:4/303: write d4/d14/d1c/f29 [486644,115121] 0 2024-03-16T09:01:10.879 INFO:tasks.workunit.client.0.smithi007.stdout:4/304: write d4/d14/d34/f45 [415369,59341] 0 2024-03-16T09:01:10.879 INFO:tasks.workunit.client.0.smithi007.stdout:4/305: fsync d4/ff 0 2024-03-16T09:01:10.881 INFO:tasks.workunit.client.1.smithi019.stdout:1/66: creat f10 x:0 0 0 2024-03-16T09:01:10.883 INFO:tasks.workunit.client.0.smithi007.stdout:7/258: rename d0/d19/d1f/l3f to d0/dc/d38/d4e/l53 0 2024-03-16T09:01:10.883 INFO:tasks.workunit.client.0.smithi007.stdout:7/259: dread - d0/d19/d1f/d42/f45 zero size 2024-03-16T09:01:10.883 INFO:tasks.workunit.client.0.smithi007.stdout:7/260: creat d0/d19/d1f/f54 x:0 0 0 2024-03-16T09:01:10.883 INFO:tasks.workunit.client.0.smithi007.stdout:7/261: fdatasync d0/dc/fe 0 2024-03-16T09:01:10.884 INFO:tasks.workunit.client.0.smithi007.stdout:8/600: rmdir d0/d26/d2d/d43/d70 39 2024-03-16T09:01:10.884 INFO:tasks.workunit.client.0.smithi007.stdout:8/601: write d0/d26/d2d/d43/d6f/f8f [291606,80924] 0 2024-03-16T09:01:10.886 INFO:tasks.workunit.client.0.smithi007.stdout:9/405: truncate f17 2032864 0 2024-03-16T09:01:10.887 INFO:tasks.workunit.client.0.smithi007.stdout:4/306: symlink d4/d14/d64/l66 0 2024-03-16T09:01:10.890 INFO:tasks.workunit.client.1.smithi019.stdout:3/128: getdents d3 0 2024-03-16T09:01:10.890 INFO:tasks.workunit.client.0.smithi007.stdout:8/602: rename d0/dc/dd/d8b/lc4 to d0/dc/d54/da4/ld4 0 2024-03-16T09:01:10.892 INFO:tasks.workunit.client.0.smithi007.stdout:9/406: link d24/f46 d24/d4a/f74 0 2024-03-16T09:01:10.893 INFO:tasks.workunit.client.1.smithi019.stdout:2/118: rename d5/de/d10 to d5/d6/d2b 0 2024-03-16T09:01:10.893 INFO:tasks.workunit.client.0.smithi007.stdout:9/407: mknod d24/d4a/d55/c75 0 2024-03-16T09:01:10.893 INFO:tasks.workunit.client.0.smithi007.stdout:9/408: fdatasync d24/d4a/d55/f5d 0 2024-03-16T09:01:10.894 INFO:tasks.workunit.client.0.smithi007.stdout:7/262: dread d0/f3c [0,4194304] 0 2024-03-16T09:01:10.894 INFO:tasks.workunit.client.0.smithi007.stdout:9/409: mknod d24/d4a/d55/c76 0 2024-03-16T09:01:10.894 INFO:tasks.workunit.client.1.smithi019.stdout:1/67: symlink l11 0 2024-03-16T09:01:10.895 INFO:tasks.workunit.client.1.smithi019.stdout:3/129: dread d3/ff [0,4194304] 0 2024-03-16T09:01:10.896 INFO:tasks.workunit.client.0.smithi007.stdout:7/263: creat d0/f55 x:0 0 0 2024-03-16T09:01:10.897 INFO:tasks.workunit.client.0.smithi007.stdout:9/410: readlink d24/d4c/l57 0 2024-03-16T09:01:10.897 INFO:tasks.workunit.client.0.smithi007.stdout:8/603: dread d0/dc/dd/f2c [0,4194304] 0 2024-03-16T09:01:10.898 INFO:tasks.workunit.client.0.smithi007.stdout:7/264: mknod d0/dc/c56 0 2024-03-16T09:01:10.898 INFO:tasks.workunit.client.0.smithi007.stdout:7/265: creat d0/d19/d1f/d42/f57 x:0 0 0 2024-03-16T09:01:10.898 INFO:tasks.workunit.client.0.smithi007.stdout:7/266: chown d0/d19/d1f/f54 3715 1 2024-03-16T09:01:10.898 INFO:tasks.workunit.client.0.smithi007.stdout:7/267: write d0/d19/d1f/f2c [4501800,25688] 0 2024-03-16T09:01:10.898 INFO:tasks.workunit.client.0.smithi007.stdout:7/268: write d0/d19/f41 [4245617,100113] 0 2024-03-16T09:01:10.898 INFO:tasks.workunit.client.1.smithi019.stdout:8/149: rename d6/c1e to d6/d2a/c33 0 2024-03-16T09:01:10.899 INFO:tasks.workunit.client.0.smithi007.stdout:8/604: rmdir d0/dc/dd/d8b/dc9 39 2024-03-16T09:01:10.900 INFO:tasks.workunit.client.1.smithi019.stdout:2/119: mknod d5/d6/d2b/c2c 0 2024-03-16T09:01:10.900 INFO:tasks.workunit.client.1.smithi019.stdout:2/120: fdatasync d5/d6/f1d 0 2024-03-16T09:01:10.901 INFO:tasks.workunit.client.1.smithi019.stdout:2/121: fdatasync d5/d6/f21 0 2024-03-16T09:01:10.903 INFO:tasks.workunit.client.1.smithi019.stdout:3/130: truncate d3/fc 6697744 0 2024-03-16T09:01:10.903 INFO:tasks.workunit.client.1.smithi019.stdout:3/131: fdatasync d3/ff 0 2024-03-16T09:01:10.903 INFO:tasks.workunit.client.1.smithi019.stdout:0/106: dwrite f2 [0,4194304] 0 2024-03-16T09:01:10.903 INFO:tasks.workunit.client.1.smithi019.stdout:6/276: dwrite de/d1c/f5d [0,4194304] 0 2024-03-16T09:01:10.903 INFO:tasks.workunit.client.1.smithi019.stdout:6/277: write de/d26/d32/f5b [919046,50527] 0 2024-03-16T09:01:10.903 INFO:tasks.workunit.client.1.smithi019.stdout:6/278: write de/f14 [4812523,120650] 0 2024-03-16T09:01:10.905 INFO:tasks.workunit.client.0.smithi007.stdout:3/436: dwrite d15/d18/d1f/f54 [0,4194304] 0 2024-03-16T09:01:10.909 INFO:tasks.workunit.client.0.smithi007.stdout:7/269: creat d0/dc/d24/d43/f58 x:0 0 0 2024-03-16T09:01:10.909 INFO:tasks.workunit.client.0.smithi007.stdout:7/270: write d0/dc/d38/d4e/f50 [911425,103798] 0 2024-03-16T09:01:10.909 INFO:tasks.workunit.client.1.smithi019.stdout:1/68: write fa [1382941,38605] 0 2024-03-16T09:01:10.910 INFO:tasks.workunit.client.1.smithi019.stdout:1/69: fdatasync ff 0 2024-03-16T09:01:10.910 INFO:tasks.workunit.client.1.smithi019.stdout:1/70: dread - ff zero size 2024-03-16T09:01:10.910 INFO:tasks.workunit.client.1.smithi019.stdout:2/122: dread d5/de/f1b [0,4194304] 0 2024-03-16T09:01:10.910 INFO:tasks.workunit.client.1.smithi019.stdout:9/133: rename d1/d7/de/ff to d1/d7/de/d2d/f30 0 2024-03-16T09:01:10.910 INFO:tasks.workunit.client.1.smithi019.stdout:6/279: mkdir de/d1c/d1d/d66 0 2024-03-16T09:01:10.910 INFO:tasks.workunit.client.1.smithi019.stdout:6/280: chown de/l31 82 1 2024-03-16T09:01:10.910 INFO:tasks.workunit.client.1.smithi019.stdout:6/281: stat de/d1c/l2d 0 2024-03-16T09:01:10.913 INFO:tasks.workunit.client.0.smithi007.stdout:2/469: rmdir dc/df/d24/d28/d71 0 2024-03-16T09:01:10.913 INFO:tasks.workunit.client.0.smithi007.stdout:2/470: truncate dc/df/f34 4833425 0 2024-03-16T09:01:10.913 INFO:tasks.workunit.client.0.smithi007.stdout:2/471: creat dc/df/d24/d28/d2e/d35/f98 x:0 0 0 2024-03-16T09:01:10.914 INFO:tasks.workunit.client.1.smithi019.stdout:3/132: unlink d3/d11/f21 0 2024-03-16T09:01:10.914 INFO:tasks.workunit.client.1.smithi019.stdout:3/133: stat d3/d11/d22/d24 0 2024-03-16T09:01:10.914 INFO:tasks.workunit.client.0.smithi007.stdout:4/307: dwrite d4/f5f [0,4194304] 0 2024-03-16T09:01:10.914 INFO:tasks.workunit.client.0.smithi007.stdout:4/308: truncate d4/d14/d1c/d1e/f4b 255375 0 2024-03-16T09:01:10.914 INFO:tasks.workunit.client.0.smithi007.stdout:4/309: dread - d4/d14/d1c/d5b/f62 zero size 2024-03-16T09:01:10.915 INFO:tasks.workunit.client.1.smithi019.stdout:1/71: stat c7 0 2024-03-16T09:01:10.916 INFO:tasks.workunit.client.0.smithi007.stdout:2/472: mkdir dc/df/d24/d28/d97/d99 0 2024-03-16T09:01:10.918 INFO:tasks.workunit.client.1.smithi019.stdout:2/123: rename d5/d6/c12 to d5/d6/d2b/c2d 0 2024-03-16T09:01:10.919 INFO:tasks.workunit.client.0.smithi007.stdout:6/272: dwrite d2/d13/d1f/f4f [0,4194304] 0 2024-03-16T09:01:10.922 INFO:tasks.workunit.client.0.smithi007.stdout:4/310: mkdir d4/d14/d1c/d5b/d67 0 2024-03-16T09:01:10.924 INFO:tasks.workunit.client.1.smithi019.stdout:3/134: rename d3/d17/l19 to d3/l27 0 2024-03-16T09:01:10.925 INFO:tasks.workunit.client.0.smithi007.stdout:2/473: creat dc/df/d24/d28/d2e/d6e/f9a x:0 0 0 2024-03-16T09:01:10.925 INFO:tasks.workunit.client.0.smithi007.stdout:2/474: chown dc/df/l12 13209 1 2024-03-16T09:01:10.925 INFO:tasks.workunit.client.0.smithi007.stdout:6/273: readlink d2/d11/l30 0 2024-03-16T09:01:10.925 INFO:tasks.workunit.client.0.smithi007.stdout:6/274: fsync d2/d11/f25 0 2024-03-16T09:01:10.927 INFO:tasks.workunit.client.0.smithi007.stdout:6/275: creat d2/f59 x:0 0 0 2024-03-16T09:01:10.927 INFO:tasks.workunit.client.0.smithi007.stdout:6/276: write d2/d10/d12/d15/f22 [4892235,3003] 0 2024-03-16T09:01:10.932 INFO:tasks.workunit.client.0.smithi007.stdout:6/277: rename d2/d10/d12/d15/l27 to d2/d13/d1f/d34/l5a 0 2024-03-16T09:01:10.932 INFO:tasks.workunit.client.0.smithi007.stdout:6/278: creat d2/d13/d1f/d4e/f5b x:0 0 0 2024-03-16T09:01:10.933 INFO:tasks.workunit.client.1.smithi019.stdout:8/150: dwrite d6/de/f1d [0,4194304] 0 2024-03-16T09:01:10.933 INFO:tasks.workunit.client.1.smithi019.stdout:8/151: chown d6/de/ff 4639208 1 2024-03-16T09:01:10.936 INFO:tasks.workunit.client.1.smithi019.stdout:9/134: dwrite d1/d7/de/f2a [0,4194304] 0 2024-03-16T09:01:10.938 INFO:tasks.workunit.client.0.smithi007.stdout:7/271: dread d0/f49 [0,4194304] 0 2024-03-16T09:01:10.938 INFO:tasks.workunit.client.0.smithi007.stdout:7/272: fsync d0/d19/d1f/d42/f45 0 2024-03-16T09:01:10.947 INFO:tasks.workunit.client.0.smithi007.stdout:2/475: write dc/df/d24/d28/f66 [1253997,102114] 0 2024-03-16T09:01:10.950 INFO:tasks.workunit.client.1.smithi019.stdout:9/135: symlink d1/d7/de/d24/l31 0 2024-03-16T09:01:10.951 INFO:tasks.workunit.client.1.smithi019.stdout:9/136: write d1/d7/d21/d27/f13 [1031810,118642] 0 2024-03-16T09:01:10.951 INFO:tasks.workunit.client.0.smithi007.stdout:2/476: creat dc/df/d24/d28/d2e/f9b x:0 0 0 2024-03-16T09:01:10.951 INFO:tasks.workunit.client.0.smithi007.stdout:2/477: getdents dc/df/d24 0 2024-03-16T09:01:10.953 INFO:tasks.workunit.client.1.smithi019.stdout:8/152: rmdir d6 39 2024-03-16T09:01:10.955 INFO:tasks.workunit.client.1.smithi019.stdout:9/137: rmdir d1/d7 39 2024-03-16T09:01:10.962 INFO:tasks.workunit.client.1.smithi019.stdout:0/107: dwrite d7/d8/fc [0,4194304] 0 2024-03-16T09:01:10.964 INFO:tasks.workunit.client.1.smithi019.stdout:1/72: fdatasync fa 0 2024-03-16T09:01:10.968 INFO:tasks.workunit.client.0.smithi007.stdout:9/411: dwrite d24/f54 [0,4194304] 0 2024-03-16T09:01:10.969 INFO:tasks.workunit.client.0.smithi007.stdout:8/605: dwrite d0/d2/d5f/fc2 [0,4194304] 0 2024-03-16T09:01:10.969 INFO:tasks.workunit.client.0.smithi007.stdout:8/606: chown d0/d26/d2d/d43/d4c 15047 1 2024-03-16T09:01:10.969 INFO:tasks.workunit.client.0.smithi007.stdout:8/607: write d0/dc/d80/fcc [877908,95726] 0 2024-03-16T09:01:10.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/437: dwrite d15/d18/d1f/d29/d43/d72/f50 [0,4194304] 0 2024-03-16T09:01:10.977 INFO:tasks.workunit.client.0.smithi007.stdout:9/412: creat d24/d4a/d55/d56/f77 x:0 0 0 2024-03-16T09:01:10.978 INFO:tasks.workunit.client.0.smithi007.stdout:9/413: write d24/d4a/f53 [15111,121832] 0 2024-03-16T09:01:10.984 INFO:tasks.workunit.client.0.smithi007.stdout:8/608: link d0/dc/d54/da4/fc7 d0/d26/d2d/d43/d4c/dba/fd5 0 2024-03-16T09:01:10.984 INFO:tasks.workunit.client.0.smithi007.stdout:4/311: dwrite d4/f22 [0,4194304] 0 2024-03-16T09:01:10.985 INFO:tasks.workunit.client.1.smithi019.stdout:2/124: dwrite d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:10.985 INFO:tasks.workunit.client.1.smithi019.stdout:2/125: write d5/fb [2807211,93351] 0 2024-03-16T09:01:10.986 INFO:tasks.workunit.client.0.smithi007.stdout:3/438: creat d15/d18/d1f/d3b/d87/f8e x:0 0 0 2024-03-16T09:01:10.987 INFO:tasks.workunit.client.0.smithi007.stdout:8/609: truncate d0/d26/f39 1633561 0 2024-03-16T09:01:10.991 INFO:tasks.workunit.client.1.smithi019.stdout:2/126: truncate f4 1512771 0 2024-03-16T09:01:10.992 INFO:tasks.workunit.client.1.smithi019.stdout:2/127: chown d5/f7 27020 1 2024-03-16T09:01:10.995 INFO:tasks.workunit.client.0.smithi007.stdout:9/414: write d24/f4f [7350370,73338] 0 2024-03-16T09:01:10.997 INFO:tasks.workunit.client.0.smithi007.stdout:1/370: sync 2024-03-16T09:01:10.997 INFO:tasks.workunit.client.0.smithi007.stdout:5/383: sync 2024-03-16T09:01:11.002 INFO:tasks.workunit.client.0.smithi007.stdout:8/610: rmdir d0/dc/dd/d8b/dc9 39 2024-03-16T09:01:11.003 INFO:tasks.workunit.client.0.smithi007.stdout:9/415: link f17 d24/d4a/f78 0 2024-03-16T09:01:11.003 INFO:tasks.workunit.client.0.smithi007.stdout:9/416: write d24/f2d [3033300,73985] 0 2024-03-16T09:01:11.005 INFO:tasks.workunit.client.0.smithi007.stdout:1/371: rename d9/c27 to d9/dd/d1b/d28/c71 0 2024-03-16T09:01:11.005 INFO:tasks.workunit.client.0.smithi007.stdout:9/417: dread d24/f25 [0,4194304] 0 2024-03-16T09:01:11.006 INFO:tasks.workunit.client.0.smithi007.stdout:9/418: fdatasync d24/d4a/d55/f5d 0 2024-03-16T09:01:11.006 INFO:tasks.workunit.client.0.smithi007.stdout:5/384: link d1/d7/d1c/c7b d1/d7/d1c/d19/d34/c8f 0 2024-03-16T09:01:11.006 INFO:tasks.workunit.client.0.smithi007.stdout:3/439: getdents d15/d18/d6a/d78 0 2024-03-16T09:01:11.007 INFO:tasks.workunit.client.0.smithi007.stdout:1/372: symlink d9/dd/d1b/d2d/l72 0 2024-03-16T09:01:11.009 INFO:tasks.workunit.client.0.smithi007.stdout:3/440: mkdir d15/d18/d1f/d3b/d63/d8f 0 2024-03-16T09:01:11.009 INFO:tasks.workunit.client.0.smithi007.stdout:1/373: mknod d9/d6b/c73 0 2024-03-16T09:01:11.010 INFO:tasks.workunit.client.0.smithi007.stdout:1/374: write d9/dd/d1b/d28/f60 [4851606,95615] 0 2024-03-16T09:01:11.011 INFO:tasks.workunit.client.0.smithi007.stdout:3/441: rename cc to d15/d18/d1f/d29/d3e/d4c/c90 0 2024-03-16T09:01:11.012 INFO:tasks.workunit.client.0.smithi007.stdout:3/442: truncate d15/d18/d1f/d29/d7c/f8c 287216 0 2024-03-16T09:01:11.017 INFO:tasks.workunit.client.1.smithi019.stdout:8/153: dwrite f5 [0,4194304] 0 2024-03-16T09:01:11.017 INFO:tasks.workunit.client.0.smithi007.stdout:7/273: dwrite d0/dc/fd [0,4194304] 0 2024-03-16T09:01:11.031 INFO:tasks.workunit.client.0.smithi007.stdout:6/279: dwrite d2/d10/d12/d15/f22 [0,4194304] 0 2024-03-16T09:01:11.031 INFO:tasks.workunit.client.0.smithi007.stdout:2/478: dwrite dc/df/d24/f80 [0,4194304] 0 2024-03-16T09:01:11.031 INFO:tasks.workunit.client.1.smithi019.stdout:3/135: dwrite d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:11.031 INFO:tasks.workunit.client.1.smithi019.stdout:0/108: dwrite d7/fd [0,4194304] 0 2024-03-16T09:01:11.032 INFO:tasks.workunit.client.0.smithi007.stdout:6/280: creat d2/d13/d1f/f5c x:0 0 0 2024-03-16T09:01:11.035 INFO:tasks.workunit.client.1.smithi019.stdout:3/136: symlink d3/l28 0 2024-03-16T09:01:11.035 INFO:tasks.workunit.client.0.smithi007.stdout:6/281: getdents d2/d10/d12 0 2024-03-16T09:01:11.036 INFO:tasks.workunit.client.1.smithi019.stdout:3/137: dread d3/ff [0,4194304] 0 2024-03-16T09:01:11.037 INFO:tasks.workunit.client.1.smithi019.stdout:0/109: getdents d7 0 2024-03-16T09:01:11.037 INFO:tasks.workunit.client.1.smithi019.stdout:0/110: chown d7/d8/c10 31358 1 2024-03-16T09:01:11.037 INFO:tasks.workunit.client.0.smithi007.stdout:6/282: symlink d2/d29/l5d 0 2024-03-16T09:01:11.037 INFO:tasks.workunit.client.0.smithi007.stdout:6/283: readlink d2/d13/d1f/d34/l5a 0 2024-03-16T09:01:11.039 INFO:tasks.workunit.client.1.smithi019.stdout:0/111: truncate d7/f19 293890 0 2024-03-16T09:01:11.041 INFO:tasks.workunit.client.1.smithi019.stdout:0/112: rename d7/d1d/d1a to d7/d8/d22 0 2024-03-16T09:01:11.055 INFO:tasks.workunit.client.0.smithi007.stdout:2/479: read dc/df/d24/d28/f39 [2169577,95791] 0 2024-03-16T09:01:11.058 INFO:tasks.workunit.client.1.smithi019.stdout:9/138: dwrite d1/d7/d21/d27/f22 [4194304,4194304] 0 2024-03-16T09:01:11.058 INFO:tasks.workunit.client.1.smithi019.stdout:1/73: dwrite fd [0,4194304] 0 2024-03-16T09:01:11.058 INFO:tasks.workunit.client.1.smithi019.stdout:1/74: dread - f10 zero size 2024-03-16T09:01:11.059 INFO:tasks.workunit.client.1.smithi019.stdout:2/128: dwrite d5/d6/d2b/f18 [0,4194304] 0 2024-03-16T09:01:11.059 INFO:tasks.workunit.client.1.smithi019.stdout:2/129: dread - d5/de/f1f zero size 2024-03-16T09:01:11.060 INFO:tasks.workunit.client.1.smithi019.stdout:9/139: unlink d1/d7/d21/c26 0 2024-03-16T09:01:11.060 INFO:tasks.workunit.client.1.smithi019.stdout:9/140: stat d1/d7/de/f10 0 2024-03-16T09:01:11.060 INFO:tasks.workunit.client.1.smithi019.stdout:9/141: fsync d1/d7/de/d24/f2c 0 2024-03-16T09:01:11.060 INFO:tasks.workunit.client.1.smithi019.stdout:9/142: dread - d1/d7/de/d24/f2c zero size 2024-03-16T09:01:11.064 INFO:tasks.workunit.client.0.smithi007.stdout:4/312: dwrite d4/d14/d1c/f2c [0,4194304] 0 2024-03-16T09:01:11.066 INFO:tasks.workunit.client.1.smithi019.stdout:2/130: mkdir d5/d2e 0 2024-03-16T09:01:11.067 INFO:tasks.workunit.client.1.smithi019.stdout:2/131: truncate f1 3263253 0 2024-03-16T09:01:11.075 INFO:tasks.workunit.client.1.smithi019.stdout:1/75: dread fa [0,4194304] 0 2024-03-16T09:01:11.075 INFO:tasks.workunit.client.1.smithi019.stdout:1/76: write f10 [848363,87861] 0 2024-03-16T09:01:11.075 INFO:tasks.workunit.client.0.smithi007.stdout:9/419: truncate d24/f29 1398664 0 2024-03-16T09:01:11.076 INFO:tasks.workunit.client.1.smithi019.stdout:2/132: dread d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:11.076 INFO:tasks.workunit.client.1.smithi019.stdout:1/77: creat f12 x:0 0 0 2024-03-16T09:01:11.076 INFO:tasks.workunit.client.1.smithi019.stdout:1/78: truncate ff 723584 0 2024-03-16T09:01:11.076 INFO:tasks.workunit.client.0.smithi007.stdout:9/420: symlink d24/d4a/l79 0 2024-03-16T09:01:11.077 INFO:tasks.workunit.client.1.smithi019.stdout:2/133: mkdir d5/d6/d2b/d2f 0 2024-03-16T09:01:11.077 INFO:tasks.workunit.client.1.smithi019.stdout:2/134: read d5/fb [643111,118329] 0 2024-03-16T09:01:11.079 INFO:tasks.workunit.client.1.smithi019.stdout:2/135: symlink d5/l30 0 2024-03-16T09:01:11.079 INFO:tasks.workunit.client.1.smithi019.stdout:2/136: fsync d5/de/fd 0 2024-03-16T09:01:11.079 INFO:tasks.workunit.client.1.smithi019.stdout:2/137: truncate d5/d6/f28 523985 0 2024-03-16T09:01:11.079 INFO:tasks.workunit.client.1.smithi019.stdout:2/138: write d5/ff [1535428,10614] 0 2024-03-16T09:01:11.083 INFO:tasks.workunit.client.1.smithi019.stdout:2/139: mkdir d5/d6/d2b/d2f/d31 0 2024-03-16T09:01:11.083 INFO:tasks.workunit.client.1.smithi019.stdout:2/140: rename d5/de/f1b to d5/de/f32 0 2024-03-16T09:01:11.083 INFO:tasks.workunit.client.1.smithi019.stdout:2/141: rename d5/d6 to d5/d6/d2b/d33 22 2024-03-16T09:01:11.083 INFO:tasks.workunit.client.0.smithi007.stdout:2/480: write dc/df/f34 [3507023,106134] 0 2024-03-16T09:01:11.083 INFO:tasks.workunit.client.0.smithi007.stdout:2/481: read - dc/df/d24/d28/d2e/d35/f98 zero size 2024-03-16T09:01:11.084 INFO:tasks.workunit.client.0.smithi007.stdout:2/482: rmdir dc/df/d24 39 2024-03-16T09:01:11.084 INFO:tasks.workunit.client.0.smithi007.stdout:2/483: creat dc/df/d24/d28/d2e/d4a/f9c x:0 0 0 2024-03-16T09:01:11.084 INFO:tasks.workunit.client.1.smithi019.stdout:8/154: dwrite d6/de/d16/f32 [0,4194304] 0 2024-03-16T09:01:11.085 INFO:tasks.workunit.client.0.smithi007.stdout:2/484: truncate dc/df/d24/d28/d2e/d4a/f7a 3827611 0 2024-03-16T09:01:11.086 INFO:tasks.workunit.client.0.smithi007.stdout:2/485: chown dc/f3a 42422 1 2024-03-16T09:01:11.086 INFO:tasks.workunit.client.0.smithi007.stdout:1/375: dwrite d9/dd/d1b/d2d/d30/d45/f4a [0,4194304] 0 2024-03-16T09:01:11.086 INFO:tasks.workunit.client.0.smithi007.stdout:1/376: fsync d9/dd/d1b/d28/d29/f2a 0 2024-03-16T09:01:11.086 INFO:tasks.workunit.client.0.smithi007.stdout:1/377: truncate d9/f12 4781178 0 2024-03-16T09:01:11.086 INFO:tasks.workunit.client.0.smithi007.stdout:8/611: dwrite d0/dc/f3a [0,4194304] 0 2024-03-16T09:01:11.086 INFO:tasks.workunit.client.0.smithi007.stdout:5/385: dwrite d1/d7/d1c/d19/d34/d3e/d41/d48/f75 [0,4194304] 0 2024-03-16T09:01:11.087 INFO:tasks.workunit.client.0.smithi007.stdout:2/486: mknod dc/df/d24/d28/c9d 0 2024-03-16T09:01:11.087 INFO:tasks.workunit.client.0.smithi007.stdout:2/487: write dc/df/d24/d28/d2e/f9b [573343,100037] 0 2024-03-16T09:01:11.087 INFO:tasks.workunit.client.1.smithi019.stdout:1/79: read fd [1063986,14213] 0 2024-03-16T09:01:11.087 INFO:tasks.workunit.client.1.smithi019.stdout:8/155: mkdir d6/de/d24/d30/d34 0 2024-03-16T09:01:11.088 INFO:tasks.workunit.client.0.smithi007.stdout:7/274: dwrite d0/f33 [0,4194304] 0 2024-03-16T09:01:11.089 INFO:tasks.workunit.client.1.smithi019.stdout:1/80: unlink fa 0 2024-03-16T09:01:11.089 INFO:tasks.workunit.client.1.smithi019.stdout:1/81: write f9 [223880,106647] 0 2024-03-16T09:01:11.090 INFO:tasks.workunit.client.0.smithi007.stdout:9/421: dread f7 [0,4194304] 0 2024-03-16T09:01:11.090 INFO:tasks.workunit.client.0.smithi007.stdout:9/422: write f7 [3087187,88017] 0 2024-03-16T09:01:11.090 INFO:tasks.workunit.client.1.smithi019.stdout:3/138: dwrite d3/ff [0,4194304] 0 2024-03-16T09:01:11.091 INFO:tasks.workunit.client.1.smithi019.stdout:8/156: rename d6/de/f1d to d6/de/d24/f35 0 2024-03-16T09:01:11.091 INFO:tasks.workunit.client.1.smithi019.stdout:0/113: dwrite f2 [0,4194304] 0 2024-03-16T09:01:11.091 INFO:tasks.workunit.client.0.smithi007.stdout:3/443: dwrite d15/d18/f19 [0,4194304] 0 2024-03-16T09:01:11.091 INFO:tasks.workunit.client.0.smithi007.stdout:6/284: dwrite d2/fd [0,4194304] 0 2024-03-16T09:01:11.092 INFO:tasks.workunit.client.0.smithi007.stdout:1/378: unlink d9/dd/ce 0 2024-03-16T09:01:11.092 INFO:tasks.workunit.client.0.smithi007.stdout:1/379: chown d9/dd/d1b/d28/f60 3737706 1 2024-03-16T09:01:11.094 INFO:tasks.workunit.client.0.smithi007.stdout:8/612: rmdir d0/dc/dd/d8b 39 2024-03-16T09:01:11.094 INFO:tasks.workunit.client.0.smithi007.stdout:8/613: chown d0/dc/c4d 11 1 2024-03-16T09:01:11.094 INFO:tasks.workunit.client.1.smithi019.stdout:3/139: symlink d3/d17/l29 0 2024-03-16T09:01:11.096 INFO:tasks.workunit.client.1.smithi019.stdout:8/157: link d6/de/d16/d19/f2e d6/d2a/f36 0 2024-03-16T09:01:11.096 INFO:tasks.workunit.client.1.smithi019.stdout:8/158: readlink d6/lc 0 2024-03-16T09:01:11.097 INFO:tasks.workunit.client.0.smithi007.stdout:5/386: creat d1/d7/d16/d1e/d2a/d81/f90 x:0 0 0 2024-03-16T09:01:11.098 INFO:tasks.workunit.client.1.smithi019.stdout:0/114: dread d7/f19 [0,4194304] 0 2024-03-16T09:01:11.098 INFO:tasks.workunit.client.1.smithi019.stdout:0/115: readlink d7/la 0 2024-03-16T09:01:11.099 INFO:tasks.workunit.client.0.smithi007.stdout:2/488: creat dc/df/d24/d28/d97/f9e x:0 0 0 2024-03-16T09:01:11.100 INFO:tasks.workunit.client.0.smithi007.stdout:4/313: write d4/d14/f1b [463279,37692] 0 2024-03-16T09:01:11.102 INFO:tasks.workunit.client.0.smithi007.stdout:1/380: dread d9/dd/d1b/d58/f6c [0,4194304] 0 2024-03-16T09:01:11.104 INFO:tasks.workunit.client.1.smithi019.stdout:3/140: mknod d3/d1b/c2a 0 2024-03-16T09:01:11.104 INFO:tasks.workunit.client.0.smithi007.stdout:7/275: mknod d0/c59 0 2024-03-16T09:01:11.104 INFO:tasks.workunit.client.0.smithi007.stdout:7/276: write d0/d19/d1f/f54 [853179,6660] 0 2024-03-16T09:01:11.106 INFO:tasks.workunit.client.1.smithi019.stdout:8/159: truncate d6/de/ff 650185 0 2024-03-16T09:01:11.107 INFO:tasks.workunit.client.0.smithi007.stdout:9/423: creat d24/d4c/f7a x:0 0 0 2024-03-16T09:01:11.108 INFO:tasks.workunit.client.1.smithi019.stdout:0/116: creat d7/d8/f23 x:0 0 0 2024-03-16T09:01:11.108 INFO:tasks.workunit.client.1.smithi019.stdout:0/117: write d7/f19 [1116257,27422] 0 2024-03-16T09:01:11.108 INFO:tasks.workunit.client.1.smithi019.stdout:3/141: rmdir d3 39 2024-03-16T09:01:11.109 INFO:tasks.workunit.client.0.smithi007.stdout:3/444: truncate d15/d18/d1f/f54 2341745 0 2024-03-16T09:01:11.109 INFO:tasks.workunit.client.1.smithi019.stdout:3/142: mknod d3/d17/c2b 0 2024-03-16T09:01:11.110 INFO:tasks.workunit.client.1.smithi019.stdout:8/160: dread d6/de/f29 [0,4194304] 0 2024-03-16T09:01:11.111 INFO:tasks.workunit.client.1.smithi019.stdout:3/143: mkdir d3/d11/d22/d24/d2c 0 2024-03-16T09:01:11.112 INFO:tasks.workunit.client.1.smithi019.stdout:3/144: creat d3/f2d x:0 0 0 2024-03-16T09:01:11.113 INFO:tasks.workunit.client.0.smithi007.stdout:2/489: dread dc/df/f14 [0,4194304] 0 2024-03-16T09:01:11.113 INFO:tasks.workunit.client.0.smithi007.stdout:2/490: fdatasync dc/d27/f8e 0 2024-03-16T09:01:11.114 INFO:tasks.workunit.client.1.smithi019.stdout:3/145: mkdir d3/d11/d22/d2e 0 2024-03-16T09:01:11.115 INFO:tasks.workunit.client.0.smithi007.stdout:2/491: dread dc/df/d6c/f86 [0,4194304] 0 2024-03-16T09:01:11.115 INFO:tasks.workunit.client.0.smithi007.stdout:2/492: write dc/df/d24/d28/d2e/f9b [675926,43146] 0 2024-03-16T09:01:11.116 INFO:tasks.workunit.client.1.smithi019.stdout:5/108: sync 2024-03-16T09:01:11.116 INFO:tasks.workunit.client.1.smithi019.stdout:4/172: sync 2024-03-16T09:01:11.116 INFO:tasks.workunit.client.1.smithi019.stdout:7/116: sync 2024-03-16T09:01:11.117 INFO:tasks.workunit.client.1.smithi019.stdout:5/109: getdents d6 0 2024-03-16T09:01:11.118 INFO:tasks.workunit.client.1.smithi019.stdout:8/161: read d6/de/d16/d19/f18 [2698971,57007] 0 2024-03-16T09:01:11.118 INFO:tasks.workunit.client.1.smithi019.stdout:3/146: link d3/d11/f14 d3/f2f 0 2024-03-16T09:01:11.119 INFO:tasks.workunit.client.1.smithi019.stdout:4/173: getdents d4/d1b 0 2024-03-16T09:01:11.120 INFO:tasks.workunit.client.0.smithi007.stdout:8/614: creat d0/dc/d50/d8e/db8/fd6 x:0 0 0 2024-03-16T09:01:11.122 INFO:tasks.workunit.client.1.smithi019.stdout:4/174: truncate f2 2951262 0 2024-03-16T09:01:11.122 INFO:tasks.workunit.client.0.smithi007.stdout:5/387: mknod d1/d7/d1c/d19/d34/c91 0 2024-03-16T09:01:11.122 INFO:tasks.workunit.client.0.smithi007.stdout:5/388: dread - d1/d7/d1c/d19/d34/f8c zero size 2024-03-16T09:01:11.124 INFO:tasks.workunit.client.1.smithi019.stdout:0/118: write d7/d8/f12 [4072668,81847] 0 2024-03-16T09:01:11.124 INFO:tasks.workunit.client.1.smithi019.stdout:4/175: mkdir d4/d1b/d2a/d3d 0 2024-03-16T09:01:11.124 INFO:tasks.workunit.client.1.smithi019.stdout:1/82: dwrite f12 [0,4194304] 0 2024-03-16T09:01:11.125 INFO:tasks.workunit.client.1.smithi019.stdout:4/176: stat d4/d6/d15/l1a 0 2024-03-16T09:01:11.125 INFO:tasks.workunit.client.1.smithi019.stdout:3/147: dread d3/f2f [0,4194304] 0 2024-03-16T09:01:11.128 INFO:tasks.workunit.client.0.smithi007.stdout:6/285: dwrite d2/d13/d1f/d34/f4a [0,4194304] 0 2024-03-16T09:01:11.128 INFO:tasks.workunit.client.1.smithi019.stdout:5/110: dread f1 [0,4194304] 0 2024-03-16T09:01:11.129 INFO:tasks.workunit.client.1.smithi019.stdout:3/148: write d3/d1b/f1c [735130,120924] 0 2024-03-16T09:01:11.129 INFO:tasks.workunit.client.0.smithi007.stdout:1/381: symlink d9/d6b/l74 0 2024-03-16T09:01:11.130 INFO:tasks.workunit.client.1.smithi019.stdout:4/177: truncate d4/d6/d15/f2e 358657 0 2024-03-16T09:01:11.131 INFO:tasks.workunit.client.0.smithi007.stdout:7/277: mkdir d0/dc/d5a 0 2024-03-16T09:01:11.131 INFO:tasks.workunit.client.1.smithi019.stdout:6/282: sync 2024-03-16T09:01:11.131 INFO:tasks.workunit.client.1.smithi019.stdout:6/283: chown de/d1c/d1d 58 1 2024-03-16T09:01:11.132 INFO:tasks.workunit.client.1.smithi019.stdout:8/162: write d6/de/d16/f32 [2996815,114310] 0 2024-03-16T09:01:11.132 INFO:tasks.workunit.client.1.smithi019.stdout:5/111: creat d6/dd/f1d x:0 0 0 2024-03-16T09:01:11.133 INFO:tasks.workunit.client.1.smithi019.stdout:1/83: symlink l13 0 2024-03-16T09:01:11.134 INFO:tasks.workunit.client.0.smithi007.stdout:9/424: truncate d24/f2a 178051 0 2024-03-16T09:01:11.134 INFO:tasks.workunit.client.0.smithi007.stdout:9/425: truncate d24/d4c/f7a 636658 0 2024-03-16T09:01:11.134 INFO:tasks.workunit.client.0.smithi007.stdout:9/426: truncate f17 2844308 0 2024-03-16T09:01:11.135 INFO:tasks.workunit.client.1.smithi019.stdout:3/149: rename d3/d11/f26 to d3/f30 0 2024-03-16T09:01:11.136 INFO:tasks.workunit.client.0.smithi007.stdout:3/445: symlink d15/d18/d1f/l91 0 2024-03-16T09:01:11.140 INFO:tasks.workunit.client.1.smithi019.stdout:8/163: creat d6/de/f37 x:0 0 0 2024-03-16T09:01:11.140 INFO:tasks.workunit.client.0.smithi007.stdout:5/389: dwrite d1/d7/f12 [4194304,4194304] 0 2024-03-16T09:01:11.140 INFO:tasks.workunit.client.0.smithi007.stdout:5/390: truncate d1/d7/d16/d1e/f6e 1149707 0 2024-03-16T09:01:11.140 INFO:tasks.workunit.client.0.smithi007.stdout:5/391: creat d1/d7/d1c/d19/d34/d5e/d6f/d72/f92 x:0 0 0 2024-03-16T09:01:11.141 INFO:tasks.workunit.client.0.smithi007.stdout:8/615: dread d0/d26/f5b [0,4194304] 0 2024-03-16T09:01:11.141 INFO:tasks.workunit.client.1.smithi019.stdout:2/142: dwrite d5/ff [0,4194304] 0 2024-03-16T09:01:11.141 INFO:tasks.workunit.client.1.smithi019.stdout:2/143: chown d5/ca 19268 1 2024-03-16T09:01:11.144 INFO:tasks.workunit.client.0.smithi007.stdout:7/278: unlink d0/dc/d24/f46 0 2024-03-16T09:01:11.145 INFO:tasks.workunit.client.1.smithi019.stdout:4/178: read d4/f12 [127840,10614] 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/179: stat d4/d6/d15/c2f 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/180: creat d4/d6/d15/d1f/f3e x:0 0 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/181: write d4/d6/d15/d1f/f38 [438354,5140] 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/182: fdatasync d4/d1b/f28 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/183: creat d4/d6/d15/d1f/d2c/f3f x:0 0 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/184: creat d4/d6/d15/d1f/d2c/f40 x:0 0 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/185: creat d4/d6/d15/d1f/d2c/f41 x:0 0 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/186: truncate d4/d6/d15/d1f/f3e 210885 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/187: write d4/d6/d15/d1f/d2c/f3f [235800,9719] 0 2024-03-16T09:01:11.146 INFO:tasks.workunit.client.1.smithi019.stdout:4/188: truncate d4/d6/d15/d1f/d2c/f31 1469384 0 2024-03-16T09:01:11.147 INFO:tasks.workunit.client.1.smithi019.stdout:6/284: getdents de 0 2024-03-16T09:01:11.149 INFO:tasks.workunit.client.1.smithi019.stdout:7/117: dwrite d9/db/f1d [0,4194304] 0 2024-03-16T09:01:11.149 INFO:tasks.workunit.client.1.smithi019.stdout:7/118: read d9/da/d13/f17 [1691469,123168] 0 2024-03-16T09:01:11.150 INFO:tasks.workunit.client.1.smithi019.stdout:7/119: write d9/f16 [357896,63507] 0 2024-03-16T09:01:11.152 INFO:tasks.workunit.client.0.smithi007.stdout:9/427: link f17 d24/d4a/d61/f7b 0 2024-03-16T09:01:11.153 INFO:tasks.workunit.client.1.smithi019.stdout:0/119: dread d7/d8/f12 [0,4194304] 0 2024-03-16T09:01:11.154 INFO:tasks.workunit.client.1.smithi019.stdout:2/144: dread d5/de/f32 [0,4194304] 0 2024-03-16T09:01:11.154 INFO:tasks.workunit.client.0.smithi007.stdout:2/493: dwrite dc/df/d24/d28/d2e/d35/f5f [0,4194304] 0 2024-03-16T09:01:11.158 INFO:tasks.workunit.client.0.smithi007.stdout:4/314: rename d4/d14/d34 to d4/d14/d64/d68 0 2024-03-16T09:01:11.159 INFO:tasks.workunit.client.0.smithi007.stdout:5/392: write d1/d7/f12 [7051459,14711] 0 2024-03-16T09:01:11.161 INFO:tasks.workunit.client.0.smithi007.stdout:1/382: getdents d9/dd/d1b/d58 0 2024-03-16T09:01:11.167 INFO:tasks.workunit.client.0.smithi007.stdout:3/446: dwrite d15/f2c [0,4194304] 0 2024-03-16T09:01:11.169 INFO:tasks.workunit.client.1.smithi019.stdout:5/112: dwrite f4 [4194304,4194304] 0 2024-03-16T09:01:11.170 INFO:tasks.workunit.client.1.smithi019.stdout:5/113: dread - d6/d19/f1b zero size 2024-03-16T09:01:11.172 INFO:tasks.workunit.client.1.smithi019.stdout:6/285: stat ca 0 2024-03-16T09:01:11.172 INFO:tasks.workunit.client.1.smithi019.stdout:2/145: write d5/f7 [2825801,83019] 0 2024-03-16T09:01:11.172 INFO:tasks.workunit.client.1.smithi019.stdout:2/146: read f1 [2591049,62013] 0 2024-03-16T09:01:11.172 INFO:tasks.workunit.client.1.smithi019.stdout:2/147: chown d5/ca 3431 1 2024-03-16T09:01:11.173 INFO:tasks.workunit.client.0.smithi007.stdout:6/286: rename d2/d10/d12/d15/f22 to d2/d13/d1f/d34/f5e 0 2024-03-16T09:01:11.173 INFO:tasks.workunit.client.0.smithi007.stdout:6/287: chown d2/d11/d3b 94 1 2024-03-16T09:01:11.173 INFO:tasks.workunit.client.0.smithi007.stdout:4/315: rename d4/d14/d1c to d4/d14/d1c/d69 22 2024-03-16T09:01:11.174 INFO:tasks.workunit.client.0.smithi007.stdout:2/494: write dc/df/f18 [3527377,107521] 0 2024-03-16T09:01:11.175 INFO:tasks.workunit.client.0.smithi007.stdout:0/351: sync 2024-03-16T09:01:11.175 INFO:tasks.workunit.client.0.smithi007.stdout:0/352: chown d3/d9/db/fc 1 1 2024-03-16T09:01:11.175 INFO:tasks.workunit.client.0.smithi007.stdout:0/353: write d3/d9/da/d48/d4b/f52 [130335,105836] 0 2024-03-16T09:01:11.175 INFO:tasks.workunit.client.0.smithi007.stdout:0/354: creat d3/d9/da/d18/d23/f78 x:0 0 0 2024-03-16T09:01:11.175 INFO:tasks.workunit.client.0.smithi007.stdout:0/355: dread - d3/d9/da/d18/d23/f75 zero size 2024-03-16T09:01:11.176 INFO:tasks.workunit.client.0.smithi007.stdout:5/393: mknod d1/d7/d1c/d19/d34/d1d/c93 0 2024-03-16T09:01:11.176 INFO:tasks.workunit.client.0.smithi007.stdout:5/394: creat d1/d7/d16/d1e/f94 x:0 0 0 2024-03-16T09:01:11.176 INFO:tasks.workunit.client.0.smithi007.stdout:5/395: write d1/d7/d1c/d19/d34/d4d/f77 [209365,14576] 0 2024-03-16T09:01:11.179 INFO:tasks.workunit.client.1.smithi019.stdout:6/286: truncate de/f49 529324 0 2024-03-16T09:01:11.180 INFO:tasks.workunit.client.1.smithi019.stdout:6/287: chown de 16093215 1 2024-03-16T09:01:11.182 INFO:tasks.workunit.client.1.smithi019.stdout:3/150: getdents d3/d11 0 2024-03-16T09:01:11.187 INFO:tasks.workunit.client.0.smithi007.stdout:6/288: truncate d2/d10/d12/f35 16965 0 2024-03-16T09:01:11.187 INFO:tasks.workunit.client.0.smithi007.stdout:6/289: fdatasync d2/d10/f36 0 2024-03-16T09:01:11.188 INFO:tasks.workunit.client.1.smithi019.stdout:3/151: rename d3/d11/l20 to d3/d11/d22/l31 0 2024-03-16T09:01:11.190 INFO:tasks.workunit.client.0.smithi007.stdout:1/383: rename d9/lb to d9/dd/d1b/d28/l75 0 2024-03-16T09:01:11.190 INFO:tasks.workunit.client.0.smithi007.stdout:1/384: dread - d9/dd/f67 zero size 2024-03-16T09:01:11.190 INFO:tasks.workunit.client.0.smithi007.stdout:3/447: dread d15/f1e [0,4194304] 0 2024-03-16T09:01:11.191 INFO:tasks.workunit.client.0.smithi007.stdout:3/448: dread - d15/d18/f4f zero size 2024-03-16T09:01:11.194 INFO:tasks.workunit.client.0.smithi007.stdout:2/495: symlink dc/df/l9f 0 2024-03-16T09:01:11.195 INFO:tasks.workunit.client.0.smithi007.stdout:2/496: chown dc/df/d24/d28/d2e/d35/f91 7 1 2024-03-16T09:01:11.195 INFO:tasks.workunit.client.0.smithi007.stdout:4/316: creat d4/d14/d1c/f6a x:0 0 0 2024-03-16T09:01:11.195 INFO:tasks.workunit.client.0.smithi007.stdout:4/317: chown d4/l3c 4 1 2024-03-16T09:01:11.198 INFO:tasks.workunit.client.0.smithi007.stdout:0/356: unlink d3/f4 0 2024-03-16T09:01:11.200 INFO:tasks.workunit.client.0.smithi007.stdout:5/396: creat d1/d7/d1c/d88/f95 x:0 0 0 2024-03-16T09:01:11.204 INFO:tasks.workunit.client.0.smithi007.stdout:6/290: rename d2/c8 to d2/d13/d3f/d50/c5f 0 2024-03-16T09:01:11.204 INFO:tasks.workunit.client.0.smithi007.stdout:2/497: mknod dc/df/d24/d28/d2e/d35/ca0 0 2024-03-16T09:01:11.204 INFO:tasks.workunit.client.0.smithi007.stdout:2/498: write f7 [1302089,45151] 0 2024-03-16T09:01:11.207 INFO:tasks.workunit.client.0.smithi007.stdout:0/357: rename d3/d9/da/d18/d27/d34/d3a/l5f to d3/d9/da/d18/d23/l79 0 2024-03-16T09:01:11.208 INFO:tasks.workunit.client.0.smithi007.stdout:6/291: creat d2/d10/d12/f60 x:0 0 0 2024-03-16T09:01:11.211 INFO:tasks.workunit.client.0.smithi007.stdout:2/499: unlink dc/df/d24/d28/l3c 0 2024-03-16T09:01:11.212 INFO:tasks.workunit.client.0.smithi007.stdout:0/358: creat d3/d9/da/d48/d25/d43/d1a/d58/d61/f7a x:0 0 0 2024-03-16T09:01:11.215 INFO:tasks.workunit.client.0.smithi007.stdout:2/500: getdents dc/df/d24/d28/d2e/d6e 0 2024-03-16T09:01:11.227 INFO:tasks.workunit.client.0.smithi007.stdout:8/616: dwrite d0/d26/d2d/d43/d6f/da5/fc8 [0,4194304] 0 2024-03-16T09:01:11.229 INFO:tasks.workunit.client.1.smithi019.stdout:8/164: dwrite d6/d2a/f36 [0,4194304] 0 2024-03-16T09:01:11.230 INFO:tasks.workunit.client.1.smithi019.stdout:0/120: dwrite d7/f15 [0,4194304] 0 2024-03-16T09:01:11.230 INFO:tasks.workunit.client.0.smithi007.stdout:8/617: creat d0/d26/d2d/d43/d4c/dba/fd7 x:0 0 0 2024-03-16T09:01:11.232 INFO:tasks.workunit.client.0.smithi007.stdout:8/618: truncate d0/dc/f89 997527 0 2024-03-16T09:01:11.232 INFO:tasks.workunit.client.0.smithi007.stdout:8/619: creat d0/dc/d54/fd8 x:0 0 0 2024-03-16T09:01:11.234 INFO:tasks.workunit.client.1.smithi019.stdout:0/121: symlink d7/l24 0 2024-03-16T09:01:11.235 INFO:tasks.workunit.client.0.smithi007.stdout:8/620: creat d0/d26/d2d/d43/d72/dad/fd9 x:0 0 0 2024-03-16T09:01:11.235 INFO:tasks.workunit.client.1.smithi019.stdout:0/122: creat d7/d8/d22/f25 x:0 0 0 2024-03-16T09:01:11.235 INFO:tasks.workunit.client.1.smithi019.stdout:0/123: truncate d7/d8/f23 357926 0 2024-03-16T09:01:11.236 INFO:tasks.workunit.client.0.smithi007.stdout:8/621: creat d0/dc/dd/fda x:0 0 0 2024-03-16T09:01:11.243 INFO:tasks.workunit.client.1.smithi019.stdout:0/124: write f2 [3095418,62716] 0 2024-03-16T09:01:11.252 INFO:tasks.workunit.client.1.smithi019.stdout:8/165: dread f2 [0,4194304] 0 2024-03-16T09:01:11.254 INFO:tasks.workunit.client.0.smithi007.stdout:8/622: read d0/dc/dd/d8b/fd0 [2907359,50248] 0 2024-03-16T09:01:11.258 INFO:tasks.workunit.client.1.smithi019.stdout:1/84: dwrite f9 [0,4194304] 0 2024-03-16T09:01:11.258 INFO:tasks.workunit.client.1.smithi019.stdout:1/85: write f10 [1921921,13771] 0 2024-03-16T09:01:11.258 INFO:tasks.workunit.client.1.smithi019.stdout:2/148: dwrite d5/f23 [0,4194304] 0 2024-03-16T09:01:11.260 INFO:tasks.workunit.client.0.smithi007.stdout:3/449: dwrite d15/d18/d6a/f7d [0,4194304] 0 2024-03-16T09:01:11.260 INFO:tasks.workunit.client.0.smithi007.stdout:3/450: truncate d15/d18/d1f/f8d 1285325 0 2024-03-16T09:01:11.260 INFO:tasks.workunit.client.0.smithi007.stdout:7/279: dwrite d0/dc/f3d [0,4194304] 0 2024-03-16T09:01:11.261 INFO:tasks.workunit.client.0.smithi007.stdout:4/318: dwrite d4/f51 [4194304,4194304] 0 2024-03-16T09:01:11.261 INFO:tasks.workunit.client.0.smithi007.stdout:3/451: dread d15/d18/d1f/d29/d7c/f8c [0,4194304] 0 2024-03-16T09:01:11.262 INFO:tasks.workunit.client.1.smithi019.stdout:0/125: write d7/d8/fc [3605895,36831] 0 2024-03-16T09:01:11.263 INFO:tasks.workunit.client.1.smithi019.stdout:5/114: dwrite f1 [4194304,4194304] 0 2024-03-16T09:01:11.264 INFO:tasks.workunit.client.0.smithi007.stdout:8/623: dread d0/d26/fa1 [0,4194304] 0 2024-03-16T09:01:11.264 INFO:tasks.workunit.client.1.smithi019.stdout:4/189: rmdir d4/d6/d15/d1f/d2c 39 2024-03-16T09:01:11.265 INFO:tasks.workunit.client.1.smithi019.stdout:4/190: chown d4/d6/l3c 18 1 2024-03-16T09:01:11.265 INFO:tasks.workunit.client.1.smithi019.stdout:5/115: mknod d6/dd/c1e 0 2024-03-16T09:01:11.267 INFO:tasks.workunit.client.1.smithi019.stdout:4/191: mknod d4/c42 0 2024-03-16T09:01:11.268 INFO:tasks.workunit.client.0.smithi007.stdout:7/280: dread d0/fa [0,4194304] 0 2024-03-16T09:01:11.268 INFO:tasks.workunit.client.0.smithi007.stdout:0/359: dwrite d3/d9/da/d18/d23/f63 [0,4194304] 0 2024-03-16T09:01:11.269 INFO:tasks.workunit.client.1.smithi019.stdout:4/192: getdents d4/d1b/d2a 0 2024-03-16T09:01:11.270 INFO:tasks.workunit.client.1.smithi019.stdout:4/193: mknod d4/d6/c43 0 2024-03-16T09:01:11.272 INFO:tasks.workunit.client.1.smithi019.stdout:2/149: write d5/d6/f1d [2747182,30201] 0 2024-03-16T09:01:11.273 INFO:tasks.workunit.client.0.smithi007.stdout:5/397: dwrite d1/fa [4194304,4194304] 0 2024-03-16T09:01:11.273 INFO:tasks.workunit.client.0.smithi007.stdout:6/292: dwrite d2/d10/f3a [0,4194304] 0 2024-03-16T09:01:11.273 INFO:tasks.workunit.client.0.smithi007.stdout:6/293: readlink d2/d13/l3c 0 2024-03-16T09:01:11.273 INFO:tasks.workunit.client.0.smithi007.stdout:4/319: mkdir d4/d7/d6b 0 2024-03-16T09:01:11.275 INFO:tasks.workunit.client.0.smithi007.stdout:8/624: symlink d0/dc/d80/ldb 0 2024-03-16T09:01:11.276 INFO:tasks.workunit.client.0.smithi007.stdout:7/281: mknod d0/dc/d24/d25/d3a/c5b 0 2024-03-16T09:01:11.276 INFO:tasks.workunit.client.0.smithi007.stdout:7/282: write d0/d19/d1f/d42/f57 [18929,88397] 0 2024-03-16T09:01:11.277 INFO:tasks.workunit.client.1.smithi019.stdout:8/166: dwrite d6/de/d16/d19/f1a [4194304,4194304] 0 2024-03-16T09:01:11.277 INFO:tasks.workunit.client.1.smithi019.stdout:8/167: stat d6/c2d 0 2024-03-16T09:01:11.278 INFO:tasks.workunit.client.1.smithi019.stdout:5/116: write d6/f17 [2046575,66673] 0 2024-03-16T09:01:11.278 INFO:tasks.workunit.client.1.smithi019.stdout:3/152: dwrite d3/d1b/f1c [0,4194304] 0 2024-03-16T09:01:11.282 INFO:tasks.workunit.client.0.smithi007.stdout:0/360: write d3/d9/db/d2c/f2f [3777480,12059] 0 2024-03-16T09:01:11.282 INFO:tasks.workunit.client.0.smithi007.stdout:0/361: fdatasync d3/d9/da/d48/d25/d43/d1a/d58/d61/f7a 0 2024-03-16T09:01:11.282 INFO:tasks.workunit.client.0.smithi007.stdout:0/362: chown d3/d9 9722 1 2024-03-16T09:01:11.286 INFO:tasks.workunit.client.1.smithi019.stdout:4/194: dread d4/d6/f39 [4194304,4194304] 0 2024-03-16T09:01:11.287 INFO:tasks.workunit.client.0.smithi007.stdout:5/398: mknod d1/d7/d1c/d19/d34/d4d/c96 0 2024-03-16T09:01:11.287 INFO:tasks.workunit.client.1.smithi019.stdout:0/126: dwrite d7/fd [4194304,4194304] 0 2024-03-16T09:01:11.289 INFO:tasks.workunit.client.0.smithi007.stdout:6/294: symlink d2/d11/d1e/l61 0 2024-03-16T09:01:11.290 INFO:tasks.workunit.client.1.smithi019.stdout:3/153: creat d3/d17/f32 x:0 0 0 2024-03-16T09:01:11.290 INFO:tasks.workunit.client.1.smithi019.stdout:8/168: write d6/de/d16/d19/f1a [3382214,125205] 0 2024-03-16T09:01:11.291 INFO:tasks.workunit.client.0.smithi007.stdout:8/625: link d0/dc/d75/fb1 d0/d26/d2d/fdc 0 2024-03-16T09:01:11.292 INFO:tasks.workunit.client.1.smithi019.stdout:4/195: rename d4/d6/d15/d1f/f38 to d4/d6/d15/d1f/f44 0 2024-03-16T09:01:11.295 INFO:tasks.workunit.client.0.smithi007.stdout:7/283: link d0/d19/f41 d0/dc/d24/d43/f5c 0 2024-03-16T09:01:11.295 INFO:tasks.workunit.client.0.smithi007.stdout:7/284: chown d0/dc/d38/d4e 95 1 2024-03-16T09:01:11.295 INFO:tasks.workunit.client.1.smithi019.stdout:0/127: symlink d7/d8/d22/l26 0 2024-03-16T09:01:11.297 INFO:tasks.workunit.client.1.smithi019.stdout:8/169: symlink d6/de/l38 0 2024-03-16T09:01:11.298 INFO:tasks.workunit.client.1.smithi019.stdout:8/170: creat d6/de/d16/d19/d28/f39 x:0 0 0 2024-03-16T09:01:11.298 INFO:tasks.workunit.client.1.smithi019.stdout:4/196: dread d4/d6/d15/d1f/f3e [0,4194304] 0 2024-03-16T09:01:11.299 INFO:tasks.workunit.client.1.smithi019.stdout:4/197: readlink d4/d6/l9 0 2024-03-16T09:01:11.300 INFO:tasks.workunit.client.1.smithi019.stdout:6/288: dwrite de/f49 [0,4194304] 0 2024-03-16T09:01:11.300 INFO:tasks.workunit.client.1.smithi019.stdout:6/289: write de/d26/f2a [997672,20559] 0 2024-03-16T09:01:11.300 INFO:tasks.workunit.client.1.smithi019.stdout:8/171: mknod d6/de/d16/c3a 0 2024-03-16T09:01:11.302 INFO:tasks.workunit.client.1.smithi019.stdout:4/198: symlink d4/d1b/l45 0 2024-03-16T09:01:11.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/399: unlink d1/d7/d1c/d19/d34/d5e/d6f/f73 0 2024-03-16T09:01:11.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/400: write d1/d7/d1c/d19/d34/d5e/d6f/d72/f92 [1019674,92802] 0 2024-03-16T09:01:11.305 INFO:tasks.workunit.client.1.smithi019.stdout:6/290: creat de/f67 x:0 0 0 2024-03-16T09:01:11.306 INFO:tasks.workunit.client.1.smithi019.stdout:8/172: creat d6/de/d24/d30/f3b x:0 0 0 2024-03-16T09:01:11.306 INFO:tasks.workunit.client.1.smithi019.stdout:8/173: readlink d6/l20 0 2024-03-16T09:01:11.307 INFO:tasks.workunit.client.0.smithi007.stdout:6/295: mkdir d2/d13/d62 0 2024-03-16T09:01:11.309 INFO:tasks.workunit.client.1.smithi019.stdout:4/199: unlink d4/f1e 0 2024-03-16T09:01:11.309 INFO:tasks.workunit.client.1.smithi019.stdout:4/200: truncate d4/d6/d15/d1f/f30 521893 0 2024-03-16T09:01:11.309 INFO:tasks.workunit.client.1.smithi019.stdout:4/201: chown d4/d1b/f1c 2584313 1 2024-03-16T09:01:11.309 INFO:tasks.workunit.client.0.smithi007.stdout:0/363: getdents d3/d9/da/d18/d27/d34 0 2024-03-16T09:01:11.311 INFO:tasks.workunit.client.0.smithi007.stdout:6/296: rmdir d2/d10 39 2024-03-16T09:01:11.312 INFO:tasks.workunit.client.1.smithi019.stdout:6/291: mknod de/d26/c68 0 2024-03-16T09:01:11.312 INFO:tasks.workunit.client.1.smithi019.stdout:6/292: dread - de/d26/d32/d4e/f59 zero size 2024-03-16T09:01:11.313 INFO:tasks.workunit.client.0.smithi007.stdout:8/626: symlink d0/d26/ldd 0 2024-03-16T09:01:11.314 INFO:tasks.workunit.client.1.smithi019.stdout:8/174: rmdir d6/de/d24/d30/d34 0 2024-03-16T09:01:11.315 INFO:tasks.workunit.client.1.smithi019.stdout:4/202: rmdir d4/d1b/d2a 39 2024-03-16T09:01:11.315 INFO:tasks.workunit.client.1.smithi019.stdout:4/203: write d4/d6/d15/d1f/f30 [762423,60901] 0 2024-03-16T09:01:11.315 INFO:tasks.workunit.client.0.smithi007.stdout:0/364: truncate d3/d9/da/d48/d25/d43/d1a/f30 3162832 0 2024-03-16T09:01:11.316 INFO:tasks.workunit.client.1.smithi019.stdout:6/293: symlink de/l69 0 2024-03-16T09:01:11.317 INFO:tasks.workunit.client.1.smithi019.stdout:8/175: mkdir d6/d2a/d3c 0 2024-03-16T09:01:11.318 INFO:tasks.workunit.client.0.smithi007.stdout:6/297: rename d2/d13/d1f/d34/f5e to d2/d11/d1e/d54/f63 0 2024-03-16T09:01:11.318 INFO:tasks.workunit.client.0.smithi007.stdout:6/298: creat d2/d13/d1f/d4e/f64 x:0 0 0 2024-03-16T09:01:11.320 INFO:tasks.workunit.client.0.smithi007.stdout:8/627: unlink d0/dc/d54/c5a 0 2024-03-16T09:01:11.320 INFO:tasks.workunit.client.1.smithi019.stdout:4/204: rename d4/d6/d15/d1f/f3e to d4/d1b/d2a/d3d/f46 0 2024-03-16T09:01:11.321 INFO:tasks.workunit.client.1.smithi019.stdout:8/176: mkdir d6/de/d16/d19/d28/d3d 0 2024-03-16T09:01:11.323 INFO:tasks.workunit.client.0.smithi007.stdout:8/628: read d0/d26/d2d/fdc [180203,22508] 0 2024-03-16T09:01:11.323 INFO:tasks.workunit.client.1.smithi019.stdout:4/205: mknod d4/d6/c47 0 2024-03-16T09:01:11.324 INFO:tasks.workunit.client.0.smithi007.stdout:6/299: unlink d2/d11/d1e/f40 0 2024-03-16T09:01:11.324 INFO:tasks.workunit.client.1.smithi019.stdout:8/177: symlink d6/de/d16/d2c/l3e 0 2024-03-16T09:01:11.325 INFO:tasks.workunit.client.1.smithi019.stdout:8/178: write d6/de/f29 [369144,42258] 0 2024-03-16T09:01:11.326 INFO:tasks.workunit.client.0.smithi007.stdout:8/629: creat d0/dc/dd/d8b/d9e/fde x:0 0 0 2024-03-16T09:01:11.326 INFO:tasks.workunit.client.0.smithi007.stdout:8/630: stat d0/dc/dd/d8b/ld1 0 2024-03-16T09:01:11.326 INFO:tasks.workunit.client.0.smithi007.stdout:8/631: chown d0/d2/d7e 15593189 1 2024-03-16T09:01:11.326 INFO:tasks.workunit.client.0.smithi007.stdout:8/632: rename d0/d26/d2d/d43/d70 to d0/d26/d2d/d43/d70/ddf 22 2024-03-16T09:01:11.326 INFO:tasks.workunit.client.0.smithi007.stdout:8/633: dread - d0/d26/d2d/d43/d4c/dba/fd5 zero size 2024-03-16T09:01:11.328 INFO:tasks.workunit.client.0.smithi007.stdout:6/300: symlink d2/d13/d1f/l65 0 2024-03-16T09:01:11.339 INFO:tasks.workunit.client.1.smithi019.stdout:6/294: dread de/d1c/d1d/d2c/f3a [0,4194304] 0 2024-03-16T09:01:11.341 INFO:tasks.workunit.client.1.smithi019.stdout:9/143: sync 2024-03-16T09:01:11.343 INFO:tasks.workunit.client.1.smithi019.stdout:9/144: truncate d1/d7/de/f10 2251275 0 2024-03-16T09:01:11.351 INFO:tasks.workunit.client.1.smithi019.stdout:3/154: dwrite d3/f2f [4194304,4194304] 0 2024-03-16T09:01:11.353 INFO:tasks.workunit.client.1.smithi019.stdout:5/117: dwrite f2 [4194304,4194304] 0 2024-03-16T09:01:11.353 INFO:tasks.workunit.client.1.smithi019.stdout:5/118: fsync d6/f17 0 2024-03-16T09:01:11.353 INFO:tasks.workunit.client.1.smithi019.stdout:5/119: getdents d6 0 2024-03-16T09:01:11.353 INFO:tasks.workunit.client.1.smithi019.stdout:1/86: dwrite f9 [4194304,4194304] 0 2024-03-16T09:01:11.355 INFO:tasks.workunit.client.1.smithi019.stdout:5/120: mkdir d6/d1f 0 2024-03-16T09:01:11.355 INFO:tasks.workunit.client.1.smithi019.stdout:5/121: write d6/f8 [4716438,26333] 0 2024-03-16T09:01:11.357 INFO:tasks.workunit.client.1.smithi019.stdout:1/87: creat f14 x:0 0 0 2024-03-16T09:01:11.358 INFO:tasks.workunit.client.1.smithi019.stdout:8/179: dread d6/de/d16/f26 [0,4194304] 0 2024-03-16T09:01:11.358 INFO:tasks.workunit.client.1.smithi019.stdout:8/180: chown d6/de/d16/d19/f1c 6926804 1 2024-03-16T09:01:11.358 INFO:tasks.workunit.client.1.smithi019.stdout:5/122: creat d6/d1f/f20 x:0 0 0 2024-03-16T09:01:11.363 INFO:tasks.workunit.client.1.smithi019.stdout:7/120: sync 2024-03-16T09:01:11.363 INFO:tasks.workunit.client.1.smithi019.stdout:7/121: dread - d9/db/d12/f20 zero size 2024-03-16T09:01:11.365 INFO:tasks.workunit.client.1.smithi019.stdout:7/122: creat d9/da/f24 x:0 0 0 2024-03-16T09:01:11.365 INFO:tasks.workunit.client.1.smithi019.stdout:7/123: write d9/da/d13/f17 [2872179,61144] 0 2024-03-16T09:01:11.374 INFO:tasks.workunit.client.1.smithi019.stdout:0/128: dwrite d7/f15 [0,4194304] 0 2024-03-16T09:01:11.374 INFO:tasks.workunit.client.1.smithi019.stdout:0/129: truncate d7/d8/fc 4668202 0 2024-03-16T09:01:11.374 INFO:tasks.workunit.client.1.smithi019.stdout:0/130: write d7/d8/fc [2097528,48062] 0 2024-03-16T09:01:11.375 INFO:tasks.workunit.client.0.smithi007.stdout:6/301: write d2/d11/d1e/d54/f63 [3992884,42421] 0 2024-03-16T09:01:11.375 INFO:tasks.workunit.client.0.smithi007.stdout:7/285: dwrite d0/d19/d1f/f54 [0,4194304] 0 2024-03-16T09:01:11.391 INFO:tasks.workunit.client.1.smithi019.stdout:0/131: dread f4 [4194304,4194304] 0 2024-03-16T09:01:11.392 INFO:tasks.workunit.client.1.smithi019.stdout:0/132: rename f2 to d7/d8/d22/f27 0 2024-03-16T09:01:11.392 INFO:tasks.workunit.client.0.smithi007.stdout:5/401: dwrite d1/d7/d1c/d19/d34/d3e/d41/f7a [0,4194304] 0 2024-03-16T09:01:11.395 INFO:tasks.workunit.client.0.smithi007.stdout:5/402: symlink d1/d7/d1c/l97 0 2024-03-16T09:01:11.395 INFO:tasks.workunit.client.0.smithi007.stdout:5/403: mknod d1/d7/d1c/d19/d34/d3e/d41/d48/c98 0 2024-03-16T09:01:11.396 INFO:tasks.workunit.client.0.smithi007.stdout:0/365: dwrite d3/d9/da/d48/d25/d43/d1a/f1b [0,4194304] 0 2024-03-16T09:01:11.398 INFO:tasks.workunit.client.0.smithi007.stdout:5/404: dread d1/d7/d1c/d19/d34/d5e/d6f/d72/f92 [0,4194304] 0 2024-03-16T09:01:11.398 INFO:tasks.workunit.client.0.smithi007.stdout:5/405: stat d1/d7/d1c/d19/d34/de/c52 0 2024-03-16T09:01:11.398 INFO:tasks.workunit.client.0.smithi007.stdout:0/366: mkdir d3/d1f/d70/d7b 0 2024-03-16T09:01:11.398 INFO:tasks.workunit.client.0.smithi007.stdout:0/367: stat d3/d9/da/d48 0 2024-03-16T09:01:11.398 INFO:tasks.workunit.client.1.smithi019.stdout:0/133: dread d7/d8/d22/f27 [0,4194304] 0 2024-03-16T09:01:11.398 INFO:tasks.workunit.client.1.smithi019.stdout:0/134: readlink d7/d14/l1f 0 2024-03-16T09:01:11.399 INFO:tasks.workunit.client.0.smithi007.stdout:0/368: mkdir d3/d1f/d70/d7c 0 2024-03-16T09:01:11.401 INFO:tasks.workunit.client.1.smithi019.stdout:0/135: rename d7/d14 to d7/d1d/d28 0 2024-03-16T09:01:11.402 INFO:tasks.workunit.client.0.smithi007.stdout:8/634: dwrite d0/d2/d74/fd3 [0,4194304] 0 2024-03-16T09:01:11.402 INFO:tasks.workunit.client.0.smithi007.stdout:8/635: fsync d0/dc/dd/d8b/d9e/fa0 0 2024-03-16T09:01:11.402 INFO:tasks.workunit.client.0.smithi007.stdout:8/636: truncate d0/dc/d50/d8e/db8/fd6 142477 0 2024-03-16T09:01:11.402 INFO:tasks.workunit.client.0.smithi007.stdout:8/637: chown d0/d26/f5b 605224901 1 2024-03-16T09:01:11.403 INFO:tasks.workunit.client.1.smithi019.stdout:4/206: dwrite d4/d1b/f28 [4194304,4194304] 0 2024-03-16T09:01:11.403 INFO:tasks.workunit.client.1.smithi019.stdout:8/181: dwrite d6/de/d16/d19/f2e [4194304,4194304] 0 2024-03-16T09:01:11.403 INFO:tasks.workunit.client.1.smithi019.stdout:8/182: chown d6/de/f29 918 1 2024-03-16T09:01:11.403 INFO:tasks.workunit.client.1.smithi019.stdout:8/183: chown d6/de/d24/f35 252 1 2024-03-16T09:01:11.403 INFO:tasks.workunit.client.1.smithi019.stdout:8/184: dread - d6/de/f37 zero size 2024-03-16T09:01:11.403 INFO:tasks.workunit.client.1.smithi019.stdout:8/185: readlink d6/l17 0 2024-03-16T09:01:11.403 INFO:tasks.workunit.client.1.smithi019.stdout:5/123: dwrite d6/f9 [0,4194304] 0 2024-03-16T09:01:11.404 INFO:tasks.workunit.client.1.smithi019.stdout:0/136: mkdir d7/d29 0 2024-03-16T09:01:11.404 INFO:tasks.workunit.client.1.smithi019.stdout:0/137: write d7/d8/d22/f25 [155594,59601] 0 2024-03-16T09:01:11.404 INFO:tasks.workunit.client.1.smithi019.stdout:6/295: dwrite de/d1c/f20 [0,4194304] 0 2024-03-16T09:01:11.404 INFO:tasks.workunit.client.1.smithi019.stdout:6/296: creat de/d1c/f6a x:0 0 0 2024-03-16T09:01:11.405 INFO:tasks.workunit.client.1.smithi019.stdout:6/297: write de/d1c/f28 [2155406,30797] 0 2024-03-16T09:01:11.407 INFO:tasks.workunit.client.1.smithi019.stdout:7/124: dwrite d9/da/d13/d18/d1a/f1f [0,4194304] 0 2024-03-16T09:01:11.408 INFO:tasks.workunit.client.1.smithi019.stdout:4/207: rename d4/d6/d15/d1f/d2c/f41 to d4/d1b/d2a/d3d/f48 0 2024-03-16T09:01:11.409 INFO:tasks.workunit.client.1.smithi019.stdout:0/138: write d7/d8/f23 [24526,109026] 0 2024-03-16T09:01:11.410 INFO:tasks.workunit.client.1.smithi019.stdout:7/125: dread d9/f16 [0,4194304] 0 2024-03-16T09:01:11.412 INFO:tasks.workunit.client.0.smithi007.stdout:0/369: dread d3/d9/da/d48/d25/d43/d1a/f1b [0,4194304] 0 2024-03-16T09:01:11.413 INFO:tasks.workunit.client.1.smithi019.stdout:0/139: write d7/fd [7471271,40990] 0 2024-03-16T09:01:11.413 INFO:tasks.workunit.client.1.smithi019.stdout:0/140: stat d7/l17 0 2024-03-16T09:01:11.414 INFO:tasks.workunit.client.1.smithi019.stdout:0/141: write f4 [5583829,121699] 0 2024-03-16T09:01:11.414 INFO:tasks.workunit.client.1.smithi019.stdout:0/142: chown d7/d8/d22/f25 1017 1 2024-03-16T09:01:11.414 INFO:tasks.workunit.client.1.smithi019.stdout:0/143: readlink d7/la 0 2024-03-16T09:01:11.414 INFO:tasks.workunit.client.1.smithi019.stdout:0/144: write f4 [7927909,1866] 0 2024-03-16T09:01:11.417 INFO:tasks.workunit.client.0.smithi007.stdout:0/370: dread d3/d9/da/d48/f12 [0,4194304] 0 2024-03-16T09:01:11.417 INFO:tasks.workunit.client.0.smithi007.stdout:0/371: chown d3/d9/db/d2c 51 1 2024-03-16T09:01:11.417 INFO:tasks.workunit.client.0.smithi007.stdout:0/372: chown d3/d9/da/d18/d27/d34 0 1 2024-03-16T09:01:11.417 INFO:tasks.workunit.client.1.smithi019.stdout:6/298: truncate de/d1c/f27 47720 0 2024-03-16T09:01:11.417 INFO:tasks.workunit.client.1.smithi019.stdout:6/299: readlink de/d26/d32/d4e/l54 0 2024-03-16T09:01:11.418 INFO:tasks.workunit.client.0.smithi007.stdout:0/373: dread d3/d9/da/d18/d23/f4d [0,4194304] 0 2024-03-16T09:01:11.420 INFO:tasks.workunit.client.0.smithi007.stdout:0/374: unlink d3/d9/da/d18/c66 0 2024-03-16T09:01:11.420 INFO:tasks.workunit.client.0.smithi007.stdout:0/375: fsync d3/d9/da/d18/d27/d34/f57 0 2024-03-16T09:01:11.421 INFO:tasks.workunit.client.0.smithi007.stdout:0/376: creat d3/d9/da/d18/f7d x:0 0 0 2024-03-16T09:01:11.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/208: mkdir d4/d1b/d49 0 2024-03-16T09:01:11.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/209: dread - d4/d6/d15/f33 zero size 2024-03-16T09:01:11.422 INFO:tasks.workunit.client.0.smithi007.stdout:0/377: dread d3/f49 [0,4194304] 0 2024-03-16T09:01:11.422 INFO:tasks.workunit.client.0.smithi007.stdout:0/378: write d3/f6f [4511,119460] 0 2024-03-16T09:01:11.422 INFO:tasks.workunit.client.0.smithi007.stdout:0/379: stat d3/d9/da/d18/d27/d34/d3a/c41 0 2024-03-16T09:01:11.423 INFO:tasks.workunit.client.1.smithi019.stdout:7/126: truncate d9/fc 1479991 0 2024-03-16T09:01:11.423 INFO:tasks.workunit.client.1.smithi019.stdout:7/127: dread - d9/da/f24 zero size 2024-03-16T09:01:11.423 INFO:tasks.workunit.client.1.smithi019.stdout:7/128: chown d9/da 488463 1 2024-03-16T09:01:11.423 INFO:tasks.workunit.client.0.smithi007.stdout:0/380: symlink d3/d9/da/d18/l7e 0 2024-03-16T09:01:11.424 INFO:tasks.workunit.client.0.smithi007.stdout:0/381: dread - d3/f22 zero size 2024-03-16T09:01:11.424 INFO:tasks.workunit.client.1.smithi019.stdout:0/145: creat d7/f2a x:0 0 0 2024-03-16T09:01:11.424 INFO:tasks.workunit.client.1.smithi019.stdout:6/300: mknod de/d1c/c6b 0 2024-03-16T09:01:11.424 INFO:tasks.workunit.client.1.smithi019.stdout:6/301: getdents de/d26/d32/d4e 0 2024-03-16T09:01:11.425 INFO:tasks.workunit.client.1.smithi019.stdout:4/210: symlink d4/d1b/d49/l4a 0 2024-03-16T09:01:11.427 INFO:tasks.workunit.client.1.smithi019.stdout:7/129: mkdir d9/d25 0 2024-03-16T09:01:11.427 INFO:tasks.workunit.client.1.smithi019.stdout:7/130: write d9/da/d13/f17 [3239794,114860] 0 2024-03-16T09:01:11.427 INFO:tasks.workunit.client.1.smithi019.stdout:0/146: getdents d7/d1d/d28 0 2024-03-16T09:01:11.427 INFO:tasks.workunit.client.1.smithi019.stdout:0/147: write d7/f2a [150483,129500] 0 2024-03-16T09:01:11.429 INFO:tasks.workunit.client.0.smithi007.stdout:7/286: dwrite d0/fa [0,4194304] 0 2024-03-16T09:01:11.429 INFO:tasks.workunit.client.0.smithi007.stdout:7/287: fsync d0/f33 0 2024-03-16T09:01:11.430 INFO:tasks.workunit.client.0.smithi007.stdout:7/288: dread - d0/d19/d1f/d42/f45 zero size 2024-03-16T09:01:11.430 INFO:tasks.workunit.client.1.smithi019.stdout:7/131: rmdir d9/da/d13/d18/d1a/d1b 0 2024-03-16T09:01:11.431 INFO:tasks.workunit.client.1.smithi019.stdout:5/124: dwrite f1 [0,4194304] 0 2024-03-16T09:01:11.432 INFO:tasks.workunit.client.0.smithi007.stdout:8/638: dwrite d0/ff [4194304,4194304] 0 2024-03-16T09:01:11.432 INFO:tasks.workunit.client.0.smithi007.stdout:7/289: mkdir d0/d19/d1f/d5d 0 2024-03-16T09:01:11.433 INFO:tasks.workunit.client.0.smithi007.stdout:8/639: symlink d0/dc/d54/le0 0 2024-03-16T09:01:11.435 INFO:tasks.workunit.client.0.smithi007.stdout:8/640: creat d0/dc/dd/d8b/fe1 x:0 0 0 2024-03-16T09:01:11.435 INFO:tasks.workunit.client.0.smithi007.stdout:8/641: stat d0/dc/d8a/f91 0 2024-03-16T09:01:11.436 INFO:tasks.workunit.client.1.smithi019.stdout:0/148: fdatasync d7/f2a 0 2024-03-16T09:01:11.436 INFO:tasks.workunit.client.0.smithi007.stdout:8/642: mkdir d0/d2/d18/da7/de2 0 2024-03-16T09:01:11.436 INFO:tasks.workunit.client.0.smithi007.stdout:8/643: stat d0/dc/d80/fcc 0 2024-03-16T09:01:11.438 INFO:tasks.workunit.client.0.smithi007.stdout:8/644: rename d0/d26/d2d/d43/d4c/l7c to d0/dc/d50/d8e/dd2/le3 0 2024-03-16T09:01:11.438 INFO:tasks.workunit.client.0.smithi007.stdout:8/645: chown d0/dc/dd/ca9 15975 1 2024-03-16T09:01:11.438 INFO:tasks.workunit.client.0.smithi007.stdout:8/646: creat d0/dc/d54/fe4 x:0 0 0 2024-03-16T09:01:11.439 INFO:tasks.workunit.client.1.smithi019.stdout:9/145: dread d1/d7/de/f10 [0,4194304] 0 2024-03-16T09:01:11.440 INFO:tasks.workunit.client.0.smithi007.stdout:0/382: dwrite d3/d9/db/d2c/f6e [0,4194304] 0 2024-03-16T09:01:11.440 INFO:tasks.workunit.client.0.smithi007.stdout:0/383: fsync d3/d9/da/d48/d25/f68 0 2024-03-16T09:01:11.440 INFO:tasks.workunit.client.0.smithi007.stdout:0/384: write d3/d9/da/d18/d23/f45 [809339,109794] 0 2024-03-16T09:01:11.440 INFO:tasks.workunit.client.0.smithi007.stdout:0/385: write d3/d9/da/d48/d25/d43/d1a/f2d [4605845,45753] 0 2024-03-16T09:01:11.440 INFO:tasks.workunit.client.0.smithi007.stdout:0/386: truncate d3/d1f/d70/f67 935942 0 2024-03-16T09:01:11.441 INFO:tasks.workunit.client.0.smithi007.stdout:5/406: dwrite d1/d7/d1c/f15 [0,4194304] 0 2024-03-16T09:01:11.441 INFO:tasks.workunit.client.0.smithi007.stdout:8/647: truncate d0/dc/dd/d8b/fd0 2010898 0 2024-03-16T09:01:11.441 INFO:tasks.workunit.client.0.smithi007.stdout:8/648: readlink d0/d26/d2d/d43/d4c/l69 0 2024-03-16T09:01:11.441 INFO:tasks.workunit.client.1.smithi019.stdout:5/125: mkdir d6/d1f/d21 0 2024-03-16T09:01:11.442 INFO:tasks.workunit.client.1.smithi019.stdout:5/126: stat d6/lc 0 2024-03-16T09:01:11.442 INFO:tasks.workunit.client.1.smithi019.stdout:5/127: dread - d6/dd/f1d zero size 2024-03-16T09:01:11.442 INFO:tasks.workunit.client.1.smithi019.stdout:5/128: creat d6/dd/f22 x:0 0 0 2024-03-16T09:01:11.442 INFO:tasks.workunit.client.1.smithi019.stdout:5/129: chown l5 256 1 2024-03-16T09:01:11.443 INFO:tasks.workunit.client.1.smithi019.stdout:0/149: link d7/d1d/d28/c20 d7/d8/c2b 0 2024-03-16T09:01:11.443 INFO:tasks.workunit.client.1.smithi019.stdout:7/132: dread d9/db/f1d [0,4194304] 0 2024-03-16T09:01:11.445 INFO:tasks.workunit.client.1.smithi019.stdout:9/146: rename d1/d7/la to d1/d7/d21/l32 0 2024-03-16T09:01:11.447 INFO:tasks.workunit.client.1.smithi019.stdout:5/130: unlink f1 0 2024-03-16T09:01:11.447 INFO:tasks.workunit.client.1.smithi019.stdout:5/131: chown d6/d1f/d21 3171354 1 2024-03-16T09:01:11.448 INFO:tasks.workunit.client.1.smithi019.stdout:0/150: link d7/f15 d7/d8/d22/f2c 0 2024-03-16T09:01:11.451 INFO:tasks.workunit.client.1.smithi019.stdout:5/132: creat d6/f23 x:0 0 0 2024-03-16T09:01:11.451 INFO:tasks.workunit.client.1.smithi019.stdout:5/133: chown d6/l11 103067 1 2024-03-16T09:01:11.452 INFO:tasks.workunit.client.1.smithi019.stdout:0/151: symlink d7/d8/d22/l2d 0 2024-03-16T09:01:11.454 INFO:tasks.workunit.client.1.smithi019.stdout:0/152: mkdir d7/d1d/d2e 0 2024-03-16T09:01:11.455 INFO:tasks.workunit.client.1.smithi019.stdout:0/153: mkdir d7/d21/d2f 0 2024-03-16T09:01:11.459 INFO:tasks.workunit.client.0.smithi007.stdout:5/407: dread d1/d7/d16/d1e/d2a/f57 [0,4194304] 0 2024-03-16T09:01:11.460 INFO:tasks.workunit.client.0.smithi007.stdout:0/387: write d3/d9/da/d48/d25/d43/d1a/f2d [147208,127921] 0 2024-03-16T09:01:11.460 INFO:tasks.workunit.client.0.smithi007.stdout:7/290: dwrite d0/d19/f2d [0,4194304] 0 2024-03-16T09:01:11.462 INFO:tasks.workunit.client.1.smithi019.stdout:6/302: dread de/d1c/f27 [0,4194304] 0 2024-03-16T09:01:11.463 INFO:tasks.workunit.client.0.smithi007.stdout:0/388: link d3/d9/da/d18/d23/f45 d3/d1f/f7f 0 2024-03-16T09:01:11.466 INFO:tasks.workunit.client.1.smithi019.stdout:6/303: unlink de/d1c/d1d/c50 0 2024-03-16T09:01:11.466 INFO:tasks.workunit.client.1.smithi019.stdout:6/304: dread - de/d1c/d1d/f5a zero size 2024-03-16T09:01:11.466 INFO:tasks.workunit.client.1.smithi019.stdout:6/305: fsync f7 0 2024-03-16T09:01:11.468 INFO:tasks.workunit.client.0.smithi007.stdout:0/389: rename d3/d9/db/d1c/l50 to d3/d9/da/d18/l80 0 2024-03-16T09:01:11.468 INFO:tasks.workunit.client.0.smithi007.stdout:0/390: creat d3/d9/da/d18/d23/f81 x:0 0 0 2024-03-16T09:01:11.468 INFO:tasks.workunit.client.0.smithi007.stdout:0/391: read - d3/d1f/d70/d73/f5c zero size 2024-03-16T09:01:11.469 INFO:tasks.workunit.client.1.smithi019.stdout:6/306: truncate fb 5935267 0 2024-03-16T09:01:11.469 INFO:tasks.workunit.client.1.smithi019.stdout:6/307: chown de/f15 500271 1 2024-03-16T09:01:11.469 INFO:tasks.workunit.client.1.smithi019.stdout:0/154: read d7/f15 [2236699,81955] 0 2024-03-16T09:01:11.471 INFO:tasks.workunit.client.1.smithi019.stdout:6/308: unlink de/d1c/f27 0 2024-03-16T09:01:11.471 INFO:tasks.workunit.client.0.smithi007.stdout:8/649: dread d0/dc/dd/d8b/fd0 [0,4194304] 0 2024-03-16T09:01:11.472 INFO:tasks.workunit.client.0.smithi007.stdout:8/650: chown d0/d2/c46 2261 1 2024-03-16T09:01:11.472 INFO:tasks.workunit.client.0.smithi007.stdout:8/651: dread - d0/dc/d54/fd8 zero size 2024-03-16T09:01:11.473 INFO:tasks.workunit.client.0.smithi007.stdout:0/392: link d3/d9/da/d18/d23/f81 d3/d9/db/d1c/d72/f82 0 2024-03-16T09:01:11.473 INFO:tasks.workunit.client.1.smithi019.stdout:6/309: symlink de/d1c/l6c 0 2024-03-16T09:01:11.474 INFO:tasks.workunit.client.1.smithi019.stdout:5/134: dwrite d6/f17 [0,4194304] 0 2024-03-16T09:01:11.474 INFO:tasks.workunit.client.1.smithi019.stdout:5/135: chown d6/dd/f16 145406411 1 2024-03-16T09:01:11.477 INFO:tasks.workunit.client.1.smithi019.stdout:2/150: sync 2024-03-16T09:01:11.478 INFO:tasks.workunit.client.1.smithi019.stdout:2/151: link d5/de/fd d5/f34 0 2024-03-16T09:01:11.480 INFO:tasks.workunit.client.1.smithi019.stdout:2/152: truncate f1 2719708 0 2024-03-16T09:01:11.481 INFO:tasks.workunit.client.0.smithi007.stdout:0/393: dread d3/d9/da/d18/d23/f32 [0,4194304] 0 2024-03-16T09:01:11.482 INFO:tasks.workunit.client.0.smithi007.stdout:0/394: dread - d3/d9/da/d18/d23/f78 zero size 2024-03-16T09:01:11.485 INFO:tasks.workunit.client.0.smithi007.stdout:0/395: creat d3/d9/da/d18/d27/f83 x:0 0 0 2024-03-16T09:01:11.490 INFO:tasks.workunit.client.1.smithi019.stdout:5/136: dread d6/f9 [0,4194304] 0 2024-03-16T09:01:11.491 INFO:tasks.workunit.client.1.smithi019.stdout:0/155: dwrite f4 [8388608,4194304] 0 2024-03-16T09:01:11.491 INFO:tasks.workunit.client.1.smithi019.stdout:0/156: unlink d7/d8/d22/l2d 0 2024-03-16T09:01:11.491 INFO:tasks.workunit.client.0.smithi007.stdout:0/396: dread d3/f6f [0,4194304] 0 2024-03-16T09:01:11.492 INFO:tasks.workunit.client.0.smithi007.stdout:0/397: mkdir d3/d9/da/d48/d25/d43/d1a/d58/d61/d84 0 2024-03-16T09:01:11.492 INFO:tasks.workunit.client.0.smithi007.stdout:0/398: link d3/d9/da/d18/d23/c64 d3/d9/db/d1c/c85 0 2024-03-16T09:01:11.493 INFO:tasks.workunit.client.1.smithi019.stdout:0/157: symlink d7/d1d/l30 0 2024-03-16T09:01:11.493 INFO:tasks.workunit.client.1.smithi019.stdout:6/310: getdents de/d1c/d1d 0 2024-03-16T09:01:11.494 INFO:tasks.workunit.client.0.smithi007.stdout:0/399: unlink d3/d9/db/d2c/f2f 0 2024-03-16T09:01:11.495 INFO:tasks.workunit.client.0.smithi007.stdout:0/400: creat d3/d9/da/d18/d23/d36/f86 x:0 0 0 2024-03-16T09:01:11.495 INFO:tasks.workunit.client.0.smithi007.stdout:0/401: write d3/d9/da/f4c [2967080,82530] 0 2024-03-16T09:01:11.495 INFO:tasks.workunit.client.0.smithi007.stdout:0/402: dread d3/d9/da/d48/f12 [0,4194304] 0 2024-03-16T09:01:11.496 INFO:tasks.workunit.client.0.smithi007.stdout:8/652: dwrite d0/dc/dd/f2c [0,4194304] 0 2024-03-16T09:01:11.497 INFO:tasks.workunit.client.1.smithi019.stdout:0/158: mknod d7/d1d/d2e/c31 0 2024-03-16T09:01:11.497 INFO:tasks.workunit.client.1.smithi019.stdout:2/153: dwrite d5/d6/d2b/f18 [0,4194304] 0 2024-03-16T09:01:11.499 INFO:tasks.workunit.client.1.smithi019.stdout:6/311: unlink de/d1c/f2e 0 2024-03-16T09:01:11.499 INFO:tasks.workunit.client.0.smithi007.stdout:8/653: rmdir d0/dc/d80 39 2024-03-16T09:01:11.503 INFO:tasks.workunit.client.0.smithi007.stdout:8/654: link d0/dc/d80/fcc d0/dc/d80/fe5 0 2024-03-16T09:01:11.503 INFO:tasks.workunit.client.1.smithi019.stdout:0/159: mkdir d7/d8/d32 0 2024-03-16T09:01:11.504 INFO:tasks.workunit.client.1.smithi019.stdout:0/160: write d7/d8/d22/f27 [3531959,18484] 0 2024-03-16T09:01:11.504 INFO:tasks.workunit.client.1.smithi019.stdout:0/161: write d7/d8/d22/f2c [2598704,20233] 0 2024-03-16T09:01:11.510 INFO:tasks.workunit.client.0.smithi007.stdout:8/655: mknod d0/dc/dd/ce6 0 2024-03-16T09:01:11.510 INFO:tasks.workunit.client.1.smithi019.stdout:2/154: dread d5/ff [0,4194304] 0 2024-03-16T09:01:11.510 INFO:tasks.workunit.client.1.smithi019.stdout:2/155: fsync d5/f34 0 2024-03-16T09:01:11.514 INFO:tasks.workunit.client.0.smithi007.stdout:0/403: dwrite d3/d9/da/d18/f59 [0,4194304] 0 2024-03-16T09:01:11.516 INFO:tasks.workunit.client.1.smithi019.stdout:6/312: rename fb to de/d1c/d1d/d66/f6d 0 2024-03-16T09:01:11.516 INFO:tasks.workunit.client.1.smithi019.stdout:6/313: write de/d1c/f5d [5060155,16588] 0 2024-03-16T09:01:11.518 INFO:tasks.workunit.client.1.smithi019.stdout:2/156: mkdir d5/d6/d2b/d2f/d35 0 2024-03-16T09:01:11.518 INFO:tasks.workunit.client.1.smithi019.stdout:2/157: readlink d5/d6/l26 0 2024-03-16T09:01:11.518 INFO:tasks.workunit.client.1.smithi019.stdout:2/158: read - d5/de/f1f zero size 2024-03-16T09:01:11.518 INFO:tasks.workunit.client.1.smithi019.stdout:2/159: stat d5/de/f1f 0 2024-03-16T09:01:11.518 INFO:tasks.workunit.client.0.smithi007.stdout:0/404: rename d3/d1f/d70/d73 to d3/d9/da/d48/d25/d87 0 2024-03-16T09:01:11.518 INFO:tasks.workunit.client.0.smithi007.stdout:0/405: stat d3/d9/da/d48/d25/d43/d1a/d58 0 2024-03-16T09:01:11.526 INFO:tasks.workunit.client.1.smithi019.stdout:6/314: readlink de/l69 0 2024-03-16T09:01:11.527 INFO:tasks.workunit.client.1.smithi019.stdout:2/160: truncate d5/f34 3857542 0 2024-03-16T09:01:11.527 INFO:tasks.workunit.client.1.smithi019.stdout:2/161: stat d5/d2e 0 2024-03-16T09:01:11.527 INFO:tasks.workunit.client.1.smithi019.stdout:6/315: fsync de/f3c 0 2024-03-16T09:01:11.533 INFO:tasks.workunit.client.1.smithi019.stdout:0/162: dwrite f4 [0,4194304] 0 2024-03-16T09:01:11.534 INFO:tasks.workunit.client.1.smithi019.stdout:0/163: symlink d7/d1d/l33 0 2024-03-16T09:01:11.535 INFO:tasks.workunit.client.1.smithi019.stdout:0/164: unlink c1 0 2024-03-16T09:01:11.547 INFO:tasks.workunit.client.1.smithi019.stdout:2/162: write d5/f23 [715695,105561] 0 2024-03-16T09:01:11.548 INFO:tasks.workunit.client.1.smithi019.stdout:6/316: write de/f3c [601317,36373] 0 2024-03-16T09:01:11.549 INFO:tasks.workunit.client.1.smithi019.stdout:6/317: rmdir de/d1c/d1d/d66 39 2024-03-16T09:01:11.551 INFO:tasks.workunit.client.1.smithi019.stdout:6/318: creat de/d26/d32/d4e/f6e x:0 0 0 2024-03-16T09:01:11.553 INFO:tasks.workunit.client.1.smithi019.stdout:5/137: dwrite f4 [4194304,4194304] 0 2024-03-16T09:01:11.553 INFO:tasks.workunit.client.0.smithi007.stdout:0/406: dwrite d3/d9/da/d18/d27/d34/f57 [0,4194304] 0 2024-03-16T09:01:11.553 INFO:tasks.workunit.client.0.smithi007.stdout:0/407: readlink d3/d9/da/d18/d27/d34/d3a/l5a 0 2024-03-16T09:01:11.553 INFO:tasks.workunit.client.0.smithi007.stdout:0/408: truncate d3/d9/da/d18/d23/f75 33337 0 2024-03-16T09:01:11.554 INFO:tasks.workunit.client.0.smithi007.stdout:0/409: readlink d3/d9/da/d48/d25/d87/l3b 0 2024-03-16T09:01:11.554 INFO:tasks.workunit.client.0.smithi007.stdout:8/656: dwrite d0/dc/d80/fcc [0,4194304] 0 2024-03-16T09:01:11.555 INFO:tasks.workunit.client.1.smithi019.stdout:5/138: chown d6/d19/l1c 3226 1 2024-03-16T09:01:11.556 INFO:tasks.workunit.client.1.smithi019.stdout:5/139: fsync f4 0 2024-03-16T09:01:11.557 INFO:tasks.workunit.client.1.smithi019.stdout:0/165: dread d7/fd [4194304,4194304] 0 2024-03-16T09:01:11.559 INFO:tasks.workunit.client.1.smithi019.stdout:6/319: unlink de/d1c/d1d/d66/f6d 0 2024-03-16T09:01:11.561 INFO:tasks.workunit.client.0.smithi007.stdout:0/410: read d3/d9/db/d2c/f6e [4002992,77096] 0 2024-03-16T09:01:11.562 INFO:tasks.workunit.client.1.smithi019.stdout:2/163: dwrite f1 [0,4194304] 0 2024-03-16T09:01:11.562 INFO:tasks.workunit.client.1.smithi019.stdout:2/164: creat d5/d6/f36 x:0 0 0 2024-03-16T09:01:11.562 INFO:tasks.workunit.client.1.smithi019.stdout:2/165: readlink d5/d6/d2b/l20 0 2024-03-16T09:01:11.562 INFO:tasks.workunit.client.1.smithi019.stdout:2/166: stat d5/d6/d2b/f1e 0 2024-03-16T09:01:11.562 INFO:tasks.workunit.client.1.smithi019.stdout:2/167: fdatasync d5/d6/f11 0 2024-03-16T09:01:11.564 INFO:tasks.workunit.client.1.smithi019.stdout:6/320: link de/d26/d32/f4f de/f6f 0 2024-03-16T09:01:11.566 INFO:tasks.workunit.client.1.smithi019.stdout:3/155: sync 2024-03-16T09:01:11.566 INFO:tasks.workunit.client.1.smithi019.stdout:3/156: creat d3/d17/f33 x:0 0 0 2024-03-16T09:01:11.566 INFO:tasks.workunit.client.1.smithi019.stdout:3/157: rename d3/d11/d22/d24 to d3/d11/d22/d24/d34 22 2024-03-16T09:01:11.567 INFO:tasks.workunit.client.0.smithi007.stdout:0/411: dread d3/d9/da/d48/d25/d43/d1a/f20 [0,4194304] 0 2024-03-16T09:01:11.567 INFO:tasks.workunit.client.0.smithi007.stdout:0/412: truncate d3/d9/db/d1c/f2a 5133590 0 2024-03-16T09:01:11.569 INFO:tasks.workunit.client.1.smithi019.stdout:6/321: mkdir de/d1c/d1d/d51/d70 0 2024-03-16T09:01:11.569 INFO:tasks.workunit.client.1.smithi019.stdout:3/158: write f2 [2463268,114930] 0 2024-03-16T09:01:11.570 INFO:tasks.workunit.client.1.smithi019.stdout:5/140: dread f2 [4194304,4194304] 0 2024-03-16T09:01:11.572 INFO:tasks.workunit.client.1.smithi019.stdout:0/166: truncate d7/fd 7506056 0 2024-03-16T09:01:11.574 INFO:tasks.workunit.client.1.smithi019.stdout:0/167: dread d7/f2a [0,4194304] 0 2024-03-16T09:01:11.574 INFO:tasks.workunit.client.1.smithi019.stdout:0/168: write d7/d8/d22/f2c [1212739,103714] 0 2024-03-16T09:01:11.575 INFO:tasks.workunit.client.1.smithi019.stdout:3/159: getdents d3/d11/d22 0 2024-03-16T09:01:11.575 INFO:tasks.workunit.client.1.smithi019.stdout:0/169: read d7/f19 [750314,75988] 0 2024-03-16T09:01:11.576 INFO:tasks.workunit.client.0.smithi007.stdout:0/413: read d3/d9/da/d18/d23/f63 [1642791,2501] 0 2024-03-16T09:01:11.576 INFO:tasks.workunit.client.0.smithi007.stdout:0/414: fdatasync d3/d9/da/d18/d27/d34/f57 0 2024-03-16T09:01:11.576 INFO:tasks.workunit.client.0.smithi007.stdout:0/415: chown d3/d9/da/d48/d25/d87/f5c 220973982 1 2024-03-16T09:01:11.577 INFO:tasks.workunit.client.1.smithi019.stdout:4/211: sync 2024-03-16T09:01:11.577 INFO:tasks.workunit.client.1.smithi019.stdout:1/88: sync 2024-03-16T09:01:11.577 INFO:tasks.workunit.client.1.smithi019.stdout:8/186: sync 2024-03-16T09:01:11.577 INFO:tasks.workunit.client.1.smithi019.stdout:1/89: write fb [338357,128161] 0 2024-03-16T09:01:11.577 INFO:tasks.workunit.client.0.smithi007.stdout:0/416: creat d3/d77/f88 x:0 0 0 2024-03-16T09:01:11.577 INFO:tasks.workunit.client.0.smithi007.stdout:0/417: truncate d3/d1f/f7f 1797546 0 2024-03-16T09:01:11.581 INFO:tasks.workunit.client.1.smithi019.stdout:5/141: creat d6/d1f/d21/f24 x:0 0 0 2024-03-16T09:01:11.581 INFO:tasks.workunit.client.1.smithi019.stdout:5/142: chown d6/lc 119 1 2024-03-16T09:01:11.581 INFO:tasks.workunit.client.1.smithi019.stdout:5/143: truncate d6/f23 245616 0 2024-03-16T09:01:11.584 INFO:tasks.workunit.client.1.smithi019.stdout:6/322: write de/f49 [3648814,109256] 0 2024-03-16T09:01:11.584 INFO:tasks.workunit.client.1.smithi019.stdout:6/323: chown de/d1c/f20 194087563 1 2024-03-16T09:01:11.584 INFO:tasks.workunit.client.1.smithi019.stdout:3/160: link d3/l27 d3/d11/d22/l35 0 2024-03-16T09:01:11.585 INFO:tasks.workunit.client.1.smithi019.stdout:0/170: rename d7/f15 to d7/d1d/d28/f34 0 2024-03-16T09:01:11.588 INFO:tasks.workunit.client.1.smithi019.stdout:4/212: link d4/lb d4/d1b/d49/l4b 0 2024-03-16T09:01:11.588 INFO:tasks.workunit.client.1.smithi019.stdout:4/213: fsync d4/d1b/f1c 0 2024-03-16T09:01:11.588 INFO:tasks.workunit.client.1.smithi019.stdout:4/214: stat d4/d6/f24 0 2024-03-16T09:01:11.588 INFO:tasks.workunit.client.1.smithi019.stdout:4/215: write d4/d6/f24 [1370975,41800] 0 2024-03-16T09:01:11.589 INFO:tasks.workunit.client.1.smithi019.stdout:1/90: dread fd [0,4194304] 0 2024-03-16T09:01:11.590 INFO:tasks.workunit.client.1.smithi019.stdout:5/144: write d6/f8 [4587029,8961] 0 2024-03-16T09:01:11.591 INFO:tasks.workunit.client.1.smithi019.stdout:6/324: symlink de/d26/d32/d4e/l71 0 2024-03-16T09:01:11.591 INFO:tasks.workunit.client.1.smithi019.stdout:6/325: chown de/f49 15 1 2024-03-16T09:01:11.593 INFO:tasks.workunit.client.1.smithi019.stdout:0/171: creat d7/d21/f35 x:0 0 0 2024-03-16T09:01:11.594 INFO:tasks.workunit.client.1.smithi019.stdout:0/172: stat d7/d1d/d28/l1f 0 2024-03-16T09:01:11.597 INFO:tasks.workunit.client.1.smithi019.stdout:1/91: rename l11 to l15 0 2024-03-16T09:01:11.600 INFO:tasks.workunit.client.1.smithi019.stdout:6/326: write de/d1c/f47 [912870,75935] 0 2024-03-16T09:01:11.600 INFO:tasks.workunit.client.1.smithi019.stdout:6/327: write de/d26/d32/d4e/f6e [754059,10988] 0 2024-03-16T09:01:11.600 INFO:tasks.workunit.client.1.smithi019.stdout:6/328: chown de/d26/c68 40743667 1 2024-03-16T09:01:11.600 INFO:tasks.workunit.client.1.smithi019.stdout:6/329: write de/d26/d32/d4e/f59 [121468,94433] 0 2024-03-16T09:01:11.602 INFO:tasks.workunit.client.1.smithi019.stdout:7/133: sync 2024-03-16T09:01:11.602 INFO:tasks.workunit.client.1.smithi019.stdout:7/134: creat d9/db/d12/f26 x:0 0 0 2024-03-16T09:01:11.602 INFO:tasks.workunit.client.1.smithi019.stdout:9/147: sync 2024-03-16T09:01:11.603 INFO:tasks.workunit.client.1.smithi019.stdout:8/187: dwrite d6/f7 [0,4194304] 0 2024-03-16T09:01:11.605 INFO:tasks.workunit.client.1.smithi019.stdout:5/145: write d6/f9 [1022272,54368] 0 2024-03-16T09:01:11.606 INFO:tasks.workunit.client.1.smithi019.stdout:7/135: rename d9/da/c10 to d9/da/c27 0 2024-03-16T09:01:11.606 INFO:tasks.workunit.client.1.smithi019.stdout:1/92: dread f9 [4194304,4194304] 0 2024-03-16T09:01:11.607 INFO:tasks.workunit.client.1.smithi019.stdout:1/93: chown f14 29420 1 2024-03-16T09:01:11.607 INFO:tasks.workunit.client.1.smithi019.stdout:1/94: chown f9 114014926 1 2024-03-16T09:01:11.607 INFO:tasks.workunit.client.1.smithi019.stdout:2/168: dwrite d5/f7 [0,4194304] 0 2024-03-16T09:01:11.607 INFO:tasks.workunit.client.1.smithi019.stdout:5/146: mknod d6/dd/c25 0 2024-03-16T09:01:11.607 INFO:tasks.workunit.client.0.smithi007.stdout:0/418: dwrite d3/d9/db/d2c/f6e [0,4194304] 0 2024-03-16T09:01:11.608 INFO:tasks.workunit.client.0.smithi007.stdout:0/419: dread - d3/d1f/f71 zero size 2024-03-16T09:01:11.608 INFO:tasks.workunit.client.1.smithi019.stdout:3/161: write d3/d11/f14 [3286234,46467] 0 2024-03-16T09:01:11.608 INFO:tasks.workunit.client.1.smithi019.stdout:3/162: chown d3/d1b/c1e 30 1 2024-03-16T09:01:11.609 INFO:tasks.workunit.client.0.smithi007.stdout:0/420: dread d3/d9/da/d18/d23/f75 [0,4194304] 0 2024-03-16T09:01:11.609 INFO:tasks.workunit.client.0.smithi007.stdout:0/421: read d3/f49 [177297,125073] 0 2024-03-16T09:01:11.609 INFO:tasks.workunit.client.0.smithi007.stdout:0/422: fsync d3/d9/da/d18/f6a 0 2024-03-16T09:01:11.610 INFO:tasks.workunit.client.1.smithi019.stdout:7/136: mkdir d9/d25/d28 0 2024-03-16T09:01:11.612 INFO:tasks.workunit.client.1.smithi019.stdout:2/169: mknod d5/d6/d2b/c37 0 2024-03-16T09:01:11.612 INFO:tasks.workunit.client.0.smithi007.stdout:0/423: rename d3/d9/db/d2c/f6e to d3/d9/da/d18/d27/d34/f89 0 2024-03-16T09:01:11.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/424: stat d3/d9/da/d18/d27/f83 0 2024-03-16T09:01:11.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/425: truncate d3/d9/da/d48/f47 1000189 0 2024-03-16T09:01:11.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/426: readlink d3/d9/db/l56 0 2024-03-16T09:01:11.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/427: mkdir d3/d9/db/d60/d8a 0 2024-03-16T09:01:11.614 INFO:tasks.workunit.client.1.smithi019.stdout:5/147: unlink d6/c10 0 2024-03-16T09:01:11.614 INFO:tasks.workunit.client.1.smithi019.stdout:5/148: creat d6/d1f/d21/f26 x:0 0 0 2024-03-16T09:01:11.614 INFO:tasks.workunit.client.1.smithi019.stdout:5/149: write d6/f9 [3815514,49696] 0 2024-03-16T09:01:11.615 INFO:tasks.workunit.client.0.smithi007.stdout:0/428: dread d3/d9/da/d18/d23/f75 [0,4194304] 0 2024-03-16T09:01:11.615 INFO:tasks.workunit.client.0.smithi007.stdout:0/429: fdatasync d3/d9/da/d18/d27/d34/f57 0 2024-03-16T09:01:11.619 INFO:tasks.workunit.client.0.smithi007.stdout:0/430: read d3/d9/da/d48/d25/d43/d1a/f1b [837326,127824] 0 2024-03-16T09:01:11.620 INFO:tasks.workunit.client.1.smithi019.stdout:7/137: mknod d9/c29 0 2024-03-16T09:01:11.621 INFO:tasks.workunit.client.0.smithi007.stdout:0/431: creat d3/f8b x:0 0 0 2024-03-16T09:01:11.621 INFO:tasks.workunit.client.1.smithi019.stdout:1/95: mkdir d16 0 2024-03-16T09:01:11.621 INFO:tasks.workunit.client.1.smithi019.stdout:8/188: dwrite d6/f15 [0,4194304] 0 2024-03-16T09:01:11.623 INFO:tasks.workunit.client.0.smithi007.stdout:0/432: link d3/d9/da/d18/d23/l79 d3/d1f/d70/l8c 0 2024-03-16T09:01:11.624 INFO:tasks.workunit.client.0.smithi007.stdout:0/433: mknod d3/d9/da/d18/d27/d34/c8d 0 2024-03-16T09:01:11.624 INFO:tasks.workunit.client.1.smithi019.stdout:5/150: mkdir d6/d1f/d27 0 2024-03-16T09:01:11.625 INFO:tasks.workunit.client.1.smithi019.stdout:3/163: rmdir d3/d11/d22/d24 39 2024-03-16T09:01:11.627 INFO:tasks.workunit.client.1.smithi019.stdout:0/173: rename d7/d21 to d7/d8/d32/d36 0 2024-03-16T09:01:11.630 INFO:tasks.workunit.client.1.smithi019.stdout:8/189: rmdir d6/de/d16 39 2024-03-16T09:01:11.630 INFO:tasks.workunit.client.1.smithi019.stdout:7/138: dread f8 [0,4194304] 0 2024-03-16T09:01:11.633 INFO:tasks.workunit.client.1.smithi019.stdout:0/174: mknod d7/d1d/d2e/c37 0 2024-03-16T09:01:11.633 INFO:tasks.workunit.client.1.smithi019.stdout:0/175: truncate d7/d8/f12 5164431 0 2024-03-16T09:01:11.634 INFO:tasks.workunit.client.1.smithi019.stdout:9/148: rename d1/d7/de/d24/d2e to d1/d7/d33 0 2024-03-16T09:01:11.634 INFO:tasks.workunit.client.1.smithi019.stdout:9/149: readlink d1/d7/d21/l32 0 2024-03-16T09:01:11.636 INFO:tasks.workunit.client.1.smithi019.stdout:7/139: dread f8 [0,4194304] 0 2024-03-16T09:01:11.637 INFO:tasks.workunit.client.1.smithi019.stdout:9/150: creat d1/d7/f34 x:0 0 0 2024-03-16T09:01:11.639 INFO:tasks.workunit.client.1.smithi019.stdout:1/96: rename c3 to d16/c17 0 2024-03-16T09:01:11.640 INFO:tasks.workunit.client.1.smithi019.stdout:7/140: symlink d9/da/d13/l2a 0 2024-03-16T09:01:11.641 INFO:tasks.workunit.client.1.smithi019.stdout:9/151: truncate d1/d7/de/f1f 3110123 0 2024-03-16T09:01:11.644 INFO:tasks.workunit.client.1.smithi019.stdout:5/151: rename d6/l11 to d6/d19/l28 0 2024-03-16T09:01:11.645 INFO:tasks.workunit.client.1.smithi019.stdout:5/152: dread d6/f23 [0,4194304] 0 2024-03-16T09:01:11.647 INFO:tasks.workunit.client.1.smithi019.stdout:9/152: rename d1/f20 to d1/d7/d21/d27/f35 0 2024-03-16T09:01:11.649 INFO:tasks.workunit.client.1.smithi019.stdout:9/153: symlink d1/l36 0 2024-03-16T09:01:11.650 INFO:tasks.workunit.client.1.smithi019.stdout:9/154: read d1/d7/fd [1395681,74821] 0 2024-03-16T09:01:11.650 INFO:tasks.workunit.client.0.smithi007.stdout:0/434: dwrite d3/f22 [0,4194304] 0 2024-03-16T09:01:11.651 INFO:tasks.workunit.client.0.smithi007.stdout:0/435: creat d3/d77/f8e x:0 0 0 2024-03-16T09:01:11.652 INFO:tasks.workunit.client.1.smithi019.stdout:9/155: mknod d1/c37 0 2024-03-16T09:01:11.653 INFO:tasks.workunit.client.1.smithi019.stdout:9/156: truncate d1/d7/d21/d27/f13 688821 0 2024-03-16T09:01:11.653 INFO:tasks.workunit.client.1.smithi019.stdout:9/157: fsync d1/f25 0 2024-03-16T09:01:11.653 INFO:tasks.workunit.client.1.smithi019.stdout:9/158: write d1/d7/de/d2d/f30 [5133558,101136] 0 2024-03-16T09:01:11.653 INFO:tasks.workunit.client.1.smithi019.stdout:9/159: write d1/d7/d21/d27/f35 [778143,6365] 0 2024-03-16T09:01:11.656 INFO:tasks.workunit.client.0.smithi007.stdout:0/436: rename d3/f8 to d3/d9/da/d48/d25/d43/d1a/d58/d61/f8f 0 2024-03-16T09:01:11.657 INFO:tasks.workunit.client.0.smithi007.stdout:0/437: creat d3/d9/db/d2c/f90 x:0 0 0 2024-03-16T09:01:11.658 INFO:tasks.workunit.client.1.smithi019.stdout:5/153: write d6/dd/f1a [2297285,7562] 0 2024-03-16T09:01:11.658 INFO:tasks.workunit.client.1.smithi019.stdout:5/154: creat d6/d19/f29 x:0 0 0 2024-03-16T09:01:11.667 INFO:tasks.workunit.client.1.smithi019.stdout:7/141: dread f4 [0,4194304] 0 2024-03-16T09:01:11.670 INFO:tasks.workunit.client.0.smithi007.stdout:0/438: dread d3/d9/da/d18/d27/d34/f57 [0,4194304] 0 2024-03-16T09:01:11.670 INFO:tasks.workunit.client.0.smithi007.stdout:0/439: chown d3/d9/da/d48/d25/d87/c5d 322200298 1 2024-03-16T09:01:11.676 INFO:tasks.workunit.client.0.smithi007.stdout:0/440: dread d3/d9/da/f4c [0,4194304] 0 2024-03-16T09:01:11.677 INFO:tasks.workunit.client.1.smithi019.stdout:0/176: dwrite d7/d8/d22/f25 [0,4194304] 0 2024-03-16T09:01:11.677 INFO:tasks.workunit.client.1.smithi019.stdout:1/97: dwrite fb [0,4194304] 0 2024-03-16T09:01:11.677 INFO:tasks.workunit.client.1.smithi019.stdout:1/98: readlink l15 0 2024-03-16T09:01:11.678 INFO:tasks.workunit.client.0.smithi007.stdout:0/441: mkdir d3/d9/da/d48/d25/d43/d91 0 2024-03-16T09:01:11.678 INFO:tasks.workunit.client.0.smithi007.stdout:0/442: getdents d3/d9/da 0 2024-03-16T09:01:11.679 INFO:tasks.workunit.client.1.smithi019.stdout:0/177: rmdir d7/d8/d32/d36/d2f 0 2024-03-16T09:01:11.681 INFO:tasks.workunit.client.1.smithi019.stdout:1/99: mkdir d16/d18 0 2024-03-16T09:01:11.681 INFO:tasks.workunit.client.1.smithi019.stdout:1/100: write f14 [638551,78243] 0 2024-03-16T09:01:11.681 INFO:tasks.workunit.client.1.smithi019.stdout:1/101: getdents d16/d18 0 2024-03-16T09:01:11.681 INFO:tasks.workunit.client.1.smithi019.stdout:1/102: stat d16 0 2024-03-16T09:01:11.682 INFO:tasks.workunit.client.1.smithi019.stdout:1/103: creat d16/f19 x:0 0 0 2024-03-16T09:01:11.682 INFO:tasks.workunit.client.1.smithi019.stdout:1/104: chown fb 22782 1 2024-03-16T09:01:11.684 INFO:tasks.workunit.client.1.smithi019.stdout:1/105: rename fd to d16/d18/f1a 0 2024-03-16T09:01:11.684 INFO:tasks.workunit.client.1.smithi019.stdout:1/106: chown f14 4715 1 2024-03-16T09:01:11.687 INFO:tasks.workunit.client.1.smithi019.stdout:3/164: dwrite d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:11.687 INFO:tasks.workunit.client.1.smithi019.stdout:5/155: dwrite f4 [4194304,4194304] 0 2024-03-16T09:01:11.688 INFO:tasks.workunit.client.1.smithi019.stdout:7/142: dwrite d9/db/f1d [0,4194304] 0 2024-03-16T09:01:11.689 INFO:tasks.workunit.client.1.smithi019.stdout:1/107: dread d16/d18/f1a [0,4194304] 0 2024-03-16T09:01:11.690 INFO:tasks.workunit.client.1.smithi019.stdout:3/165: symlink d3/d11/d22/d24/d2c/l36 0 2024-03-16T09:01:11.694 INFO:tasks.workunit.client.1.smithi019.stdout:1/108: creat d16/d18/f1b x:0 0 0 2024-03-16T09:01:11.695 INFO:tasks.workunit.client.1.smithi019.stdout:5/156: getdents d6/dd 0 2024-03-16T09:01:11.695 INFO:tasks.workunit.client.1.smithi019.stdout:5/157: stat d6/f8 0 2024-03-16T09:01:11.695 INFO:tasks.workunit.client.1.smithi019.stdout:5/158: read d6/dd/f1a [3694583,123520] 0 2024-03-16T09:01:11.696 INFO:tasks.workunit.client.0.smithi007.stdout:0/443: rename d3/d77 to d3/d9/da/d92 0 2024-03-16T09:01:11.697 INFO:tasks.workunit.client.0.smithi007.stdout:0/444: unlink d3/d9/da/d48/d25/d43/d1a/f2d 0 2024-03-16T09:01:11.697 INFO:tasks.workunit.client.0.smithi007.stdout:0/445: fdatasync d3/d9/db/d1c/f2a 0 2024-03-16T09:01:11.699 INFO:tasks.workunit.client.0.smithi007.stdout:0/446: creat d3/d9/f93 x:0 0 0 2024-03-16T09:01:11.700 INFO:tasks.workunit.client.0.smithi007.stdout:0/447: creat d3/d9/db/f94 x:0 0 0 2024-03-16T09:01:11.700 INFO:tasks.workunit.client.0.smithi007.stdout:0/448: dread - d3/d9/db/d1c/d72/f82 zero size 2024-03-16T09:01:11.700 INFO:tasks.workunit.client.0.smithi007.stdout:0/449: dread - d3/d9/da/d18/d23/f78 zero size 2024-03-16T09:01:11.700 INFO:tasks.workunit.client.0.smithi007.stdout:0/450: write d3/d9/db/d2c/f90 [192529,35609] 0 2024-03-16T09:01:11.701 INFO:tasks.workunit.client.1.smithi019.stdout:3/166: rename d3/d17/f33 to d3/d11/f37 0 2024-03-16T09:01:11.701 INFO:tasks.workunit.client.0.smithi007.stdout:0/451: creat d3/d9/da/d48/f95 x:0 0 0 2024-03-16T09:01:11.701 INFO:tasks.workunit.client.0.smithi007.stdout:0/452: write d3/d9/da/d92/f8e [646444,106515] 0 2024-03-16T09:01:11.702 INFO:tasks.workunit.client.1.smithi019.stdout:1/109: link f14 d16/f1c 0 2024-03-16T09:01:11.704 INFO:tasks.workunit.client.1.smithi019.stdout:5/159: symlink d6/d19/l2a 0 2024-03-16T09:01:11.704 INFO:tasks.workunit.client.0.smithi007.stdout:0/453: rename d3/d9/da/d48/d25/d87/f5c to d3/d9/db/d2c/d5b/f96 0 2024-03-16T09:01:11.704 INFO:tasks.workunit.client.0.smithi007.stdout:0/454: stat d3/d9/da/d48/d25/d43/d1a/d58 0 2024-03-16T09:01:11.705 INFO:tasks.workunit.client.0.smithi007.stdout:0/455: fsync d3/d9/da/d48/f47 0 2024-03-16T09:01:11.705 INFO:tasks.workunit.client.0.smithi007.stdout:0/456: dread - d3/d9/da/d18/f7d zero size 2024-03-16T09:01:11.714 INFO:tasks.workunit.client.0.smithi007.stdout:1/385: sync 2024-03-16T09:01:11.714 INFO:tasks.workunit.client.0.smithi007.stdout:9/428: sync 2024-03-16T09:01:11.714 INFO:tasks.workunit.client.0.smithi007.stdout:2/501: sync 2024-03-16T09:01:11.715 INFO:tasks.workunit.client.0.smithi007.stdout:1/386: creat d9/dd/d1b/d2d/d30/d45/f76 x:0 0 0 2024-03-16T09:01:11.716 INFO:tasks.workunit.client.0.smithi007.stdout:9/429: readlink d24/d4c/l51 0 2024-03-16T09:01:11.718 INFO:tasks.workunit.client.0.smithi007.stdout:9/430: unlink d24/f71 0 2024-03-16T09:01:11.728 INFO:tasks.workunit.client.0.smithi007.stdout:1/387: write d9/dd/d1b/f25 [16957,94926] 0 2024-03-16T09:01:11.728 INFO:tasks.workunit.client.0.smithi007.stdout:2/502: dread dc/f10 [0,4194304] 0 2024-03-16T09:01:11.729 INFO:tasks.workunit.client.0.smithi007.stdout:2/503: mknod dc/df/d24/d28/d2e/d4a/ca1 0 2024-03-16T09:01:11.729 INFO:tasks.workunit.client.0.smithi007.stdout:1/388: symlink d9/dd/d5e/l77 0 2024-03-16T09:01:11.730 INFO:tasks.workunit.client.0.smithi007.stdout:2/504: mkdir dc/df/d24/d28/d2e/d35/da2 0 2024-03-16T09:01:11.731 INFO:tasks.workunit.client.0.smithi007.stdout:2/505: mkdir dc/d27/da3 0 2024-03-16T09:01:11.731 INFO:tasks.workunit.client.0.smithi007.stdout:2/506: readlink lb 0 2024-03-16T09:01:11.732 INFO:tasks.workunit.client.0.smithi007.stdout:2/507: mkdir dc/df/d24/d28/d97/da4 0 2024-03-16T09:01:11.733 INFO:tasks.workunit.client.1.smithi019.stdout:1/110: dwrite f9 [0,4194304] 0 2024-03-16T09:01:11.733 INFO:tasks.workunit.client.1.smithi019.stdout:3/167: dwrite d3/f2f [4194304,4194304] 0 2024-03-16T09:01:11.733 INFO:tasks.workunit.client.0.smithi007.stdout:2/508: creat dc/fa5 x:0 0 0 2024-03-16T09:01:11.733 INFO:tasks.workunit.client.0.smithi007.stdout:2/509: readlink dc/df/d6c/l77 0 2024-03-16T09:01:11.734 INFO:tasks.workunit.client.0.smithi007.stdout:2/510: truncate dc/df/d24/d28/d2e/f85 601057 0 2024-03-16T09:01:11.734 INFO:tasks.workunit.client.0.smithi007.stdout:0/457: dwrite d3/d9/da/d92/f88 [0,4194304] 0 2024-03-16T09:01:11.734 INFO:tasks.workunit.client.0.smithi007.stdout:9/431: dwrite d24/f44 [0,4194304] 0 2024-03-16T09:01:11.734 INFO:tasks.workunit.client.0.smithi007.stdout:2/511: unlink dc/df/c3b 0 2024-03-16T09:01:11.735 INFO:tasks.workunit.client.1.smithi019.stdout:3/168: creat d3/d11/d22/d24/d2c/f38 x:0 0 0 2024-03-16T09:01:11.735 INFO:tasks.workunit.client.1.smithi019.stdout:3/169: readlink d3/l27 0 2024-03-16T09:01:11.736 INFO:tasks.workunit.client.1.smithi019.stdout:1/111: rmdir d16 39 2024-03-16T09:01:11.737 INFO:tasks.workunit.client.1.smithi019.stdout:1/112: mknod d16/c1d 0 2024-03-16T09:01:11.738 INFO:tasks.workunit.client.0.smithi007.stdout:9/432: mkdir d24/d4a/d55/d56/d7c 0 2024-03-16T09:01:11.738 INFO:tasks.workunit.client.0.smithi007.stdout:9/433: readlink l20 0 2024-03-16T09:01:11.738 INFO:tasks.workunit.client.1.smithi019.stdout:3/170: getdents d3/d17 0 2024-03-16T09:01:11.739 INFO:tasks.workunit.client.0.smithi007.stdout:2/512: creat dc/df/d24/d28/d97/da4/fa6 x:0 0 0 2024-03-16T09:01:11.739 INFO:tasks.workunit.client.1.smithi019.stdout:1/113: truncate f12 2596210 0 2024-03-16T09:01:11.740 INFO:tasks.workunit.client.0.smithi007.stdout:9/434: rename cb to d24/d4a/d61/c7d 0 2024-03-16T09:01:11.742 INFO:tasks.workunit.client.0.smithi007.stdout:2/513: mkdir dc/df/d24/d28/d2e/d35/da7 0 2024-03-16T09:01:11.742 INFO:tasks.workunit.client.0.smithi007.stdout:2/514: chown dc/d27 42954148 1 2024-03-16T09:01:11.742 INFO:tasks.workunit.client.1.smithi019.stdout:1/114: truncate f9 211635 0 2024-03-16T09:01:11.744 INFO:tasks.workunit.client.1.smithi019.stdout:1/115: creat d16/d18/f1e x:0 0 0 2024-03-16T09:01:11.749 INFO:tasks.workunit.client.0.smithi007.stdout:0/458: write d3/d9/da/d92/f88 [2119377,20063] 0 2024-03-16T09:01:11.749 INFO:tasks.workunit.client.0.smithi007.stdout:0/459: fsync d3/d9/da/d48/d25/d43/d1a/d58/d61/f8f 0 2024-03-16T09:01:11.751 INFO:tasks.workunit.client.0.smithi007.stdout:0/460: mknod d3/d9/da/d18/d27/d34/d3a/c97 0 2024-03-16T09:01:11.753 INFO:tasks.workunit.client.0.smithi007.stdout:0/461: creat d3/d9/da/d48/d25/d87/f98 x:0 0 0 2024-03-16T09:01:11.760 INFO:tasks.workunit.client.1.smithi019.stdout:1/116: dread f12 [0,4194304] 0 2024-03-16T09:01:11.760 INFO:tasks.workunit.client.0.smithi007.stdout:0/462: dread d3/d9/da/d48/d25/d43/d1a/d58/d61/f8f [0,4194304] 0 2024-03-16T09:01:11.760 INFO:tasks.workunit.client.0.smithi007.stdout:0/463: chown d3/d9/da/d18/f6a 90 1 2024-03-16T09:01:11.761 INFO:tasks.workunit.client.0.smithi007.stdout:0/464: mkdir d3/d9/da/d92/d99 0 2024-03-16T09:01:11.774 INFO:tasks.workunit.client.0.smithi007.stdout:9/435: dwrite d24/d4a/d55/d56/f5a [0,4194304] 0 2024-03-16T09:01:11.780 INFO:tasks.workunit.client.0.smithi007.stdout:9/436: dread d24/f31 [0,4194304] 0 2024-03-16T09:01:11.781 INFO:tasks.workunit.client.1.smithi019.stdout:4/216: sync 2024-03-16T09:01:11.781 INFO:tasks.workunit.client.1.smithi019.stdout:6/330: sync 2024-03-16T09:01:11.781 INFO:tasks.workunit.client.0.smithi007.stdout:9/437: symlink d24/d4a/d55/d56/d7c/l7e 0 2024-03-16T09:01:11.782 INFO:tasks.workunit.client.1.smithi019.stdout:3/171: dwrite d3/d17/f32 [0,4194304] 0 2024-03-16T09:01:11.782 INFO:tasks.workunit.client.1.smithi019.stdout:6/331: mkdir de/d1c/d1d/d66/d72 0 2024-03-16T09:01:11.783 INFO:tasks.workunit.client.1.smithi019.stdout:6/332: stat de/d26/d32/d4e/c61 0 2024-03-16T09:01:11.785 INFO:tasks.workunit.client.1.smithi019.stdout:3/172: rename d3/d11/d22/l35 to d3/d11/l39 0 2024-03-16T09:01:11.785 INFO:tasks.workunit.client.1.smithi019.stdout:6/333: mkdir de/d26/d73 0 2024-03-16T09:01:11.786 INFO:tasks.workunit.client.1.smithi019.stdout:3/173: symlink d3/d11/d22/d24/l3a 0 2024-03-16T09:01:11.787 INFO:tasks.workunit.client.1.smithi019.stdout:6/334: creat de/d1c/d1d/d66/f74 x:0 0 0 2024-03-16T09:01:11.787 INFO:tasks.workunit.client.1.smithi019.stdout:3/174: creat d3/d11/f3b x:0 0 0 2024-03-16T09:01:11.790 INFO:tasks.workunit.client.0.smithi007.stdout:0/465: dwrite d3/f8b [0,4194304] 0 2024-03-16T09:01:11.790 INFO:tasks.workunit.client.0.smithi007.stdout:0/466: chown d3/d9/db/d2c/f90 58562 1 2024-03-16T09:01:11.790 INFO:tasks.workunit.client.0.smithi007.stdout:0/467: creat d3/d9/da/d18/d27/f9a x:0 0 0 2024-03-16T09:01:11.790 INFO:tasks.workunit.client.1.smithi019.stdout:1/117: dwrite f14 [0,4194304] 0 2024-03-16T09:01:11.791 INFO:tasks.workunit.client.1.smithi019.stdout:1/118: write d16/d18/f1a [3350457,110526] 0 2024-03-16T09:01:11.792 INFO:tasks.workunit.client.1.smithi019.stdout:4/217: write d4/d6/f39 [8132594,67415] 0 2024-03-16T09:01:11.792 INFO:tasks.workunit.client.0.smithi007.stdout:9/438: dread f17 [0,4194304] 0 2024-03-16T09:01:11.794 INFO:tasks.workunit.client.1.smithi019.stdout:1/119: mkdir d16/d1f 0 2024-03-16T09:01:11.794 INFO:tasks.workunit.client.1.smithi019.stdout:4/218: symlink d4/d6/l4c 0 2024-03-16T09:01:11.796 INFO:tasks.workunit.client.0.smithi007.stdout:9/439: mkdir d24/d4a/d55/d56/d7c/d7f 0 2024-03-16T09:01:11.796 INFO:tasks.workunit.client.0.smithi007.stdout:9/440: fsync d24/d4a/d61/f6f 0 2024-03-16T09:01:11.797 INFO:tasks.workunit.client.0.smithi007.stdout:9/441: read - d24/d4a/d55/f5d zero size 2024-03-16T09:01:11.797 INFO:tasks.workunit.client.0.smithi007.stdout:9/442: chown d24/f50 3 1 2024-03-16T09:01:11.797 INFO:tasks.workunit.client.1.smithi019.stdout:4/219: write d4/d6/f24 [1364049,110835] 0 2024-03-16T09:01:11.797 INFO:tasks.workunit.client.0.smithi007.stdout:9/443: rmdir d24/d4a/d55/d56/d7c 39 2024-03-16T09:01:11.806 INFO:tasks.workunit.client.1.smithi019.stdout:1/120: write f14 [1018734,24726] 0 2024-03-16T09:01:11.808 INFO:tasks.workunit.client.1.smithi019.stdout:1/121: creat d16/f20 x:0 0 0 2024-03-16T09:01:11.809 INFO:tasks.workunit.client.1.smithi019.stdout:4/220: write d4/d6/f39 [1384370,82081] 0 2024-03-16T09:01:11.809 INFO:tasks.workunit.client.1.smithi019.stdout:4/221: dread - d4/d6/d15/d1f/d2c/f40 zero size 2024-03-16T09:01:11.809 INFO:tasks.workunit.client.1.smithi019.stdout:4/222: write d4/d6/d15/d1f/d2c/f40 [192089,66266] 0 2024-03-16T09:01:11.809 INFO:tasks.workunit.client.1.smithi019.stdout:1/122: symlink d16/l21 0 2024-03-16T09:01:11.813 INFO:tasks.workunit.client.1.smithi019.stdout:3/175: dwrite d3/f2f [0,4194304] 0 2024-03-16T09:01:11.815 INFO:tasks.workunit.client.0.smithi007.stdout:0/468: dwrite d3/d9/da/d18/d23/f75 [0,4194304] 0 2024-03-16T09:01:11.815 INFO:tasks.workunit.client.0.smithi007.stdout:0/469: dread - d3/d9/db/f94 zero size 2024-03-16T09:01:11.815 INFO:tasks.workunit.client.0.smithi007.stdout:0/470: dread - d3/d9/da/d18/d27/f9a zero size 2024-03-16T09:01:11.816 INFO:tasks.workunit.client.1.smithi019.stdout:3/176: rename d3/d11/d22/d2e to d3/d11/d22/d3c 0 2024-03-16T09:01:11.817 INFO:tasks.workunit.client.1.smithi019.stdout:3/177: symlink d3/d11/d22/d24/l3d 0 2024-03-16T09:01:11.818 INFO:tasks.workunit.client.0.smithi007.stdout:0/471: link d3/d9/l69 d3/d9/da/d18/d27/l9b 0 2024-03-16T09:01:11.821 INFO:tasks.workunit.client.0.smithi007.stdout:9/444: dwrite f1e [4194304,4194304] 0 2024-03-16T09:01:11.821 INFO:tasks.workunit.client.0.smithi007.stdout:9/445: readlink lc 0 2024-03-16T09:01:11.822 INFO:tasks.workunit.client.0.smithi007.stdout:0/472: mknod d3/d1f/d70/d7c/c9c 0 2024-03-16T09:01:11.826 INFO:tasks.workunit.client.0.smithi007.stdout:9/446: mkdir d24/d4a/d55/d56/d7c/d7f/d80 0 2024-03-16T09:01:11.828 INFO:tasks.workunit.client.0.smithi007.stdout:0/473: unlink d3/d9/da/d92/f88 0 2024-03-16T09:01:11.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/474: symlink d3/d9/da/d18/l9d 0 2024-03-16T09:01:11.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/475: dread - d3/d9/da/d18/f6a zero size 2024-03-16T09:01:11.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/476: write d3/f6f [592436,56823] 0 2024-03-16T09:01:11.831 INFO:tasks.workunit.client.0.smithi007.stdout:9/447: getdents d24/d4a/d55/d56 0 2024-03-16T09:01:11.831 INFO:tasks.workunit.client.0.smithi007.stdout:9/448: fdatasync d24/d4a/d55/d56/f77 0 2024-03-16T09:01:11.834 INFO:tasks.workunit.client.0.smithi007.stdout:0/477: rmdir d3/d9/db/d60/d8a 0 2024-03-16T09:01:11.834 INFO:tasks.workunit.client.0.smithi007.stdout:9/449: mknod d24/d4a/c81 0 2024-03-16T09:01:11.835 INFO:tasks.workunit.client.0.smithi007.stdout:0/478: unlink d3/d9/da/d18/d27/d34/d3a/c97 0 2024-03-16T09:01:11.835 INFO:tasks.workunit.client.0.smithi007.stdout:0/479: truncate d3/d9/da/d48/f4e 873608 0 2024-03-16T09:01:11.836 INFO:tasks.workunit.client.0.smithi007.stdout:0/480: mkdir d3/d9/da/d48/d9e 0 2024-03-16T09:01:11.836 INFO:tasks.workunit.client.0.smithi007.stdout:0/481: readlink d3/d9/l69 0 2024-03-16T09:01:11.842 INFO:tasks.workunit.client.1.smithi019.stdout:3/178: dwrite d3/f2d [0,4194304] 0 2024-03-16T09:01:11.842 INFO:tasks.workunit.client.1.smithi019.stdout:3/179: readlink d3/d11/d22/d24/l3a 0 2024-03-16T09:01:11.843 INFO:tasks.workunit.client.0.smithi007.stdout:9/450: write d24/f50 [2427873,119003] 0 2024-03-16T09:01:11.844 INFO:tasks.workunit.client.0.smithi007.stdout:9/451: symlink d24/d4a/d55/l82 0 2024-03-16T09:01:11.844 INFO:tasks.workunit.client.0.smithi007.stdout:9/452: chown d24/l6d 230151155 1 2024-03-16T09:01:11.845 INFO:tasks.workunit.client.0.smithi007.stdout:9/453: symlink d24/d4a/d55/d56/d7c/d7f/d80/l83 0 2024-03-16T09:01:11.846 INFO:tasks.workunit.client.0.smithi007.stdout:9/454: creat d24/d4a/f84 x:0 0 0 2024-03-16T09:01:11.850 INFO:tasks.workunit.client.0.smithi007.stdout:0/482: dread d3/f22 [0,4194304] 0 2024-03-16T09:01:11.851 INFO:tasks.workunit.client.0.smithi007.stdout:0/483: mkdir d3/d9/da/d92/d9f 0 2024-03-16T09:01:11.852 INFO:tasks.workunit.client.1.smithi019.stdout:4/223: dwrite d4/d1b/d2a/d3d/f48 [0,4194304] 0 2024-03-16T09:01:11.852 INFO:tasks.workunit.client.1.smithi019.stdout:1/123: dwrite f9 [0,4194304] 0 2024-03-16T09:01:11.855 INFO:tasks.workunit.client.1.smithi019.stdout:4/224: creat d4/d1b/d2a/f4d x:0 0 0 2024-03-16T09:01:11.856 INFO:tasks.workunit.client.1.smithi019.stdout:3/180: dread d3/f2d [0,4194304] 0 2024-03-16T09:01:11.856 INFO:tasks.workunit.client.1.smithi019.stdout:3/181: read - d3/f25 zero size 2024-03-16T09:01:11.856 INFO:tasks.workunit.client.1.smithi019.stdout:3/182: dread - d3/d11/f3b zero size 2024-03-16T09:01:11.857 INFO:tasks.workunit.client.0.smithi007.stdout:0/484: getdents d3/d9/da/d18 0 2024-03-16T09:01:11.857 INFO:tasks.workunit.client.0.smithi007.stdout:0/485: chown d3/d9/db/d2c/d5b 130940 1 2024-03-16T09:01:11.859 INFO:tasks.workunit.client.1.smithi019.stdout:4/225: dread d4/f26 [0,4194304] 0 2024-03-16T09:01:11.860 INFO:tasks.workunit.client.1.smithi019.stdout:4/226: dread d4/d1b/d2a/d3d/f46 [0,4194304] 0 2024-03-16T09:01:11.860 INFO:tasks.workunit.client.1.smithi019.stdout:4/227: chown d4/d6/d15/c2f 33 1 2024-03-16T09:01:11.861 INFO:tasks.workunit.client.1.smithi019.stdout:3/183: dread d3/f2d [0,4194304] 0 2024-03-16T09:01:11.861 INFO:tasks.workunit.client.1.smithi019.stdout:3/184: stat d3/d17/c2b 0 2024-03-16T09:01:11.861 INFO:tasks.workunit.client.1.smithi019.stdout:3/185: fsync d3/d11/f3b 0 2024-03-16T09:01:11.863 INFO:tasks.workunit.client.0.smithi007.stdout:9/455: dwrite d24/d4a/f66 [0,4194304] 0 2024-03-16T09:01:11.864 INFO:tasks.workunit.client.1.smithi019.stdout:4/228: rename d4/d1b/d2a/d3d/f48 to d4/d1b/d2a/f4e 0 2024-03-16T09:01:11.864 INFO:tasks.workunit.client.1.smithi019.stdout:4/229: readlink d4/d6/d15/l25 0 2024-03-16T09:01:11.866 INFO:tasks.workunit.client.0.smithi007.stdout:9/456: stat d24/l30 0 2024-03-16T09:01:11.866 INFO:tasks.workunit.client.0.smithi007.stdout:9/457: chown d24/f33 12 1 2024-03-16T09:01:11.868 INFO:tasks.workunit.client.1.smithi019.stdout:2/170: sync 2024-03-16T09:01:11.868 INFO:tasks.workunit.client.1.smithi019.stdout:8/190: sync 2024-03-16T09:01:11.868 INFO:tasks.workunit.client.1.smithi019.stdout:8/191: write d6/de/f37 [490273,99908] 0 2024-03-16T09:01:11.868 INFO:tasks.workunit.client.1.smithi019.stdout:3/186: write d3/f30 [4830468,15743] 0 2024-03-16T09:01:11.868 INFO:tasks.workunit.client.1.smithi019.stdout:3/187: creat d3/f3e x:0 0 0 2024-03-16T09:01:11.869 INFO:tasks.workunit.client.1.smithi019.stdout:2/171: mknod d5/d6/c38 0 2024-03-16T09:01:11.869 INFO:tasks.workunit.client.1.smithi019.stdout:8/192: mkdir d6/d2a/d3f 0 2024-03-16T09:01:11.870 INFO:tasks.workunit.client.1.smithi019.stdout:1/124: dwrite d16/f1c [4194304,4194304] 0 2024-03-16T09:01:11.871 INFO:tasks.workunit.client.0.smithi007.stdout:9/458: dread d24/d4a/f78 [0,4194304] 0 2024-03-16T09:01:11.872 INFO:tasks.workunit.client.1.smithi019.stdout:8/193: rename d6/d2a/f36 to d6/de/d24/f40 0 2024-03-16T09:01:11.872 INFO:tasks.workunit.client.1.smithi019.stdout:8/194: write d6/de/f1b [2831217,312] 0 2024-03-16T09:01:11.872 INFO:tasks.workunit.client.1.smithi019.stdout:8/195: fdatasync d6/f15 0 2024-03-16T09:01:11.872 INFO:tasks.workunit.client.1.smithi019.stdout:8/196: chown d6/f27 2016 1 2024-03-16T09:01:11.873 INFO:tasks.workunit.client.1.smithi019.stdout:4/230: dread f2 [0,4194304] 0 2024-03-16T09:01:11.874 INFO:tasks.workunit.client.0.smithi007.stdout:9/459: link d24/l6d d24/d4a/d55/d56/l85 0 2024-03-16T09:01:11.878 INFO:tasks.workunit.client.0.smithi007.stdout:0/486: dwrite d3/d9/da/d18/d23/f32 [0,4194304] 0 2024-03-16T09:01:11.883 INFO:tasks.workunit.client.1.smithi019.stdout:3/188: read d3/ff [2459880,99165] 0 2024-03-16T09:01:11.883 INFO:tasks.workunit.client.1.smithi019.stdout:3/189: readlink d3/d1b/l1d 0 2024-03-16T09:01:11.884 INFO:tasks.workunit.client.0.smithi007.stdout:4/320: sync 2024-03-16T09:01:11.884 INFO:tasks.workunit.client.0.smithi007.stdout:4/321: truncate d4/d14/d64/d68/f63 455505 0 2024-03-16T09:01:11.886 INFO:tasks.workunit.client.0.smithi007.stdout:6/302: sync 2024-03-16T09:01:11.886 INFO:tasks.workunit.client.0.smithi007.stdout:3/452: sync 2024-03-16T09:01:11.886 INFO:tasks.workunit.client.0.smithi007.stdout:6/303: read - d2/d10/d12/f60 zero size 2024-03-16T09:01:11.886 INFO:tasks.workunit.client.0.smithi007.stdout:9/460: write fd [1892411,12687] 0 2024-03-16T09:01:11.886 INFO:tasks.workunit.client.0.smithi007.stdout:4/322: write d4/d14/f35 [422005,38455] 0 2024-03-16T09:01:11.888 INFO:tasks.workunit.client.0.smithi007.stdout:3/453: rename d15/d18/f19 to d15/d18/d1f/d3b/d87/f92 0 2024-03-16T09:01:11.889 INFO:tasks.workunit.client.0.smithi007.stdout:4/323: symlink d4/d14/d64/l6c 0 2024-03-16T09:01:11.898 INFO:tasks.workunit.client.0.smithi007.stdout:3/454: write d15/d18/f35 [4644293,60752] 0 2024-03-16T09:01:11.898 INFO:tasks.workunit.client.0.smithi007.stdout:3/455: write d15/d18/d1f/d29/d43/f7a [839320,66929] 0 2024-03-16T09:01:11.898 INFO:tasks.workunit.client.1.smithi019.stdout:3/190: dread d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:11.899 INFO:tasks.workunit.client.0.smithi007.stdout:4/324: read d4/d14/d1c/f3e [1014201,85370] 0 2024-03-16T09:01:11.899 INFO:tasks.workunit.client.1.smithi019.stdout:3/191: mkdir d3/d17/d3f 0 2024-03-16T09:01:11.900 INFO:tasks.workunit.client.0.smithi007.stdout:4/325: creat d4/d7/d6b/f6d x:0 0 0 2024-03-16T09:01:11.903 INFO:tasks.workunit.client.1.smithi019.stdout:3/192: rename d3/d11/d22 to d3/d1b/d40 0 2024-03-16T09:01:11.903 INFO:tasks.workunit.client.1.smithi019.stdout:3/193: truncate d3/f30 6956266 0 2024-03-16T09:01:11.906 INFO:tasks.workunit.client.0.smithi007.stdout:0/487: dwrite d3/d9/da/d18/d23/d36/f86 [0,4194304] 0 2024-03-16T09:01:11.908 INFO:tasks.workunit.client.1.smithi019.stdout:2/172: dwrite d5/de/fd [0,4194304] 0 2024-03-16T09:01:11.908 INFO:tasks.workunit.client.1.smithi019.stdout:2/173: fsync d5/d6/d2b/f18 0 2024-03-16T09:01:11.908 INFO:tasks.workunit.client.1.smithi019.stdout:2/174: dread - d5/f13 zero size 2024-03-16T09:01:11.908 INFO:tasks.workunit.client.1.smithi019.stdout:2/175: truncate d5/d6/f36 922223 0 2024-03-16T09:01:11.908 INFO:tasks.workunit.client.0.smithi007.stdout:6/304: dwrite d2/d11/d1e/d54/f63 [0,4194304] 0 2024-03-16T09:01:11.909 INFO:tasks.workunit.client.1.smithi019.stdout:8/197: dwrite d6/de/d16/f32 [0,4194304] 0 2024-03-16T09:01:11.910 INFO:tasks.workunit.client.1.smithi019.stdout:2/176: creat d5/d6/d2b/d2f/d35/f39 x:0 0 0 2024-03-16T09:01:11.910 INFO:tasks.workunit.client.1.smithi019.stdout:2/177: fsync d5/d6/d2b/f1e 0 2024-03-16T09:01:11.913 INFO:tasks.workunit.client.1.smithi019.stdout:8/198: unlink d6/de/f29 0 2024-03-16T09:01:11.913 INFO:tasks.workunit.client.1.smithi019.stdout:8/199: write d6/de/d16/d19/f1a [2515081,28223] 0 2024-03-16T09:01:11.913 INFO:tasks.workunit.client.1.smithi019.stdout:8/200: readlink d6/l20 0 2024-03-16T09:01:11.913 INFO:tasks.workunit.client.1.smithi019.stdout:3/194: write d3/d17/f32 [2112636,72877] 0 2024-03-16T09:01:11.915 INFO:tasks.workunit.client.0.smithi007.stdout:0/488: truncate d3/d9/db/fc 2212300 0 2024-03-16T09:01:11.916 INFO:tasks.workunit.client.1.smithi019.stdout:8/201: rename d6/de/d16/d19/c2f to d6/de/d16/d2c/c41 0 2024-03-16T09:01:11.919 INFO:tasks.workunit.client.0.smithi007.stdout:6/305: symlink d2/d29/l66 0 2024-03-16T09:01:11.919 INFO:tasks.workunit.client.1.smithi019.stdout:3/195: getdents d3/d1b/d40/d24 0 2024-03-16T09:01:11.924 INFO:tasks.workunit.client.0.smithi007.stdout:3/456: dread d15/d18/f35 [0,4194304] 0 2024-03-16T09:01:11.924 INFO:tasks.workunit.client.0.smithi007.stdout:3/457: fdatasync d15/d18/d1f/d29/d43/f55 0 2024-03-16T09:01:11.924 INFO:tasks.workunit.client.0.smithi007.stdout:3/458: write d15/d18/f35 [4829003,12192] 0 2024-03-16T09:01:11.926 INFO:tasks.workunit.client.0.smithi007.stdout:4/326: dwrite d4/d14/d1c/f26 [0,4194304] 0 2024-03-16T09:01:11.931 INFO:tasks.workunit.client.1.smithi019.stdout:3/196: write d3/f2f [7111391,8009] 0 2024-03-16T09:01:11.934 INFO:tasks.workunit.client.1.smithi019.stdout:2/178: read d5/d6/d2b/f1e [1012723,65460] 0 2024-03-16T09:01:11.934 INFO:tasks.workunit.client.1.smithi019.stdout:2/179: fdatasync d5/d6/f11 0 2024-03-16T09:01:11.934 INFO:tasks.workunit.client.1.smithi019.stdout:2/180: write d5/d6/d2b/f1e [155967,66379] 0 2024-03-16T09:01:11.935 INFO:tasks.workunit.client.0.smithi007.stdout:6/306: dread d2/d13/d1f/f4f [0,4194304] 0 2024-03-16T09:01:11.936 INFO:tasks.workunit.client.0.smithi007.stdout:6/307: creat d2/d13/d1f/f67 x:0 0 0 2024-03-16T09:01:11.938 INFO:tasks.workunit.client.1.smithi019.stdout:3/197: rename d3/f2d to d3/d1b/d40/d24/f41 0 2024-03-16T09:01:11.942 INFO:tasks.workunit.client.0.smithi007.stdout:9/461: dwrite d24/f3d [0,4194304] 0 2024-03-16T09:01:11.952 INFO:tasks.workunit.client.1.smithi019.stdout:2/181: read d5/f7 [3304377,51297] 0 2024-03-16T09:01:11.953 INFO:tasks.workunit.client.0.smithi007.stdout:6/308: write d2/d11/f26 [2660156,11335] 0 2024-03-16T09:01:11.967 INFO:tasks.workunit.client.0.smithi007.stdout:0/489: dwrite d3/d9/da/d48/f95 [0,4194304] 0 2024-03-16T09:01:11.974 INFO:tasks.workunit.client.0.smithi007.stdout:6/309: dread d2/d11/f25 [0,4194304] 0 2024-03-16T09:01:11.986 INFO:tasks.workunit.client.1.smithi019.stdout:8/202: dwrite d6/de/f1b [4194304,4194304] 0 2024-03-16T09:01:11.987 INFO:tasks.workunit.client.1.smithi019.stdout:8/203: getdents d6 0 2024-03-16T09:01:11.988 INFO:tasks.workunit.client.0.smithi007.stdout:6/310: dread d2/fd [0,4194304] 0 2024-03-16T09:01:11.988 INFO:tasks.workunit.client.0.smithi007.stdout:6/311: write d2/d13/d3f/d50/f53 [4693420,39179] 0 2024-03-16T09:01:11.988 INFO:tasks.workunit.client.0.smithi007.stdout:6/312: chown d2/d11/l30 97300 1 2024-03-16T09:01:11.989 INFO:tasks.workunit.client.0.smithi007.stdout:3/459: dwrite d15/d18/d1f/f2e [4194304,4194304] 0 2024-03-16T09:01:11.989 INFO:tasks.workunit.client.0.smithi007.stdout:3/460: truncate d15/d18/d1f/d29/d43/f81 1042845 0 2024-03-16T09:01:11.989 INFO:tasks.workunit.client.0.smithi007.stdout:3/461: read - d15/d18/d1f/d29/f5b zero size 2024-03-16T09:01:11.990 INFO:tasks.workunit.client.1.smithi019.stdout:3/198: dwrite d3/fc [4194304,4194304] 0 2024-03-16T09:01:11.991 INFO:tasks.workunit.client.0.smithi007.stdout:6/313: chown d2/d11/l32 4 1 2024-03-16T09:01:11.991 INFO:tasks.workunit.client.0.smithi007.stdout:6/314: fdatasync d2/f7 0 2024-03-16T09:01:11.991 INFO:tasks.workunit.client.0.smithi007.stdout:6/315: write d2/d13/d1f/f5c [36055,77443] 0 2024-03-16T09:01:11.991 INFO:tasks.workunit.client.0.smithi007.stdout:0/490: dwrite d3/d9/db/d2c/d5b/f96 [0,4194304] 0 2024-03-16T09:01:11.991 INFO:tasks.workunit.client.0.smithi007.stdout:0/491: write d3/d9/da/d18/d27/f9a [363007,83730] 0 2024-03-16T09:01:11.991 INFO:tasks.workunit.client.0.smithi007.stdout:9/462: dwrite d24/f3d [0,4194304] 0 2024-03-16T09:01:11.991 INFO:tasks.workunit.client.0.smithi007.stdout:9/463: chown d24/c2f 1 1 2024-03-16T09:01:11.992 INFO:tasks.workunit.client.0.smithi007.stdout:9/464: chown f7 205 1 2024-03-16T09:01:11.992 INFO:tasks.workunit.client.0.smithi007.stdout:4/327: dwrite d4/f10 [0,4194304] 0 2024-03-16T09:01:11.992 INFO:tasks.workunit.client.0.smithi007.stdout:4/328: write d4/d14/d1c/f3e [227305,118065] 0 2024-03-16T09:01:11.992 INFO:tasks.workunit.client.0.smithi007.stdout:4/329: chown d4/d14/d1c/d1e/f53 9426 1 2024-03-16T09:01:11.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/462: truncate d15/d18/d1f/d29/f1b 125497 0 2024-03-16T09:01:11.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/463: write d15/d18/f22 [3400967,24622] 0 2024-03-16T09:01:11.993 INFO:tasks.workunit.client.0.smithi007.stdout:3/464: creat d15/d18/d1f/d3b/d63/f93 x:0 0 0 2024-03-16T09:01:11.993 INFO:tasks.workunit.client.1.smithi019.stdout:3/199: rmdir d3/d1b/d40/d3c 0 2024-03-16T09:01:11.993 INFO:tasks.workunit.client.1.smithi019.stdout:3/200: rename d3 to d3/d1b/d40/d24/d2c/d42 22 2024-03-16T09:01:11.993 INFO:tasks.workunit.client.1.smithi019.stdout:3/201: chown d3/d11/f1f 0 1 2024-03-16T09:01:11.994 INFO:tasks.workunit.client.0.smithi007.stdout:6/316: creat d2/d13/d62/f68 x:0 0 0 2024-03-16T09:01:11.994 INFO:tasks.workunit.client.0.smithi007.stdout:6/317: dread - d2/d13/d62/f68 zero size 2024-03-16T09:01:11.994 INFO:tasks.workunit.client.0.smithi007.stdout:6/318: creat d2/d13/d1f/d34/f69 x:0 0 0 2024-03-16T09:01:11.997 INFO:tasks.workunit.client.1.smithi019.stdout:3/202: rename d3/f2f to d3/f43 0 2024-03-16T09:01:11.997 INFO:tasks.workunit.client.1.smithi019.stdout:3/203: dread - d3/d1b/d40/d24/d2c/f38 zero size 2024-03-16T09:01:11.997 INFO:tasks.workunit.client.0.smithi007.stdout:0/492: link d3/d9/da/d48/d25/f68 d3/d6c/fa0 0 2024-03-16T09:01:11.997 INFO:tasks.workunit.client.0.smithi007.stdout:0/493: dread - d3/d9/da/d48/d25/d43/d1a/d58/d61/f7a zero size 2024-03-16T09:01:12.002 INFO:tasks.workunit.client.1.smithi019.stdout:9/160: sync 2024-03-16T09:01:12.002 INFO:tasks.workunit.client.1.smithi019.stdout:7/143: sync 2024-03-16T09:01:12.002 INFO:tasks.workunit.client.1.smithi019.stdout:0/178: sync 2024-03-16T09:01:12.002 INFO:tasks.workunit.client.1.smithi019.stdout:5/160: sync 2024-03-16T09:01:12.004 INFO:tasks.workunit.client.1.smithi019.stdout:0/179: mknod d7/c38 0 2024-03-16T09:01:12.004 INFO:tasks.workunit.client.1.smithi019.stdout:7/144: write d9/da/d13/f17 [2688763,4571] 0 2024-03-16T09:01:12.005 INFO:tasks.workunit.client.1.smithi019.stdout:5/161: rename d6/lc to d6/l2b 0 2024-03-16T09:01:12.006 INFO:tasks.workunit.client.1.smithi019.stdout:0/180: truncate d7/fd 4442272 0 2024-03-16T09:01:12.007 INFO:tasks.workunit.client.1.smithi019.stdout:3/204: dread f2 [0,4194304] 0 2024-03-16T09:01:12.009 INFO:tasks.workunit.client.0.smithi007.stdout:4/330: mknod d4/d7/c6e 0 2024-03-16T09:01:12.010 INFO:tasks.workunit.client.1.smithi019.stdout:7/145: unlink d9/da/d13/d18/d1a/l22 0 2024-03-16T09:01:12.010 INFO:tasks.workunit.client.0.smithi007.stdout:3/465: dread d15/d18/d1f/d3b/d87/f92 [0,4194304] 0 2024-03-16T09:01:12.011 INFO:tasks.workunit.client.0.smithi007.stdout:3/466: readlink d15/l40 0 2024-03-16T09:01:12.011 INFO:tasks.workunit.client.0.smithi007.stdout:3/467: readlink d15/d18/d1f/d77/l89 0 2024-03-16T09:01:12.011 INFO:tasks.workunit.client.0.smithi007.stdout:3/468: creat d15/d18/d1f/d29/d7c/f94 x:0 0 0 2024-03-16T09:01:12.011 INFO:tasks.workunit.client.1.smithi019.stdout:5/162: rename d6/l14 to d6/d19/l2c 0 2024-03-16T09:01:12.011 INFO:tasks.workunit.client.0.smithi007.stdout:6/319: creat d2/d13/d24/f6a x:0 0 0 2024-03-16T09:01:12.012 INFO:tasks.workunit.client.0.smithi007.stdout:9/465: dwrite d24/d4a/d55/f5b [0,4194304] 0 2024-03-16T09:01:12.013 INFO:tasks.workunit.client.0.smithi007.stdout:4/331: rmdir d4/d14/d64/d68 39 2024-03-16T09:01:12.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:11 smithi007 ceph-mon[29470]: pgmap v19: 65 pgs: 65 active+clean; 932 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 89 MiB/s rd, 118 MiB/s wr, 1.10k op/s 2024-03-16T09:01:12.019 INFO:tasks.workunit.client.1.smithi019.stdout:3/205: dread d3/d17/f32 [0,4194304] 0 2024-03-16T09:01:12.019 INFO:tasks.workunit.client.1.smithi019.stdout:8/204: dwrite d6/f15 [4194304,4194304] 0 2024-03-16T09:01:12.021 INFO:tasks.workunit.client.0.smithi007.stdout:3/469: write d15/d18/d1f/d29/d43/f4e [923564,11801] 0 2024-03-16T09:01:12.021 INFO:tasks.workunit.client.0.smithi007.stdout:0/494: dwrite d3/d9/da/d18/d23/f63 [0,4194304] 0 2024-03-16T09:01:12.021 INFO:tasks.workunit.client.0.smithi007.stdout:0/495: creat d3/fa1 x:0 0 0 2024-03-16T09:01:12.021 INFO:tasks.workunit.client.0.smithi007.stdout:0/496: readlink d3/d9/da/d48/d25/d87/l3b 0 2024-03-16T09:01:12.022 INFO:tasks.workunit.client.0.smithi007.stdout:4/332: creat d4/d14/d64/d68/f6f x:0 0 0 2024-03-16T09:01:12.022 INFO:tasks.workunit.client.0.smithi007.stdout:4/333: dread - d4/d7/d6b/f6d zero size 2024-03-16T09:01:12.023 INFO:tasks.workunit.client.1.smithi019.stdout:7/146: dread f8 [0,4194304] 0 2024-03-16T09:01:12.023 INFO:tasks.workunit.client.1.smithi019.stdout:3/206: rename d3/d1b/d40/l31 to d3/d17/d3f/l44 0 2024-03-16T09:01:12.023 INFO:tasks.workunit.client.1.smithi019.stdout:3/207: readlink d3/d1b/d40/d24/d2c/l36 0 2024-03-16T09:01:12.023 INFO:tasks.workunit.client.0.smithi007.stdout:6/320: rmdir d2/d13/d1f/d4e 39 2024-03-16T09:01:12.024 INFO:tasks.workunit.client.1.smithi019.stdout:3/208: truncate d3/f30 5644266 0 2024-03-16T09:01:12.024 INFO:tasks.workunit.client.1.smithi019.stdout:3/209: fsync d3/d1b/f1c 0 2024-03-16T09:01:12.024 INFO:tasks.workunit.client.1.smithi019.stdout:3/210: chown d3/d11/f37 31419654 1 2024-03-16T09:01:12.025 INFO:tasks.workunit.client.0.smithi007.stdout:6/321: dread d2/d10/d12/f35 [0,4194304] 0 2024-03-16T09:01:12.025 INFO:tasks.workunit.client.0.smithi007.stdout:6/322: creat d2/d13/d62/f6b x:0 0 0 2024-03-16T09:01:12.026 INFO:tasks.workunit.client.1.smithi019.stdout:3/211: mknod d3/d17/d3f/c45 0 2024-03-16T09:01:12.026 INFO:tasks.workunit.client.1.smithi019.stdout:3/212: dread - d3/d11/f3b zero size 2024-03-16T09:01:12.026 INFO:tasks.workunit.client.1.smithi019.stdout:3/213: chown d3/d11/f3b 682 1 2024-03-16T09:01:12.026 INFO:tasks.workunit.client.1.smithi019.stdout:3/214: readlink d3/d17/d3f/l44 0 2024-03-16T09:01:12.027 INFO:tasks.workunit.client.0.smithi007.stdout:9/466: mkdir d24/d4c/d86 0 2024-03-16T09:01:12.029 INFO:tasks.workunit.client.0.smithi007.stdout:9/467: dread d24/f5c [0,4194304] 0 2024-03-16T09:01:12.030 INFO:tasks.workunit.client.0.smithi007.stdout:0/497: unlink d3/d9/da/d92/f8e 0 2024-03-16T09:01:12.031 INFO:tasks.workunit.client.0.smithi007.stdout:5/408: sync 2024-03-16T09:01:12.031 INFO:tasks.workunit.client.0.smithi007.stdout:7/291: sync 2024-03-16T09:01:12.031 INFO:tasks.workunit.client.0.smithi007.stdout:8/657: sync 2024-03-16T09:01:12.031 INFO:tasks.workunit.client.0.smithi007.stdout:1/389: sync 2024-03-16T09:01:12.031 INFO:tasks.workunit.client.0.smithi007.stdout:2/515: sync 2024-03-16T09:01:12.031 INFO:tasks.workunit.client.1.smithi019.stdout:0/181: read d7/fd [3279916,86178] 0 2024-03-16T09:01:12.033 INFO:tasks.workunit.client.0.smithi007.stdout:4/334: creat d4/d7/f70 x:0 0 0 2024-03-16T09:01:12.034 INFO:tasks.workunit.client.0.smithi007.stdout:9/468: mkdir d24/d4a/d55/d56/d7c/d7f/d80/d87 0 2024-03-16T09:01:12.035 INFO:tasks.workunit.client.1.smithi019.stdout:0/182: creat d7/d1d/d2e/f39 x:0 0 0 2024-03-16T09:01:12.035 INFO:tasks.workunit.client.1.smithi019.stdout:0/183: read d7/fd [3484101,56514] 0 2024-03-16T09:01:12.035 INFO:tasks.workunit.client.1.smithi019.stdout:8/205: dread f2 [0,4194304] 0 2024-03-16T09:01:12.039 INFO:tasks.workunit.client.0.smithi007.stdout:9/469: rmdir d24/d4a/d55/d56/d7c/d7f/d80 39 2024-03-16T09:01:12.040 INFO:tasks.workunit.client.1.smithi019.stdout:0/184: creat d7/d29/f3a x:0 0 0 2024-03-16T09:01:12.040 INFO:tasks.workunit.client.0.smithi007.stdout:8/658: symlink d0/d2/le7 0 2024-03-16T09:01:12.041 INFO:tasks.workunit.client.0.smithi007.stdout:6/323: write d2/d10/d12/d15/f45 [2034317,84608] 0 2024-03-16T09:01:12.041 INFO:tasks.workunit.client.1.smithi019.stdout:8/206: creat d6/de/d16/f42 x:0 0 0 2024-03-16T09:01:12.041 INFO:tasks.workunit.client.1.smithi019.stdout:8/207: fdatasync d6/de/d24/d30/f3b 0 2024-03-16T09:01:12.043 INFO:tasks.workunit.client.0.smithi007.stdout:1/390: truncate d9/dd/d1b/d2d/f44 3879397 0 2024-03-16T09:01:12.043 INFO:tasks.workunit.client.0.smithi007.stdout:1/391: chown d9/dd/d1b/d28/f60 0 1 2024-03-16T09:01:12.044 INFO:tasks.workunit.client.0.smithi007.stdout:7/292: dread d0/d19/f2d [0,4194304] 0 2024-03-16T09:01:12.044 INFO:tasks.workunit.client.0.smithi007.stdout:7/293: fdatasync d0/f11 0 2024-03-16T09:01:12.046 INFO:tasks.workunit.client.0.smithi007.stdout:8/659: rename d0/dc/d54/le0 to d0/d26/d2d/le8 0 2024-03-16T09:01:12.051 INFO:tasks.workunit.client.0.smithi007.stdout:1/392: chown d9/dd/d1b/d2d/f3f 15 1 2024-03-16T09:01:12.051 INFO:tasks.workunit.client.0.smithi007.stdout:7/294: creat d0/dc/d24/f5e x:0 0 0 2024-03-16T09:01:12.055 INFO:tasks.workunit.client.0.smithi007.stdout:8/660: rename d0/c34 to d0/d2/d7e/ce9 0 2024-03-16T09:01:12.055 INFO:tasks.workunit.client.1.smithi019.stdout:0/185: read d7/d8/d22/f25 [1510845,112124] 0 2024-03-16T09:01:12.056 INFO:tasks.workunit.client.0.smithi007.stdout:9/470: dread d24/d4a/d55/f5b [0,4194304] 0 2024-03-16T09:01:12.056 INFO:tasks.workunit.client.0.smithi007.stdout:9/471: read d24/f27 [1134341,11591] 0 2024-03-16T09:01:12.056 INFO:tasks.workunit.client.0.smithi007.stdout:9/472: write d24/f54 [4469017,127492] 0 2024-03-16T09:01:12.056 INFO:tasks.workunit.client.0.smithi007.stdout:9/473: readlink d24/d4a/d55/l82 0 2024-03-16T09:01:12.057 INFO:tasks.workunit.client.1.smithi019.stdout:0/186: creat d7/d8/d32/d36/f3b x:0 0 0 2024-03-16T09:01:12.057 INFO:tasks.workunit.client.1.smithi019.stdout:0/187: creat d7/d1d/d2e/f3c x:0 0 0 2024-03-16T09:01:12.058 INFO:tasks.workunit.client.0.smithi007.stdout:9/474: creat d24/d4a/d55/d56/d7c/f88 x:0 0 0 2024-03-16T09:01:12.059 INFO:tasks.workunit.client.1.smithi019.stdout:0/188: truncate d7/d1d/d28/f34 385905 0 2024-03-16T09:01:12.059 INFO:tasks.workunit.client.0.smithi007.stdout:6/324: write d2/f46 [292834,53034] 0 2024-03-16T09:01:12.060 INFO:tasks.workunit.client.1.smithi019.stdout:0/189: symlink d7/d1d/d28/l3d 0 2024-03-16T09:01:12.060 INFO:tasks.workunit.client.1.smithi019.stdout:0/190: readlink d7/l24 0 2024-03-16T09:01:12.063 INFO:tasks.workunit.client.0.smithi007.stdout:9/475: dread d24/f4d [0,4194304] 0 2024-03-16T09:01:12.065 INFO:tasks.workunit.client.0.smithi007.stdout:9/476: mkdir d24/d4a/d55/d56/d7c/d7f/d80/d87/d89 0 2024-03-16T09:01:12.067 INFO:tasks.workunit.client.0.smithi007.stdout:9/477: creat d24/d4a/d55/d56/d7c/d7f/d80/d87/d89/f8a x:0 0 0 2024-03-16T09:01:12.070 INFO:tasks.workunit.client.1.smithi019.stdout:5/163: dwrite d6/f18 [0,4194304] 0 2024-03-16T09:01:12.070 INFO:tasks.workunit.client.1.smithi019.stdout:5/164: fdatasync d6/dd/f1a 0 2024-03-16T09:01:12.070 INFO:tasks.workunit.client.1.smithi019.stdout:5/165: creat d6/d1f/f2d x:0 0 0 2024-03-16T09:01:12.070 INFO:tasks.workunit.client.1.smithi019.stdout:5/166: fsync d6/f18 0 2024-03-16T09:01:12.071 INFO:tasks.workunit.client.0.smithi007.stdout:9/478: rename d24/c52 to d24/d4c/d86/c8b 0 2024-03-16T09:01:12.073 INFO:tasks.workunit.client.0.smithi007.stdout:9/479: truncate d24/f3d 3357745 0 2024-03-16T09:01:12.073 INFO:tasks.workunit.client.0.smithi007.stdout:9/480: fdatasync d24/f3c 0 2024-03-16T09:01:12.073 INFO:tasks.workunit.client.0.smithi007.stdout:9/481: chown d24/d4a/d61/f68 23 1 2024-03-16T09:01:12.076 INFO:tasks.workunit.client.0.smithi007.stdout:9/482: rename f17 to d24/d4a/d55/d56/d7c/d7f/d80/f8c 0 2024-03-16T09:01:12.076 INFO:tasks.workunit.client.0.smithi007.stdout:9/483: fdatasync d24/f29 0 2024-03-16T09:01:12.077 INFO:tasks.workunit.client.0.smithi007.stdout:8/661: dread d0/dc/d80/fe5 [0,4194304] 0 2024-03-16T09:01:12.077 INFO:tasks.workunit.client.0.smithi007.stdout:8/662: truncate d0/dc/dd/fda 644625 0 2024-03-16T09:01:12.079 INFO:tasks.workunit.client.0.smithi007.stdout:8/663: rmdir d0/d26/d2d/d43/d4c/dba 39 2024-03-16T09:01:12.086 INFO:tasks.workunit.client.0.smithi007.stdout:8/664: write d0/d26/d2d/d43/d6f/da5/fc8 [3320204,7558] 0 2024-03-16T09:01:12.086 INFO:tasks.workunit.client.0.smithi007.stdout:3/470: dwrite d15/f2c [0,4194304] 0 2024-03-16T09:01:12.089 INFO:tasks.workunit.client.1.smithi019.stdout:7/147: dwrite f6 [0,4194304] 0 2024-03-16T09:01:12.089 INFO:tasks.workunit.client.1.smithi019.stdout:7/148: creat d9/da/d13/d18/d1a/f2b x:0 0 0 2024-03-16T09:01:12.089 INFO:tasks.workunit.client.1.smithi019.stdout:7/149: stat d9/da/d13/d18/d1a/l21 0 2024-03-16T09:01:12.091 INFO:tasks.workunit.client.1.smithi019.stdout:7/150: getdents d9/da 0 2024-03-16T09:01:12.095 INFO:tasks.workunit.client.1.smithi019.stdout:7/151: write d9/fc [1376513,110704] 0 2024-03-16T09:01:12.095 INFO:tasks.workunit.client.1.smithi019.stdout:7/152: creat d9/db/f2c x:0 0 0 2024-03-16T09:01:12.095 INFO:tasks.workunit.client.1.smithi019.stdout:7/153: write d9/f23 [1009257,14198] 0 2024-03-16T09:01:12.095 INFO:tasks.workunit.client.1.smithi019.stdout:7/154: chown d9/da/d13/d18 4013070 1 2024-03-16T09:01:12.095 INFO:tasks.workunit.client.1.smithi019.stdout:7/155: chown d9/f23 102893921 1 2024-03-16T09:01:12.095 INFO:tasks.workunit.client.1.smithi019.stdout:7/156: write d9/fc [2406627,125264] 0 2024-03-16T09:01:12.095 INFO:tasks.workunit.client.1.smithi019.stdout:7/157: dread - d9/da/d13/d18/d1a/f2b zero size 2024-03-16T09:01:12.095 INFO:tasks.workunit.client.1.smithi019.stdout:7/158: write d9/da/f24 [826902,103052] 0 2024-03-16T09:01:12.098 INFO:tasks.workunit.client.0.smithi007.stdout:2/516: dwrite dc/df/d24/d28/d2e/f45 [0,4194304] 0 2024-03-16T09:01:12.100 INFO:tasks.workunit.client.0.smithi007.stdout:2/517: symlink dc/df/d24/d28/d2e/d6e/la8 0 2024-03-16T09:01:12.101 INFO:tasks.workunit.client.1.smithi019.stdout:8/208: dwrite d6/f27 [0,4194304] 0 2024-03-16T09:01:12.102 INFO:tasks.workunit.client.0.smithi007.stdout:6/325: dwrite d2/d13/d1f/d4e/f52 [0,4194304] 0 2024-03-16T09:01:12.102 INFO:tasks.workunit.client.0.smithi007.stdout:7/295: dwrite d0/dc/fe [4194304,4194304] 0 2024-03-16T09:01:12.102 INFO:tasks.workunit.client.0.smithi007.stdout:7/296: chown d0/l14 589 1 2024-03-16T09:01:12.102 INFO:tasks.workunit.client.0.smithi007.stdout:0/498: dwrite d3/d9/da/d18/d23/f45 [0,4194304] 0 2024-03-16T09:01:12.103 INFO:tasks.workunit.client.0.smithi007.stdout:2/518: unlink dc/df/d24/d28/d4c/f57 0 2024-03-16T09:01:12.103 INFO:tasks.workunit.client.0.smithi007.stdout:2/519: chown dc/df/d24/d28/d2e 13 1 2024-03-16T09:01:12.103 INFO:tasks.workunit.client.1.smithi019.stdout:5/167: dwrite f4 [8388608,4194304] 0 2024-03-16T09:01:12.104 INFO:tasks.workunit.client.0.smithi007.stdout:0/499: dread d3/d9/db/d2c/f90 [0,4194304] 0 2024-03-16T09:01:12.105 INFO:tasks.workunit.client.1.smithi019.stdout:8/209: creat d6/de/d16/d19/f43 x:0 0 0 2024-03-16T09:01:12.106 INFO:tasks.workunit.client.0.smithi007.stdout:6/326: creat d2/d10/d3d/f6c x:0 0 0 2024-03-16T09:01:12.106 INFO:tasks.workunit.client.0.smithi007.stdout:6/327: fdatasync d2/f59 0 2024-03-16T09:01:12.114 INFO:tasks.workunit.client.1.smithi019.stdout:0/191: link d7/d1d/d28/f34 d7/d29/f3e 0 2024-03-16T09:01:12.114 INFO:tasks.workunit.client.1.smithi019.stdout:0/192: fdatasync f4 0 2024-03-16T09:01:12.114 INFO:tasks.workunit.client.1.smithi019.stdout:0/193: write d7/d1d/d28/f34 [1387561,4474] 0 2024-03-16T09:01:12.114 INFO:tasks.workunit.client.0.smithi007.stdout:2/520: link dc/df/d24/d28/f66 dc/df/d6c/fa9 0 2024-03-16T09:01:12.115 INFO:tasks.workunit.client.1.smithi019.stdout:0/194: symlink d7/d8/d32/l3f 0 2024-03-16T09:01:12.115 INFO:tasks.workunit.client.0.smithi007.stdout:0/500: mknod d3/d9/da/d48/d4b/ca2 0 2024-03-16T09:01:12.118 INFO:tasks.workunit.client.0.smithi007.stdout:2/521: mkdir dc/df/daa 0 2024-03-16T09:01:12.118 INFO:tasks.workunit.client.0.smithi007.stdout:2/522: truncate dc/df/d24/d28/d2e/d4a/f9c 41733 0 2024-03-16T09:01:12.120 INFO:tasks.workunit.client.0.smithi007.stdout:0/501: creat d3/d1f/d70/fa3 x:0 0 0 2024-03-16T09:01:12.125 INFO:tasks.workunit.client.1.smithi019.stdout:8/210: dread f4 [4194304,4194304] 0 2024-03-16T09:01:12.125 INFO:tasks.workunit.client.1.smithi019.stdout:8/211: write d6/de/d16/f32 [4625490,84350] 0 2024-03-16T09:01:12.127 INFO:tasks.workunit.client.0.smithi007.stdout:8/665: rename d0/d2 to d0/dc/d50/d8e/dd2/dea 0 2024-03-16T09:01:12.127 INFO:tasks.workunit.client.1.smithi019.stdout:8/212: dread d6/de/f37 [0,4194304] 0 2024-03-16T09:01:12.127 INFO:tasks.workunit.client.1.smithi019.stdout:8/213: fsync d6/de/d24/f35 0 2024-03-16T09:01:12.128 INFO:tasks.workunit.client.0.smithi007.stdout:8/666: mkdir d0/d26/d2d/d43/d4c/deb 0 2024-03-16T09:01:12.128 INFO:tasks.workunit.client.0.smithi007.stdout:8/667: chown d0/dc/d50/d8e/dd2/dea/fa 0 1 2024-03-16T09:01:12.130 INFO:tasks.workunit.client.0.smithi007.stdout:8/668: rename d0/l44 to d0/dc/d50/d8e/db8/lec 0 2024-03-16T09:01:12.130 INFO:tasks.workunit.client.0.smithi007.stdout:8/669: chown d0/d26/d2d/d43/d4c/dba/fd7 560 1 2024-03-16T09:01:12.134 INFO:tasks.workunit.client.0.smithi007.stdout:9/484: getdents d24/d4a/d55/d56/d7c/d7f/d80 0 2024-03-16T09:01:12.138 INFO:tasks.workunit.client.0.smithi007.stdout:8/670: dread d0/f3c [0,4194304] 0 2024-03-16T09:01:12.138 INFO:tasks.workunit.client.0.smithi007.stdout:8/671: write d0/d26/f30 [1470116,117630] 0 2024-03-16T09:01:12.146 INFO:tasks.workunit.client.0.smithi007.stdout:9/485: dread d24/f44 [0,4194304] 0 2024-03-16T09:01:12.151 INFO:tasks.workunit.client.1.smithi019.stdout:7/159: dwrite d9/f16 [0,4194304] 0 2024-03-16T09:01:12.152 INFO:tasks.workunit.client.1.smithi019.stdout:5/168: dwrite d6/d1f/f20 [0,4194304] 0 2024-03-16T09:01:12.152 INFO:tasks.workunit.client.1.smithi019.stdout:5/169: readlink d6/d19/l28 0 2024-03-16T09:01:12.152 INFO:tasks.workunit.client.1.smithi019.stdout:5/170: fsync d6/dd/f16 0 2024-03-16T09:01:12.152 INFO:tasks.workunit.client.0.smithi007.stdout:0/502: dwrite d3/d9/da/d48/f38 [4194304,4194304] 0 2024-03-16T09:01:12.153 INFO:tasks.workunit.client.0.smithi007.stdout:6/328: dwrite d2/d13/d24/f6a [0,4194304] 0 2024-03-16T09:01:12.153 INFO:tasks.workunit.client.0.smithi007.stdout:7/297: dwrite d0/d19/d1f/d42/f57 [0,4194304] 0 2024-03-16T09:01:12.153 INFO:tasks.workunit.client.1.smithi019.stdout:5/171: write d6/fb [4531512,93364] 0 2024-03-16T09:01:12.153 INFO:tasks.workunit.client.1.smithi019.stdout:8/214: dwrite d6/de/f1b [0,4194304] 0 2024-03-16T09:01:12.155 INFO:tasks.workunit.client.0.smithi007.stdout:0/503: mknod d3/d1f/d70/ca4 0 2024-03-16T09:01:12.155 INFO:tasks.workunit.client.0.smithi007.stdout:0/504: creat d3/d9/da/d48/fa5 x:0 0 0 2024-03-16T09:01:12.158 INFO:tasks.workunit.client.0.smithi007.stdout:7/298: symlink d0/d19/d1f/d5d/l5f 0 2024-03-16T09:01:12.161 INFO:tasks.workunit.client.0.smithi007.stdout:3/471: dwrite d15/d18/f35 [0,4194304] 0 2024-03-16T09:01:12.161 INFO:tasks.workunit.client.0.smithi007.stdout:3/472: write d15/d18/d1f/d29/d43/f81 [1426488,15773] 0 2024-03-16T09:01:12.161 INFO:tasks.workunit.client.0.smithi007.stdout:9/486: dread d24/f34 [0,4194304] 0 2024-03-16T09:01:12.165 INFO:tasks.workunit.client.1.smithi019.stdout:5/172: rename d6/f23 to d6/d19/f2e 0 2024-03-16T09:01:12.165 INFO:tasks.workunit.client.1.smithi019.stdout:5/173: creat d6/f2f x:0 0 0 2024-03-16T09:01:12.166 INFO:tasks.workunit.client.1.smithi019.stdout:1/125: sync 2024-03-16T09:01:12.166 INFO:tasks.workunit.client.1.smithi019.stdout:1/126: stat d16 0 2024-03-16T09:01:12.166 INFO:tasks.workunit.client.1.smithi019.stdout:6/335: sync 2024-03-16T09:01:12.167 INFO:tasks.workunit.client.0.smithi007.stdout:0/505: mknod d3/d1f/d70/d7c/ca6 0 2024-03-16T09:01:12.167 INFO:tasks.workunit.client.1.smithi019.stdout:8/215: creat d6/d2a/d3c/f44 x:0 0 0 2024-03-16T09:01:12.168 INFO:tasks.workunit.client.0.smithi007.stdout:9/487: rename d24/l4b to d24/l8d 0 2024-03-16T09:01:12.170 INFO:tasks.workunit.client.1.smithi019.stdout:5/174: mknod d6/d19/c30 0 2024-03-16T09:01:12.171 INFO:tasks.workunit.client.0.smithi007.stdout:6/329: dwrite d2/d13/d1f/f4d [0,4194304] 0 2024-03-16T09:01:12.171 INFO:tasks.workunit.client.1.smithi019.stdout:1/127: truncate d16/f1c 4125022 0 2024-03-16T09:01:12.172 INFO:tasks.workunit.client.0.smithi007.stdout:6/330: write d2/d10/d12/d15/f45 [3128528,106283] 0 2024-03-16T09:01:12.172 INFO:tasks.workunit.client.0.smithi007.stdout:7/299: rmdir d0/dc/d38/d4e 39 2024-03-16T09:01:12.172 INFO:tasks.workunit.client.0.smithi007.stdout:7/300: truncate d0/dc/d24/d43/f58 11979 0 2024-03-16T09:01:12.172 INFO:tasks.workunit.client.1.smithi019.stdout:0/195: dwrite d7/d8/d22/f2c [0,4194304] 0 2024-03-16T09:01:12.173 INFO:tasks.workunit.client.1.smithi019.stdout:8/216: getdents d6/d2a 0 2024-03-16T09:01:12.173 INFO:tasks.workunit.client.0.smithi007.stdout:9/488: mkdir d24/d8e 0 2024-03-16T09:01:12.175 INFO:tasks.workunit.client.1.smithi019.stdout:1/128: rename ff to d16/f22 0 2024-03-16T09:01:12.177 INFO:tasks.workunit.client.1.smithi019.stdout:0/196: dread d7/fd [0,4194304] 0 2024-03-16T09:01:12.177 INFO:tasks.workunit.client.1.smithi019.stdout:6/336: dread de/d1c/d1d/f46 [0,4194304] 0 2024-03-16T09:01:12.180 INFO:tasks.workunit.client.0.smithi007.stdout:6/331: mkdir d2/d10/d3d/d6d 0 2024-03-16T09:01:12.180 INFO:tasks.workunit.client.0.smithi007.stdout:6/332: readlink d2/d11/l32 0 2024-03-16T09:01:12.181 INFO:tasks.workunit.client.1.smithi019.stdout:4/231: sync 2024-03-16T09:01:12.182 INFO:tasks.workunit.client.1.smithi019.stdout:4/232: write d4/d6/d15/d1f/d2c/f3f [73241,90752] 0 2024-03-16T09:01:12.183 INFO:tasks.workunit.client.0.smithi007.stdout:7/301: symlink d0/d19/l60 0 2024-03-16T09:01:12.183 INFO:tasks.workunit.client.1.smithi019.stdout:0/197: unlink d7/d29/f3a 0 2024-03-16T09:01:12.184 INFO:tasks.workunit.client.1.smithi019.stdout:6/337: stat de/d1c/d1d/d2c/c33 0 2024-03-16T09:01:12.184 INFO:tasks.workunit.client.1.smithi019.stdout:6/338: stat de/d26/d32/f37 0 2024-03-16T09:01:12.186 INFO:tasks.workunit.client.0.smithi007.stdout:8/672: rmdir d0/dc/d50 39 2024-03-16T09:01:12.186 INFO:tasks.workunit.client.1.smithi019.stdout:7/160: dwrite d9/da/f24 [0,4194304] 0 2024-03-16T09:01:12.186 INFO:tasks.workunit.client.1.smithi019.stdout:7/161: dread - d9/db/d12/f26 zero size 2024-03-16T09:01:12.186 INFO:tasks.workunit.client.1.smithi019.stdout:5/175: write f4 [2202955,100155] 0 2024-03-16T09:01:12.187 INFO:tasks.workunit.client.1.smithi019.stdout:9/161: sync 2024-03-16T09:01:12.187 INFO:tasks.workunit.client.1.smithi019.stdout:2/182: sync 2024-03-16T09:01:12.188 INFO:tasks.workunit.client.1.smithi019.stdout:8/217: rename d6/de/d16 to d6/d2a/d45 0 2024-03-16T09:01:12.188 INFO:tasks.workunit.client.1.smithi019.stdout:3/215: sync 2024-03-16T09:01:12.188 INFO:tasks.workunit.client.1.smithi019.stdout:3/216: write d3/d17/f32 [2183951,66871] 0 2024-03-16T09:01:12.188 INFO:tasks.workunit.client.1.smithi019.stdout:3/217: fdatasync d3/f43 0 2024-03-16T09:01:12.188 INFO:tasks.workunit.client.1.smithi019.stdout:4/233: mkdir d4/d1b/d2a/d4f 0 2024-03-16T09:01:12.190 INFO:tasks.workunit.client.0.smithi007.stdout:7/302: symlink d0/dc/d24/d25/d3a/l61 0 2024-03-16T09:01:12.190 INFO:tasks.workunit.client.0.smithi007.stdout:7/303: readlink d0/dc/lf 0 2024-03-16T09:01:12.190 INFO:tasks.workunit.client.0.smithi007.stdout:7/304: creat d0/d19/d1f/d42/f62 x:0 0 0 2024-03-16T09:01:12.191 INFO:tasks.workunit.client.1.smithi019.stdout:6/339: truncate de/d1c/d1d/f46 903242 0 2024-03-16T09:01:12.191 INFO:tasks.workunit.client.1.smithi019.stdout:6/340: write de/d1c/f56 [1015286,79732] 0 2024-03-16T09:01:12.194 INFO:tasks.workunit.client.1.smithi019.stdout:9/162: creat d1/d7/d21/f38 x:0 0 0 2024-03-16T09:01:12.195 INFO:tasks.workunit.client.0.smithi007.stdout:8/673: dread d0/dc/d50/f6b [0,4194304] 0 2024-03-16T09:01:12.195 INFO:tasks.workunit.client.0.smithi007.stdout:7/305: mkdir d0/dc/d24/d25/d3a/d63 0 2024-03-16T09:01:12.196 INFO:tasks.workunit.client.0.smithi007.stdout:7/306: dread - d0/dc/d24/f5e zero size 2024-03-16T09:01:12.198 INFO:tasks.workunit.client.0.smithi007.stdout:3/473: dwrite d15/d18/d1f/d29/d43/f81 [0,4194304] 0 2024-03-16T09:01:12.198 INFO:tasks.workunit.client.0.smithi007.stdout:8/674: creat d0/dc/d50/d8e/dd2/dea/d18/da7/fed x:0 0 0 2024-03-16T09:01:12.199 INFO:tasks.workunit.client.0.smithi007.stdout:7/307: creat d0/dc/d24/f64 x:0 0 0 2024-03-16T09:01:12.202 INFO:tasks.workunit.client.1.smithi019.stdout:6/341: mkdir de/d26/d32/d4e/d75 0 2024-03-16T09:01:12.202 INFO:tasks.workunit.client.0.smithi007.stdout:8/675: symlink d0/dc/d50/d8e/dd2/dea/d5f/lee 0 2024-03-16T09:01:12.203 INFO:tasks.workunit.client.0.smithi007.stdout:7/308: mknod d0/dc/d24/d25/d3a/c65 0 2024-03-16T09:01:12.203 INFO:tasks.workunit.client.1.smithi019.stdout:9/163: stat d1/d7/d21/d27/c18 0 2024-03-16T09:01:12.204 INFO:tasks.workunit.client.0.smithi007.stdout:8/676: getdents d0/dc/d50/d8e/dd2 0 2024-03-16T09:01:12.206 INFO:tasks.workunit.client.0.smithi007.stdout:7/309: mknod d0/dc/d24/d25/d3a/c66 0 2024-03-16T09:01:12.206 INFO:tasks.workunit.client.0.smithi007.stdout:7/310: truncate d0/dc/d24/f5e 1029539 0 2024-03-16T09:01:12.207 INFO:tasks.workunit.client.1.smithi019.stdout:1/129: rename d16/f1c to d16/d18/f23 0 2024-03-16T09:01:12.207 INFO:tasks.workunit.client.1.smithi019.stdout:1/130: creat d16/d18/f24 x:0 0 0 2024-03-16T09:01:12.207 INFO:tasks.workunit.client.1.smithi019.stdout:6/342: getdents de 0 2024-03-16T09:01:12.208 INFO:tasks.workunit.client.0.smithi007.stdout:8/677: mkdir d0/dc/d50/d8e/dd2/dea/d18/da7/dc5/def 0 2024-03-16T09:01:12.208 INFO:tasks.workunit.client.0.smithi007.stdout:8/678: readlink d0/dc/d50/d8e/db8/lbc 0 2024-03-16T09:01:12.208 INFO:tasks.workunit.client.0.smithi007.stdout:8/679: write d0/dc/d50/d8e/dd2/dea/d5f/fc2 [4967513,126765] 0 2024-03-16T09:01:12.209 INFO:tasks.workunit.client.1.smithi019.stdout:9/164: unlink d1/d7/de/f1f 0 2024-03-16T09:01:12.210 INFO:tasks.workunit.client.1.smithi019.stdout:1/131: symlink d16/l25 0 2024-03-16T09:01:12.210 INFO:tasks.workunit.client.1.smithi019.stdout:1/132: getdents d16/d1f 0 2024-03-16T09:01:12.210 INFO:tasks.workunit.client.1.smithi019.stdout:6/343: creat de/d1c/d1d/d66/d72/f76 x:0 0 0 2024-03-16T09:01:12.212 INFO:tasks.workunit.client.1.smithi019.stdout:5/176: dread d6/fb [0,4194304] 0 2024-03-16T09:01:12.212 INFO:tasks.workunit.client.1.smithi019.stdout:5/177: chown d6/dd/l15 248 1 2024-03-16T09:01:12.213 INFO:tasks.workunit.client.1.smithi019.stdout:5/178: chown d6/c13 1492214 1 2024-03-16T09:01:12.213 INFO:tasks.workunit.client.1.smithi019.stdout:3/218: rename d3/l27 to d3/d11/l46 0 2024-03-16T09:01:12.213 INFO:tasks.workunit.client.1.smithi019.stdout:3/219: fsync d3/d11/f37 0 2024-03-16T09:01:12.213 INFO:tasks.workunit.client.1.smithi019.stdout:3/220: fdatasync d3/d11/f14 0 2024-03-16T09:01:12.213 INFO:tasks.workunit.client.1.smithi019.stdout:3/221: readlink d3/d11/l39 0 2024-03-16T09:01:12.213 INFO:tasks.workunit.client.1.smithi019.stdout:3/222: stat d3/d11/f3b 0 2024-03-16T09:01:12.213 INFO:tasks.workunit.client.1.smithi019.stdout:3/223: stat d3/d17/d3f 0 2024-03-16T09:01:12.214 INFO:tasks.workunit.client.1.smithi019.stdout:3/224: fdatasync d3/ff 0 2024-03-16T09:01:12.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:11 smithi019 ceph-mon[36407]: pgmap v19: 65 pgs: 65 active+clean; 932 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 89 MiB/s rd, 118 MiB/s wr, 1.10k op/s 2024-03-16T09:01:12.215 INFO:tasks.workunit.client.1.smithi019.stdout:1/133: link d16/l21 d16/d18/l26 0 2024-03-16T09:01:12.215 INFO:tasks.workunit.client.1.smithi019.stdout:6/344: symlink de/d26/d32/d4e/d75/l77 0 2024-03-16T09:01:12.215 INFO:tasks.workunit.client.1.smithi019.stdout:5/179: symlink d6/dd/l31 0 2024-03-16T09:01:12.215 INFO:tasks.workunit.client.1.smithi019.stdout:5/180: write d6/d19/f1b [330702,53334] 0 2024-03-16T09:01:12.215 INFO:tasks.workunit.client.1.smithi019.stdout:5/181: readlink d6/d19/l28 0 2024-03-16T09:01:12.216 INFO:tasks.workunit.client.0.smithi007.stdout:9/489: dwrite d24/d4a/f74 [0,4194304] 0 2024-03-16T09:01:12.216 INFO:tasks.workunit.client.0.smithi007.stdout:7/311: rename d0/f33 to d0/d19/d1f/d5d/f67 0 2024-03-16T09:01:12.216 INFO:tasks.workunit.client.1.smithi019.stdout:6/345: mknod de/d1c/c78 0 2024-03-16T09:01:12.216 INFO:tasks.workunit.client.1.smithi019.stdout:6/346: write de/d1c/f28 [2150918,95365] 0 2024-03-16T09:01:12.216 INFO:tasks.workunit.client.1.smithi019.stdout:7/162: dwrite d9/db/d12/f20 [0,4194304] 0 2024-03-16T09:01:12.218 INFO:tasks.workunit.client.1.smithi019.stdout:5/182: rename d6/dd/f22 to d6/f32 0 2024-03-16T09:01:12.218 INFO:tasks.workunit.client.1.smithi019.stdout:5/183: readlink d6/l2b 0 2024-03-16T09:01:12.218 INFO:tasks.workunit.client.1.smithi019.stdout:5/184: dread - d6/d1f/d21/f24 zero size 2024-03-16T09:01:12.220 INFO:tasks.workunit.client.1.smithi019.stdout:0/198: rmdir d7 39 2024-03-16T09:01:12.220 INFO:tasks.workunit.client.1.smithi019.stdout:0/199: chown f4 1243 1 2024-03-16T09:01:12.220 INFO:tasks.workunit.client.1.smithi019.stdout:0/200: read f4 [1957284,43588] 0 2024-03-16T09:01:12.224 INFO:tasks.workunit.client.0.smithi007.stdout:6/333: truncate d2/d10/d12/d15/f45 1655368 0 2024-03-16T09:01:12.225 INFO:tasks.workunit.client.0.smithi007.stdout:7/312: dread d0/d19/d1f/d42/f57 [0,4194304] 0 2024-03-16T09:01:12.225 INFO:tasks.workunit.client.0.smithi007.stdout:7/313: chown d0/dc/d5a 239758940 1 2024-03-16T09:01:12.225 INFO:tasks.workunit.client.1.smithi019.stdout:1/134: read f9 [3160333,83906] 0 2024-03-16T09:01:12.227 INFO:tasks.workunit.client.1.smithi019.stdout:3/225: dread d3/d11/f14 [4194304,4194304] 0 2024-03-16T09:01:12.227 INFO:tasks.workunit.client.1.smithi019.stdout:3/226: fdatasync d3/f43 0 2024-03-16T09:01:12.231 INFO:tasks.workunit.client.0.smithi007.stdout:6/334: link d2/d11/l30 d2/d13/d62/l6e 0 2024-03-16T09:01:12.231 INFO:tasks.workunit.client.0.smithi007.stdout:6/335: write d2/d10/d3d/f6c [348380,35992] 0 2024-03-16T09:01:12.231 INFO:tasks.workunit.client.0.smithi007.stdout:6/336: chown d2/c3 0 1 2024-03-16T09:01:12.231 INFO:tasks.workunit.client.0.smithi007.stdout:6/337: write d2/d13/d1f/d4e/f5b [451100,104947] 0 2024-03-16T09:01:12.231 INFO:tasks.workunit.client.0.smithi007.stdout:6/338: stat d2/d13/d24/f38 0 2024-03-16T09:01:12.232 INFO:tasks.workunit.client.0.smithi007.stdout:6/339: mkdir d2/d11/d1e/d6f 0 2024-03-16T09:01:12.233 INFO:tasks.workunit.client.1.smithi019.stdout:7/163: rmdir d9 39 2024-03-16T09:01:12.233 INFO:tasks.workunit.client.1.smithi019.stdout:7/164: fsync d9/db/f2c 0 2024-03-16T09:01:12.235 INFO:tasks.workunit.client.1.smithi019.stdout:1/135: creat d16/d1f/f27 x:0 0 0 2024-03-16T09:01:12.235 INFO:tasks.workunit.client.0.smithi007.stdout:8/680: write d0/dc/d50/d8e/dd2/dea/d74/fd3 [2937520,11897] 0 2024-03-16T09:01:12.235 INFO:tasks.workunit.client.0.smithi007.stdout:8/681: truncate d0/dc/d50/d8e/dd2/dea/d18/f52 785632 0 2024-03-16T09:01:12.236 INFO:tasks.workunit.client.0.smithi007.stdout:6/340: creat d2/d10/f70 x:0 0 0 2024-03-16T09:01:12.236 INFO:tasks.workunit.client.1.smithi019.stdout:7/165: mknod d9/d25/c2d 0 2024-03-16T09:01:12.240 INFO:tasks.workunit.client.1.smithi019.stdout:7/166: mknod d9/db/d12/c2e 0 2024-03-16T09:01:12.241 INFO:tasks.workunit.client.1.smithi019.stdout:7/167: readlink d9/da/d13/d18/d1a/l21 0 2024-03-16T09:01:12.241 INFO:tasks.workunit.client.1.smithi019.stdout:7/168: fdatasync d9/da/f24 0 2024-03-16T09:01:12.245 INFO:tasks.workunit.client.1.smithi019.stdout:3/227: read d3/d1b/f1c [2248534,36237] 0 2024-03-16T09:01:12.250 INFO:tasks.workunit.client.1.smithi019.stdout:3/228: rename d3/fc to d3/d1b/d40/d24/d2c/f47 0 2024-03-16T09:01:12.251 INFO:tasks.workunit.client.1.smithi019.stdout:3/229: mknod d3/c48 0 2024-03-16T09:01:12.251 INFO:tasks.workunit.client.1.smithi019.stdout:2/183: dwrite d5/f7 [0,4194304] 0 2024-03-16T09:01:12.253 INFO:tasks.workunit.client.1.smithi019.stdout:3/230: truncate d3/d1b/d40/d24/d2c/f47 4324797 0 2024-03-16T09:01:12.255 INFO:tasks.workunit.client.1.smithi019.stdout:2/184: rmdir d5/d6/d2b/d2f/d31 0 2024-03-16T09:01:12.255 INFO:tasks.workunit.client.1.smithi019.stdout:2/185: dread - d5/f13 zero size 2024-03-16T09:01:12.256 INFO:tasks.workunit.client.1.smithi019.stdout:2/186: creat d5/d6/d2b/d2f/f3a x:0 0 0 2024-03-16T09:01:12.257 INFO:tasks.workunit.client.1.smithi019.stdout:2/187: getdents d5/d6/d2b 0 2024-03-16T09:01:12.260 INFO:tasks.workunit.client.1.smithi019.stdout:2/188: unlink d5/c25 0 2024-03-16T09:01:12.262 INFO:tasks.workunit.client.1.smithi019.stdout:2/189: creat d5/d2e/f3b x:0 0 0 2024-03-16T09:01:12.282 INFO:tasks.workunit.client.1.smithi019.stdout:9/165: dwrite d1/d7/d21/f38 [0,4194304] 0 2024-03-16T09:01:12.283 INFO:tasks.workunit.client.1.smithi019.stdout:9/166: fsync d1/d7/de/f10 0 2024-03-16T09:01:12.283 INFO:tasks.workunit.client.1.smithi019.stdout:9/167: creat d1/d7/de/d24/f39 x:0 0 0 2024-03-16T09:01:12.284 INFO:tasks.workunit.client.1.smithi019.stdout:6/347: dread de/d1c/d1d/f46 [0,4194304] 0 2024-03-16T09:01:12.284 INFO:tasks.workunit.client.1.smithi019.stdout:6/348: read - de/d26/f39 zero size 2024-03-16T09:01:12.284 INFO:tasks.workunit.client.0.smithi007.stdout:3/474: dwrite d15/d18/d1f/d29/d43/d72/f50 [0,4194304] 0 2024-03-16T09:01:12.284 INFO:tasks.workunit.client.0.smithi007.stdout:3/475: creat d15/d18/d1f/d56/f95 x:0 0 0 2024-03-16T09:01:12.285 INFO:tasks.workunit.client.0.smithi007.stdout:3/476: chown d15/d18/d1f/d29/d3e/d4c 6213215 1 2024-03-16T09:01:12.285 INFO:tasks.workunit.client.0.smithi007.stdout:3/477: fsync d15/d18/d1f/d29/f31 0 2024-03-16T09:01:12.288 INFO:tasks.workunit.client.1.smithi019.stdout:9/168: dread d1/d7/fd [0,4194304] 0 2024-03-16T09:01:12.297 INFO:tasks.workunit.client.1.smithi019.stdout:1/136: dwrite d16/d1f/f27 [0,4194304] 0 2024-03-16T09:01:12.297 INFO:tasks.workunit.client.1.smithi019.stdout:0/201: dwrite d7/f2a [0,4194304] 0 2024-03-16T09:01:12.297 INFO:tasks.workunit.client.1.smithi019.stdout:0/202: truncate d7/d8/d32/d36/f3b 784157 0 2024-03-16T09:01:12.298 INFO:tasks.workunit.client.0.smithi007.stdout:8/682: dwrite d0/dc/d54/da4/fc7 [0,4194304] 0 2024-03-16T09:01:12.298 INFO:tasks.workunit.client.0.smithi007.stdout:9/490: dwrite d24/f31 [0,4194304] 0 2024-03-16T09:01:12.298 INFO:tasks.workunit.client.0.smithi007.stdout:9/491: fsync ff 0 2024-03-16T09:01:12.298 INFO:tasks.workunit.client.0.smithi007.stdout:9/492: dread - d24/d4a/d55/d56/d7c/d7f/d80/d87/d89/f8a zero size 2024-03-16T09:01:12.298 INFO:tasks.workunit.client.0.smithi007.stdout:3/478: write d15/d18/d1f/f58 [3361512,51044] 0 2024-03-16T09:01:12.299 INFO:tasks.workunit.client.0.smithi007.stdout:3/479: chown d15/d18/d1f/d29/d3e/d4c 5968 1 2024-03-16T09:01:12.299 INFO:tasks.workunit.client.0.smithi007.stdout:3/480: rename d15/d18/d1f/d29 to d15/d18/d1f/d29/d43/d72/d96 22 2024-03-16T09:01:12.299 INFO:tasks.workunit.client.0.smithi007.stdout:3/481: creat d15/d18/d1f/d3b/d87/f97 x:0 0 0 2024-03-16T09:01:12.299 INFO:tasks.workunit.client.0.smithi007.stdout:3/482: dread - d15/d18/d1f/d3b/f44 zero size 2024-03-16T09:01:12.299 INFO:tasks.workunit.client.0.smithi007.stdout:3/483: fdatasync d15/d18/d1f/d3b/d63/f93 0 2024-03-16T09:01:12.299 INFO:tasks.workunit.client.0.smithi007.stdout:0/506: dwrite d3/d9/da/f1e [0,4194304] 0 2024-03-16T09:01:12.300 INFO:tasks.workunit.client.1.smithi019.stdout:2/190: dwrite d5/ff [0,4194304] 0 2024-03-16T09:01:12.300 INFO:tasks.workunit.client.1.smithi019.stdout:2/191: fsync d5/d6/d2b/f18 0 2024-03-16T09:01:12.300 INFO:tasks.workunit.client.1.smithi019.stdout:2/192: fsync d5/d6/f1d 0 2024-03-16T09:01:12.300 INFO:tasks.workunit.client.1.smithi019.stdout:2/193: fdatasync d5/de/f1f 0 2024-03-16T09:01:12.300 INFO:tasks.workunit.client.1.smithi019.stdout:2/194: write d5/d6/f28 [1280397,6364] 0 2024-03-16T09:01:12.301 INFO:tasks.workunit.client.1.smithi019.stdout:2/195: write d5/de/f32 [1647393,19003] 0 2024-03-16T09:01:12.301 INFO:tasks.workunit.client.1.smithi019.stdout:1/137: mknod d16/d1f/c28 0 2024-03-16T09:01:12.301 INFO:tasks.workunit.client.1.smithi019.stdout:5/185: dwrite d6/f9 [4194304,4194304] 0 2024-03-16T09:01:12.301 INFO:tasks.workunit.client.0.smithi007.stdout:7/314: dwrite d0/d19/f41 [4194304,4194304] 0 2024-03-16T09:01:12.301 INFO:tasks.workunit.client.1.smithi019.stdout:3/231: dwrite d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:12.306 INFO:tasks.workunit.client.1.smithi019.stdout:0/203: symlink d7/d1d/d28/l40 0 2024-03-16T09:01:12.310 INFO:tasks.workunit.client.1.smithi019.stdout:9/169: dwrite d1/d7/d21/d27/f22 [4194304,4194304] 0 2024-03-16T09:01:12.312 INFO:tasks.workunit.client.0.smithi007.stdout:3/484: mkdir d15/d18/d1f/d56/d98 0 2024-03-16T09:01:12.312 INFO:tasks.workunit.client.0.smithi007.stdout:3/485: getdents d15 0 2024-03-16T09:01:12.312 INFO:tasks.workunit.client.0.smithi007.stdout:3/486: readlink d15/l79 0 2024-03-16T09:01:12.312 INFO:tasks.workunit.client.0.smithi007.stdout:3/487: fdatasync d15/d18/d1f/d56/f65 0 2024-03-16T09:01:12.312 INFO:tasks.workunit.client.0.smithi007.stdout:3/488: fsync d15/d18/d1f/f39 0 2024-03-16T09:01:12.312 INFO:tasks.workunit.client.0.smithi007.stdout:9/493: dread d24/d4a/d55/d56/f5a [0,4194304] 0 2024-03-16T09:01:12.312 INFO:tasks.workunit.client.0.smithi007.stdout:9/494: stat d24/d4a/d55/d56/d7c/d7f/d80/d87/d89/f8a 0 2024-03-16T09:01:12.314 INFO:tasks.workunit.client.1.smithi019.stdout:1/138: mknod d16/d1f/c29 0 2024-03-16T09:01:12.314 INFO:tasks.workunit.client.1.smithi019.stdout:1/139: chown f14 847283 1 2024-03-16T09:01:12.314 INFO:tasks.workunit.client.1.smithi019.stdout:1/140: fdatasync d16/d18/f1a 0 2024-03-16T09:01:12.314 INFO:tasks.workunit.client.1.smithi019.stdout:1/141: chown d16/c1d 21661354 1 2024-03-16T09:01:12.315 INFO:tasks.workunit.client.1.smithi019.stdout:2/196: dread d5/fb [0,4194304] 0 2024-03-16T09:01:12.315 INFO:tasks.workunit.client.1.smithi019.stdout:2/197: chown f1 2 1 2024-03-16T09:01:12.315 INFO:tasks.workunit.client.1.smithi019.stdout:2/198: chown d5/c16 557 1 2024-03-16T09:01:12.319 INFO:tasks.workunit.client.0.smithi007.stdout:0/507: mknod d3/d9/da/d48/d9e/ca7 0 2024-03-16T09:01:12.319 INFO:tasks.workunit.client.0.smithi007.stdout:0/508: readlink d3/d9/da/d18/d27/l9b 0 2024-03-16T09:01:12.321 INFO:tasks.workunit.client.0.smithi007.stdout:3/489: symlink d15/d18/d1f/d3b/d87/l99 0 2024-03-16T09:01:12.321 INFO:tasks.workunit.client.0.smithi007.stdout:9/495: truncate d24/f3c 341406 0 2024-03-16T09:01:12.321 INFO:tasks.workunit.client.0.smithi007.stdout:9/496: truncate d24/f2a 424182 0 2024-03-16T09:01:12.323 INFO:tasks.workunit.client.0.smithi007.stdout:3/490: symlink d15/d18/d1f/d3b/l9a 0 2024-03-16T09:01:12.324 INFO:tasks.workunit.client.0.smithi007.stdout:9/497: mkdir d24/d4a/d55/d56/d7c/d7f/d8f 0 2024-03-16T09:01:12.324 INFO:tasks.workunit.client.0.smithi007.stdout:9/498: write d24/f2a [810759,106416] 0 2024-03-16T09:01:12.324 INFO:tasks.workunit.client.0.smithi007.stdout:3/491: symlink d15/d18/d1f/d29/d43/d72/l9b 0 2024-03-16T09:01:12.324 INFO:tasks.workunit.client.0.smithi007.stdout:3/492: chown d15/d18/d1f/d29/d3e/d4c/c90 2740831 1 2024-03-16T09:01:12.326 INFO:tasks.workunit.client.1.smithi019.stdout:5/186: getdents d6/d1f/d21 0 2024-03-16T09:01:12.326 INFO:tasks.workunit.client.0.smithi007.stdout:9/499: unlink f1b 0 2024-03-16T09:01:12.328 INFO:tasks.workunit.client.0.smithi007.stdout:9/500: write d24/f26 [111406,113353] 0 2024-03-16T09:01:12.328 INFO:tasks.workunit.client.0.smithi007.stdout:3/493: truncate d15/d18/d1f/d29/d43/f7a 641999 0 2024-03-16T09:01:12.328 INFO:tasks.workunit.client.0.smithi007.stdout:3/494: fsync d15/d18/d1f/d29/d7c/f94 0 2024-03-16T09:01:12.329 INFO:tasks.workunit.client.1.smithi019.stdout:9/170: getdents d1/d7 0 2024-03-16T09:01:12.329 INFO:tasks.workunit.client.1.smithi019.stdout:9/171: creat d1/d7/de/f3a x:0 0 0 2024-03-16T09:01:12.329 INFO:tasks.workunit.client.0.smithi007.stdout:9/501: truncate d24/d4a/d55/d56/d7c/d7f/d80/f8c 169295 0 2024-03-16T09:01:12.329 INFO:tasks.workunit.client.0.smithi007.stdout:9/502: chown d24/d4a/d55/d56/d7c 3328 1 2024-03-16T09:01:12.330 INFO:tasks.workunit.client.1.smithi019.stdout:5/187: mkdir d6/d1f/d33 0 2024-03-16T09:01:12.330 INFO:tasks.workunit.client.0.smithi007.stdout:7/315: dwrite d0/dc/d24/f5e [0,4194304] 0 2024-03-16T09:01:12.331 INFO:tasks.workunit.client.0.smithi007.stdout:7/316: read - d0/dc/d24/d25/d3a/f4f zero size 2024-03-16T09:01:12.333 INFO:tasks.workunit.client.1.smithi019.stdout:0/204: dwrite d7/d8/d32/d36/f35 [0,4194304] 0 2024-03-16T09:01:12.334 INFO:tasks.workunit.client.1.smithi019.stdout:5/188: rmdir d6/d1f/d27 0 2024-03-16T09:01:12.334 INFO:tasks.workunit.client.1.smithi019.stdout:5/189: write d6/f9 [429330,7625] 0 2024-03-16T09:01:12.338 INFO:tasks.workunit.client.1.smithi019.stdout:5/190: mkdir d6/d19/d34 0 2024-03-16T09:01:12.340 INFO:tasks.workunit.client.1.smithi019.stdout:5/191: truncate f2 584360 0 2024-03-16T09:01:12.340 INFO:tasks.workunit.client.1.smithi019.stdout:5/192: chown d6/d1f/d21/f24 298154625 1 2024-03-16T09:01:12.340 INFO:tasks.workunit.client.1.smithi019.stdout:5/193: creat d6/f35 x:0 0 0 2024-03-16T09:01:12.340 INFO:tasks.workunit.client.1.smithi019.stdout:5/194: fsync d6/dd/f16 0 2024-03-16T09:01:12.341 INFO:tasks.workunit.client.0.smithi007.stdout:9/503: dread f8 [0,4194304] 0 2024-03-16T09:01:12.341 INFO:tasks.workunit.client.0.smithi007.stdout:9/504: mkdir d24/d4a/d55/d56/d90 0 2024-03-16T09:01:12.342 INFO:tasks.workunit.client.1.smithi019.stdout:5/195: mknod d6/c36 0 2024-03-16T09:01:12.343 INFO:tasks.workunit.client.0.smithi007.stdout:9/505: mkdir d24/d4c/d91 0 2024-03-16T09:01:12.343 INFO:tasks.workunit.client.0.smithi007.stdout:9/506: chown d24/l67 37175 1 2024-03-16T09:01:12.343 INFO:tasks.workunit.client.0.smithi007.stdout:9/507: dread d24/d4c/f4e [0,4194304] 0 2024-03-16T09:01:12.345 INFO:tasks.workunit.client.1.smithi019.stdout:9/172: write d1/d7/de/d2d/f30 [3998780,72159] 0 2024-03-16T09:01:12.345 INFO:tasks.workunit.client.1.smithi019.stdout:9/173: write d1/d7/d21/d27/f35 [1778670,8016] 0 2024-03-16T09:01:12.345 INFO:tasks.workunit.client.1.smithi019.stdout:9/174: getdents d1/d7 0 2024-03-16T09:01:12.346 INFO:tasks.workunit.client.1.smithi019.stdout:9/175: mknod d1/d7/d21/d27/c3b 0 2024-03-16T09:01:12.348 INFO:tasks.workunit.client.1.smithi019.stdout:9/176: creat d1/d7/d21/d27/f3c x:0 0 0 2024-03-16T09:01:12.349 INFO:tasks.workunit.client.0.smithi007.stdout:8/683: dwrite d0/dc/d50/d8e/dd2/dea/d5f/fc2 [0,4194304] 0 2024-03-16T09:01:12.349 INFO:tasks.workunit.client.0.smithi007.stdout:3/495: read fa [1721105,17803] 0 2024-03-16T09:01:12.349 INFO:tasks.workunit.client.0.smithi007.stdout:3/496: fdatasync d15/d18/d1f/d29/d43/d72/f50 0 2024-03-16T09:01:12.350 INFO:tasks.workunit.client.1.smithi019.stdout:3/232: dwrite d3/f43 [0,4194304] 0 2024-03-16T09:01:12.350 INFO:tasks.workunit.client.0.smithi007.stdout:8/684: mkdir d0/dc/d50/d8e/dd2/dea/d74/df0 0 2024-03-16T09:01:12.351 INFO:tasks.workunit.client.0.smithi007.stdout:8/685: readlink d0/l5c 0 2024-03-16T09:01:12.352 INFO:tasks.workunit.client.0.smithi007.stdout:3/497: rename d15/d18/c46 to d15/d18/d1f/d56/d98/c9c 0 2024-03-16T09:01:12.355 INFO:tasks.workunit.client.0.smithi007.stdout:8/686: unlink d0/dc/d54/da4/ld4 0 2024-03-16T09:01:12.359 INFO:tasks.workunit.client.1.smithi019.stdout:9/177: dread d1/f3 [0,4194304] 0 2024-03-16T09:01:12.360 INFO:tasks.workunit.client.1.smithi019.stdout:9/178: dread - d1/d7/de/d24/f2c zero size 2024-03-16T09:01:12.362 INFO:tasks.workunit.client.1.smithi019.stdout:0/205: dread d7/d29/f3e [0,4194304] 0 2024-03-16T09:01:12.366 INFO:tasks.workunit.client.0.smithi007.stdout:9/508: truncate d24/d4a/d61/f7b 407824 0 2024-03-16T09:01:12.366 INFO:tasks.workunit.client.0.smithi007.stdout:9/509: dread - d24/d4a/d55/d56/f77 zero size 2024-03-16T09:01:12.367 INFO:tasks.workunit.client.0.smithi007.stdout:8/687: dread d0/d26/fa2 [0,4194304] 0 2024-03-16T09:01:12.368 INFO:tasks.workunit.client.0.smithi007.stdout:9/510: mknod d24/d4a/d55/d56/d90/c92 0 2024-03-16T09:01:12.369 INFO:tasks.workunit.client.1.smithi019.stdout:3/233: dread d3/ff [0,4194304] 0 2024-03-16T09:01:12.369 INFO:tasks.workunit.client.1.smithi019.stdout:3/234: chown d3/d1b/f1c 3 1 2024-03-16T09:01:12.369 INFO:tasks.workunit.client.1.smithi019.stdout:3/235: creat d3/d11/f49 x:0 0 0 2024-03-16T09:01:12.369 INFO:tasks.workunit.client.1.smithi019.stdout:3/236: write d3/ff [5187000,124638] 0 2024-03-16T09:01:12.371 INFO:tasks.workunit.client.1.smithi019.stdout:3/237: rename d3/d1b/d40/d24 to d3/d1b/d40/d4a 0 2024-03-16T09:01:12.371 INFO:tasks.workunit.client.1.smithi019.stdout:3/238: symlink d3/d17/l4b 0 2024-03-16T09:01:12.371 INFO:tasks.workunit.client.1.smithi019.stdout:3/239: creat d3/d11/f4c x:0 0 0 2024-03-16T09:01:12.371 INFO:tasks.workunit.client.0.smithi007.stdout:8/688: creat d0/dc/ff1 x:0 0 0 2024-03-16T09:01:12.372 INFO:tasks.workunit.client.0.smithi007.stdout:8/689: creat d0/d26/d2d/ff2 x:0 0 0 2024-03-16T09:01:12.372 INFO:tasks.workunit.client.0.smithi007.stdout:8/690: fdatasync d0/d26/d2d/ff2 0 2024-03-16T09:01:12.372 INFO:tasks.workunit.client.1.smithi019.stdout:3/240: symlink d3/d1b/l4d 0 2024-03-16T09:01:12.375 INFO:tasks.workunit.client.1.smithi019.stdout:0/206: dread d7/d8/d22/f25 [0,4194304] 0 2024-03-16T09:01:12.376 INFO:tasks.workunit.client.1.smithi019.stdout:0/207: creat d7/d29/f41 x:0 0 0 2024-03-16T09:01:12.376 INFO:tasks.workunit.client.1.smithi019.stdout:0/208: truncate d7/f2a 450861 0 2024-03-16T09:01:12.378 INFO:tasks.workunit.client.1.smithi019.stdout:9/179: write d1/d7/d21/d27/f22 [3635547,10291] 0 2024-03-16T09:01:12.384 INFO:tasks.workunit.client.1.smithi019.stdout:1/142: dwrite d16/d18/f1a [0,4194304] 0 2024-03-16T09:01:12.385 INFO:tasks.workunit.client.1.smithi019.stdout:5/196: dwrite d6/dd/f1d [0,4194304] 0 2024-03-16T09:01:12.385 INFO:tasks.workunit.client.1.smithi019.stdout:5/197: chown d6/d1f/d21/f26 63646285 1 2024-03-16T09:01:12.385 INFO:tasks.workunit.client.0.smithi007.stdout:9/511: read d24/f3c [139895,45330] 0 2024-03-16T09:01:12.386 INFO:tasks.workunit.client.0.smithi007.stdout:7/317: dwrite d0/f49 [0,4194304] 0 2024-03-16T09:01:12.386 INFO:tasks.workunit.client.0.smithi007.stdout:7/318: fsync d0/dc/d24/d25/d3a/f4f 0 2024-03-16T09:01:12.386 INFO:tasks.workunit.client.0.smithi007.stdout:0/509: dwrite d3/d9/db/d2c/f5e [0,4194304] 0 2024-03-16T09:01:12.389 INFO:tasks.workunit.client.1.smithi019.stdout:9/180: write d1/d7/de/f2a [2797791,67259] 0 2024-03-16T09:01:12.390 INFO:tasks.workunit.client.0.smithi007.stdout:0/510: dread d3/f6f [0,4194304] 0 2024-03-16T09:01:12.390 INFO:tasks.workunit.client.0.smithi007.stdout:0/511: write d3/d9/db/fc [273627,27960] 0 2024-03-16T09:01:12.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/512: unlink d24/d4a/f66 0 2024-03-16T09:01:12.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/513: chown d24/f27 1 1 2024-03-16T09:01:12.390 INFO:tasks.workunit.client.0.smithi007.stdout:9/514: readlink d24/d4c/l57 0 2024-03-16T09:01:12.392 INFO:tasks.workunit.client.0.smithi007.stdout:7/319: creat d0/dc/d38/d4e/f68 x:0 0 0 2024-03-16T09:01:12.392 INFO:tasks.workunit.client.0.smithi007.stdout:0/512: rename d3/d9/da/d48/c17 to d3/d9/da/d48/d25/ca8 0 2024-03-16T09:01:12.392 INFO:tasks.workunit.client.0.smithi007.stdout:0/513: stat d3/d9/da/d18/d23 0 2024-03-16T09:01:12.392 INFO:tasks.workunit.client.0.smithi007.stdout:0/514: creat d3/d9/da/d48/fa9 x:0 0 0 2024-03-16T09:01:12.392 INFO:tasks.workunit.client.0.smithi007.stdout:0/515: getdents d3/d9/da/d92/d9f 0 2024-03-16T09:01:12.393 INFO:tasks.workunit.client.1.smithi019.stdout:5/198: rename d6/d1f/d21 to d6/dd/d37 0 2024-03-16T09:01:12.394 INFO:tasks.workunit.client.0.smithi007.stdout:7/320: mkdir d0/d19/d1f/d69 0 2024-03-16T09:01:12.395 INFO:tasks.workunit.client.1.smithi019.stdout:9/181: link d1/l2b d1/d7/l3d 0 2024-03-16T09:01:12.396 INFO:tasks.workunit.client.1.smithi019.stdout:0/209: dwrite d7/d8/d22/f27 [4194304,4194304] 0 2024-03-16T09:01:12.396 INFO:tasks.workunit.client.1.smithi019.stdout:5/199: creat d6/dd/f38 x:0 0 0 2024-03-16T09:01:12.397 INFO:tasks.workunit.client.0.smithi007.stdout:0/516: mkdir d3/d9/da/d48/d25/d43/d1a/d58/d61/d84/daa 0 2024-03-16T09:01:12.398 INFO:tasks.workunit.client.1.smithi019.stdout:1/143: dread d16/d1f/f27 [0,4194304] 0 2024-03-16T09:01:12.398 INFO:tasks.workunit.client.0.smithi007.stdout:0/517: write d3/d1f/d70/f67 [199756,23859] 0 2024-03-16T09:01:12.398 INFO:tasks.workunit.client.1.smithi019.stdout:9/182: creat d1/d7/f3e x:0 0 0 2024-03-16T09:01:12.402 INFO:tasks.workunit.client.1.smithi019.stdout:0/210: rmdir d7/d8 39 2024-03-16T09:01:12.403 INFO:tasks.workunit.client.1.smithi019.stdout:0/211: fsync d7/fd 0 2024-03-16T09:01:12.405 INFO:tasks.workunit.client.1.smithi019.stdout:1/144: rename l15 to d16/d18/l2a 0 2024-03-16T09:01:12.405 INFO:tasks.workunit.client.1.smithi019.stdout:1/145: stat d16/d1f/c28 0 2024-03-16T09:01:12.405 INFO:tasks.workunit.client.1.smithi019.stdout:1/146: readlink d16/l25 0 2024-03-16T09:01:12.406 INFO:tasks.workunit.client.1.smithi019.stdout:1/147: creat d16/f2b x:0 0 0 2024-03-16T09:01:12.406 INFO:tasks.workunit.client.1.smithi019.stdout:1/148: readlink d16/d18/l26 0 2024-03-16T09:01:12.406 INFO:tasks.workunit.client.1.smithi019.stdout:1/149: fdatasync d16/d18/f1b 0 2024-03-16T09:01:12.406 INFO:tasks.workunit.client.1.smithi019.stdout:1/150: readlink d16/l25 0 2024-03-16T09:01:12.406 INFO:tasks.workunit.client.0.smithi007.stdout:0/518: read d3/d9/da/d18/d27/d34/f89 [3721419,85326] 0 2024-03-16T09:01:12.406 INFO:tasks.workunit.client.0.smithi007.stdout:0/519: write d3/d9/db/d1c/d72/f82 [568102,91226] 0 2024-03-16T09:01:12.407 INFO:tasks.workunit.client.0.smithi007.stdout:0/520: symlink d3/d9/da/d92/d9f/lab 0 2024-03-16T09:01:12.408 INFO:tasks.workunit.client.0.smithi007.stdout:7/321: write d0/dc/fd [928245,45296] 0 2024-03-16T09:01:12.409 INFO:tasks.workunit.client.1.smithi019.stdout:1/151: link l13 d16/l2c 0 2024-03-16T09:01:12.410 INFO:tasks.workunit.client.1.smithi019.stdout:9/183: rename d1/l36 to d1/l3f 0 2024-03-16T09:01:12.411 INFO:tasks.workunit.client.0.smithi007.stdout:9/515: dwrite d24/f2a [0,4194304] 0 2024-03-16T09:01:12.414 INFO:tasks.workunit.client.1.smithi019.stdout:3/241: rmdir d3/d11 39 2024-03-16T09:01:12.414 INFO:tasks.workunit.client.1.smithi019.stdout:3/242: chown d3/c48 480 1 2024-03-16T09:01:12.416 INFO:tasks.workunit.client.1.smithi019.stdout:3/243: unlink d3/d17/l29 0 2024-03-16T09:01:12.418 INFO:tasks.workunit.client.1.smithi019.stdout:3/244: unlink d3/d17/f32 0 2024-03-16T09:01:12.419 INFO:tasks.workunit.client.0.smithi007.stdout:7/322: read d0/f3 [168995,20562] 0 2024-03-16T09:01:12.419 INFO:tasks.workunit.client.0.smithi007.stdout:7/323: readlink d0/dc/d24/d25/d3a/l47 0 2024-03-16T09:01:12.419 INFO:tasks.workunit.client.0.smithi007.stdout:7/324: dread - d0/dc/d24/d25/d3a/f4f zero size 2024-03-16T09:01:12.419 INFO:tasks.workunit.client.0.smithi007.stdout:7/325: chown d0/dc/d24/d25/d3a/d63 22456670 1 2024-03-16T09:01:12.420 INFO:tasks.workunit.client.0.smithi007.stdout:7/326: rmdir d0/d19/d1f/d5d 39 2024-03-16T09:01:12.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/234: sync 2024-03-16T09:01:12.421 INFO:tasks.workunit.client.1.smithi019.stdout:8/218: sync 2024-03-16T09:01:12.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/235: chown d4/d6/d15/f33 3645620 1 2024-03-16T09:01:12.421 INFO:tasks.workunit.client.1.smithi019.stdout:8/219: creat d6/d2a/d3c/f46 x:0 0 0 2024-03-16T09:01:12.422 INFO:tasks.workunit.client.0.smithi007.stdout:7/327: unlink d0/dc/d24/d25/l23 0 2024-03-16T09:01:12.424 INFO:tasks.workunit.client.0.smithi007.stdout:7/328: creat d0/d19/f6a x:0 0 0 2024-03-16T09:01:12.424 INFO:tasks.workunit.client.1.smithi019.stdout:5/200: dwrite d6/f9 [0,4194304] 0 2024-03-16T09:01:12.424 INFO:tasks.workunit.client.1.smithi019.stdout:5/201: write d6/dd/d37/f24 [537720,27900] 0 2024-03-16T09:01:12.425 INFO:tasks.workunit.client.1.smithi019.stdout:5/202: read - d6/d1f/f2d zero size 2024-03-16T09:01:12.425 INFO:tasks.workunit.client.0.smithi007.stdout:9/516: write d24/d4a/d55/f5b [2881421,4943] 0 2024-03-16T09:01:12.427 INFO:tasks.workunit.client.0.smithi007.stdout:9/517: rename d24/d4a/d55/d56/d7c/l7e to d24/d4c/d91/l93 0 2024-03-16T09:01:12.428 INFO:tasks.workunit.client.1.smithi019.stdout:5/203: rename d6/d19/l1c to d6/d1f/l39 0 2024-03-16T09:01:12.429 INFO:tasks.workunit.client.1.smithi019.stdout:0/212: truncate d7/d8/d22/f27 6841486 0 2024-03-16T09:01:12.431 INFO:tasks.workunit.client.1.smithi019.stdout:3/245: read d3/f43 [205865,2284] 0 2024-03-16T09:01:12.431 INFO:tasks.workunit.client.1.smithi019.stdout:3/246: write d3/d11/f3b [712506,1563] 0 2024-03-16T09:01:12.432 INFO:tasks.workunit.client.1.smithi019.stdout:7/169: sync 2024-03-16T09:01:12.432 INFO:tasks.workunit.client.1.smithi019.stdout:6/349: sync 2024-03-16T09:01:12.432 INFO:tasks.workunit.client.1.smithi019.stdout:3/247: creat d3/d1b/d40/d4a/f4e x:0 0 0 2024-03-16T09:01:12.433 INFO:tasks.workunit.client.1.smithi019.stdout:3/248: chown d3/f43 133791841 1 2024-03-16T09:01:12.434 INFO:tasks.workunit.client.0.smithi007.stdout:7/329: dread d0/d19/d1f/d42/f57 [0,4194304] 0 2024-03-16T09:01:12.434 INFO:tasks.workunit.client.0.smithi007.stdout:0/521: dwrite d3/f6f [0,4194304] 0 2024-03-16T09:01:12.434 INFO:tasks.workunit.client.0.smithi007.stdout:0/522: chown d3/d9/da/d48/f13 222005 1 2024-03-16T09:01:12.435 INFO:tasks.workunit.client.1.smithi019.stdout:8/220: dread d6/f7 [4194304,4194304] 0 2024-03-16T09:01:12.439 INFO:tasks.workunit.client.1.smithi019.stdout:6/350: rename de/d26/d32/d4e/f6e to de/d26/d32/d4e/f79 0 2024-03-16T09:01:12.439 INFO:tasks.workunit.client.1.smithi019.stdout:6/351: write de/d1c/f58 [59540,110367] 0 2024-03-16T09:01:12.439 INFO:tasks.workunit.client.1.smithi019.stdout:6/352: fdatasync de/f1a 0 2024-03-16T09:01:12.441 INFO:tasks.workunit.client.1.smithi019.stdout:5/204: dread d6/fb [0,4194304] 0 2024-03-16T09:01:12.441 INFO:tasks.workunit.client.1.smithi019.stdout:1/152: dwrite d16/d18/f23 [0,4194304] 0 2024-03-16T09:01:12.441 INFO:tasks.workunit.client.1.smithi019.stdout:6/353: truncate de/f14 4547299 0 2024-03-16T09:01:12.441 INFO:tasks.workunit.client.1.smithi019.stdout:6/354: fdatasync de/d26/f2a 0 2024-03-16T09:01:12.445 INFO:tasks.workunit.client.1.smithi019.stdout:3/249: fdatasync d3/d11/f3b 0 2024-03-16T09:01:12.445 INFO:tasks.workunit.client.0.smithi007.stdout:0/523: dread d3/d1f/f7f [0,4194304] 0 2024-03-16T09:01:12.447 INFO:tasks.workunit.client.1.smithi019.stdout:3/250: mknod d3/c4f 0 2024-03-16T09:01:12.451 INFO:tasks.workunit.client.1.smithi019.stdout:0/213: dread d7/d1d/d28/f34 [0,4194304] 0 2024-03-16T09:01:12.453 INFO:tasks.workunit.client.1.smithi019.stdout:0/214: read d7/f2a [145307,112030] 0 2024-03-16T09:01:12.454 INFO:tasks.workunit.client.1.smithi019.stdout:8/221: write f5 [6461312,61402] 0 2024-03-16T09:01:12.455 INFO:tasks.workunit.client.1.smithi019.stdout:0/215: symlink d7/d8/d32/d36/l42 0 2024-03-16T09:01:12.455 INFO:tasks.workunit.client.1.smithi019.stdout:0/216: chown d7/le 13 1 2024-03-16T09:01:12.456 INFO:tasks.workunit.client.1.smithi019.stdout:0/217: stat d7/d8/c2b 0 2024-03-16T09:01:12.458 INFO:tasks.workunit.client.1.smithi019.stdout:0/218: chown l5 13097 1 2024-03-16T09:01:12.460 INFO:tasks.workunit.client.1.smithi019.stdout:0/219: rmdir d7 39 2024-03-16T09:01:12.463 INFO:tasks.workunit.client.1.smithi019.stdout:4/236: dwrite d4/d6/d15/d1f/f44 [0,4194304] 0 2024-03-16T09:01:12.465 INFO:tasks.workunit.client.1.smithi019.stdout:4/237: link d4/d6/d15/d1f/c27 d4/d1b/d49/c50 0 2024-03-16T09:01:12.466 INFO:tasks.workunit.client.0.smithi007.stdout:7/330: rmdir d0/dc/d38/d4e 39 2024-03-16T09:01:12.467 INFO:tasks.workunit.client.1.smithi019.stdout:4/238: symlink d4/d6/d15/l51 0 2024-03-16T09:01:12.467 INFO:tasks.workunit.client.1.smithi019.stdout:4/239: chown d4/d1b/d2a/f4e 323782121 1 2024-03-16T09:01:12.467 INFO:tasks.workunit.client.1.smithi019.stdout:4/240: write d4/d1b/d2a/f4e [5123333,109586] 0 2024-03-16T09:01:12.467 INFO:tasks.workunit.client.1.smithi019.stdout:4/241: chown d4/d6/d15/d1f/c27 495783568 1 2024-03-16T09:01:12.467 INFO:tasks.workunit.client.1.smithi019.stdout:4/242: write d4/f35 [126424,56026] 0 2024-03-16T09:01:12.468 INFO:tasks.workunit.client.1.smithi019.stdout:4/243: mknod d4/d1b/c52 0 2024-03-16T09:01:12.468 INFO:tasks.workunit.client.1.smithi019.stdout:4/244: read d4/d6/f24 [1128751,1201] 0 2024-03-16T09:01:12.469 INFO:tasks.workunit.client.1.smithi019.stdout:8/222: dread d6/d2a/d45/f32 [0,4194304] 0 2024-03-16T09:01:12.469 INFO:tasks.workunit.client.0.smithi007.stdout:7/331: creat d0/d19/d1f/d69/f6b x:0 0 0 2024-03-16T09:01:12.469 INFO:tasks.workunit.client.1.smithi019.stdout:8/223: creat d6/d2a/d45/f47 x:0 0 0 2024-03-16T09:01:12.470 INFO:tasks.workunit.client.1.smithi019.stdout:8/224: readlink l1 0 2024-03-16T09:01:12.471 INFO:tasks.workunit.client.1.smithi019.stdout:8/225: unlink d6/f15 0 2024-03-16T09:01:12.473 INFO:tasks.workunit.client.1.smithi019.stdout:8/226: rename d6/de/l38 to d6/d2a/d45/d19/l48 0 2024-03-16T09:01:12.487 INFO:tasks.workunit.client.0.smithi007.stdout:9/518: dwrite ff [0,4194304] 0 2024-03-16T09:01:12.496 INFO:tasks.workunit.client.0.smithi007.stdout:5/409: sync 2024-03-16T09:01:12.502 INFO:tasks.workunit.client.0.smithi007.stdout:0/524: dwrite d3/f6 [0,4194304] 0 2024-03-16T09:01:12.502 INFO:tasks.workunit.client.0.smithi007.stdout:0/525: chown d3/d9/da/d48/d25/d43/d1a/f30 4940 1 2024-03-16T09:01:12.502 INFO:tasks.workunit.client.0.smithi007.stdout:7/332: dwrite d0/dc/f2f [0,4194304] 0 2024-03-16T09:01:12.503 INFO:tasks.workunit.client.0.smithi007.stdout:7/333: write d0/d19/f6a [867064,55419] 0 2024-03-16T09:01:12.503 INFO:tasks.workunit.client.1.smithi019.stdout:4/245: dread d4/f35 [0,4194304] 0 2024-03-16T09:01:12.505 INFO:tasks.workunit.client.1.smithi019.stdout:4/246: creat d4/d1b/d49/f53 x:0 0 0 2024-03-16T09:01:12.505 INFO:tasks.workunit.client.1.smithi019.stdout:1/153: dwrite d16/d18/f23 [0,4194304] 0 2024-03-16T09:01:12.506 INFO:tasks.workunit.client.1.smithi019.stdout:8/227: dwrite d6/d2a/d45/d19/f43 [0,4194304] 0 2024-03-16T09:01:12.506 INFO:tasks.workunit.client.1.smithi019.stdout:4/247: mknod d4/d1b/d49/c54 0 2024-03-16T09:01:12.506 INFO:tasks.workunit.client.1.smithi019.stdout:4/248: read - d4/d1b/d49/f53 zero size 2024-03-16T09:01:12.507 INFO:tasks.workunit.client.1.smithi019.stdout:2/199: sync 2024-03-16T09:01:12.507 INFO:tasks.workunit.client.1.smithi019.stdout:3/251: dwrite d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:12.511 INFO:tasks.workunit.client.1.smithi019.stdout:8/228: unlink d6/de/f37 0 2024-03-16T09:01:12.513 INFO:tasks.workunit.client.1.smithi019.stdout:4/249: mknod d4/d6/c55 0 2024-03-16T09:01:12.513 INFO:tasks.workunit.client.1.smithi019.stdout:4/250: chown d4/d6/c47 992 1 2024-03-16T09:01:12.513 INFO:tasks.workunit.client.1.smithi019.stdout:4/251: fdatasync d4/d6/d15/f33 0 2024-03-16T09:01:12.514 INFO:tasks.workunit.client.1.smithi019.stdout:2/200: creat d5/d6/d2b/f3c x:0 0 0 2024-03-16T09:01:12.514 INFO:tasks.workunit.client.1.smithi019.stdout:2/201: creat d5/d2e/f3d x:0 0 0 2024-03-16T09:01:12.521 INFO:tasks.workunit.client.1.smithi019.stdout:8/229: dread f4 [0,4194304] 0 2024-03-16T09:01:12.524 INFO:tasks.workunit.client.0.smithi007.stdout:7/334: read d0/dc/f3d [3020360,61948] 0 2024-03-16T09:01:12.525 INFO:tasks.workunit.client.0.smithi007.stdout:7/335: truncate d0/f49 2287341 0 2024-03-16T09:01:12.530 INFO:tasks.workunit.client.0.smithi007.stdout:7/336: rename d0/dc/d24 to d0/d6c 0 2024-03-16T09:01:12.530 INFO:tasks.workunit.client.0.smithi007.stdout:7/337: read - d0/d19/d1f/f4b zero size 2024-03-16T09:01:12.530 INFO:tasks.workunit.client.0.smithi007.stdout:7/338: chown d0/dc/fd 1183342 1 2024-03-16T09:01:12.530 INFO:tasks.workunit.client.1.smithi019.stdout:1/154: dwrite d16/d18/f23 [0,4194304] 0 2024-03-16T09:01:12.530 INFO:tasks.workunit.client.1.smithi019.stdout:1/155: readlink l13 0 2024-03-16T09:01:12.530 INFO:tasks.workunit.client.1.smithi019.stdout:1/156: fsync d16/d18/f1a 0 2024-03-16T09:01:12.531 INFO:tasks.workunit.client.0.smithi007.stdout:7/339: rmdir d0/dc/d38/d4e 39 2024-03-16T09:01:12.531 INFO:tasks.workunit.client.0.smithi007.stdout:0/526: dwrite d3/f8b [0,4194304] 0 2024-03-16T09:01:12.532 INFO:tasks.workunit.client.0.smithi007.stdout:0/527: mknod d3/d1f/d70/cac 0 2024-03-16T09:01:12.534 INFO:tasks.workunit.client.1.smithi019.stdout:1/157: link d16/c1d d16/d18/c2d 0 2024-03-16T09:01:12.534 INFO:tasks.workunit.client.1.smithi019.stdout:1/158: creat d16/d1f/f2e x:0 0 0 2024-03-16T09:01:12.534 INFO:tasks.workunit.client.1.smithi019.stdout:1/159: write f12 [3444845,67473] 0 2024-03-16T09:01:12.570 INFO:tasks.workunit.client.1.smithi019.stdout:4/252: dwrite d4/d6/d15/d1f/f44 [0,4194304] 0 2024-03-16T09:01:12.571 INFO:tasks.workunit.client.1.smithi019.stdout:1/160: dwrite f14 [0,4194304] 0 2024-03-16T09:01:12.572 INFO:tasks.workunit.client.1.smithi019.stdout:1/161: creat d16/f2f x:0 0 0 2024-03-16T09:01:12.573 INFO:tasks.workunit.client.0.smithi007.stdout:7/340: dwrite d0/dc/f15 [4194304,4194304] 0 2024-03-16T09:01:12.573 INFO:tasks.workunit.client.0.smithi007.stdout:0/528: dwrite d3/d9/da/d48/f38 [0,4194304] 0 2024-03-16T09:01:12.573 INFO:tasks.workunit.client.1.smithi019.stdout:8/230: dwrite d6/fa [0,4194304] 0 2024-03-16T09:01:12.573 INFO:tasks.workunit.client.1.smithi019.stdout:4/253: truncate d4/d1b/f28 4166165 0 2024-03-16T09:01:12.573 INFO:tasks.workunit.client.1.smithi019.stdout:4/254: chown d4/d6/d15/l2b 8885958 1 2024-03-16T09:01:12.575 INFO:tasks.workunit.client.0.smithi007.stdout:7/341: rename d0/c51 to d0/d6c/d43/c6d 0 2024-03-16T09:01:12.575 INFO:tasks.workunit.client.1.smithi019.stdout:1/162: link d16/d18/f23 d16/d1f/f30 0 2024-03-16T09:01:12.576 INFO:tasks.workunit.client.0.smithi007.stdout:0/529: mknod d3/d9/db/d1c/cad 0 2024-03-16T09:01:12.579 INFO:tasks.workunit.client.0.smithi007.stdout:0/530: rmdir d3/d9/da/d48/d25/d43/d1a/d58/d61/d84/daa 0 2024-03-16T09:01:12.579 INFO:tasks.workunit.client.0.smithi007.stdout:0/531: fdatasync d3/d9/db/d2c/f5e 0 2024-03-16T09:01:12.580 INFO:tasks.workunit.client.1.smithi019.stdout:8/231: rename d6/d2a/d45/d19/f18 to d6/de/f49 0 2024-03-16T09:01:12.580 INFO:tasks.workunit.client.1.smithi019.stdout:8/232: chown d6/d2a/d45/d19/f31 1462 1 2024-03-16T09:01:12.580 INFO:tasks.workunit.client.1.smithi019.stdout:8/233: fsync d6/d2a/d45/d19/f1a 0 2024-03-16T09:01:12.580 INFO:tasks.workunit.client.1.smithi019.stdout:1/163: mkdir d16/d31 0 2024-03-16T09:01:12.583 INFO:tasks.workunit.client.1.smithi019.stdout:8/234: rename d6/d2a/d45/d19/d28/d3d to d6/de/d24/d30/d4a 0 2024-03-16T09:01:12.584 INFO:tasks.workunit.client.1.smithi019.stdout:4/255: dread d4/d1b/d2a/f4e [0,4194304] 0 2024-03-16T09:01:12.584 INFO:tasks.workunit.client.1.smithi019.stdout:4/256: creat d4/d1b/d49/f56 x:0 0 0 2024-03-16T09:01:12.587 INFO:tasks.workunit.client.1.smithi019.stdout:8/235: rmdir d6/d2a/d45/d19/d28 39 2024-03-16T09:01:12.590 INFO:tasks.workunit.client.1.smithi019.stdout:4/257: creat d4/d1b/f57 x:0 0 0 2024-03-16T09:01:12.591 INFO:tasks.workunit.client.1.smithi019.stdout:8/236: mknod d6/d2a/c4b 0 2024-03-16T09:01:12.593 INFO:tasks.workunit.client.1.smithi019.stdout:4/258: unlink d4/d1b/d49/c54 0 2024-03-16T09:01:12.595 INFO:tasks.workunit.client.1.smithi019.stdout:8/237: rename d6/d2a/d45/d2c/c41 to d6/d2a/c4c 0 2024-03-16T09:01:12.597 INFO:tasks.workunit.client.1.smithi019.stdout:4/259: mkdir d4/d1b/d49/d58 0 2024-03-16T09:01:12.597 INFO:tasks.workunit.client.1.smithi019.stdout:4/260: chown d4/d6/d15/l2b 6 1 2024-03-16T09:01:12.597 INFO:tasks.workunit.client.1.smithi019.stdout:4/261: truncate d4/d6/f32 758680 0 2024-03-16T09:01:12.597 INFO:tasks.workunit.client.1.smithi019.stdout:4/262: chown d4/f35 12789528 1 2024-03-16T09:01:12.597 INFO:tasks.workunit.client.1.smithi019.stdout:4/263: chown d4/d6/c43 496 1 2024-03-16T09:01:12.598 INFO:tasks.workunit.client.1.smithi019.stdout:8/238: mknod d6/de/d24/d30/c4d 0 2024-03-16T09:01:12.600 INFO:tasks.workunit.client.1.smithi019.stdout:8/239: truncate d6/d2a/d45/d19/f1a 3537232 0 2024-03-16T09:01:12.601 INFO:tasks.workunit.client.1.smithi019.stdout:4/264: getdents d4/d6/d15/d1f 0 2024-03-16T09:01:12.601 INFO:tasks.workunit.client.1.smithi019.stdout:4/265: readlink d4/d1b/d2a/l37 0 2024-03-16T09:01:12.602 INFO:tasks.workunit.client.1.smithi019.stdout:8/240: creat d6/de/d24/d30/d4a/f4e x:0 0 0 2024-03-16T09:01:12.602 INFO:tasks.workunit.client.1.smithi019.stdout:8/241: creat d6/de/d24/d30/f4f x:0 0 0 2024-03-16T09:01:12.603 INFO:tasks.workunit.client.1.smithi019.stdout:4/266: symlink d4/d6/d15/d1f/l59 0 2024-03-16T09:01:12.605 INFO:tasks.workunit.client.1.smithi019.stdout:4/267: truncate d4/d1b/f1c 884012 0 2024-03-16T09:01:12.605 INFO:tasks.workunit.client.1.smithi019.stdout:4/268: stat d4/d6/d15/f2e 0 2024-03-16T09:01:12.606 INFO:tasks.workunit.client.1.smithi019.stdout:4/269: symlink d4/d1b/d2a/d4f/l5a 0 2024-03-16T09:01:12.608 INFO:tasks.workunit.client.1.smithi019.stdout:4/270: creat d4/d6/d15/d1f/d2c/f5b x:0 0 0 2024-03-16T09:01:12.611 INFO:tasks.workunit.client.1.smithi019.stdout:9/184: sync 2024-03-16T09:01:12.612 INFO:tasks.workunit.client.1.smithi019.stdout:4/271: dread d4/f26 [0,4194304] 0 2024-03-16T09:01:12.612 INFO:tasks.workunit.client.0.smithi007.stdout:7/342: dwrite d0/d6c/f52 [0,4194304] 0 2024-03-16T09:01:12.613 INFO:tasks.workunit.client.1.smithi019.stdout:1/164: dwrite d16/f2f [0,4194304] 0 2024-03-16T09:01:12.613 INFO:tasks.workunit.client.0.smithi007.stdout:0/532: dwrite d3/d9/da/d18/f59 [0,4194304] 0 2024-03-16T09:01:12.615 INFO:tasks.workunit.client.0.smithi007.stdout:0/533: symlink d3/d9/da/d18/lae 0 2024-03-16T09:01:12.625 INFO:tasks.workunit.client.1.smithi019.stdout:1/165: write fb [2688415,124375] 0 2024-03-16T09:01:12.632 INFO:tasks.workunit.client.0.smithi007.stdout:7/343: write d0/dc/fe [563131,75382] 0 2024-03-16T09:01:12.632 INFO:tasks.workunit.client.1.smithi019.stdout:8/242: dwrite d6/d2a/d3c/f44 [0,4194304] 0 2024-03-16T09:01:12.633 INFO:tasks.workunit.client.1.smithi019.stdout:8/243: dread d6/de/ff [0,4194304] 0 2024-03-16T09:01:12.635 INFO:tasks.workunit.client.1.smithi019.stdout:9/185: dwrite d1/d7/de/d24/f39 [0,4194304] 0 2024-03-16T09:01:12.635 INFO:tasks.workunit.client.1.smithi019.stdout:9/186: creat d1/d7/de/d24/f40 x:0 0 0 2024-03-16T09:01:12.635 INFO:tasks.workunit.client.1.smithi019.stdout:9/187: chown d1/d7/d21/d27/c2f 2 1 2024-03-16T09:01:12.635 INFO:tasks.workunit.client.1.smithi019.stdout:9/188: creat d1/d7/f41 x:0 0 0 2024-03-16T09:01:12.635 INFO:tasks.workunit.client.1.smithi019.stdout:9/189: creat d1/d7/de/d24/f42 x:0 0 0 2024-03-16T09:01:12.636 INFO:tasks.workunit.client.0.smithi007.stdout:6/341: sync 2024-03-16T09:01:12.636 INFO:tasks.workunit.client.0.smithi007.stdout:4/335: sync 2024-03-16T09:01:12.637 INFO:tasks.workunit.client.0.smithi007.stdout:6/342: mknod d2/d11/d3b/c71 0 2024-03-16T09:01:12.637 INFO:tasks.workunit.client.0.smithi007.stdout:6/343: truncate d2/d13/d1f/f5c 370191 0 2024-03-16T09:01:12.645 INFO:tasks.workunit.client.1.smithi019.stdout:1/166: mkdir d16/d31/d32 0 2024-03-16T09:01:12.648 INFO:tasks.workunit.client.1.smithi019.stdout:8/244: truncate d6/f7 5386341 0 2024-03-16T09:01:12.648 INFO:tasks.workunit.client.1.smithi019.stdout:8/245: truncate d6/d2a/d45/f47 323555 0 2024-03-16T09:01:12.655 INFO:tasks.workunit.client.1.smithi019.stdout:1/167: write d16/d18/f1a [4111139,119741] 0 2024-03-16T09:01:12.655 INFO:tasks.workunit.client.1.smithi019.stdout:1/168: write f10 [899069,9188] 0 2024-03-16T09:01:12.656 INFO:tasks.workunit.client.0.smithi007.stdout:7/344: dwrite d0/fa [0,4194304] 0 2024-03-16T09:01:12.658 INFO:tasks.workunit.client.0.smithi007.stdout:7/345: creat d0/d6c/d25/d3a/d63/f6e x:0 0 0 2024-03-16T09:01:12.659 INFO:tasks.workunit.client.0.smithi007.stdout:4/336: dwrite d4/d7/d6b/f6d [0,4194304] 0 2024-03-16T09:01:12.659 INFO:tasks.workunit.client.0.smithi007.stdout:4/337: creat d4/d7/d6b/f71 x:0 0 0 2024-03-16T09:01:12.659 INFO:tasks.workunit.client.1.smithi019.stdout:8/246: read d6/de/d24/f35 [1660038,101444] 0 2024-03-16T09:01:12.659 INFO:tasks.workunit.client.0.smithi007.stdout:7/346: symlink d0/d19/l6f 0 2024-03-16T09:01:12.660 INFO:tasks.workunit.client.0.smithi007.stdout:4/338: dread d4/d14/d1c/f37 [0,4194304] 0 2024-03-16T09:01:12.666 INFO:tasks.workunit.client.0.smithi007.stdout:4/339: dread d4/ff [0,4194304] 0 2024-03-16T09:01:12.669 INFO:tasks.workunit.client.0.smithi007.stdout:4/340: getdents d4/d14/d1c/d5b 0 2024-03-16T09:01:12.670 INFO:tasks.workunit.client.0.smithi007.stdout:4/341: symlink d4/l72 0 2024-03-16T09:01:12.670 INFO:tasks.workunit.client.0.smithi007.stdout:7/347: dread d0/d6c/f3b [0,4194304] 0 2024-03-16T09:01:12.671 INFO:tasks.workunit.client.0.smithi007.stdout:7/348: mkdir d0/d6c/d25/d3a/d70 0 2024-03-16T09:01:12.679 INFO:tasks.workunit.client.0.smithi007.stdout:4/342: dread d4/d7/d6b/f6d [0,4194304] 0 2024-03-16T09:01:12.679 INFO:tasks.workunit.client.0.smithi007.stdout:4/343: rename d4/d14 to d4/d14/d1c/d5b/d67/d73 22 2024-03-16T09:01:12.684 INFO:tasks.workunit.client.0.smithi007.stdout:7/349: dwrite d0/f3 [0,4194304] 0 2024-03-16T09:01:12.685 INFO:tasks.workunit.client.0.smithi007.stdout:7/350: link d0/d6c/d43/f58 d0/d6c/d25/d3a/d63/f71 0 2024-03-16T09:01:12.686 INFO:tasks.workunit.client.0.smithi007.stdout:7/351: symlink d0/dc/d5a/l72 0 2024-03-16T09:01:12.686 INFO:tasks.workunit.client.0.smithi007.stdout:7/352: chown d0/d19/d1f/d42/f62 5 1 2024-03-16T09:01:12.687 INFO:tasks.workunit.client.0.smithi007.stdout:4/344: dread d4/fa [0,4194304] 0 2024-03-16T09:01:12.689 INFO:tasks.workunit.client.0.smithi007.stdout:4/345: creat d4/d14/d1c/d5b/f74 x:0 0 0 2024-03-16T09:01:12.691 INFO:tasks.workunit.client.0.smithi007.stdout:7/353: rename d0/d6c/d43 to d0/d6c/d25/d73 0 2024-03-16T09:01:12.691 INFO:tasks.workunit.client.0.smithi007.stdout:7/354: fsync d0/f11 0 2024-03-16T09:01:12.692 INFO:tasks.workunit.client.0.smithi007.stdout:4/346: rename d4/c16 to d4/d7/c75 0 2024-03-16T09:01:12.708 INFO:tasks.workunit.client.0.smithi007.stdout:7/355: dwrite d0/d19/f48 [0,4194304] 0 2024-03-16T09:01:12.708 INFO:tasks.workunit.client.0.smithi007.stdout:7/356: chown d0/d6c/l30 93 1 2024-03-16T09:01:12.715 INFO:tasks.workunit.client.0.smithi007.stdout:7/357: creat d0/dc/d5a/f74 x:0 0 0 2024-03-16T09:01:12.715 INFO:tasks.workunit.client.0.smithi007.stdout:7/358: symlink d0/d19/d1f/d69/l75 0 2024-03-16T09:01:12.715 INFO:tasks.workunit.client.0.smithi007.stdout:7/359: truncate d0/dc/f15 2878206 0 2024-03-16T09:01:12.715 INFO:tasks.workunit.client.0.smithi007.stdout:7/360: write d0/d19/d1f/d42/f45 [624115,7245] 0 2024-03-16T09:01:12.715 INFO:tasks.workunit.client.0.smithi007.stdout:7/361: rmdir d0/d6c/d25/d3a/d70 0 2024-03-16T09:01:12.715 INFO:tasks.workunit.client.0.smithi007.stdout:7/362: truncate d0/d19/f2d 457126 0 2024-03-16T09:01:12.715 INFO:tasks.workunit.client.0.smithi007.stdout:7/363: mkdir d0/d19/d1f/d76 0 2024-03-16T09:01:12.717 INFO:tasks.workunit.client.0.smithi007.stdout:7/364: creat d0/d6c/d25/d3a/f77 x:0 0 0 2024-03-16T09:01:12.717 INFO:tasks.workunit.client.0.smithi007.stdout:4/347: dwrite d4/d14/d1c/d1e/f4c [0,4194304] 0 2024-03-16T09:01:12.717 INFO:tasks.workunit.client.0.smithi007.stdout:4/348: chown d4/d14/f35 26748796 1 2024-03-16T09:01:12.718 INFO:tasks.workunit.client.0.smithi007.stdout:7/365: rename d0/d19/d1f/l2e to d0/d19/d1f/l78 0 2024-03-16T09:01:12.720 INFO:tasks.workunit.client.1.smithi019.stdout:9/190: mkdir d1/d7/de/d43 0 2024-03-16T09:01:12.720 INFO:tasks.workunit.client.0.smithi007.stdout:4/349: rename d4/d14/c55 to d4/d7/d6b/c76 0 2024-03-16T09:01:12.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/169: creat d16/d31/f33 x:0 0 0 2024-03-16T09:01:12.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/170: creat d16/d1f/f34 x:0 0 0 2024-03-16T09:01:12.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/171: write d16/f2b [611481,38832] 0 2024-03-16T09:01:12.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/172: dread - d16/d1f/f2e zero size 2024-03-16T09:01:12.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/173: chown f14 27054218 1 2024-03-16T09:01:12.726 INFO:tasks.workunit.client.1.smithi019.stdout:6/355: sync 2024-03-16T09:01:12.726 INFO:tasks.workunit.client.1.smithi019.stdout:7/170: sync 2024-03-16T09:01:12.727 INFO:tasks.workunit.client.1.smithi019.stdout:9/191: mkdir d1/d7/de/d24/d44 0 2024-03-16T09:01:12.727 INFO:tasks.workunit.client.1.smithi019.stdout:9/192: creat d1/d7/f45 x:0 0 0 2024-03-16T09:01:12.727 INFO:tasks.workunit.client.1.smithi019.stdout:9/193: dread - d1/d7/f41 zero size 2024-03-16T09:01:12.728 INFO:tasks.workunit.client.1.smithi019.stdout:0/220: sync 2024-03-16T09:01:12.728 INFO:tasks.workunit.client.1.smithi019.stdout:5/205: sync 2024-03-16T09:01:12.728 INFO:tasks.workunit.client.1.smithi019.stdout:5/206: fsync f4 0 2024-03-16T09:01:12.728 INFO:tasks.workunit.client.1.smithi019.stdout:6/356: dread de/d26/d32/f37 [0,4194304] 0 2024-03-16T09:01:12.728 INFO:tasks.workunit.client.1.smithi019.stdout:1/174: stat d16/d18/c2d 0 2024-03-16T09:01:12.729 INFO:tasks.workunit.client.0.smithi007.stdout:7/366: rename d0/c5 to d0/dc/c79 0 2024-03-16T09:01:12.729 INFO:tasks.workunit.client.0.smithi007.stdout:7/367: chown d0/d6c/d25/d73/f58 4025116 1 2024-03-16T09:01:12.729 INFO:tasks.workunit.client.0.smithi007.stdout:7/368: write d0/dc/f2f [4593190,45697] 0 2024-03-16T09:01:12.731 INFO:tasks.workunit.client.0.smithi007.stdout:4/350: mkdir d4/d14/d64/d68/d77 0 2024-03-16T09:01:12.731 INFO:tasks.workunit.client.0.smithi007.stdout:4/351: fsync d4/d7/d6b/f6d 0 2024-03-16T09:01:12.731 INFO:tasks.workunit.client.1.smithi019.stdout:9/194: unlink d1/d7/d21/d27/c3b 0 2024-03-16T09:01:12.732 INFO:tasks.workunit.client.1.smithi019.stdout:5/207: symlink d6/d1f/l3a 0 2024-03-16T09:01:12.733 INFO:tasks.workunit.client.0.smithi007.stdout:4/352: mkdir d4/d7/d78 0 2024-03-16T09:01:12.733 INFO:tasks.workunit.client.0.smithi007.stdout:4/353: chown d4/d7/l5a 2 1 2024-03-16T09:01:12.733 INFO:tasks.workunit.client.0.smithi007.stdout:4/354: chown d4/d14/l57 396878 1 2024-03-16T09:01:12.734 INFO:tasks.workunit.client.1.smithi019.stdout:2/202: sync 2024-03-16T09:01:12.734 INFO:tasks.workunit.client.1.smithi019.stdout:3/252: sync 2024-03-16T09:01:12.734 INFO:tasks.workunit.client.1.smithi019.stdout:4/272: sync 2024-03-16T09:01:12.734 INFO:tasks.workunit.client.1.smithi019.stdout:8/247: sync 2024-03-16T09:01:12.734 INFO:tasks.workunit.client.1.smithi019.stdout:2/203: write d5/d6/f11 [1472716,108554] 0 2024-03-16T09:01:12.736 INFO:tasks.workunit.client.1.smithi019.stdout:3/253: creat d3/d17/f50 x:0 0 0 2024-03-16T09:01:12.736 INFO:tasks.workunit.client.1.smithi019.stdout:3/254: creat d3/f51 x:0 0 0 2024-03-16T09:01:12.736 INFO:tasks.workunit.client.1.smithi019.stdout:6/357: creat de/f7a x:0 0 0 2024-03-16T09:01:12.736 INFO:tasks.workunit.client.0.smithi007.stdout:7/369: rename d0/d6c/d25/d3a/d63 to d0/d6c/d25/d7a 0 2024-03-16T09:01:12.737 INFO:tasks.workunit.client.1.smithi019.stdout:5/208: mknod d6/d19/d34/c3b 0 2024-03-16T09:01:12.738 INFO:tasks.workunit.client.1.smithi019.stdout:2/204: link d5/ca d5/d6/d2b/d2f/c3e 0 2024-03-16T09:01:12.740 INFO:tasks.workunit.client.1.smithi019.stdout:7/171: read d9/f16 [3739746,101642] 0 2024-03-16T09:01:12.740 INFO:tasks.workunit.client.1.smithi019.stdout:3/255: write d3/d11/f3b [173255,56991] 0 2024-03-16T09:01:12.742 INFO:tasks.workunit.client.1.smithi019.stdout:2/205: truncate d5/d6/f21 2495551 0 2024-03-16T09:01:12.742 INFO:tasks.workunit.client.1.smithi019.stdout:7/172: creat d9/db/d12/f2f x:0 0 0 2024-03-16T09:01:12.743 INFO:tasks.workunit.client.1.smithi019.stdout:0/221: dread d7/d1d/d28/f34 [0,4194304] 0 2024-03-16T09:01:12.745 INFO:tasks.workunit.client.0.smithi007.stdout:7/370: read d0/d19/d1f/f54 [3041169,118391] 0 2024-03-16T09:01:12.745 INFO:tasks.workunit.client.0.smithi007.stdout:7/371: readlink d0/d6c/d25/d3a/l47 0 2024-03-16T09:01:12.746 INFO:tasks.workunit.client.0.smithi007.stdout:7/372: creat d0/d19/d1f/d42/f7b x:0 0 0 2024-03-16T09:01:12.747 INFO:tasks.workunit.client.1.smithi019.stdout:3/256: link d3/d17/l4b d3/d1b/d40/d4a/d2c/l52 0 2024-03-16T09:01:12.747 INFO:tasks.workunit.client.1.smithi019.stdout:2/206: symlink d5/d6/d2b/l3f 0 2024-03-16T09:01:12.748 INFO:tasks.workunit.client.0.smithi007.stdout:4/355: truncate d4/d14/d1c/d1e/f4c 2856888 0 2024-03-16T09:01:12.749 INFO:tasks.workunit.client.1.smithi019.stdout:1/175: write d16/d18/f23 [3662752,19779] 0 2024-03-16T09:01:12.749 INFO:tasks.workunit.client.1.smithi019.stdout:7/173: mkdir d9/da/d13/d18/d30 0 2024-03-16T09:01:12.749 INFO:tasks.workunit.client.1.smithi019.stdout:7/174: stat d9/db 0 2024-03-16T09:01:12.749 INFO:tasks.workunit.client.1.smithi019.stdout:1/176: dread - d16/f19 zero size 2024-03-16T09:01:12.752 INFO:tasks.workunit.client.1.smithi019.stdout:7/175: creat d9/d25/f31 x:0 0 0 2024-03-16T09:01:12.754 INFO:tasks.workunit.client.0.smithi007.stdout:4/356: creat d4/d14/d1c/d5b/d67/f79 x:0 0 0 2024-03-16T09:01:12.757 INFO:tasks.workunit.client.0.smithi007.stdout:4/357: mknod d4/d7/d78/c7a 0 2024-03-16T09:01:12.760 INFO:tasks.workunit.client.1.smithi019.stdout:7/176: mkdir d9/da/d13/d18/d1a/d32 0 2024-03-16T09:01:12.769 INFO:tasks.workunit.client.0.smithi007.stdout:4/358: dread d4/d14/f19 [0,4194304] 0 2024-03-16T09:01:12.772 INFO:tasks.workunit.client.1.smithi019.stdout:7/177: dread d9/da/d13/f17 [0,4194304] 0 2024-03-16T09:01:12.772 INFO:tasks.workunit.client.1.smithi019.stdout:7/178: fdatasync f6 0 2024-03-16T09:01:12.773 INFO:tasks.workunit.client.1.smithi019.stdout:7/179: truncate d9/da/f24 508726 0 2024-03-16T09:01:12.774 INFO:tasks.workunit.client.1.smithi019.stdout:7/180: mkdir d9/da/d13/d18/d1a/d32/d33 0 2024-03-16T09:01:12.776 INFO:tasks.workunit.client.1.smithi019.stdout:0/222: dwrite d7/d1d/d28/f34 [0,4194304] 0 2024-03-16T09:01:12.776 INFO:tasks.workunit.client.1.smithi019.stdout:1/177: dread d16/d18/f23 [0,4194304] 0 2024-03-16T09:01:12.776 INFO:tasks.workunit.client.1.smithi019.stdout:8/248: dwrite d6/d2a/d45/d19/f1c [0,4194304] 0 2024-03-16T09:01:12.776 INFO:tasks.workunit.client.1.smithi019.stdout:7/181: creat d9/d25/d28/f34 x:0 0 0 2024-03-16T09:01:12.776 INFO:tasks.workunit.client.1.smithi019.stdout:0/223: truncate d7/d8/d22/f27 5856776 0 2024-03-16T09:01:12.776 INFO:tasks.workunit.client.1.smithi019.stdout:0/224: chown d7/d29 41214 1 2024-03-16T09:01:12.778 INFO:tasks.workunit.client.1.smithi019.stdout:8/249: truncate d6/de/f1b 3386325 0 2024-03-16T09:01:12.778 INFO:tasks.workunit.client.1.smithi019.stdout:8/250: readlink d6/l17 0 2024-03-16T09:01:12.779 INFO:tasks.workunit.client.1.smithi019.stdout:7/182: symlink d9/da/d13/d18/d1a/d32/l35 0 2024-03-16T09:01:12.779 INFO:tasks.workunit.client.1.smithi019.stdout:7/183: chown d9/f23 25344 1 2024-03-16T09:01:12.781 INFO:tasks.workunit.client.1.smithi019.stdout:0/225: link d7/d1d/d28/c20 d7/d29/c43 0 2024-03-16T09:01:12.781 INFO:tasks.workunit.client.1.smithi019.stdout:0/226: write d7/d8/f12 [4266673,78327] 0 2024-03-16T09:01:12.781 INFO:tasks.workunit.client.1.smithi019.stdout:8/251: creat d6/d2a/d45/d19/d28/f50 x:0 0 0 2024-03-16T09:01:12.782 INFO:tasks.workunit.client.1.smithi019.stdout:7/184: rename d9/da/cf to d9/d25/c36 0 2024-03-16T09:01:12.783 INFO:tasks.workunit.client.1.smithi019.stdout:8/252: unlink d6/d2a/d45/d19/d28/f39 0 2024-03-16T09:01:12.784 INFO:tasks.workunit.client.1.smithi019.stdout:0/227: creat d7/d1d/d28/f44 x:0 0 0 2024-03-16T09:01:12.785 INFO:tasks.workunit.client.0.smithi007.stdout:4/359: dread d4/f6 [0,4194304] 0 2024-03-16T09:01:12.785 INFO:tasks.workunit.client.1.smithi019.stdout:7/185: creat d9/da/d13/d18/d30/f37 x:0 0 0 2024-03-16T09:01:12.786 INFO:tasks.workunit.client.1.smithi019.stdout:0/228: mknod d7/d29/c45 0 2024-03-16T09:01:12.786 INFO:tasks.workunit.client.1.smithi019.stdout:0/229: stat d7/f19 0 2024-03-16T09:01:12.786 INFO:tasks.workunit.client.1.smithi019.stdout:0/230: readlink d7/d1d/d28/l1f 0 2024-03-16T09:01:12.787 INFO:tasks.workunit.client.1.smithi019.stdout:0/231: creat d7/f46 x:0 0 0 2024-03-16T09:01:12.787 INFO:tasks.workunit.client.1.smithi019.stdout:0/232: chown d7/d1d/d28/c20 2 1 2024-03-16T09:01:12.788 INFO:tasks.workunit.client.1.smithi019.stdout:0/233: rmdir d7/d8/d32 39 2024-03-16T09:01:12.797 INFO:tasks.workunit.client.1.smithi019.stdout:7/186: dread f6 [0,4194304] 0 2024-03-16T09:01:12.797 INFO:tasks.workunit.client.0.smithi007.stdout:7/373: dwrite d0/d19/d1f/f54 [4194304,4194304] 0 2024-03-16T09:01:12.798 INFO:tasks.workunit.client.1.smithi019.stdout:6/358: dwrite de/d1c/d1d/d2c/f3a [4194304,4194304] 0 2024-03-16T09:01:12.798 INFO:tasks.workunit.client.1.smithi019.stdout:6/359: dread - de/f45 zero size 2024-03-16T09:01:12.798 INFO:tasks.workunit.client.1.smithi019.stdout:3/257: dwrite d3/d1b/f1c [0,4194304] 0 2024-03-16T09:01:12.799 INFO:tasks.workunit.client.1.smithi019.stdout:1/178: dwrite d16/f19 [0,4194304] 0 2024-03-16T09:01:12.799 INFO:tasks.workunit.client.1.smithi019.stdout:6/360: write de/d26/d32/d4e/f79 [261310,100027] 0 2024-03-16T09:01:12.799 INFO:tasks.workunit.client.1.smithi019.stdout:5/209: dwrite d6/d19/f29 [0,4194304] 0 2024-03-16T09:01:12.801 INFO:tasks.workunit.client.1.smithi019.stdout:5/210: write d6/dd/d37/f24 [171523,4668] 0 2024-03-16T09:01:12.801 INFO:tasks.workunit.client.1.smithi019.stdout:5/211: creat d6/d19/f3c x:0 0 0 2024-03-16T09:01:12.803 INFO:tasks.workunit.client.0.smithi007.stdout:4/360: dread d4/d14/d1c/d1e/f4c [0,4194304] 0 2024-03-16T09:01:12.806 INFO:tasks.workunit.client.1.smithi019.stdout:1/179: dread f10 [0,4194304] 0 2024-03-16T09:01:12.806 INFO:tasks.workunit.client.1.smithi019.stdout:1/180: dread - d16/d1f/f2e zero size 2024-03-16T09:01:12.808 INFO:tasks.workunit.client.1.smithi019.stdout:7/187: write f8 [3624396,9794] 0 2024-03-16T09:01:12.809 INFO:tasks.workunit.client.1.smithi019.stdout:7/188: dread - d9/da/d13/d18/d1a/f2b zero size 2024-03-16T09:01:12.809 INFO:tasks.workunit.client.0.smithi007.stdout:7/374: rename d0/dc/f2f to d0/dc/d38/f7c 0 2024-03-16T09:01:12.810 INFO:tasks.workunit.client.0.smithi007.stdout:4/361: unlink d4/d7/f50 0 2024-03-16T09:01:12.811 INFO:tasks.workunit.client.1.smithi019.stdout:5/212: rename d6/f18 to d6/dd/d37/f3d 0 2024-03-16T09:01:12.812 INFO:tasks.workunit.client.0.smithi007.stdout:7/375: rmdir d0/d6c 39 2024-03-16T09:01:12.813 INFO:tasks.workunit.client.1.smithi019.stdout:7/189: getdents d9/da/d13/d18/d1a 0 2024-03-16T09:01:12.814 INFO:tasks.workunit.client.0.smithi007.stdout:8/691: sync 2024-03-16T09:01:12.814 INFO:tasks.workunit.client.0.smithi007.stdout:5/410: sync 2024-03-16T09:01:12.815 INFO:tasks.workunit.client.0.smithi007.stdout:1/393: sync 2024-03-16T09:01:12.815 INFO:tasks.workunit.client.0.smithi007.stdout:1/394: write d9/dd/d1b/f25 [5459301,100356] 0 2024-03-16T09:01:12.815 INFO:tasks.workunit.client.0.smithi007.stdout:8/692: write d0/d26/d2d/d43/d4c/dba/fd7 [630048,98983] 0 2024-03-16T09:01:12.815 INFO:tasks.workunit.client.0.smithi007.stdout:9/519: sync 2024-03-16T09:01:12.815 INFO:tasks.workunit.client.0.smithi007.stdout:3/498: sync 2024-03-16T09:01:12.815 INFO:tasks.workunit.client.0.smithi007.stdout:2/523: sync 2024-03-16T09:01:12.816 INFO:tasks.workunit.client.0.smithi007.stdout:9/520: read d24/f43 [2737300,104092] 0 2024-03-16T09:01:12.816 INFO:tasks.workunit.client.0.smithi007.stdout:2/524: creat dc/fab x:0 0 0 2024-03-16T09:01:12.816 INFO:tasks.workunit.client.0.smithi007.stdout:2/525: dread - dc/df/d24/f63 zero size 2024-03-16T09:01:12.816 INFO:tasks.workunit.client.0.smithi007.stdout:2/526: dread - dc/df/d24/f63 zero size 2024-03-16T09:01:12.816 INFO:tasks.workunit.client.0.smithi007.stdout:2/527: fsync dc/df/d24/d28/f66 0 2024-03-16T09:01:12.816 INFO:tasks.workunit.client.0.smithi007.stdout:7/376: symlink d0/d6c/d25/d3a/l7d 0 2024-03-16T09:01:12.816 INFO:tasks.workunit.client.0.smithi007.stdout:4/362: symlink d4/d7/d6b/l7b 0 2024-03-16T09:01:12.817 INFO:tasks.workunit.client.0.smithi007.stdout:7/377: chown d0/d19/f2d 32990 1 2024-03-16T09:01:12.817 INFO:tasks.workunit.client.0.smithi007.stdout:7/378: truncate d0/f55 112216 0 2024-03-16T09:01:12.818 INFO:tasks.workunit.client.0.smithi007.stdout:5/411: mkdir d1/d7/d1c/d19/d34/d4d/d99 0 2024-03-16T09:01:12.818 INFO:tasks.workunit.client.0.smithi007.stdout:1/395: mknod d9/c78 0 2024-03-16T09:01:12.818 INFO:tasks.workunit.client.0.smithi007.stdout:1/396: creat d9/dd/d5e/f79 x:0 0 0 2024-03-16T09:01:12.819 INFO:tasks.workunit.client.1.smithi019.stdout:1/181: write d16/f19 [138064,86237] 0 2024-03-16T09:01:12.820 INFO:tasks.workunit.client.0.smithi007.stdout:8/693: rename d0/dc/f82 to d0/dc/dd/ff3 0 2024-03-16T09:01:12.820 INFO:tasks.workunit.client.0.smithi007.stdout:1/397: rename d9/dd to d9/dd/d1b/d2d/d30/d45/d7a 22 2024-03-16T09:01:12.820 INFO:tasks.workunit.client.0.smithi007.stdout:3/499: stat d15/d18/d1f/d29/d3e/c60 0 2024-03-16T09:01:12.820 INFO:tasks.workunit.client.0.smithi007.stdout:3/500: fsync d15/d18/d1f/d29/d7c/f8c 0 2024-03-16T09:01:12.821 INFO:tasks.workunit.client.1.smithi019.stdout:1/182: truncate d16/d1f/f30 277946 0 2024-03-16T09:01:12.822 INFO:tasks.workunit.client.0.smithi007.stdout:9/521: unlink d24/d4a/l79 0 2024-03-16T09:01:12.822 INFO:tasks.workunit.client.0.smithi007.stdout:5/412: dread d1/d7/d16/d1b/f67 [0,4194304] 0 2024-03-16T09:01:12.823 INFO:tasks.workunit.client.0.smithi007.stdout:2/528: symlink dc/df/d24/d28/d2e/lac 0 2024-03-16T09:01:12.824 INFO:tasks.workunit.client.0.smithi007.stdout:4/363: symlink d4/d14/d1c/l7c 0 2024-03-16T09:01:12.824 INFO:tasks.workunit.client.1.smithi019.stdout:1/183: link d16/d1f/c28 d16/c35 0 2024-03-16T09:01:12.824 INFO:tasks.workunit.client.1.smithi019.stdout:1/184: fdatasync f12 0 2024-03-16T09:01:12.825 INFO:tasks.workunit.client.0.smithi007.stdout:7/379: mknod d0/dc/d38/d4e/c7e 0 2024-03-16T09:01:12.827 INFO:tasks.workunit.client.1.smithi019.stdout:1/185: getdents d16/d31 0 2024-03-16T09:01:12.827 INFO:tasks.workunit.client.0.smithi007.stdout:8/694: symlink d0/dc/dd/d8b/lf4 0 2024-03-16T09:01:12.827 INFO:tasks.workunit.client.0.smithi007.stdout:8/695: creat d0/dc/d50/d8e/dd2/dea/d18/da7/ff5 x:0 0 0 2024-03-16T09:01:12.828 INFO:tasks.workunit.client.0.smithi007.stdout:3/501: mkdir d15/d18/d1f/d77/d9d 0 2024-03-16T09:01:12.828 INFO:tasks.workunit.client.0.smithi007.stdout:3/502: write d15/d18/f22 [3984925,53146] 0 2024-03-16T09:01:12.829 INFO:tasks.workunit.client.0.smithi007.stdout:9/522: mkdir d24/d4c/d91/d94 0 2024-03-16T09:01:12.830 INFO:tasks.workunit.client.1.smithi019.stdout:3/258: dwrite f2 [0,4194304] 0 2024-03-16T09:01:12.830 INFO:tasks.workunit.client.0.smithi007.stdout:5/413: mkdir d1/d7/d16/d9a 0 2024-03-16T09:01:12.831 INFO:tasks.workunit.client.1.smithi019.stdout:0/234: dwrite d7/d1d/d28/f34 [4194304,4194304] 0 2024-03-16T09:01:12.831 INFO:tasks.workunit.client.1.smithi019.stdout:0/235: stat d7/d1d/d28/f44 0 2024-03-16T09:01:12.831 INFO:tasks.workunit.client.1.smithi019.stdout:0/236: write d7/d8/f12 [3059287,94544] 0 2024-03-16T09:01:12.831 INFO:tasks.workunit.client.1.smithi019.stdout:0/237: creat d7/d29/f47 x:0 0 0 2024-03-16T09:01:12.832 INFO:tasks.workunit.client.0.smithi007.stdout:2/529: mkdir dc/df/d24/d28/d2e/dad 0 2024-03-16T09:01:12.832 INFO:tasks.workunit.client.0.smithi007.stdout:2/530: write dc/f68 [3538963,105704] 0 2024-03-16T09:01:12.832 INFO:tasks.workunit.client.0.smithi007.stdout:2/531: stat dc/df/d24/d28/d2e/d6e/f76 0 2024-03-16T09:01:12.833 INFO:tasks.workunit.client.1.smithi019.stdout:3/259: truncate d3/d1b/d40/d4a/f41 1596902 0 2024-03-16T09:01:12.833 INFO:tasks.workunit.client.1.smithi019.stdout:3/260: write d3/d17/f50 [1008537,86059] 0 2024-03-16T09:01:12.833 INFO:tasks.workunit.client.1.smithi019.stdout:3/261: write d3/d11/f14 [472694,18960] 0 2024-03-16T09:01:12.833 INFO:tasks.workunit.client.0.smithi007.stdout:0/534: sync 2024-03-16T09:01:12.833 INFO:tasks.workunit.client.0.smithi007.stdout:0/535: creat d3/faf x:0 0 0 2024-03-16T09:01:12.834 INFO:tasks.workunit.client.0.smithi007.stdout:8/696: creat d0/d26/d2d/d43/ff6 x:0 0 0 2024-03-16T09:01:12.835 INFO:tasks.workunit.client.1.smithi019.stdout:5/213: dwrite d6/dd/d37/f26 [0,4194304] 0 2024-03-16T09:01:12.835 INFO:tasks.workunit.client.1.smithi019.stdout:5/214: dread - d6/d1f/f2d zero size 2024-03-16T09:01:12.835 INFO:tasks.workunit.client.1.smithi019.stdout:6/361: dwrite de/d1c/d1d/d2c/f3a [4194304,4194304] 0 2024-03-16T09:01:12.835 INFO:tasks.workunit.client.0.smithi007.stdout:1/398: getdents d9/dd 0 2024-03-16T09:01:12.837 INFO:tasks.workunit.client.0.smithi007.stdout:6/344: sync 2024-03-16T09:01:12.838 INFO:tasks.workunit.client.0.smithi007.stdout:7/380: dread d0/dc/f15 [0,4194304] 0 2024-03-16T09:01:12.839 INFO:tasks.workunit.client.0.smithi007.stdout:3/503: creat d15/d18/d6a/d78/f9e x:0 0 0 2024-03-16T09:01:12.839 INFO:tasks.workunit.client.1.smithi019.stdout:0/238: truncate d7/d8/fc 480439 0 2024-03-16T09:01:12.839 INFO:tasks.workunit.client.1.smithi019.stdout:1/186: dread f9 [0,4194304] 0 2024-03-16T09:01:12.841 INFO:tasks.workunit.client.0.smithi007.stdout:4/364: dwrite d4/d14/d1c/f2c [0,4194304] 0 2024-03-16T09:01:12.842 INFO:tasks.workunit.client.1.smithi019.stdout:3/262: mknod d3/d1b/d40/d4a/d2c/c53 0 2024-03-16T09:01:12.844 INFO:tasks.workunit.client.0.smithi007.stdout:0/536: mknod d3/cb0 0 2024-03-16T09:01:12.845 INFO:tasks.workunit.client.0.smithi007.stdout:9/523: write d24/f2a [2697485,99827] 0 2024-03-16T09:01:12.845 INFO:tasks.workunit.client.0.smithi007.stdout:1/399: mkdir d9/dd/d1b/d28/d29/d7b 0 2024-03-16T09:01:12.845 INFO:tasks.workunit.client.0.smithi007.stdout:1/400: write d9/f62 [242039,65375] 0 2024-03-16T09:01:12.845 INFO:tasks.workunit.client.0.smithi007.stdout:1/401: chown d9/dd/d1b/d28/c6f 44 1 2024-03-16T09:01:12.846 INFO:tasks.workunit.client.1.smithi019.stdout:1/187: write d16/d1f/f27 [2177505,78019] 0 2024-03-16T09:01:12.846 INFO:tasks.workunit.client.1.smithi019.stdout:1/188: write d16/d1f/f2e [857368,33779] 0 2024-03-16T09:01:12.848 INFO:tasks.workunit.client.1.smithi019.stdout:6/362: rename de/d26/c68 to de/d1c/c7b 0 2024-03-16T09:01:12.848 INFO:tasks.workunit.client.1.smithi019.stdout:6/363: chown de/d1c/d1d 7741 1 2024-03-16T09:01:12.848 INFO:tasks.workunit.client.1.smithi019.stdout:6/364: chown de/d1c/d1d/d66/d72 2378 1 2024-03-16T09:01:12.849 INFO:tasks.workunit.client.0.smithi007.stdout:7/381: truncate d0/fa 416396 0 2024-03-16T09:01:12.850 INFO:tasks.workunit.client.1.smithi019.stdout:0/239: mkdir d7/d1d/d28/d48 0 2024-03-16T09:01:12.850 INFO:tasks.workunit.client.1.smithi019.stdout:0/240: write d7/d29/f41 [855589,95347] 0 2024-03-16T09:01:12.850 INFO:tasks.workunit.client.1.smithi019.stdout:3/263: symlink d3/d17/d3f/l54 0 2024-03-16T09:01:12.853 INFO:tasks.workunit.client.0.smithi007.stdout:4/365: creat d4/d14/d64/f7d x:0 0 0 2024-03-16T09:01:12.854 INFO:tasks.workunit.client.1.smithi019.stdout:3/264: dread d3/d11/f3b [0,4194304] 0 2024-03-16T09:01:12.854 INFO:tasks.workunit.client.1.smithi019.stdout:3/265: write d3/f3e [630499,117818] 0 2024-03-16T09:01:12.855 INFO:tasks.workunit.client.1.smithi019.stdout:6/365: symlink de/d1c/d1d/d51/l7c 0 2024-03-16T09:01:12.855 INFO:tasks.workunit.client.1.smithi019.stdout:6/366: chown de/d1c/d1d/d51/d70 3686149 1 2024-03-16T09:01:12.855 INFO:tasks.workunit.client.1.smithi019.stdout:6/367: truncate de/d26/f39 501586 0 2024-03-16T09:01:12.856 INFO:tasks.workunit.client.1.smithi019.stdout:0/241: symlink d7/d8/l49 0 2024-03-16T09:01:12.856 INFO:tasks.workunit.client.0.smithi007.stdout:0/537: dread d3/d9/da/d18/d27/d34/f89 [0,4194304] 0 2024-03-16T09:01:12.857 INFO:tasks.workunit.client.1.smithi019.stdout:1/189: truncate d16/d18/f23 530959 0 2024-03-16T09:01:12.858 INFO:tasks.workunit.client.0.smithi007.stdout:7/382: creat d0/d6c/d25/f7f x:0 0 0 2024-03-16T09:01:12.859 INFO:tasks.workunit.client.0.smithi007.stdout:4/366: dread d4/d14/d1c/f3e [0,4194304] 0 2024-03-16T09:01:12.859 INFO:tasks.workunit.client.0.smithi007.stdout:6/345: rename d2/d13/d1f/d34 to d2/d13/d3f/d50/d72 0 2024-03-16T09:01:12.859 INFO:tasks.workunit.client.0.smithi007.stdout:4/367: write d4/f17 [1237238,119968] 0 2024-03-16T09:01:12.860 INFO:tasks.workunit.client.0.smithi007.stdout:6/346: readlink d2/d11/d1e/l61 0 2024-03-16T09:01:12.860 INFO:tasks.workunit.client.0.smithi007.stdout:4/368: truncate d4/d14/d1c/d5b/f74 650958 0 2024-03-16T09:01:12.860 INFO:tasks.workunit.client.1.smithi019.stdout:6/368: creat de/d1c/d1d/d51/f7d x:0 0 0 2024-03-16T09:01:12.860 INFO:tasks.workunit.client.1.smithi019.stdout:6/369: chown de/f3c 0 1 2024-03-16T09:01:12.861 INFO:tasks.workunit.client.1.smithi019.stdout:0/242: mkdir d7/d8/d32/d4a 0 2024-03-16T09:01:12.862 INFO:tasks.workunit.client.0.smithi007.stdout:0/538: symlink d3/d9/da/lb1 0 2024-03-16T09:01:12.862 INFO:tasks.workunit.client.1.smithi019.stdout:6/370: creat de/d26/d73/f7e x:0 0 0 2024-03-16T09:01:12.863 INFO:tasks.workunit.client.1.smithi019.stdout:6/371: truncate de/d1c/d1d/f36 997460 0 2024-03-16T09:01:12.863 INFO:tasks.workunit.client.0.smithi007.stdout:1/402: rename d9/dd/d1b/d28/d4c/c56 to d9/dd/d1b/d28/c7c 0 2024-03-16T09:01:12.863 INFO:tasks.workunit.client.0.smithi007.stdout:6/347: mknod d2/d13/d3f/d50/d72/c73 0 2024-03-16T09:01:12.863 INFO:tasks.workunit.client.0.smithi007.stdout:6/348: truncate d2/d11/d1e/d54/f56 124846 0 2024-03-16T09:01:12.863 INFO:tasks.workunit.client.0.smithi007.stdout:6/349: chown d2/d11/d3b/c48 18890 1 2024-03-16T09:01:12.863 INFO:tasks.workunit.client.1.smithi019.stdout:0/243: dread d7/f2a [0,4194304] 0 2024-03-16T09:01:12.865 INFO:tasks.workunit.client.0.smithi007.stdout:7/383: rename d0/d6c/d25/c32 to d0/d19/d1f/d42/c80 0 2024-03-16T09:01:12.865 INFO:tasks.workunit.client.0.smithi007.stdout:1/403: creat d9/dd/d1b/d58/f7d x:0 0 0 2024-03-16T09:01:12.865 INFO:tasks.workunit.client.0.smithi007.stdout:1/404: write d9/dd/d1b/d2d/d30/d45/f5b [805602,54885] 0 2024-03-16T09:01:12.866 INFO:tasks.workunit.client.1.smithi019.stdout:6/372: unlink de/f7a 0 2024-03-16T09:01:12.866 INFO:tasks.workunit.client.0.smithi007.stdout:9/524: dwrite d24/f39 [0,4194304] 0 2024-03-16T09:01:12.866 INFO:tasks.workunit.client.0.smithi007.stdout:1/405: mkdir d9/d6b/d7e 0 2024-03-16T09:01:12.866 INFO:tasks.workunit.client.0.smithi007.stdout:1/406: dread - d9/dd/d1b/d58/f66 zero size 2024-03-16T09:01:12.867 INFO:tasks.workunit.client.1.smithi019.stdout:6/373: dread de/d26/f39 [0,4194304] 0 2024-03-16T09:01:12.869 INFO:tasks.workunit.client.1.smithi019.stdout:6/374: mkdir de/d1c/d1d/d66/d72/d7f 0 2024-03-16T09:01:12.869 INFO:tasks.workunit.client.1.smithi019.stdout:6/375: fsync de/f67 0 2024-03-16T09:01:12.869 INFO:tasks.workunit.client.0.smithi007.stdout:9/525: rename d24/d4a/d61 to d24/d4a/d55/d56/d7c/d95 0 2024-03-16T09:01:12.870 INFO:tasks.workunit.client.0.smithi007.stdout:4/369: dread d4/d7/d6b/f6d [0,4194304] 0 2024-03-16T09:01:12.871 INFO:tasks.workunit.client.1.smithi019.stdout:6/376: unlink de/d1c/f6a 0 2024-03-16T09:01:12.873 INFO:tasks.workunit.client.0.smithi007.stdout:9/526: mkdir d24/d4a/d96 0 2024-03-16T09:01:12.877 INFO:tasks.workunit.client.0.smithi007.stdout:1/407: rename d9/dd/d1b/c3d to d9/dd/d1b/d2d/c7f 0 2024-03-16T09:01:12.880 INFO:tasks.workunit.client.1.smithi019.stdout:0/244: dread d7/d8/d22/f27 [0,4194304] 0 2024-03-16T09:01:12.881 INFO:tasks.workunit.client.1.smithi019.stdout:0/245: chown d7/d8/c2b 8769134 1 2024-03-16T09:01:12.881 INFO:tasks.workunit.client.1.smithi019.stdout:0/246: fsync d7/d8/d32/d36/f35 0 2024-03-16T09:01:12.883 INFO:tasks.workunit.client.1.smithi019.stdout:0/247: rmdir d7/d1d/d2e 39 2024-03-16T09:01:12.883 INFO:tasks.workunit.client.1.smithi019.stdout:0/248: stat d7/d8/fc 0 2024-03-16T09:01:12.886 INFO:tasks.workunit.client.1.smithi019.stdout:0/249: creat d7/d1d/f4b x:0 0 0 2024-03-16T09:01:12.888 INFO:tasks.workunit.client.1.smithi019.stdout:0/250: symlink d7/d8/d32/d36/l4c 0 2024-03-16T09:01:12.890 INFO:tasks.workunit.client.0.smithi007.stdout:4/370: dread d4/d7/f1d [0,4194304] 0 2024-03-16T09:01:12.890 INFO:tasks.workunit.client.1.smithi019.stdout:0/251: dread d7/f2a [0,4194304] 0 2024-03-16T09:01:12.890 INFO:tasks.workunit.client.1.smithi019.stdout:0/252: truncate d7/f2a 1480992 0 2024-03-16T09:01:12.891 INFO:tasks.workunit.client.1.smithi019.stdout:0/253: creat d7/d8/d32/d4a/f4d x:0 0 0 2024-03-16T09:01:12.903 INFO:tasks.workunit.client.0.smithi007.stdout:4/371: dread d4/d14/d1c/d5b/f5e [0,4194304] 0 2024-03-16T09:01:12.905 INFO:tasks.workunit.client.0.smithi007.stdout:4/372: write d4/d14/d1c/f37 [1056225,98429] 0 2024-03-16T09:01:12.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/532: dwrite dc/df/d24/d28/d4c/f59 [0,4194304] 0 2024-03-16T09:01:12.909 INFO:tasks.workunit.client.0.smithi007.stdout:2/533: write dc/df/d24/f63 [465900,53309] 0 2024-03-16T09:01:12.911 INFO:tasks.workunit.client.0.smithi007.stdout:2/534: truncate dc/d27/f65 2375772 0 2024-03-16T09:01:12.911 INFO:tasks.workunit.client.0.smithi007.stdout:2/535: write dc/fab [128824,6914] 0 2024-03-16T09:01:12.911 INFO:tasks.workunit.client.0.smithi007.stdout:2/536: stat dc/d27/f7f 0 2024-03-16T09:01:12.911 INFO:tasks.workunit.client.0.smithi007.stdout:2/537: write dc/df/d24/d28/d2e/d6e/f9a [901594,18287] 0 2024-03-16T09:01:12.913 INFO:tasks.workunit.client.0.smithi007.stdout:2/538: symlink dc/df/d24/d28/d2e/d6e/lae 0 2024-03-16T09:01:12.913 INFO:tasks.workunit.client.0.smithi007.stdout:5/414: dread d1/d7/f12 [0,4194304] 0 2024-03-16T09:01:12.913 INFO:tasks.workunit.client.0.smithi007.stdout:5/415: write d1/d7/d1c/f80 [332621,55697] 0 2024-03-16T09:01:12.913 INFO:tasks.workunit.client.0.smithi007.stdout:1/408: dread d9/dd/d1b/d28/f50 [0,4194304] 0 2024-03-16T09:01:12.913 INFO:tasks.workunit.client.0.smithi007.stdout:1/409: readlink d9/dd/d1b/d28/d29/l41 0 2024-03-16T09:01:12.913 INFO:tasks.workunit.client.0.smithi007.stdout:1/410: stat d9/dd/d1b/d2d/d30/f61 0 2024-03-16T09:01:12.914 INFO:tasks.workunit.client.0.smithi007.stdout:1/411: read d9/dd/d1b/d28/f50 [59126,35431] 0 2024-03-16T09:01:12.918 INFO:tasks.workunit.client.0.smithi007.stdout:2/539: mkdir dc/df/d24/d28/d4c/daf 0 2024-03-16T09:01:12.918 INFO:tasks.workunit.client.0.smithi007.stdout:5/416: mkdir d1/d7/d1c/d19/d34/d1d/d36/d89/d9b 0 2024-03-16T09:01:12.918 INFO:tasks.workunit.client.1.smithi019.stdout:6/377: dwrite de/d26/d32/f5b [0,4194304] 0 2024-03-16T09:01:12.918 INFO:tasks.workunit.client.1.smithi019.stdout:6/378: write de/f49 [2952860,85597] 0 2024-03-16T09:01:12.920 INFO:tasks.workunit.client.0.smithi007.stdout:6/350: dwrite d2/d11/f21 [0,4194304] 0 2024-03-16T09:01:12.920 INFO:tasks.workunit.client.0.smithi007.stdout:5/417: rmdir d1/d7/d16/d9a 0 2024-03-16T09:01:12.921 INFO:tasks.workunit.client.1.smithi019.stdout:3/266: dwrite d3/f30 [0,4194304] 0 2024-03-16T09:01:12.921 INFO:tasks.workunit.client.1.smithi019.stdout:3/267: write d3/d1b/d40/d4a/f4e [301051,13351] 0 2024-03-16T09:01:12.922 INFO:tasks.workunit.client.0.smithi007.stdout:5/418: mknod d1/d7/d1c/d19/d34/d5e/c9c 0 2024-03-16T09:01:12.922 INFO:tasks.workunit.client.0.smithi007.stdout:5/419: chown d1/f2 926854 1 2024-03-16T09:01:12.923 INFO:tasks.workunit.client.1.smithi019.stdout:5/215: dwrite d6/f2f [0,4194304] 0 2024-03-16T09:01:12.928 INFO:tasks.workunit.client.0.smithi007.stdout:5/420: mknod d1/d7/d1c/d19/d34/d4d/c9d 0 2024-03-16T09:01:12.928 INFO:tasks.workunit.client.0.smithi007.stdout:5/421: write d1/d7/d16/d1b/f8d [1744039,129984] 0 2024-03-16T09:01:12.932 INFO:tasks.workunit.client.0.smithi007.stdout:1/412: write d9/dd/d1b/d28/d29/f42 [7530259,59194] 0 2024-03-16T09:01:12.932 INFO:tasks.workunit.client.0.smithi007.stdout:1/413: fdatasync d9/dd/d1b/d28/d4c/f63 0 2024-03-16T09:01:12.932 INFO:tasks.workunit.client.0.smithi007.stdout:7/384: dwrite d0/d19/d1f/f2c [4194304,4194304] 0 2024-03-16T09:01:12.932 INFO:tasks.workunit.client.0.smithi007.stdout:1/414: write d9/dd/d1b/d2d/d30/d45/f5b [1641790,121187] 0 2024-03-16T09:01:12.933 INFO:tasks.workunit.client.1.smithi019.stdout:6/379: dread de/d1c/f47 [0,4194304] 0 2024-03-16T09:01:12.933 INFO:tasks.workunit.client.1.smithi019.stdout:6/380: truncate de/d1c/d1d/f46 1696398 0 2024-03-16T09:01:12.934 INFO:tasks.workunit.client.0.smithi007.stdout:9/527: dwrite d24/f43 [0,4194304] 0 2024-03-16T09:01:12.935 INFO:tasks.workunit.client.0.smithi007.stdout:7/385: link d0/d19/d1f/d69/l75 d0/d6c/d25/d73/l81 0 2024-03-16T09:01:12.938 INFO:tasks.workunit.client.1.smithi019.stdout:5/216: dread d6/d19/f2e [0,4194304] 0 2024-03-16T09:01:12.939 INFO:tasks.workunit.client.0.smithi007.stdout:9/528: symlink d24/d4a/d55/d56/d7c/d7f/d8f/l97 0 2024-03-16T09:01:12.939 INFO:tasks.workunit.client.0.smithi007.stdout:9/529: write d24/d4a/d55/d56/d7c/d7f/d80/d87/d89/f8a [1036627,75859] 0 2024-03-16T09:01:12.940 INFO:tasks.workunit.client.1.smithi019.stdout:0/254: dwrite f4 [0,4194304] 0 2024-03-16T09:01:12.940 INFO:tasks.workunit.client.1.smithi019.stdout:0/255: dread - d7/d1d/d28/f44 zero size 2024-03-16T09:01:12.940 INFO:tasks.workunit.client.1.smithi019.stdout:5/217: mkdir d6/d19/d34/d3e 0 2024-03-16T09:01:12.941 INFO:tasks.workunit.client.0.smithi007.stdout:0/539: dwrite d3/d1f/f71 [0,4194304] 0 2024-03-16T09:01:12.941 INFO:tasks.workunit.client.0.smithi007.stdout:0/540: write d3/d9/da/d48/d4b/f51 [64238,101333] 0 2024-03-16T09:01:12.942 INFO:tasks.workunit.client.1.smithi019.stdout:3/268: write f2 [433481,37088] 0 2024-03-16T09:01:12.942 INFO:tasks.workunit.client.1.smithi019.stdout:3/269: dread - d3/d11/f49 zero size 2024-03-16T09:01:12.943 INFO:tasks.workunit.client.0.smithi007.stdout:0/541: symlink d3/d9/db/d2c/lb2 0 2024-03-16T09:01:12.943 INFO:tasks.workunit.client.0.smithi007.stdout:6/351: dread d2/d13/d3f/d50/d72/f4c [0,4194304] 0 2024-03-16T09:01:12.944 INFO:tasks.workunit.client.1.smithi019.stdout:0/256: rename f4 to d7/d1d/d28/d48/f4e 0 2024-03-16T09:01:12.948 INFO:tasks.workunit.client.0.smithi007.stdout:4/373: dwrite d4/d14/d1c/f3b [0,4194304] 0 2024-03-16T09:01:12.948 INFO:tasks.workunit.client.0.smithi007.stdout:4/374: readlink d4/d14/d64/l66 0 2024-03-16T09:01:12.948 INFO:tasks.workunit.client.0.smithi007.stdout:4/375: truncate d4/d14/d1c/f29 647206 0 2024-03-16T09:01:12.951 INFO:tasks.workunit.client.0.smithi007.stdout:4/376: mknod d4/d14/c7e 0 2024-03-16T09:01:12.951 INFO:tasks.workunit.client.0.smithi007.stdout:1/415: dread d9/dd/d1b/d28/f3e [0,4194304] 0 2024-03-16T09:01:12.951 INFO:tasks.workunit.client.0.smithi007.stdout:4/377: write d4/d7/f20 [4492264,19527] 0 2024-03-16T09:01:12.951 INFO:tasks.workunit.client.0.smithi007.stdout:1/416: chown d9/dd/c17 0 1 2024-03-16T09:01:12.956 INFO:tasks.workunit.client.0.smithi007.stdout:1/417: creat d9/dd/d1b/d28/d29/f80 x:0 0 0 2024-03-16T09:01:12.959 INFO:tasks.workunit.client.1.smithi019.stdout:0/257: write d7/d8/d32/d36/f35 [2592880,13664] 0 2024-03-16T09:01:12.959 INFO:tasks.workunit.client.0.smithi007.stdout:1/418: rename d9/f48 to d9/dd/f81 0 2024-03-16T09:01:12.959 INFO:tasks.workunit.client.0.smithi007.stdout:2/540: dwrite dc/df/f34 [0,4194304] 0 2024-03-16T09:01:12.959 INFO:tasks.workunit.client.0.smithi007.stdout:2/541: write dc/f10 [4734883,58218] 0 2024-03-16T09:01:12.961 INFO:tasks.workunit.client.1.smithi019.stdout:1/190: dwrite d16/f2b [0,4194304] 0 2024-03-16T09:01:12.961 INFO:tasks.workunit.client.1.smithi019.stdout:1/191: write f9 [1827637,17347] 0 2024-03-16T09:01:12.961 INFO:tasks.workunit.client.1.smithi019.stdout:1/192: truncate f10 1946290 0 2024-03-16T09:01:12.962 INFO:tasks.workunit.client.0.smithi007.stdout:2/542: truncate dc/f11 1008975 0 2024-03-16T09:01:12.965 INFO:tasks.workunit.client.0.smithi007.stdout:2/543: link dc/df/d24/d28/d4c/d5e/c79 dc/df/d24/d28/d97/d99/cb0 0 2024-03-16T09:01:12.968 INFO:tasks.workunit.client.0.smithi007.stdout:2/544: dread dc/df/d24/d28/d2e/f9b [0,4194304] 0 2024-03-16T09:01:12.971 INFO:tasks.workunit.client.1.smithi019.stdout:1/193: rmdir d16/d31/d32 0 2024-03-16T09:01:12.972 INFO:tasks.workunit.client.0.smithi007.stdout:2/545: creat dc/df/d24/fb1 x:0 0 0 2024-03-16T09:01:12.972 INFO:tasks.workunit.client.1.smithi019.stdout:1/194: mknod d16/d18/c36 0 2024-03-16T09:01:12.972 INFO:tasks.workunit.client.1.smithi019.stdout:1/195: write d16/d18/f1a [2556490,78334] 0 2024-03-16T09:01:12.973 INFO:tasks.workunit.client.0.smithi007.stdout:2/546: link dc/df/d24/c2c dc/d27/da3/cb2 0 2024-03-16T09:01:12.973 INFO:tasks.workunit.client.0.smithi007.stdout:2/547: chown dc/df/d24/d28/d4c/d5e 0 1 2024-03-16T09:01:12.974 INFO:tasks.workunit.client.0.smithi007.stdout:2/548: creat dc/df/daa/fb3 x:0 0 0 2024-03-16T09:01:12.975 INFO:tasks.workunit.client.0.smithi007.stdout:2/549: chown dc/df/l21 94 1 2024-03-16T09:01:12.975 INFO:tasks.workunit.client.0.smithi007.stdout:2/550: creat dc/df/d24/d28/d2e/d35/fb4 x:0 0 0 2024-03-16T09:01:12.976 INFO:tasks.workunit.client.0.smithi007.stdout:2/551: mknod dc/df/d24/d28/d2e/d35/da2/cb5 0 2024-03-16T09:01:12.977 INFO:tasks.workunit.client.0.smithi007.stdout:2/552: creat dc/df/d24/d28/d2e/dad/fb6 x:0 0 0 2024-03-16T09:01:12.979 INFO:tasks.workunit.client.0.smithi007.stdout:2/553: mkdir dc/db7 0 2024-03-16T09:01:12.979 INFO:tasks.workunit.client.0.smithi007.stdout:2/554: read - dc/df/d24/d28/d2e/d6e/f76 zero size 2024-03-16T09:01:12.980 INFO:tasks.workunit.client.0.smithi007.stdout:2/555: truncate dc/df/d24/d28/d2e/f5c 1518417 0 2024-03-16T09:01:12.983 INFO:tasks.workunit.client.0.smithi007.stdout:1/419: dread d9/dd/f81 [0,4194304] 0 2024-03-16T09:01:12.986 INFO:tasks.workunit.client.0.smithi007.stdout:5/422: dwrite d1/d7/d1c/d19/d34/d3e/f6a [0,4194304] 0 2024-03-16T09:01:12.986 INFO:tasks.workunit.client.0.smithi007.stdout:1/420: rename d9/dd/d1b/d58/f6c to d9/dd/d1b/d28/d29/d7b/f82 0 2024-03-16T09:01:12.986 INFO:tasks.workunit.client.0.smithi007.stdout:1/421: fsync d9/dd/f59 0 2024-03-16T09:01:12.996 INFO:tasks.workunit.client.1.smithi019.stdout:6/381: dwrite de/d1c/d1d/f5a [0,4194304] 0 2024-03-16T09:01:12.996 INFO:tasks.workunit.client.1.smithi019.stdout:6/382: chown de/d26/d32 2974554 1 2024-03-16T09:01:12.997 INFO:tasks.workunit.client.0.smithi007.stdout:1/422: getdents d9/dd/d1b/d28/d4c 0 2024-03-16T09:01:13.000 INFO:tasks.workunit.client.0.smithi007.stdout:7/386: getdents d0/d19/d1f/d42 0 2024-03-16T09:01:13.000 INFO:tasks.workunit.client.0.smithi007.stdout:7/387: fdatasync d0/dc/fe 0 2024-03-16T09:01:13.000 INFO:tasks.workunit.client.0.smithi007.stdout:7/388: creat d0/d19/d1f/f82 x:0 0 0 2024-03-16T09:01:13.001 INFO:tasks.workunit.client.0.smithi007.stdout:1/423: getdents d9/d6b 0 2024-03-16T09:01:13.004 INFO:tasks.workunit.client.0.smithi007.stdout:1/424: creat d9/dd/d1b/f83 x:0 0 0 2024-03-16T09:01:13.013 INFO:tasks.workunit.client.1.smithi019.stdout:5/218: dwrite d6/d19/f2e [0,4194304] 0 2024-03-16T09:01:13.013 INFO:tasks.workunit.client.1.smithi019.stdout:3/270: dwrite d3/d11/f49 [0,4194304] 0 2024-03-16T09:01:13.014 INFO:tasks.workunit.client.1.smithi019.stdout:3/271: read - d3/f25 zero size 2024-03-16T09:01:13.016 INFO:tasks.workunit.client.1.smithi019.stdout:5/219: rename f2 to d6/d1f/f3f 0 2024-03-16T09:01:13.016 INFO:tasks.workunit.client.1.smithi019.stdout:5/220: creat d6/dd/f40 x:0 0 0 2024-03-16T09:01:13.017 INFO:tasks.workunit.client.0.smithi007.stdout:9/530: dwrite f8 [0,4194304] 0 2024-03-16T09:01:13.017 INFO:tasks.workunit.client.1.smithi019.stdout:0/258: dwrite d7/d8/d22/f25 [0,4194304] 0 2024-03-16T09:01:13.020 INFO:tasks.workunit.client.1.smithi019.stdout:3/272: mknod d3/d17/d3f/c55 0 2024-03-16T09:01:13.021 INFO:tasks.workunit.client.0.smithi007.stdout:6/352: dwrite d2/d11/d1e/d54/f63 [0,4194304] 0 2024-03-16T09:01:13.023 INFO:tasks.workunit.client.0.smithi007.stdout:2/556: dwrite dc/df/d24/d28/d2e/d4a/f7a [0,4194304] 0 2024-03-16T09:01:13.023 INFO:tasks.workunit.client.0.smithi007.stdout:2/557: write dc/df/d24/d28/d4c/f58 [214033,110388] 0 2024-03-16T09:01:13.023 INFO:tasks.workunit.client.0.smithi007.stdout:7/389: dwrite d0/dc/f3d [0,4194304] 0 2024-03-16T09:01:13.023 INFO:tasks.workunit.client.0.smithi007.stdout:7/390: fdatasync d0/d6c/d25/d3a/f44 0 2024-03-16T09:01:13.025 INFO:tasks.workunit.client.1.smithi019.stdout:3/273: creat d3/d1b/d40/d4a/d2c/f56 x:0 0 0 2024-03-16T09:01:13.026 INFO:tasks.workunit.client.1.smithi019.stdout:5/221: dread d6/dd/f1d [0,4194304] 0 2024-03-16T09:01:13.027 INFO:tasks.workunit.client.1.smithi019.stdout:4/273: sync 2024-03-16T09:01:13.027 INFO:tasks.workunit.client.1.smithi019.stdout:2/207: sync 2024-03-16T09:01:13.027 INFO:tasks.workunit.client.1.smithi019.stdout:9/195: sync 2024-03-16T09:01:13.028 INFO:tasks.workunit.client.0.smithi007.stdout:6/353: mkdir d2/d11/d3b/d74 0 2024-03-16T09:01:13.029 INFO:tasks.workunit.client.1.smithi019.stdout:2/208: creat d5/f40 x:0 0 0 2024-03-16T09:01:13.031 INFO:tasks.workunit.client.1.smithi019.stdout:1/196: getdents d16/d31 0 2024-03-16T09:01:13.032 INFO:tasks.workunit.client.1.smithi019.stdout:1/197: creat d16/f37 x:0 0 0 2024-03-16T09:01:13.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/558: unlink lb 0 2024-03-16T09:01:13.032 INFO:tasks.workunit.client.0.smithi007.stdout:9/531: dread d24/f43 [0,4194304] 0 2024-03-16T09:01:13.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/559: fsync dc/f17 0 2024-03-16T09:01:13.032 INFO:tasks.workunit.client.0.smithi007.stdout:9/532: getdents d24/d4c/d91/d94 0 2024-03-16T09:01:13.033 INFO:tasks.workunit.client.0.smithi007.stdout:2/560: dread - dc/df/d24/d28/d2e/d6e/f6f zero size 2024-03-16T09:01:13.033 INFO:tasks.workunit.client.0.smithi007.stdout:5/423: dwrite d1/d7/d1c/d19/d34/d5e/d6f/d72/f92 [0,4194304] 0 2024-03-16T09:01:13.033 INFO:tasks.workunit.client.0.smithi007.stdout:9/533: creat d24/d4a/d55/d56/d7c/f98 x:0 0 0 2024-03-16T09:01:13.033 INFO:tasks.workunit.client.0.smithi007.stdout:9/534: stat d24/d4a/d55/d56/d7c/d7f/d80/d87 0 2024-03-16T09:01:13.035 INFO:tasks.workunit.client.1.smithi019.stdout:9/196: symlink d1/d7/l46 0 2024-03-16T09:01:13.035 INFO:tasks.workunit.client.1.smithi019.stdout:9/197: truncate d1/d7/de/d24/f2c 812869 0 2024-03-16T09:01:13.035 INFO:tasks.workunit.client.1.smithi019.stdout:9/198: creat d1/d7/de/d24/f47 x:0 0 0 2024-03-16T09:01:13.035 INFO:tasks.workunit.client.1.smithi019.stdout:9/199: readlink d1/d7/d21/d27/l17 0 2024-03-16T09:01:13.035 INFO:tasks.workunit.client.1.smithi019.stdout:2/209: dread d5/d6/f11 [0,4194304] 0 2024-03-16T09:01:13.036 INFO:tasks.workunit.client.1.smithi019.stdout:2/210: stat d5/fb 0 2024-03-16T09:01:13.036 INFO:tasks.workunit.client.1.smithi019.stdout:2/211: truncate d5/f13 752468 0 2024-03-16T09:01:13.036 INFO:tasks.workunit.client.0.smithi007.stdout:4/378: dread d4/d7/f20 [0,4194304] 0 2024-03-16T09:01:13.037 INFO:tasks.workunit.client.0.smithi007.stdout:7/391: symlink d0/d19/d1f/d69/l83 0 2024-03-16T09:01:13.037 INFO:tasks.workunit.client.1.smithi019.stdout:1/198: dread d16/d1f/f2e [0,4194304] 0 2024-03-16T09:01:13.037 INFO:tasks.workunit.client.1.smithi019.stdout:5/222: dread d6/dd/f16 [0,4194304] 0 2024-03-16T09:01:13.038 INFO:tasks.workunit.client.0.smithi007.stdout:6/354: mknod d2/c75 0 2024-03-16T09:01:13.039 INFO:tasks.workunit.client.1.smithi019.stdout:9/200: link d1/d7/l3d d1/d7/l48 0 2024-03-16T09:01:13.039 INFO:tasks.workunit.client.1.smithi019.stdout:1/199: dread d16/d18/f23 [0,4194304] 0 2024-03-16T09:01:13.039 INFO:tasks.workunit.client.1.smithi019.stdout:1/200: dread - d16/d1f/f34 zero size 2024-03-16T09:01:13.040 INFO:tasks.workunit.client.1.smithi019.stdout:2/212: mknod d5/d6/d2b/d2f/c41 0 2024-03-16T09:01:13.040 INFO:tasks.workunit.client.1.smithi019.stdout:2/213: chown d5/d6/d2b/d2f/c3e 303 1 2024-03-16T09:01:13.040 INFO:tasks.workunit.client.1.smithi019.stdout:2/214: dread - d5/d2e/f3b zero size 2024-03-16T09:01:13.040 INFO:tasks.workunit.client.1.smithi019.stdout:9/201: symlink d1/d7/de/d2d/l49 0 2024-03-16T09:01:13.042 INFO:tasks.workunit.client.1.smithi019.stdout:1/201: mknod d16/c38 0 2024-03-16T09:01:13.042 INFO:tasks.workunit.client.1.smithi019.stdout:1/202: write d16/d18/f23 [878302,29687] 0 2024-03-16T09:01:13.043 INFO:tasks.workunit.client.1.smithi019.stdout:2/215: unlink d5/d6/f21 0 2024-03-16T09:01:13.043 INFO:tasks.workunit.client.1.smithi019.stdout:2/216: chown f1 0 1 2024-03-16T09:01:13.043 INFO:tasks.workunit.client.0.smithi007.stdout:2/561: mkdir dc/df/d24/d28/d97/da4/db8 0 2024-03-16T09:01:13.044 INFO:tasks.workunit.client.1.smithi019.stdout:9/202: mknod d1/d7/c4a 0 2024-03-16T09:01:13.045 INFO:tasks.workunit.client.1.smithi019.stdout:8/253: sync 2024-03-16T09:01:13.045 INFO:tasks.workunit.client.1.smithi019.stdout:8/254: stat d6/d2a/d45/d2c 0 2024-03-16T09:01:13.045 INFO:tasks.workunit.client.1.smithi019.stdout:2/217: mknod d5/d6/d2b/c42 0 2024-03-16T09:01:13.046 INFO:tasks.workunit.client.0.smithi007.stdout:9/535: creat d24/d8e/f99 x:0 0 0 2024-03-16T09:01:13.046 INFO:tasks.workunit.client.0.smithi007.stdout:9/536: fsync d24/d4a/d55/d56/d7c/f88 0 2024-03-16T09:01:13.046 INFO:tasks.workunit.client.0.smithi007.stdout:9/537: fsync d24/d4a/d55/f5b 0 2024-03-16T09:01:13.047 INFO:tasks.workunit.client.1.smithi019.stdout:8/255: mkdir d6/de/d51 0 2024-03-16T09:01:13.047 INFO:tasks.workunit.client.1.smithi019.stdout:8/256: write d6/d2a/d45/f47 [879016,91406] 0 2024-03-16T09:01:13.047 INFO:tasks.workunit.client.1.smithi019.stdout:2/218: mknod d5/de/c43 0 2024-03-16T09:01:13.049 INFO:tasks.workunit.client.0.smithi007.stdout:5/424: rename d1/d7/d16/d1b/f67 to d1/d7/d1c/d19/d34/d1d/f9e 0 2024-03-16T09:01:13.049 INFO:tasks.workunit.client.0.smithi007.stdout:5/425: write d1/d7/d1c/d19/d34/d5e/d6f/d72/f85 [932777,34655] 0 2024-03-16T09:01:13.049 INFO:tasks.workunit.client.0.smithi007.stdout:5/426: creat d1/d7/d1c/d19/d34/d3e/d41/f9f x:0 0 0 2024-03-16T09:01:13.049 INFO:tasks.workunit.client.0.smithi007.stdout:5/427: write d1/d7/d1c/d19/d34/d1d/f1f [4047326,64724] 0 2024-03-16T09:01:13.049 INFO:tasks.workunit.client.0.smithi007.stdout:5/428: creat d1/d7/d16/d1e/fa0 x:0 0 0 2024-03-16T09:01:13.050 INFO:tasks.workunit.client.0.smithi007.stdout:4/379: symlink d4/d7/l7f 0 2024-03-16T09:01:13.051 INFO:tasks.workunit.client.0.smithi007.stdout:7/392: creat d0/d6c/d25/d7a/f84 x:0 0 0 2024-03-16T09:01:13.052 INFO:tasks.workunit.client.1.smithi019.stdout:5/223: write d6/d19/f2e [4012854,49740] 0 2024-03-16T09:01:13.052 INFO:tasks.workunit.client.1.smithi019.stdout:2/219: truncate d5/de/fd 303586 0 2024-03-16T09:01:13.053 INFO:tasks.workunit.client.1.smithi019.stdout:2/220: write d5/f13 [1652053,95156] 0 2024-03-16T09:01:13.053 INFO:tasks.workunit.client.1.smithi019.stdout:2/221: fdatasync d5/d2e/f3b 0 2024-03-16T09:01:13.053 INFO:tasks.workunit.client.1.smithi019.stdout:7/190: sync 2024-03-16T09:01:13.053 INFO:tasks.workunit.client.1.smithi019.stdout:3/274: dwrite d3/d1b/d40/d4a/d2c/f38 [0,4194304] 0 2024-03-16T09:01:13.053 INFO:tasks.workunit.client.1.smithi019.stdout:3/275: chown d3/c8 0 1 2024-03-16T09:01:13.053 INFO:tasks.workunit.client.1.smithi019.stdout:5/224: symlink d6/dd/d37/l41 0 2024-03-16T09:01:13.053 INFO:tasks.workunit.client.1.smithi019.stdout:4/274: dwrite d4/d1b/d49/f56 [0,4194304] 0 2024-03-16T09:01:13.053 INFO:tasks.workunit.client.1.smithi019.stdout:0/259: dwrite d7/d8/d32/d36/f3b [0,4194304] 0 2024-03-16T09:01:13.055 INFO:tasks.workunit.client.1.smithi019.stdout:2/222: rename d5/d6/d2b/c17 to d5/d6/d2b/d2f/d35/c44 0 2024-03-16T09:01:13.055 INFO:tasks.workunit.client.1.smithi019.stdout:2/223: fsync d5/d6/f28 0 2024-03-16T09:01:13.056 INFO:tasks.workunit.client.1.smithi019.stdout:8/257: write d6/d2a/d3c/f44 [967611,12076] 0 2024-03-16T09:01:13.056 INFO:tasks.workunit.client.0.smithi007.stdout:6/355: creat d2/d11/d3b/f76 x:0 0 0 2024-03-16T09:01:13.056 INFO:tasks.workunit.client.0.smithi007.stdout:6/356: chown d2/d11/d1e/d6f 215 1 2024-03-16T09:01:13.056 INFO:tasks.workunit.client.0.smithi007.stdout:6/357: chown d2/d13/d1f 167 1 2024-03-16T09:01:13.056 INFO:tasks.workunit.client.0.smithi007.stdout:6/358: dread - d2/d13/d3f/d50/d72/f69 zero size 2024-03-16T09:01:13.057 INFO:tasks.workunit.client.0.smithi007.stdout:2/562: mknod dc/df/d24/d28/cb9 0 2024-03-16T09:01:13.058 INFO:tasks.workunit.client.1.smithi019.stdout:7/191: link d9/db/d12/f2f d9/da/d13/d18/d1a/d32/f38 0 2024-03-16T09:01:13.058 INFO:tasks.workunit.client.0.smithi007.stdout:9/538: rename d24/d4a/c81 to d24/d4a/d55/d56/d7c/d7f/d80/d87/d89/c9a 0 2024-03-16T09:01:13.060 INFO:tasks.workunit.client.1.smithi019.stdout:5/225: dread d6/d19/f29 [0,4194304] 0 2024-03-16T09:01:13.062 INFO:tasks.workunit.client.0.smithi007.stdout:5/429: symlink d1/d7/d16/d1e/la1 0 2024-03-16T09:01:13.063 INFO:tasks.workunit.client.1.smithi019.stdout:4/275: symlink d4/d1b/d2a/d4f/l5c 0 2024-03-16T09:01:13.065 INFO:tasks.workunit.client.1.smithi019.stdout:8/258: write d6/d2a/d45/d19/f2e [1376260,81792] 0 2024-03-16T09:01:13.066 INFO:tasks.workunit.client.1.smithi019.stdout:4/276: dread d4/d6/f32 [0,4194304] 0 2024-03-16T09:01:13.066 INFO:tasks.workunit.client.1.smithi019.stdout:4/277: write d4/f12 [3863052,129069] 0 2024-03-16T09:01:13.066 INFO:tasks.workunit.client.1.smithi019.stdout:4/278: creat d4/d1b/f5d x:0 0 0 2024-03-16T09:01:13.067 INFO:tasks.workunit.client.0.smithi007.stdout:6/359: write d2/d11/f21 [2848512,13564] 0 2024-03-16T09:01:13.067 INFO:tasks.workunit.client.1.smithi019.stdout:0/260: symlink d7/d8/d32/d36/l4f 0 2024-03-16T09:01:13.067 INFO:tasks.workunit.client.1.smithi019.stdout:0/261: truncate d7/f19 1179718 0 2024-03-16T09:01:13.067 INFO:tasks.workunit.client.0.smithi007.stdout:9/539: link d24/l35 d24/d8e/l9b 0 2024-03-16T09:01:13.068 INFO:tasks.workunit.client.1.smithi019.stdout:3/276: rename d3/c48 to d3/d1b/c57 0 2024-03-16T09:01:13.068 INFO:tasks.workunit.client.1.smithi019.stdout:2/224: dread d5/d6/d2b/f18 [0,4194304] 0 2024-03-16T09:01:13.070 INFO:tasks.workunit.client.0.smithi007.stdout:5/430: truncate d1/d7/d16/d1e/d2a/f57 3851678 0 2024-03-16T09:01:13.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/192: write f4 [2854674,38122] 0 2024-03-16T09:01:13.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/193: creat d9/d25/d28/f39 x:0 0 0 2024-03-16T09:01:13.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/194: dread - d9/db/f2c zero size 2024-03-16T09:01:13.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/195: dread - d9/db/d12/f2f zero size 2024-03-16T09:01:13.071 INFO:tasks.workunit.client.1.smithi019.stdout:5/226: symlink d6/d1f/l42 0 2024-03-16T09:01:13.071 INFO:tasks.workunit.client.1.smithi019.stdout:5/227: fsync d6/fb 0 2024-03-16T09:01:13.071 INFO:tasks.workunit.client.1.smithi019.stdout:5/228: dread - d6/dd/f38 zero size 2024-03-16T09:01:13.071 INFO:tasks.workunit.client.0.smithi007.stdout:1/425: rmdir d9/dd/d1b/d58 39 2024-03-16T09:01:13.072 INFO:tasks.workunit.client.0.smithi007.stdout:1/426: fdatasync d9/dd/d1b/d2d/d30/d45/f51 0 2024-03-16T09:01:13.072 INFO:tasks.workunit.client.0.smithi007.stdout:1/427: creat d9/dd/d5e/f84 x:0 0 0 2024-03-16T09:01:13.072 INFO:tasks.workunit.client.0.smithi007.stdout:1/428: dread - d9/dd/d1b/d28/d29/f5d zero size 2024-03-16T09:01:13.072 INFO:tasks.workunit.client.0.smithi007.stdout:4/380: rmdir d4/d14/d1c/d1e 39 2024-03-16T09:01:13.073 INFO:tasks.workunit.client.0.smithi007.stdout:2/563: dread dc/df/d24/d28/d4c/f58 [0,4194304] 0 2024-03-16T09:01:13.074 INFO:tasks.workunit.client.1.smithi019.stdout:4/279: rename d4/d1b/c52 to d4/d1b/c5e 0 2024-03-16T09:01:13.074 INFO:tasks.workunit.client.1.smithi019.stdout:4/280: creat d4/d6/f5f x:0 0 0 2024-03-16T09:01:13.075 INFO:tasks.workunit.client.0.smithi007.stdout:6/360: link d2/d10/d12/l23 d2/d10/d12/l77 0 2024-03-16T09:01:13.075 INFO:tasks.workunit.client.0.smithi007.stdout:6/361: write d2/d13/d1f/d4e/f5b [496149,79746] 0 2024-03-16T09:01:13.076 INFO:tasks.workunit.client.0.smithi007.stdout:9/540: dread d24/f39 [0,4194304] 0 2024-03-16T09:01:13.076 INFO:tasks.workunit.client.1.smithi019.stdout:0/262: creat d7/d8/f50 x:0 0 0 2024-03-16T09:01:13.077 INFO:tasks.workunit.client.0.smithi007.stdout:1/429: getdents d9/dd/d1b/d28/d29/d7b 0 2024-03-16T09:01:13.079 INFO:tasks.workunit.client.1.smithi019.stdout:8/259: rename d6/l20 to d6/de/l52 0 2024-03-16T09:01:13.079 INFO:tasks.workunit.client.0.smithi007.stdout:4/381: write d4/d7/f20 [3000247,5716] 0 2024-03-16T09:01:13.080 INFO:tasks.workunit.client.0.smithi007.stdout:2/564: mkdir dc/dba 0 2024-03-16T09:01:13.082 INFO:tasks.workunit.client.0.smithi007.stdout:6/362: unlink d2/d11/d3b/c71 0 2024-03-16T09:01:13.083 INFO:tasks.workunit.client.1.smithi019.stdout:2/225: creat d5/de/f45 x:0 0 0 2024-03-16T09:01:13.083 INFO:tasks.workunit.client.1.smithi019.stdout:2/226: chown d5/d6/d2b/d2f/d35/c44 42 1 2024-03-16T09:01:13.083 INFO:tasks.workunit.client.1.smithi019.stdout:2/227: dread - d5/de/f45 zero size 2024-03-16T09:01:13.083 INFO:tasks.workunit.client.1.smithi019.stdout:2/228: creat d5/d6/d2b/d2f/f46 x:0 0 0 2024-03-16T09:01:13.087 INFO:tasks.workunit.client.1.smithi019.stdout:7/196: symlink d9/l3a 0 2024-03-16T09:01:13.087 INFO:tasks.workunit.client.1.smithi019.stdout:7/197: stat d9/da/d13/d18/d1a/l21 0 2024-03-16T09:01:13.087 INFO:tasks.workunit.client.0.smithi007.stdout:1/430: creat d9/f85 x:0 0 0 2024-03-16T09:01:13.088 INFO:tasks.workunit.client.0.smithi007.stdout:1/431: fsync d9/dd/d1b/d28/d29/f5d 0 2024-03-16T09:01:13.088 INFO:tasks.workunit.client.0.smithi007.stdout:1/432: chown d9/f85 3 1 2024-03-16T09:01:13.088 INFO:tasks.workunit.client.0.smithi007.stdout:1/433: write d9/dd/d1b/d58/f66 [479936,83271] 0 2024-03-16T09:01:13.088 INFO:tasks.workunit.client.1.smithi019.stdout:5/229: truncate d6/d1f/f20 1425218 0 2024-03-16T09:01:13.088 INFO:tasks.workunit.client.1.smithi019.stdout:5/230: readlink d6/dd/l12 0 2024-03-16T09:01:13.089 INFO:tasks.workunit.client.0.smithi007.stdout:4/382: mkdir d4/d7/d78/d80 0 2024-03-16T09:01:13.093 INFO:tasks.workunit.client.0.smithi007.stdout:5/431: rename d1/d7/d1c/d19/d34/d1d/d36 to d1/d7/d1c/d88/da2 0 2024-03-16T09:01:13.093 INFO:tasks.workunit.client.0.smithi007.stdout:9/541: dread d24/f34 [0,4194304] 0 2024-03-16T09:01:13.093 INFO:tasks.workunit.client.0.smithi007.stdout:9/542: write d24/d4a/d55/d56/d7c/d95/f7b [989324,61806] 0 2024-03-16T09:01:13.093 INFO:tasks.workunit.client.1.smithi019.stdout:4/281: link d4/d1b/d2a/f4e d4/d6/d15/f60 0 2024-03-16T09:01:13.095 INFO:tasks.workunit.client.1.smithi019.stdout:8/260: link d6/lc d6/de/d24/d30/d4a/l53 0 2024-03-16T09:01:13.095 INFO:tasks.workunit.client.1.smithi019.stdout:8/261: dread - d6/d2a/d45/f42 zero size 2024-03-16T09:01:13.095 INFO:tasks.workunit.client.1.smithi019.stdout:8/262: truncate d6/d2a/d3c/f46 976651 0 2024-03-16T09:01:13.095 INFO:tasks.workunit.client.1.smithi019.stdout:0/263: dwrite d7/d8/d32/d4a/f4d [0,4194304] 0 2024-03-16T09:01:13.096 INFO:tasks.workunit.client.1.smithi019.stdout:3/277: mknod d3/d11/c58 0 2024-03-16T09:01:13.097 INFO:tasks.workunit.client.1.smithi019.stdout:3/278: creat d3/d17/f59 x:0 0 0 2024-03-16T09:01:13.097 INFO:tasks.workunit.client.1.smithi019.stdout:3/279: fdatasync d3/d1b/d40/d4a/f41 0 2024-03-16T09:01:13.099 INFO:tasks.workunit.client.1.smithi019.stdout:7/198: symlink d9/db/l3b 0 2024-03-16T09:01:13.100 INFO:tasks.workunit.client.1.smithi019.stdout:5/231: symlink d6/d1f/l43 0 2024-03-16T09:01:13.100 INFO:tasks.workunit.client.0.smithi007.stdout:1/434: mknod d9/dd/d5e/c86 0 2024-03-16T09:01:13.101 INFO:tasks.workunit.client.1.smithi019.stdout:4/282: symlink d4/d1b/d2a/d3d/l61 0 2024-03-16T09:01:13.102 INFO:tasks.workunit.client.1.smithi019.stdout:8/263: mknod d6/d2a/d45/d19/d28/c54 0 2024-03-16T09:01:13.103 INFO:tasks.workunit.client.1.smithi019.stdout:2/229: dwrite d5/de/f45 [0,4194304] 0 2024-03-16T09:01:13.103 INFO:tasks.workunit.client.1.smithi019.stdout:2/230: fdatasync d5/d6/f1d 0 2024-03-16T09:01:13.104 INFO:tasks.workunit.client.0.smithi007.stdout:6/363: getdents d2/d11/d1e/d54 0 2024-03-16T09:01:13.105 INFO:tasks.workunit.client.1.smithi019.stdout:0/264: symlink d7/l51 0 2024-03-16T09:01:13.105 INFO:tasks.workunit.client.1.smithi019.stdout:0/265: dread - d7/f46 zero size 2024-03-16T09:01:13.105 INFO:tasks.workunit.client.1.smithi019.stdout:0/266: chown d7/d1d/f4b 66431 1 2024-03-16T09:01:13.105 INFO:tasks.workunit.client.1.smithi019.stdout:0/267: write d7/d29/f47 [759422,36559] 0 2024-03-16T09:01:13.106 INFO:tasks.workunit.client.0.smithi007.stdout:2/565: rename dc/df/d6c/c84 to dc/df/d24/d28/d4c/cbb 0 2024-03-16T09:01:13.107 INFO:tasks.workunit.client.1.smithi019.stdout:3/280: mkdir d3/d17/d5a 0 2024-03-16T09:01:13.107 INFO:tasks.workunit.client.1.smithi019.stdout:3/281: write f2 [3005639,78590] 0 2024-03-16T09:01:13.107 INFO:tasks.workunit.client.0.smithi007.stdout:5/432: creat d1/d7/d1c/d19/d34/d4d/d99/fa3 x:0 0 0 2024-03-16T09:01:13.109 INFO:tasks.workunit.client.1.smithi019.stdout:7/199: unlink d9/db/f1d 0 2024-03-16T09:01:13.110 INFO:tasks.workunit.client.1.smithi019.stdout:5/232: rename d6/dd/d37/f24 to d6/dd/f44 0 2024-03-16T09:01:13.111 INFO:tasks.workunit.client.1.smithi019.stdout:5/233: fdatasync d6/f8 0 2024-03-16T09:01:13.111 INFO:tasks.workunit.client.1.smithi019.stdout:4/283: truncate d4/d6/d15/d1f/d2c/f31 1257340 0 2024-03-16T09:01:13.112 INFO:tasks.workunit.client.1.smithi019.stdout:4/284: dread - d4/d1b/f5d zero size 2024-03-16T09:01:13.113 INFO:tasks.workunit.client.0.smithi007.stdout:9/543: dwrite d24/d4a/d55/d56/d7c/f98 [0,4194304] 0 2024-03-16T09:01:13.115 INFO:tasks.workunit.client.0.smithi007.stdout:6/364: mkdir d2/d13/d3f/d50/d78 0 2024-03-16T09:01:13.116 INFO:tasks.workunit.client.1.smithi019.stdout:8/264: mkdir d6/d2a/d55 0 2024-03-16T09:01:13.116 INFO:tasks.workunit.client.1.smithi019.stdout:8/265: creat d6/d2a/d3c/f56 x:0 0 0 2024-03-16T09:01:13.116 INFO:tasks.workunit.client.1.smithi019.stdout:5/234: dread d6/d19/f1b [0,4194304] 0 2024-03-16T09:01:13.117 INFO:tasks.workunit.client.0.smithi007.stdout:1/435: read d9/dd/d1b/f2c [3302110,94279] 0 2024-03-16T09:01:13.117 INFO:tasks.workunit.client.0.smithi007.stdout:5/433: dread d1/d7/d1c/f15 [0,4194304] 0 2024-03-16T09:01:13.117 INFO:tasks.workunit.client.0.smithi007.stdout:5/434: fdatasync d1/d7/d16/d1e/f31 0 2024-03-16T09:01:13.121 INFO:tasks.workunit.client.1.smithi019.stdout:0/268: symlink d7/l52 0 2024-03-16T09:01:13.121 INFO:tasks.workunit.client.0.smithi007.stdout:4/383: rename d4/d14/d1c/d5b/f5d to d4/d14/d1c/d5b/d67/f81 0 2024-03-16T09:01:13.126 INFO:tasks.workunit.client.0.smithi007.stdout:7/393: getdents d0/d6c/d25/d7a 0 2024-03-16T09:01:13.127 INFO:tasks.workunit.client.1.smithi019.stdout:7/200: symlink d9/da/d13/l3c 0 2024-03-16T09:01:13.127 INFO:tasks.workunit.client.0.smithi007.stdout:1/436: stat d9/dd/d1b/d28/l75 0 2024-03-16T09:01:13.127 INFO:tasks.workunit.client.0.smithi007.stdout:6/365: dread d2/d11/d1e/f2f [0,4194304] 0 2024-03-16T09:01:13.127 INFO:tasks.workunit.client.0.smithi007.stdout:5/435: creat d1/d7/d1c/d19/d34/fa4 x:0 0 0 2024-03-16T09:01:13.128 INFO:tasks.workunit.client.1.smithi019.stdout:3/282: rename d3/d1b/c1e to d3/d17/c5b 0 2024-03-16T09:01:13.128 INFO:tasks.workunit.client.1.smithi019.stdout:3/283: truncate d3/f25 699165 0 2024-03-16T09:01:13.129 INFO:tasks.workunit.client.0.smithi007.stdout:9/544: rename d24/d4c/d91/d94 to d24/d4a/d96/d9c 0 2024-03-16T09:01:13.132 INFO:tasks.workunit.client.1.smithi019.stdout:8/266: mkdir d6/d2a/d45/d57 0 2024-03-16T09:01:13.132 INFO:tasks.workunit.client.1.smithi019.stdout:8/267: stat d6/d2a/d55 0 2024-03-16T09:01:13.132 INFO:tasks.workunit.client.0.smithi007.stdout:4/384: mkdir d4/d14/d64/d68/d82 0 2024-03-16T09:01:13.133 INFO:tasks.workunit.client.1.smithi019.stdout:5/235: creat d6/d19/d34/d3e/f45 x:0 0 0 2024-03-16T09:01:13.133 INFO:tasks.workunit.client.1.smithi019.stdout:7/201: dread f8 [0,4194304] 0 2024-03-16T09:01:13.133 INFO:tasks.workunit.client.1.smithi019.stdout:7/202: chown d9/da/d13/d18/d30 10 1 2024-03-16T09:01:13.133 INFO:tasks.workunit.client.1.smithi019.stdout:7/203: stat d9/da/d13/d18/d1a/d32 0 2024-03-16T09:01:13.134 INFO:tasks.workunit.client.0.smithi007.stdout:7/394: symlink d0/d19/d1f/d42/l85 0 2024-03-16T09:01:13.134 INFO:tasks.workunit.client.0.smithi007.stdout:7/395: creat d0/d6c/d25/d3a/f86 x:0 0 0 2024-03-16T09:01:13.134 INFO:tasks.workunit.client.1.smithi019.stdout:2/231: rmdir d5/de 39 2024-03-16T09:01:13.135 INFO:tasks.workunit.client.1.smithi019.stdout:3/284: truncate d3/f30 1713314 0 2024-03-16T09:01:13.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/236: creat d6/d1f/d33/f46 x:0 0 0 2024-03-16T09:01:13.137 INFO:tasks.workunit.client.0.smithi007.stdout:5/436: dread d1/d7/d1c/d19/d34/d5e/d6f/f54 [0,4194304] 0 2024-03-16T09:01:13.137 INFO:tasks.workunit.client.0.smithi007.stdout:5/437: readlink d1/d7/l42 0 2024-03-16T09:01:13.138 INFO:tasks.workunit.client.1.smithi019.stdout:2/232: rmdir d5/de 39 2024-03-16T09:01:13.138 INFO:tasks.workunit.client.1.smithi019.stdout:2/233: fsync d5/d2e/f3b 0 2024-03-16T09:01:13.142 INFO:tasks.workunit.client.0.smithi007.stdout:4/385: creat d4/d7/d78/d80/f83 x:0 0 0 2024-03-16T09:01:13.147 INFO:tasks.workunit.client.0.smithi007.stdout:1/437: rmdir d9/dd/d1b/d2d/d30/d45 39 2024-03-16T09:01:13.150 INFO:tasks.workunit.client.1.smithi019.stdout:8/268: read d6/f7 [2667241,106007] 0 2024-03-16T09:01:13.150 INFO:tasks.workunit.client.0.smithi007.stdout:2/566: dwrite dc/d27/f65 [0,4194304] 0 2024-03-16T09:01:13.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/438: unlink d1/d7/d16/d1b/c84 0 2024-03-16T09:01:13.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/439: write d1/d7/d1c/d19/d34/f8c [620034,16039] 0 2024-03-16T09:01:13.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/440: creat d1/d7/d1c/d19/d34/d5e/d6f/d72/fa5 x:0 0 0 2024-03-16T09:01:13.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/441: chown d1/d7/d1c/d19/d34/d3e/d41/f4c 16 1 2024-03-16T09:01:13.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/442: readlink d1/d7/d16/d1e/l21 0 2024-03-16T09:01:13.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/443: dread - d1/d7/d1c/d19/d34/d5e/f8e zero size 2024-03-16T09:01:13.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/444: chown d1/d7/d1c/d19 26660722 1 2024-03-16T09:01:13.155 INFO:tasks.workunit.client.0.smithi007.stdout:6/366: rmdir d2/d10 39 2024-03-16T09:01:13.156 INFO:tasks.workunit.client.0.smithi007.stdout:4/386: mkdir d4/d7/d78/d80/d84 0 2024-03-16T09:01:13.158 INFO:tasks.workunit.client.0.smithi007.stdout:2/567: getdents dc/df/d24/d28/d2e/d35/da7 0 2024-03-16T09:01:13.161 INFO:tasks.workunit.client.0.smithi007.stdout:6/367: creat d2/d11/f79 x:0 0 0 2024-03-16T09:01:13.161 INFO:tasks.workunit.client.0.smithi007.stdout:6/368: chown d2/d13/d1f/f67 3568 1 2024-03-16T09:01:13.162 INFO:tasks.workunit.client.0.smithi007.stdout:4/387: symlink d4/d14/d1c/d5b/d67/l85 0 2024-03-16T09:01:13.163 INFO:tasks.workunit.client.0.smithi007.stdout:6/369: mkdir d2/d11/d1e/d54/d7a 0 2024-03-16T09:01:13.163 INFO:tasks.workunit.client.0.smithi007.stdout:6/370: dread - d2/d13/d62/f6b zero size 2024-03-16T09:01:13.166 INFO:tasks.workunit.client.0.smithi007.stdout:6/371: rename d2/d13/d3f/d50/d72/f4a to d2/d13/d62/f7b 0 2024-03-16T09:01:13.186 INFO:tasks.workunit.client.0.smithi007.stdout:8/697: sync 2024-03-16T09:01:13.186 INFO:tasks.workunit.client.0.smithi007.stdout:8/698: creat d0/dc/dd/ff7 x:0 0 0 2024-03-16T09:01:13.186 INFO:tasks.workunit.client.0.smithi007.stdout:8/699: dread - d0/dc/d50/d8e/dd2/dea/d18/f8d zero size 2024-03-16T09:01:13.186 INFO:tasks.workunit.client.0.smithi007.stdout:3/504: sync 2024-03-16T09:01:13.186 INFO:tasks.workunit.client.0.smithi007.stdout:0/542: sync 2024-03-16T09:01:13.187 INFO:tasks.workunit.client.1.smithi019.stdout:3/285: unlink d3/d17/c2b 0 2024-03-16T09:01:13.188 INFO:tasks.workunit.client.0.smithi007.stdout:8/700: mknod d0/dc/d50/cf8 0 2024-03-16T09:01:13.188 INFO:tasks.workunit.client.0.smithi007.stdout:3/505: symlink d15/d18/d1f/d3b/d63/l9f 0 2024-03-16T09:01:13.189 INFO:tasks.workunit.client.1.smithi019.stdout:3/286: dread d3/d17/f50 [0,4194304] 0 2024-03-16T09:01:13.189 INFO:tasks.workunit.client.1.smithi019.stdout:3/287: chown f2 14 1 2024-03-16T09:01:13.189 INFO:tasks.workunit.client.1.smithi019.stdout:3/288: chown d3/d17 3512655 1 2024-03-16T09:01:13.189 INFO:tasks.workunit.client.1.smithi019.stdout:4/285: dwrite d4/d6/d15/f33 [0,4194304] 0 2024-03-16T09:01:13.189 INFO:tasks.workunit.client.1.smithi019.stdout:4/286: write d4/d6/d15/d1f/d2c/f5b [582993,26520] 0 2024-03-16T09:01:13.189 INFO:tasks.workunit.client.0.smithi007.stdout:0/543: symlink d3/d9/da/d48/d9e/lb3 0 2024-03-16T09:01:13.191 INFO:tasks.workunit.client.0.smithi007.stdout:3/506: creat d15/d18/d1f/d29/fa0 x:0 0 0 2024-03-16T09:01:13.192 INFO:tasks.workunit.client.1.smithi019.stdout:8/269: creat d6/d2a/d45/d2c/f58 x:0 0 0 2024-03-16T09:01:13.193 INFO:tasks.workunit.client.0.smithi007.stdout:7/396: dwrite d0/d6c/d25/d3a/f77 [0,4194304] 0 2024-03-16T09:01:13.193 INFO:tasks.workunit.client.0.smithi007.stdout:7/397: creat d0/d19/d1f/d69/f87 x:0 0 0 2024-03-16T09:01:13.194 INFO:tasks.workunit.client.1.smithi019.stdout:3/289: rename d3/f25 to d3/d17/d5a/f5c 0 2024-03-16T09:01:13.194 INFO:tasks.workunit.client.1.smithi019.stdout:3/290: fdatasync d3/d11/f49 0 2024-03-16T09:01:13.195 INFO:tasks.workunit.client.0.smithi007.stdout:0/544: rename d3/d9/db to d3/d9/db4 0 2024-03-16T09:01:13.198 INFO:tasks.workunit.client.0.smithi007.stdout:7/398: creat d0/d19/d1f/d76/f88 x:0 0 0 2024-03-16T09:01:13.199 INFO:tasks.workunit.client.0.smithi007.stdout:7/399: dread - d0/d19/d1f/d76/f88 zero size 2024-03-16T09:01:13.199 INFO:tasks.workunit.client.0.smithi007.stdout:7/400: truncate d0/d6c/d25/d3a/f4f 299071 0 2024-03-16T09:01:13.211 INFO:tasks.workunit.client.0.smithi007.stdout:7/401: dread d0/d6c/f52 [0,4194304] 0 2024-03-16T09:01:13.230 INFO:tasks.workunit.client.1.smithi019.stdout:0/269: dwrite d7/fd [0,4194304] 0 2024-03-16T09:01:13.231 INFO:tasks.workunit.client.0.smithi007.stdout:8/701: write d0/d26/d95/f9b [4778952,20399] 0 2024-03-16T09:01:13.233 INFO:tasks.workunit.client.0.smithi007.stdout:8/702: creat d0/dc/d50/d8e/dd2/dea/d18/da7/de2/ff9 x:0 0 0 2024-03-16T09:01:13.233 INFO:tasks.workunit.client.0.smithi007.stdout:8/703: truncate d0/d26/d2d/d43/d4c/dba/fd7 987846 0 2024-03-16T09:01:13.233 INFO:tasks.workunit.client.0.smithi007.stdout:8/704: creat d0/d26/d2d/d43/d72/dad/ffa x:0 0 0 2024-03-16T09:01:13.235 INFO:tasks.workunit.client.1.smithi019.stdout:7/204: dwrite d9/d25/f31 [0,4194304] 0 2024-03-16T09:01:13.235 INFO:tasks.workunit.client.1.smithi019.stdout:7/205: write d9/da/d13/d18/d1a/d32/f38 [308264,99172] 0 2024-03-16T09:01:13.236 INFO:tasks.workunit.client.0.smithi007.stdout:8/705: rename d0/dc/d50/d8e/dd2/dea/d18/da7/ff5 to d0/dc/d50/d8e/db8/ffb 0 2024-03-16T09:01:13.236 INFO:tasks.workunit.client.1.smithi019.stdout:0/270: mknod d7/d8/d32/d36/c53 0 2024-03-16T09:01:13.237 INFO:tasks.workunit.client.1.smithi019.stdout:4/287: dread d4/d6/d15/d1f/f30 [0,4194304] 0 2024-03-16T09:01:13.239 INFO:tasks.workunit.client.0.smithi007.stdout:2/568: dwrite dc/d27/f4f [0,4194304] 0 2024-03-16T09:01:13.239 INFO:tasks.workunit.client.0.smithi007.stdout:2/569: write dc/df/d24/d28/d2e/f5c [754495,12418] 0 2024-03-16T09:01:13.240 INFO:tasks.workunit.client.0.smithi007.stdout:8/706: creat d0/dc/d50/ffc x:0 0 0 2024-03-16T09:01:13.241 INFO:tasks.workunit.client.1.smithi019.stdout:4/288: symlink d4/l62 0 2024-03-16T09:01:13.243 INFO:tasks.workunit.client.1.smithi019.stdout:7/206: dread d9/f16 [0,4194304] 0 2024-03-16T09:01:13.243 INFO:tasks.workunit.client.1.smithi019.stdout:7/207: truncate d9/da/d13/d18/d1a/d32/f38 1000668 0 2024-03-16T09:01:13.243 INFO:tasks.workunit.client.1.smithi019.stdout:7/208: truncate d9/d25/d28/f39 315854 0 2024-03-16T09:01:13.243 INFO:tasks.workunit.client.1.smithi019.stdout:4/289: mknod d4/d6/d15/d1f/d2c/c63 0 2024-03-16T09:01:13.244 INFO:tasks.workunit.client.0.smithi007.stdout:2/570: chown dc/df/f14 32529 1 2024-03-16T09:01:13.244 INFO:tasks.workunit.client.1.smithi019.stdout:4/290: write d4/d6/d15/d1f/d2c/f40 [1078187,121238] 0 2024-03-16T09:01:13.247 INFO:tasks.workunit.client.0.smithi007.stdout:2/571: mkdir dc/df/d24/d28/d4c/dbc 0 2024-03-16T09:01:13.247 INFO:tasks.workunit.client.1.smithi019.stdout:3/291: dwrite d3/f51 [0,4194304] 0 2024-03-16T09:01:13.248 INFO:tasks.workunit.client.0.smithi007.stdout:0/545: dread d3/d9/da/d18/d27/f9a [0,4194304] 0 2024-03-16T09:01:13.250 INFO:tasks.workunit.client.0.smithi007.stdout:6/372: dwrite d2/f20 [0,4194304] 0 2024-03-16T09:01:13.251 INFO:tasks.workunit.client.0.smithi007.stdout:2/572: symlink dc/d27/da3/lbd 0 2024-03-16T09:01:13.252 INFO:tasks.workunit.client.0.smithi007.stdout:5/445: dwrite d1/d7/d1c/d19/d34/d1d/f4f [0,4194304] 0 2024-03-16T09:01:13.252 INFO:tasks.workunit.client.0.smithi007.stdout:4/388: dwrite d4/d14/d1c/d5b/d67/f79 [0,4194304] 0 2024-03-16T09:01:13.252 INFO:tasks.workunit.client.0.smithi007.stdout:3/507: dwrite d15/d18/f23 [4194304,4194304] 0 2024-03-16T09:01:13.253 INFO:tasks.workunit.client.1.smithi019.stdout:5/237: dwrite d6/f9 [4194304,4194304] 0 2024-03-16T09:01:13.253 INFO:tasks.workunit.client.1.smithi019.stdout:5/238: chown d6/fb 4856032 1 2024-03-16T09:01:13.253 INFO:tasks.workunit.client.1.smithi019.stdout:5/239: dread - d6/d19/f3c zero size 2024-03-16T09:01:13.253 INFO:tasks.workunit.client.1.smithi019.stdout:5/240: fsync d6/d1f/f2d 0 2024-03-16T09:01:13.253 INFO:tasks.workunit.client.0.smithi007.stdout:8/707: rename d0/dc/d54/da4 to d0/dc/dd/d8b/dc9/dce/dfd 0 2024-03-16T09:01:13.256 INFO:tasks.workunit.client.0.smithi007.stdout:5/446: mkdir d1/d7/d1c/d19/d34/d3e/d41/da6 0 2024-03-16T09:01:13.258 INFO:tasks.workunit.client.1.smithi019.stdout:3/292: mknod d3/d17/d5a/c5d 0 2024-03-16T09:01:13.258 INFO:tasks.workunit.client.1.smithi019.stdout:3/293: fdatasync d3/d1b/d40/d4a/d2c/f47 0 2024-03-16T09:01:13.261 INFO:tasks.workunit.client.1.smithi019.stdout:5/241: mkdir d6/dd/d47 0 2024-03-16T09:01:13.261 INFO:tasks.workunit.client.1.smithi019.stdout:5/242: creat d6/d19/d34/d3e/f48 x:0 0 0 2024-03-16T09:01:13.261 INFO:tasks.workunit.client.1.smithi019.stdout:5/243: read - d6/dd/f38 zero size 2024-03-16T09:01:13.263 INFO:tasks.workunit.client.0.smithi007.stdout:3/508: symlink d15/d18/d1f/d3b/d63/d8f/la1 0 2024-03-16T09:01:13.264 INFO:tasks.workunit.client.1.smithi019.stdout:5/244: getdents d6/d19 0 2024-03-16T09:01:13.264 INFO:tasks.workunit.client.1.smithi019.stdout:5/245: stat d6/f8 0 2024-03-16T09:01:13.264 INFO:tasks.workunit.client.1.smithi019.stdout:5/246: chown d6/d19/f1b 3 1 2024-03-16T09:01:13.265 INFO:tasks.workunit.client.0.smithi007.stdout:6/373: dread d2/d11/f26 [0,4194304] 0 2024-03-16T09:01:13.266 INFO:tasks.workunit.client.0.smithi007.stdout:5/447: creat d1/d7/d16/d1b/d3c/fa7 x:0 0 0 2024-03-16T09:01:13.269 INFO:tasks.workunit.client.0.smithi007.stdout:5/448: read d1/d7/d1c/d19/d34/f8c [229729,52654] 0 2024-03-16T09:01:13.270 INFO:tasks.workunit.client.0.smithi007.stdout:8/708: mkdir d0/d26/dfe 0 2024-03-16T09:01:13.270 INFO:tasks.workunit.client.0.smithi007.stdout:8/709: truncate d0/dc/dd/d8b/fcf 938930 0 2024-03-16T09:01:13.271 INFO:tasks.workunit.client.0.smithi007.stdout:8/710: fdatasync d0/d26/fa2 0 2024-03-16T09:01:13.272 INFO:tasks.workunit.client.1.smithi019.stdout:4/291: dwrite d4/d6/d15/d1f/f30 [0,4194304] 0 2024-03-16T09:01:13.272 INFO:tasks.workunit.client.0.smithi007.stdout:5/449: write d1/d7/d1c/d19/d34/d1d/f9e [1449484,109587] 0 2024-03-16T09:01:13.272 INFO:tasks.workunit.client.0.smithi007.stdout:2/573: write dc/df/d24/d28/d4c/d5e/f8b [3965467,46517] 0 2024-03-16T09:01:13.272 INFO:tasks.workunit.client.0.smithi007.stdout:8/711: dread d0/dc/d8a/f91 [0,4194304] 0 2024-03-16T09:01:13.272 INFO:tasks.workunit.client.0.smithi007.stdout:8/712: write d0/dc/dd/ff7 [197260,111337] 0 2024-03-16T09:01:13.273 INFO:tasks.workunit.client.0.smithi007.stdout:8/713: truncate d0/dc/d50/ffc 582507 0 2024-03-16T09:01:13.273 INFO:tasks.workunit.client.0.smithi007.stdout:8/714: dread - d0/dc/d50/d8e/dd2/dea/d18/da7/fed zero size 2024-03-16T09:01:13.274 INFO:tasks.workunit.client.1.smithi019.stdout:0/271: dwrite d7/d29/f41 [0,4194304] 0 2024-03-16T09:01:13.274 INFO:tasks.workunit.client.0.smithi007.stdout:6/374: unlink d2/d29/c37 0 2024-03-16T09:01:13.275 INFO:tasks.workunit.client.0.smithi007.stdout:3/509: rmdir d15/d18/d6a/d78 39 2024-03-16T09:01:13.275 INFO:tasks.workunit.client.0.smithi007.stdout:6/375: chown d2/d29/l5d 489472657 1 2024-03-16T09:01:13.275 INFO:tasks.workunit.client.0.smithi007.stdout:3/510: chown d15/l79 1 1 2024-03-16T09:01:13.275 INFO:tasks.workunit.client.0.smithi007.stdout:6/376: write d2/d13/d1f/d4e/f5b [1063462,70818] 0 2024-03-16T09:01:13.275 INFO:tasks.workunit.client.0.smithi007.stdout:6/377: truncate d2/d10/f3a 5062355 0 2024-03-16T09:01:13.275 INFO:tasks.workunit.client.0.smithi007.stdout:6/378: creat d2/d13/d4b/f7c x:0 0 0 2024-03-16T09:01:13.276 INFO:tasks.workunit.client.1.smithi019.stdout:3/294: dread d3/d11/f49 [0,4194304] 0 2024-03-16T09:01:13.277 INFO:tasks.workunit.client.0.smithi007.stdout:5/450: mkdir d1/d7/d1c/d19/d34/d1d/da8 0 2024-03-16T09:01:13.278 INFO:tasks.workunit.client.0.smithi007.stdout:2/574: getdents dc/df/d24/d28/d2e/d6e 0 2024-03-16T09:01:13.278 INFO:tasks.workunit.client.1.smithi019.stdout:0/272: rmdir d7/d1d/d2e 39 2024-03-16T09:01:13.280 INFO:tasks.workunit.client.0.smithi007.stdout:8/715: rename d0/dc/d50/d8e/dd2/dea/fb0 to d0/fff 0 2024-03-16T09:01:13.281 INFO:tasks.workunit.client.0.smithi007.stdout:8/716: write d0/dc/d50/d8e/dd2/dea/f66 [1020526,88561] 0 2024-03-16T09:01:13.283 INFO:tasks.workunit.client.1.smithi019.stdout:0/273: mknod d7/d29/c54 0 2024-03-16T09:01:13.283 INFO:tasks.workunit.client.1.smithi019.stdout:0/274: write d7/d8/fc [178763,51605] 0 2024-03-16T09:01:13.283 INFO:tasks.workunit.client.0.smithi007.stdout:5/451: mkdir d1/d7/d1c/d19/d34/d5e/d6f/da9 0 2024-03-16T09:01:13.285 INFO:tasks.workunit.client.0.smithi007.stdout:8/717: truncate d0/dc/d50/d8e/dd2/dea/d18/f48 135845 0 2024-03-16T09:01:13.288 INFO:tasks.workunit.client.0.smithi007.stdout:5/452: creat d1/faa x:0 0 0 2024-03-16T09:01:13.290 INFO:tasks.workunit.client.0.smithi007.stdout:8/718: dread d0/dc/d50/d8e/dd2/dea/d18/f3f [0,4194304] 0 2024-03-16T09:01:13.292 INFO:tasks.workunit.client.0.smithi007.stdout:6/379: dread d2/d10/f3a [0,4194304] 0 2024-03-16T09:01:13.296 INFO:tasks.workunit.client.1.smithi019.stdout:3/295: write d3/f51 [1897711,95239] 0 2024-03-16T09:01:13.296 INFO:tasks.workunit.client.1.smithi019.stdout:3/296: write d3/f43 [1302885,114065] 0 2024-03-16T09:01:13.304 INFO:tasks.workunit.client.1.smithi019.stdout:3/297: mknod d3/d1b/d40/d4a/c5e 0 2024-03-16T09:01:13.305 INFO:tasks.workunit.client.1.smithi019.stdout:3/298: fdatasync d3/d11/f49 0 2024-03-16T09:01:13.305 INFO:tasks.workunit.client.1.smithi019.stdout:3/299: fdatasync d3/d1b/d40/d4a/d2c/f38 0 2024-03-16T09:01:13.306 INFO:tasks.workunit.client.1.smithi019.stdout:3/300: unlink d3/d1b/d40/d4a/d2c/f56 0 2024-03-16T09:01:13.306 INFO:tasks.workunit.client.1.smithi019.stdout:3/301: chown d3/f51 366221 1 2024-03-16T09:01:13.306 INFO:tasks.workunit.client.1.smithi019.stdout:3/302: stat d3/d1b/d40/d4a/c5e 0 2024-03-16T09:01:13.327 INFO:tasks.workunit.client.0.smithi007.stdout:4/389: dwrite d4/d7/d78/d80/f83 [0,4194304] 0 2024-03-16T09:01:13.327 INFO:tasks.workunit.client.0.smithi007.stdout:4/390: dread - d4/d14/d64/f7d zero size 2024-03-16T09:01:13.327 INFO:tasks.workunit.client.0.smithi007.stdout:0/546: dwrite d3/d9/da/d48/f44 [0,4194304] 0 2024-03-16T09:01:13.334 INFO:tasks.workunit.client.0.smithi007.stdout:0/547: rename d3/d9/db4/d2c/lb2 to d3/d9/da/d18/d23/lb5 0 2024-03-16T09:01:13.341 INFO:tasks.workunit.client.1.smithi019.stdout:4/292: dwrite d4/d6/d15/d1f/d2c/f5b [0,4194304] 0 2024-03-16T09:01:13.342 INFO:tasks.workunit.client.0.smithi007.stdout:3/511: dwrite d15/d18/d1f/f8d [0,4194304] 0 2024-03-16T09:01:13.342 INFO:tasks.workunit.client.0.smithi007.stdout:3/512: write f10 [4541273,51930] 0 2024-03-16T09:01:13.342 INFO:tasks.workunit.client.0.smithi007.stdout:3/513: fsync d15/d18/d1f/f41 0 2024-03-16T09:01:13.342 INFO:tasks.workunit.client.0.smithi007.stdout:0/548: dread d3/d9/da/d18/d27/d34/f89 [0,4194304] 0 2024-03-16T09:01:13.342 INFO:tasks.workunit.client.0.smithi007.stdout:0/549: read - d3/d9/da/d48/fa5 zero size 2024-03-16T09:01:13.342 INFO:tasks.workunit.client.0.smithi007.stdout:6/380: dwrite d2/d10/d12/f35 [0,4194304] 0 2024-03-16T09:01:13.344 INFO:tasks.workunit.client.0.smithi007.stdout:3/514: dread d15/d18/d1f/d56/f65 [0,4194304] 0 2024-03-16T09:01:13.345 INFO:tasks.workunit.client.1.smithi019.stdout:6/383: sync 2024-03-16T09:01:13.345 INFO:tasks.workunit.client.1.smithi019.stdout:4/293: dread d4/d6/f24 [0,4194304] 0 2024-03-16T09:01:13.346 INFO:tasks.workunit.client.0.smithi007.stdout:0/550: symlink d3/d9/db4/d60/lb6 0 2024-03-16T09:01:13.346 INFO:tasks.workunit.client.1.smithi019.stdout:6/384: symlink de/d26/l80 0 2024-03-16T09:01:13.346 INFO:tasks.workunit.client.1.smithi019.stdout:6/385: dread - de/d1c/d1d/d66/f74 zero size 2024-03-16T09:01:13.346 INFO:tasks.workunit.client.1.smithi019.stdout:6/386: fsync de/d26/f39 0 2024-03-16T09:01:13.347 INFO:tasks.workunit.client.1.smithi019.stdout:0/275: dwrite d7/fd [0,4194304] 0 2024-03-16T09:01:13.347 INFO:tasks.workunit.client.1.smithi019.stdout:0/276: truncate d7/d1d/d2e/f3c 740469 0 2024-03-16T09:01:13.347 INFO:tasks.workunit.client.1.smithi019.stdout:0/277: write d7/d8/d32/d36/f3b [4353232,10909] 0 2024-03-16T09:01:13.347 INFO:tasks.workunit.client.1.smithi019.stdout:0/278: truncate d7/f46 648533 0 2024-03-16T09:01:13.347 INFO:tasks.workunit.client.1.smithi019.stdout:0/279: write d7/d8/d32/d36/f35 [2725366,64093] 0 2024-03-16T09:01:13.347 INFO:tasks.workunit.client.1.smithi019.stdout:4/294: mkdir d4/d64 0 2024-03-16T09:01:13.348 INFO:tasks.workunit.client.1.smithi019.stdout:6/387: creat de/d1c/d1d/d66/d72/d7f/f81 x:0 0 0 2024-03-16T09:01:13.349 INFO:tasks.workunit.client.0.smithi007.stdout:5/453: dwrite d1/d7/d1c/d19/d34/f8c [0,4194304] 0 2024-03-16T09:01:13.349 INFO:tasks.workunit.client.0.smithi007.stdout:8/719: dwrite d0/dc/d50/ffc [0,4194304] 0 2024-03-16T09:01:13.349 INFO:tasks.workunit.client.0.smithi007.stdout:6/381: creat d2/d10/d12/d15/f7d x:0 0 0 2024-03-16T09:01:13.349 INFO:tasks.workunit.client.0.smithi007.stdout:2/575: dwrite dc/df/d24/d28/f39 [0,4194304] 0 2024-03-16T09:01:13.349 INFO:tasks.workunit.client.0.smithi007.stdout:4/391: dwrite d4/d14/d1c/d5b/f74 [0,4194304] 0 2024-03-16T09:01:13.351 INFO:tasks.workunit.client.0.smithi007.stdout:3/515: symlink d15/d18/d1f/d29/d43/d72/la2 0 2024-03-16T09:01:13.351 INFO:tasks.workunit.client.1.smithi019.stdout:0/280: mknod d7/d1d/d2e/c55 0 2024-03-16T09:01:13.352 INFO:tasks.workunit.client.1.smithi019.stdout:0/281: write d7/d8/fc [550669,122497] 0 2024-03-16T09:01:13.352 INFO:tasks.workunit.client.1.smithi019.stdout:0/282: readlink d7/d1d/l30 0 2024-03-16T09:01:13.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/392: dread d4/d7/fe [0,4194304] 0 2024-03-16T09:01:13.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/393: fsync d4/d14/d1c/f60 0 2024-03-16T09:01:13.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/394: truncate d4/d7/d6b/f71 382557 0 2024-03-16T09:01:13.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/395: stat d4/d14/d1c/f32 0 2024-03-16T09:01:13.358 INFO:tasks.workunit.client.1.smithi019.stdout:0/283: unlink d7/la 0 2024-03-16T09:01:13.359 INFO:tasks.workunit.client.1.smithi019.stdout:4/295: getdents d4/d1b 0 2024-03-16T09:01:13.360 INFO:tasks.workunit.client.0.smithi007.stdout:3/516: dread d15/d18/f23 [0,4194304] 0 2024-03-16T09:01:13.360 INFO:tasks.workunit.client.0.smithi007.stdout:5/454: rename d1/d7/d1c/d19/d34/d3e/d41/f7a to d1/d7/d1c/d19/d34/d5e/d6f/d72/fab 0 2024-03-16T09:01:13.361 INFO:tasks.workunit.client.0.smithi007.stdout:6/382: dread d2/f20 [0,4194304] 0 2024-03-16T09:01:13.361 INFO:tasks.workunit.client.0.smithi007.stdout:6/383: chown d2/d29 12502 1 2024-03-16T09:01:13.362 INFO:tasks.workunit.client.1.smithi019.stdout:3/303: rename d3/d1b/d40/d4a/d2c to d3/d1b/d5f 0 2024-03-16T09:01:13.363 INFO:tasks.workunit.client.1.smithi019.stdout:4/296: rename d4/l62 to d4/l65 0 2024-03-16T09:01:13.364 INFO:tasks.workunit.client.1.smithi019.stdout:3/304: symlink d3/d1b/d40/d4a/l60 0 2024-03-16T09:01:13.365 INFO:tasks.workunit.client.1.smithi019.stdout:4/297: dread d4/d6/d15/d1f/d2c/f3f [0,4194304] 0 2024-03-16T09:01:13.366 INFO:tasks.workunit.client.1.smithi019.stdout:3/305: creat d3/d17/d5a/f61 x:0 0 0 2024-03-16T09:01:13.366 INFO:tasks.workunit.client.1.smithi019.stdout:3/306: stat d3/d11/f4c 0 2024-03-16T09:01:13.366 INFO:tasks.workunit.client.1.smithi019.stdout:4/298: truncate d4/d6/f32 367636 0 2024-03-16T09:01:13.366 INFO:tasks.workunit.client.1.smithi019.stdout:4/299: creat d4/d6/d15/f66 x:0 0 0 2024-03-16T09:01:13.367 INFO:tasks.workunit.client.0.smithi007.stdout:4/396: dread d4/fa [0,4194304] 0 2024-03-16T09:01:13.368 INFO:tasks.workunit.client.1.smithi019.stdout:3/307: unlink d3/d1b/d40/d4a/f4e 0 2024-03-16T09:01:13.368 INFO:tasks.workunit.client.1.smithi019.stdout:3/308: fdatasync d3/d11/f49 0 2024-03-16T09:01:13.368 INFO:tasks.workunit.client.1.smithi019.stdout:3/309: write f2 [2366037,127694] 0 2024-03-16T09:01:13.368 INFO:tasks.workunit.client.1.smithi019.stdout:3/310: dread - d3/d11/f37 zero size 2024-03-16T09:01:13.371 INFO:tasks.workunit.client.1.smithi019.stdout:0/284: dread d7/fd [0,4194304] 0 2024-03-16T09:01:13.375 INFO:tasks.workunit.client.0.smithi007.stdout:0/551: dwrite d3/d9/da/d18/d27/d34/d3a/f4f [0,4194304] 0 2024-03-16T09:01:13.375 INFO:tasks.workunit.client.0.smithi007.stdout:2/576: dwrite dc/f3a [4194304,4194304] 0 2024-03-16T09:01:13.375 INFO:tasks.workunit.client.0.smithi007.stdout:2/577: write dc/f7e [4758026,94492] 0 2024-03-16T09:01:13.376 INFO:tasks.workunit.client.0.smithi007.stdout:2/578: chown la 12332 1 2024-03-16T09:01:13.376 INFO:tasks.workunit.client.0.smithi007.stdout:8/720: rename d0/dc/d50/d8e/dd2/dea/d18/da7/lb4 to d0/d26/d2d/d64/l100 0 2024-03-16T09:01:13.377 INFO:tasks.workunit.client.0.smithi007.stdout:5/455: mknod d1/d7/d1c/d19/d34/d4d/d99/cac 0 2024-03-16T09:01:13.377 INFO:tasks.workunit.client.0.smithi007.stdout:3/517: truncate d15/d18/d1f/d29/f1b 379871 0 2024-03-16T09:01:13.379 INFO:tasks.workunit.client.0.smithi007.stdout:5/456: dread d1/d7/d1c/d88/da2/f45 [0,4194304] 0 2024-03-16T09:01:13.379 INFO:tasks.workunit.client.0.smithi007.stdout:5/457: fdatasync d1/d7/d1c/d19/d34/de/f49 0 2024-03-16T09:01:13.381 INFO:tasks.workunit.client.0.smithi007.stdout:0/552: getdents d3/d9/db4 0 2024-03-16T09:01:13.384 INFO:tasks.workunit.client.0.smithi007.stdout:2/579: write dc/df/d6c/fa9 [1342882,19564] 0 2024-03-16T09:01:13.386 INFO:tasks.workunit.client.0.smithi007.stdout:4/397: rename d4/d7/l5a to d4/d14/d1c/d5b/d67/l86 0 2024-03-16T09:01:13.388 INFO:tasks.workunit.client.0.smithi007.stdout:5/458: dread d1/d7/d16/d1e/f47 [0,4194304] 0 2024-03-16T09:01:13.388 INFO:tasks.workunit.client.0.smithi007.stdout:5/459: chown d1/d7/d1c/d19/d34/d5e/c9c 2 1 2024-03-16T09:01:13.388 INFO:tasks.workunit.client.0.smithi007.stdout:5/460: fdatasync d1/d7/d1c/d19/d34/d5e/d6f/d72/fa5 0 2024-03-16T09:01:13.388 INFO:tasks.workunit.client.0.smithi007.stdout:3/518: creat d15/d18/d1f/d77/d9d/fa3 x:0 0 0 2024-03-16T09:01:13.389 INFO:tasks.workunit.client.0.smithi007.stdout:8/721: dread d0/dc/d80/fcc [0,4194304] 0 2024-03-16T09:01:13.389 INFO:tasks.workunit.client.0.smithi007.stdout:8/722: chown d0/d26/c4f 35972461 1 2024-03-16T09:01:13.389 INFO:tasks.workunit.client.1.smithi019.stdout:3/311: rmdir d3/d17/d5a 39 2024-03-16T09:01:13.389 INFO:tasks.workunit.client.1.smithi019.stdout:3/312: fsync d3/f51 0 2024-03-16T09:01:13.390 INFO:tasks.workunit.client.1.smithi019.stdout:3/313: chown d3/d11/f4c 18111551 1 2024-03-16T09:01:13.390 INFO:tasks.workunit.client.0.smithi007.stdout:4/398: dread d4/d14/f35 [0,4194304] 0 2024-03-16T09:01:13.390 INFO:tasks.workunit.client.0.smithi007.stdout:5/461: rename d1/d7/d1c/d19/d34/d1d/f1f to d1/d7/d1c/d19/fad 0 2024-03-16T09:01:13.390 INFO:tasks.workunit.client.0.smithi007.stdout:5/462: chown d1/d7/d1c/d19/d34/d5e/d6f/d72/fab 61050 1 2024-03-16T09:01:13.391 INFO:tasks.workunit.client.1.smithi019.stdout:4/300: write d4/d6/f32 [8879,8254] 0 2024-03-16T09:01:13.392 INFO:tasks.workunit.client.0.smithi007.stdout:3/519: creat d15/fa4 x:0 0 0 2024-03-16T09:01:13.393 INFO:tasks.workunit.client.0.smithi007.stdout:3/520: truncate d15/d18/d1f/d29/d43/f7a 1233019 0 2024-03-16T09:01:13.393 INFO:tasks.workunit.client.0.smithi007.stdout:2/580: read dc/df/d24/d28/d2e/f45 [520239,34076] 0 2024-03-16T09:01:13.394 INFO:tasks.workunit.client.0.smithi007.stdout:8/723: mknod d0/dc/d75/dc3/c101 0 2024-03-16T09:01:13.394 INFO:tasks.workunit.client.0.smithi007.stdout:8/724: creat d0/dc/d50/d8e/dd2/dea/d18/da7/de2/f102 x:0 0 0 2024-03-16T09:01:13.394 INFO:tasks.workunit.client.0.smithi007.stdout:8/725: write d0/dc/d50/d8e/dd2/dea/f5d [3412073,54902] 0 2024-03-16T09:01:13.395 INFO:tasks.workunit.client.1.smithi019.stdout:0/285: dwrite d7/d29/f3e [0,4194304] 0 2024-03-16T09:01:13.397 INFO:tasks.workunit.client.0.smithi007.stdout:0/553: dread d3/f22 [0,4194304] 0 2024-03-16T09:01:13.397 INFO:tasks.workunit.client.0.smithi007.stdout:5/463: mkdir d1/d7/d1c/d88/da2/dae 0 2024-03-16T09:01:13.397 INFO:tasks.workunit.client.0.smithi007.stdout:5/464: dread - d1/d7/d16/d1e/fa0 zero size 2024-03-16T09:01:13.397 INFO:tasks.workunit.client.0.smithi007.stdout:0/554: write d3/d9/db4/d1c/f2a [6016157,22161] 0 2024-03-16T09:01:13.397 INFO:tasks.workunit.client.0.smithi007.stdout:4/399: mknod d4/d14/d64/d68/c87 0 2024-03-16T09:01:13.399 INFO:tasks.workunit.client.1.smithi019.stdout:0/286: creat d7/d1d/d2e/f56 x:0 0 0 2024-03-16T09:01:13.400 INFO:tasks.workunit.client.0.smithi007.stdout:5/465: symlink d1/d7/d1c/laf 0 2024-03-16T09:01:13.402 INFO:tasks.workunit.client.0.smithi007.stdout:4/400: rename d4/d14/d64/d68/f45 to d4/d14/d64/f88 0 2024-03-16T09:01:13.402 INFO:tasks.workunit.client.0.smithi007.stdout:4/401: write d4/d14/d1c/d1e/f4b [1120440,66674] 0 2024-03-16T09:01:13.403 INFO:tasks.workunit.client.0.smithi007.stdout:0/555: dread d3/d9/da/d48/d25/d43/d1a/f30 [0,4194304] 0 2024-03-16T09:01:13.403 INFO:tasks.workunit.client.0.smithi007.stdout:3/521: getdents d15/d18/d1f/d29/d43 0 2024-03-16T09:01:13.403 INFO:tasks.workunit.client.0.smithi007.stdout:3/522: stat d15/d18/d1f/c6c 0 2024-03-16T09:01:13.404 INFO:tasks.workunit.client.1.smithi019.stdout:3/314: write d3/d1b/d5f/f38 [3287953,4849] 0 2024-03-16T09:01:13.405 INFO:tasks.workunit.client.0.smithi007.stdout:2/581: dread dc/df/f14 [0,4194304] 0 2024-03-16T09:01:13.407 INFO:tasks.workunit.client.0.smithi007.stdout:5/466: mknod d1/d7/d16/d1e/d7d/cb0 0 2024-03-16T09:01:13.407 INFO:tasks.workunit.client.1.smithi019.stdout:3/315: link d3/d11/f37 d3/d17/d5a/f62 0 2024-03-16T09:01:13.408 INFO:tasks.workunit.client.1.smithi019.stdout:0/287: truncate d7/d29/f3e 1620585 0 2024-03-16T09:01:13.409 INFO:tasks.workunit.client.1.smithi019.stdout:3/316: mknod d3/d17/d5a/c63 0 2024-03-16T09:01:13.412 INFO:tasks.workunit.client.1.smithi019.stdout:3/317: symlink d3/d1b/l64 0 2024-03-16T09:01:13.412 INFO:tasks.workunit.client.1.smithi019.stdout:3/318: fsync d3/d1b/d5f/f47 0 2024-03-16T09:01:13.412 INFO:tasks.workunit.client.1.smithi019.stdout:3/319: write d3/d11/f49 [4440586,73240] 0 2024-03-16T09:01:13.413 INFO:tasks.workunit.client.1.smithi019.stdout:0/288: link d7/d8/c10 d7/d29/c57 0 2024-03-16T09:01:13.418 INFO:tasks.workunit.client.1.smithi019.stdout:3/320: link d3/d17/d3f/c55 d3/d1b/d40/c65 0 2024-03-16T09:01:13.420 INFO:tasks.workunit.client.1.smithi019.stdout:3/321: unlink d3/d17/d5a/c63 0 2024-03-16T09:01:13.420 INFO:tasks.workunit.client.1.smithi019.stdout:3/322: write d3/d11/f14 [7928493,34213] 0 2024-03-16T09:01:13.422 INFO:tasks.workunit.client.1.smithi019.stdout:3/323: symlink d3/l66 0 2024-03-16T09:01:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:8/726: dwrite d0/dc/f12 [0,4194304] 0 2024-03-16T09:01:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:5/467: creat d1/d7/d1c/d88/da2/fb1 x:0 0 0 2024-03-16T09:01:13.431 INFO:tasks.workunit.client.1.smithi019.stdout:3/324: dread d3/f51 [0,4194304] 0 2024-03-16T09:01:13.431 INFO:tasks.workunit.client.0.smithi007.stdout:2/582: unlink dc/df/f34 0 2024-03-16T09:01:13.432 INFO:tasks.workunit.client.1.smithi019.stdout:3/325: getdents d3/d11 0 2024-03-16T09:01:13.433 INFO:tasks.workunit.client.1.smithi019.stdout:3/326: fsync d3/d11/f4c 0 2024-03-16T09:01:13.439 INFO:tasks.workunit.client.1.smithi019.stdout:9/203: sync 2024-03-16T09:01:13.439 INFO:tasks.workunit.client.1.smithi019.stdout:2/234: sync 2024-03-16T09:01:13.439 INFO:tasks.workunit.client.1.smithi019.stdout:2/235: dread - d5/d6/d2b/d2f/d35/f39 zero size 2024-03-16T09:01:13.440 INFO:tasks.workunit.client.0.smithi007.stdout:4/402: rmdir d4/d14/d64/d68 39 2024-03-16T09:01:13.447 INFO:tasks.workunit.client.1.smithi019.stdout:1/203: sync 2024-03-16T09:01:13.447 INFO:tasks.workunit.client.1.smithi019.stdout:1/204: creat d16/d18/f39 x:0 0 0 2024-03-16T09:01:13.448 INFO:tasks.workunit.client.1.smithi019.stdout:9/204: dread d1/d7/de/f2a [0,4194304] 0 2024-03-16T09:01:13.462 INFO:tasks.workunit.client.1.smithi019.stdout:2/236: write d5/de/f45 [117337,90774] 0 2024-03-16T09:01:13.465 INFO:tasks.workunit.client.1.smithi019.stdout:2/237: symlink d5/d2e/l47 0 2024-03-16T09:01:13.467 INFO:tasks.workunit.client.1.smithi019.stdout:2/238: mknod d5/d2e/c48 0 2024-03-16T09:01:13.467 INFO:tasks.workunit.client.1.smithi019.stdout:2/239: write d5/de/fd [944689,58265] 0 2024-03-16T09:01:13.467 INFO:tasks.workunit.client.1.smithi019.stdout:2/240: chown d5/de 285 1 2024-03-16T09:01:13.468 INFO:tasks.workunit.client.1.smithi019.stdout:2/241: dread d5/d6/f36 [0,4194304] 0 2024-03-16T09:01:13.470 INFO:tasks.workunit.client.0.smithi007.stdout:3/523: dwrite d15/d18/d1f/f58 [4194304,4194304] 0 2024-03-16T09:01:13.470 INFO:tasks.workunit.client.1.smithi019.stdout:2/242: link d5/d6/d2b/l2a d5/l49 0 2024-03-16T09:01:13.472 INFO:tasks.workunit.client.0.smithi007.stdout:3/524: unlink d15/d18/d1f/d3b/c48 0 2024-03-16T09:01:13.472 INFO:tasks.workunit.client.0.smithi007.stdout:3/525: write d15/d18/d1f/d56/f8a [750945,22809] 0 2024-03-16T09:01:13.473 INFO:tasks.workunit.client.0.smithi007.stdout:3/526: fdatasync d15/d18/d1f/d3b/d87/f97 0 2024-03-16T09:01:13.473 INFO:tasks.workunit.client.0.smithi007.stdout:3/527: mkdir d15/d18/d1f/d3b/d63/d8f/da5 0 2024-03-16T09:01:13.475 INFO:tasks.workunit.client.0.smithi007.stdout:0/556: dwrite d3/d1f/f71 [0,4194304] 0 2024-03-16T09:01:13.478 INFO:tasks.workunit.client.1.smithi019.stdout:0/289: dwrite d7/d8/fc [0,4194304] 0 2024-03-16T09:01:13.479 INFO:tasks.workunit.client.0.smithi007.stdout:5/468: dwrite d1/d7/d1c/d19/d34/d5e/d6f/f54 [0,4194304] 0 2024-03-16T09:01:13.479 INFO:tasks.workunit.client.1.smithi019.stdout:3/327: dwrite f2 [0,4194304] 0 2024-03-16T09:01:13.479 INFO:tasks.workunit.client.0.smithi007.stdout:4/403: dwrite d4/d14/f35 [0,4194304] 0 2024-03-16T09:01:13.480 INFO:tasks.workunit.client.1.smithi019.stdout:0/290: truncate d7/d8/d32/d4a/f4d 4086124 0 2024-03-16T09:01:13.480 INFO:tasks.workunit.client.1.smithi019.stdout:0/291: fdatasync d7/d8/f50 0 2024-03-16T09:01:13.480 INFO:tasks.workunit.client.1.smithi019.stdout:1/205: dwrite d16/f19 [0,4194304] 0 2024-03-16T09:01:13.481 INFO:tasks.workunit.client.0.smithi007.stdout:2/583: dwrite dc/df/d24/d28/d4c/d5e/f8b [0,4194304] 0 2024-03-16T09:01:13.482 INFO:tasks.workunit.client.0.smithi007.stdout:2/584: dread - dc/df/d24/d28/d2e/f78 zero size 2024-03-16T09:01:13.482 INFO:tasks.workunit.client.1.smithi019.stdout:3/328: rmdir d3/d17/d3f 39 2024-03-16T09:01:13.482 INFO:tasks.workunit.client.1.smithi019.stdout:3/329: write d3/d11/f14 [7354326,124177] 0 2024-03-16T09:01:13.482 INFO:tasks.workunit.client.1.smithi019.stdout:0/292: fsync d7/f19 0 2024-03-16T09:01:13.484 INFO:tasks.workunit.client.1.smithi019.stdout:1/206: link d16/d18/l26 d16/d31/l3a 0 2024-03-16T09:01:13.487 INFO:tasks.workunit.client.1.smithi019.stdout:3/330: unlink d3/d11/l46 0 2024-03-16T09:01:13.487 INFO:tasks.workunit.client.0.smithi007.stdout:2/585: write dc/df/f18 [3826282,96474] 0 2024-03-16T09:01:13.487 INFO:tasks.workunit.client.1.smithi019.stdout:0/293: mknod d7/d8/d32/d36/c58 0 2024-03-16T09:01:13.489 INFO:tasks.workunit.client.0.smithi007.stdout:4/404: write d4/d14/d1c/f29 [2503964,83688] 0 2024-03-16T09:01:13.490 INFO:tasks.workunit.client.0.smithi007.stdout:3/528: write d15/d18/d1f/d3b/f47 [2563565,109766] 0 2024-03-16T09:01:13.491 INFO:tasks.workunit.client.0.smithi007.stdout:3/529: chown d15/d18/d1f/d77/d9d 82705 1 2024-03-16T09:01:13.491 INFO:tasks.workunit.client.1.smithi019.stdout:0/294: mknod d7/d1d/d28/d48/c59 0 2024-03-16T09:01:13.492 INFO:tasks.workunit.client.1.smithi019.stdout:3/331: dread d3/d1b/d5f/f47 [0,4194304] 0 2024-03-16T09:01:13.492 INFO:tasks.workunit.client.1.smithi019.stdout:1/207: getdents d16/d18 0 2024-03-16T09:01:13.492 INFO:tasks.workunit.client.1.smithi019.stdout:1/208: write d16/d18/f24 [408228,125422] 0 2024-03-16T09:01:13.492 INFO:tasks.workunit.client.1.smithi019.stdout:1/209: chown d16/l2c 131807212 1 2024-03-16T09:01:13.493 INFO:tasks.workunit.client.1.smithi019.stdout:1/210: symlink d16/l3b 0 2024-03-16T09:01:13.493 INFO:tasks.workunit.client.1.smithi019.stdout:1/211: getdents d16/d18 0 2024-03-16T09:01:13.495 INFO:tasks.workunit.client.1.smithi019.stdout:1/212: mkdir d16/d1f/d3c 0 2024-03-16T09:01:13.495 INFO:tasks.workunit.client.1.smithi019.stdout:1/213: creat d16/f3d x:0 0 0 2024-03-16T09:01:13.496 INFO:tasks.workunit.client.0.smithi007.stdout:2/586: getdents dc/d27/da3 0 2024-03-16T09:01:13.499 INFO:tasks.workunit.client.0.smithi007.stdout:3/530: rename d15/d18/d1f/d29/d3e/d4c/f52 to d15/d18/d1f/d29/d3e/d85/fa6 0 2024-03-16T09:01:13.500 INFO:tasks.workunit.client.0.smithi007.stdout:3/531: readlink d15/d18/d1f/l84 0 2024-03-16T09:01:13.502 INFO:tasks.workunit.client.0.smithi007.stdout:3/532: rename d15/d18/d1f/d29/d43/f4e to d15/d18/d1f/d77/d9d/fa7 0 2024-03-16T09:01:13.504 INFO:tasks.workunit.client.0.smithi007.stdout:3/533: link d15/d18/d1f/d3b/d87/f92 d15/d18/d6a/d78/fa8 0 2024-03-16T09:01:13.505 INFO:tasks.workunit.client.0.smithi007.stdout:3/534: chown d15/d18/d1f/d29/d43/d72/f50 14 1 2024-03-16T09:01:13.506 INFO:tasks.workunit.client.0.smithi007.stdout:3/535: link d15/d18/d1f/d29/d43/d72/c4b d15/d18/d6a/ca9 0 2024-03-16T09:01:13.507 INFO:tasks.workunit.client.0.smithi007.stdout:3/536: truncate d15/d18/d1f/f41 1525899 0 2024-03-16T09:01:13.507 INFO:tasks.workunit.client.0.smithi007.stdout:3/537: stat d15/d18/f23 0 2024-03-16T09:01:13.508 INFO:tasks.workunit.client.0.smithi007.stdout:3/538: symlink d15/d18/d1f/d29/laa 0 2024-03-16T09:01:13.510 INFO:tasks.workunit.client.0.smithi007.stdout:3/539: symlink d15/d18/d1f/lab 0 2024-03-16T09:01:13.510 INFO:tasks.workunit.client.0.smithi007.stdout:3/540: fsync d15/d18/d6a/d78/f9e 0 2024-03-16T09:01:13.520 INFO:tasks.workunit.client.1.smithi019.stdout:2/243: dwrite d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:13.529 INFO:tasks.workunit.client.0.smithi007.stdout:5/469: dwrite d1/d7/d1c/f80 [0,4194304] 0 2024-03-16T09:01:13.530 INFO:tasks.workunit.client.1.smithi019.stdout:2/244: dread d5/ff [0,4194304] 0 2024-03-16T09:01:13.530 INFO:tasks.workunit.client.0.smithi007.stdout:4/405: dwrite d4/d14/d1c/d1e/f4b [0,4194304] 0 2024-03-16T09:01:13.532 INFO:tasks.workunit.client.1.smithi019.stdout:3/332: dwrite d3/d1b/d5f/f47 [0,4194304] 0 2024-03-16T09:01:13.533 INFO:tasks.workunit.client.1.smithi019.stdout:3/333: dread - d3/d17/d5a/f61 zero size 2024-03-16T09:01:13.536 INFO:tasks.workunit.client.0.smithi007.stdout:3/541: dwrite d15/d18/d1f/f58 [4194304,4194304] 0 2024-03-16T09:01:13.537 INFO:tasks.workunit.client.0.smithi007.stdout:3/542: dread d15/d18/f22 [4194304,4194304] 0 2024-03-16T09:01:13.542 INFO:tasks.workunit.client.0.smithi007.stdout:3/543: unlink d15/d18/d1f/d3b/d63/d8f/la1 0 2024-03-16T09:01:13.542 INFO:tasks.workunit.client.0.smithi007.stdout:5/470: write d1/fa [1048896,49299] 0 2024-03-16T09:01:13.543 INFO:tasks.workunit.client.0.smithi007.stdout:5/471: mknod d1/d7/d1c/d19/d34/de/cb2 0 2024-03-16T09:01:13.546 INFO:tasks.workunit.client.1.smithi019.stdout:2/245: dwrite d5/f13 [0,4194304] 0 2024-03-16T09:01:13.557 INFO:tasks.workunit.client.0.smithi007.stdout:5/472: dwrite d1/d7/d1c/d19/d34/d5e/d6f/d72/f92 [0,4194304] 0 2024-03-16T09:01:13.563 INFO:tasks.workunit.client.1.smithi019.stdout:1/214: rmdir d16/d1f 39 2024-03-16T09:01:13.564 INFO:tasks.workunit.client.1.smithi019.stdout:3/334: creat d3/d1b/d40/f67 x:0 0 0 2024-03-16T09:01:13.569 INFO:tasks.workunit.client.1.smithi019.stdout:2/246: rename f1 to d5/de/f4a 0 2024-03-16T09:01:13.569 INFO:tasks.workunit.client.0.smithi007.stdout:5/473: write d1/fc [3404483,130774] 0 2024-03-16T09:01:13.570 INFO:tasks.workunit.client.1.smithi019.stdout:3/335: write d3/d11/f37 [866266,73879] 0 2024-03-16T09:01:13.570 INFO:tasks.workunit.client.1.smithi019.stdout:3/336: dread - d3/d17/d5a/f61 zero size 2024-03-16T09:01:13.570 INFO:tasks.workunit.client.1.smithi019.stdout:8/270: sync 2024-03-16T09:01:13.570 INFO:tasks.workunit.client.1.smithi019.stdout:8/271: chown d6/d2a/d45 908 1 2024-03-16T09:01:13.570 INFO:tasks.workunit.client.1.smithi019.stdout:3/337: symlink d3/d1b/d5f/l68 0 2024-03-16T09:01:13.572 INFO:tasks.workunit.client.1.smithi019.stdout:2/247: write f4 [1093825,25425] 0 2024-03-16T09:01:13.572 INFO:tasks.workunit.client.1.smithi019.stdout:8/272: rename d6/d2a/d45/c3a to d6/de/d51/c59 0 2024-03-16T09:01:13.574 INFO:tasks.workunit.client.1.smithi019.stdout:8/273: dread d6/de/ff [0,4194304] 0 2024-03-16T09:01:13.575 INFO:tasks.workunit.client.1.smithi019.stdout:2/248: mkdir d5/d6/d2b/d2f/d35/d4b 0 2024-03-16T09:01:13.575 INFO:tasks.workunit.client.0.smithi007.stdout:5/474: dread d1/d7/d1c/d19/f26 [0,4194304] 0 2024-03-16T09:01:13.576 INFO:tasks.workunit.client.1.smithi019.stdout:3/338: getdents d3 0 2024-03-16T09:01:13.576 INFO:tasks.workunit.client.1.smithi019.stdout:3/339: stat d3/d11/f37 0 2024-03-16T09:01:13.578 INFO:tasks.workunit.client.1.smithi019.stdout:8/274: mknod d6/de/d24/d30/d4a/c5a 0 2024-03-16T09:01:13.578 INFO:tasks.workunit.client.0.smithi007.stdout:5/475: rename d1/d7/d16/d1e to d1/d7/d1c/d88/da2/d89/db3 0 2024-03-16T09:01:13.580 INFO:tasks.workunit.client.1.smithi019.stdout:2/249: rename d5/l27 to d5/d2e/l4c 0 2024-03-16T09:01:13.580 INFO:tasks.workunit.client.1.smithi019.stdout:2/250: creat d5/f4d x:0 0 0 2024-03-16T09:01:13.581 INFO:tasks.workunit.client.1.smithi019.stdout:8/275: rename d6/d2a/d45/d19/d28/f50 to d6/d2a/d45/d19/d28/f5b 0 2024-03-16T09:01:13.581 INFO:tasks.workunit.client.1.smithi019.stdout:8/276: stat d6/de/c13 0 2024-03-16T09:01:13.581 INFO:tasks.workunit.client.1.smithi019.stdout:8/277: creat d6/d2a/d45/d2c/f5c x:0 0 0 2024-03-16T09:01:13.582 INFO:tasks.workunit.client.1.smithi019.stdout:8/278: truncate d6/de/d24/d30/f4f 372997 0 2024-03-16T09:01:13.582 INFO:tasks.workunit.client.1.smithi019.stdout:8/279: readlink d6/lc 0 2024-03-16T09:01:13.582 INFO:tasks.workunit.client.1.smithi019.stdout:2/251: mkdir d5/d4e 0 2024-03-16T09:01:13.583 INFO:tasks.workunit.client.1.smithi019.stdout:2/252: read - d5/f4d zero size 2024-03-16T09:01:13.583 INFO:tasks.workunit.client.1.smithi019.stdout:2/253: truncate d5/d6/d2b/f3c 592051 0 2024-03-16T09:01:13.583 INFO:tasks.workunit.client.1.smithi019.stdout:2/254: mkdir d5/d2e/d4f 0 2024-03-16T09:01:13.585 INFO:tasks.workunit.client.1.smithi019.stdout:2/255: dread d5/d6/d2b/f3c [0,4194304] 0 2024-03-16T09:01:13.587 INFO:tasks.workunit.client.1.smithi019.stdout:3/340: dread d3/f30 [0,4194304] 0 2024-03-16T09:01:13.589 INFO:tasks.workunit.client.0.smithi007.stdout:9/545: sync 2024-03-16T09:01:13.589 INFO:tasks.workunit.client.0.smithi007.stdout:7/402: sync 2024-03-16T09:01:13.589 INFO:tasks.workunit.client.0.smithi007.stdout:1/438: sync 2024-03-16T09:01:13.589 INFO:tasks.workunit.client.0.smithi007.stdout:7/403: creat d0/dc/d5a/f89 x:0 0 0 2024-03-16T09:01:13.589 INFO:tasks.workunit.client.0.smithi007.stdout:9/546: creat d24/d4a/d55/d56/d7c/d7f/d80/d87/d89/f9d x:0 0 0 2024-03-16T09:01:13.590 INFO:tasks.workunit.client.1.smithi019.stdout:3/341: creat d3/d11/f69 x:0 0 0 2024-03-16T09:01:13.591 INFO:tasks.workunit.client.0.smithi007.stdout:1/439: symlink d9/dd/d5e/l87 0 2024-03-16T09:01:13.591 INFO:tasks.workunit.client.0.smithi007.stdout:1/440: write d9/dd/d1b/d28/f46 [4801880,24044] 0 2024-03-16T09:01:13.591 INFO:tasks.workunit.client.1.smithi019.stdout:2/256: dread d5/de/f45 [0,4194304] 0 2024-03-16T09:01:13.594 INFO:tasks.workunit.client.0.smithi007.stdout:7/404: mkdir d0/d19/d1f/d5d/d8a 0 2024-03-16T09:01:13.594 INFO:tasks.workunit.client.0.smithi007.stdout:7/405: stat d0/d19/d1f/d42/l85 0 2024-03-16T09:01:13.595 INFO:tasks.workunit.client.0.smithi007.stdout:9/547: mkdir d24/d4a/d9e 0 2024-03-16T09:01:13.595 INFO:tasks.workunit.client.0.smithi007.stdout:9/548: dread - d24/d4a/d55/d56/d7c/f88 zero size 2024-03-16T09:01:13.597 INFO:tasks.workunit.client.0.smithi007.stdout:1/441: link d9/dd/d1b/d2d/f3f d9/dd/d1b/d28/d29/d7b/f88 0 2024-03-16T09:01:13.597 INFO:tasks.workunit.client.0.smithi007.stdout:1/442: write d9/dd/d1b/f49 [124344,6747] 0 2024-03-16T09:01:13.598 INFO:tasks.workunit.client.0.smithi007.stdout:1/443: mkdir d9/dd/d5e/d89 0 2024-03-16T09:01:13.599 INFO:tasks.workunit.client.0.smithi007.stdout:1/444: write d9/dd/d1b/f49 [27320,55918] 0 2024-03-16T09:01:13.602 INFO:tasks.workunit.client.1.smithi019.stdout:3/342: write d3/d11/f1f [618115,114778] 0 2024-03-16T09:01:13.602 INFO:tasks.workunit.client.1.smithi019.stdout:3/343: fsync d3/d1b/d40/d4a/f41 0 2024-03-16T09:01:13.603 INFO:tasks.workunit.client.1.smithi019.stdout:3/344: stat d3/d11/l39 0 2024-03-16T09:01:13.603 INFO:tasks.workunit.client.1.smithi019.stdout:3/345: stat d3/l66 0 2024-03-16T09:01:13.604 INFO:tasks.workunit.client.0.smithi007.stdout:9/549: dread d24/d4a/d55/d56/d7c/d7f/d80/f8c [0,4194304] 0 2024-03-16T09:01:13.604 INFO:tasks.workunit.client.0.smithi007.stdout:9/550: chown c14 204720712 1 2024-03-16T09:01:13.604 INFO:tasks.workunit.client.1.smithi019.stdout:3/346: mkdir d3/d17/d6a 0 2024-03-16T09:01:13.605 INFO:tasks.workunit.client.0.smithi007.stdout:9/551: dread d24/f25 [0,4194304] 0 2024-03-16T09:01:13.606 INFO:tasks.workunit.client.0.smithi007.stdout:9/552: mknod d24/d4a/d55/d56/d7c/d95/c9f 0 2024-03-16T09:01:13.608 INFO:tasks.workunit.client.0.smithi007.stdout:9/553: rmdir d24/d4a/d55/d56/d7c/d7f/d8f 39 2024-03-16T09:01:13.609 INFO:tasks.workunit.client.1.smithi019.stdout:8/280: dwrite d6/f27 [0,4194304] 0 2024-03-16T09:01:13.609 INFO:tasks.workunit.client.1.smithi019.stdout:8/281: write d6/d2a/d45/f42 [618675,86526] 0 2024-03-16T09:01:13.610 INFO:tasks.workunit.client.0.smithi007.stdout:9/554: symlink d24/d4a/d55/d56/la0 0 2024-03-16T09:01:13.613 INFO:tasks.workunit.client.0.smithi007.stdout:5/476: dwrite d1/d7/d1c/d88/da2/d89/db3/f31 [0,4194304] 0 2024-03-16T09:01:13.613 INFO:tasks.workunit.client.0.smithi007.stdout:7/406: dread d0/d6c/f52 [0,4194304] 0 2024-03-16T09:01:13.619 INFO:tasks.workunit.client.1.smithi019.stdout:2/257: dwrite d5/d6/f11 [0,4194304] 0 2024-03-16T09:01:13.628 INFO:tasks.workunit.client.0.smithi007.stdout:5/477: dread d1/d7/d1c/d19/fad [0,4194304] 0 2024-03-16T09:01:13.639 INFO:tasks.workunit.client.0.smithi007.stdout:1/445: dwrite d9/dd/d1b/d2d/f6a [0,4194304] 0 2024-03-16T09:01:13.643 INFO:tasks.workunit.client.1.smithi019.stdout:3/347: dwrite d3/d1b/d40/f67 [0,4194304] 0 2024-03-16T09:01:13.644 INFO:tasks.workunit.client.1.smithi019.stdout:3/348: fdatasync d3/f43 0 2024-03-16T09:01:13.644 INFO:tasks.workunit.client.1.smithi019.stdout:3/349: write d3/ff [2449444,115966] 0 2024-03-16T09:01:13.644 INFO:tasks.workunit.client.0.smithi007.stdout:7/407: dwrite d0/d6c/d25/d7a/f84 [0,4194304] 0 2024-03-16T09:01:13.644 INFO:tasks.workunit.client.0.smithi007.stdout:7/408: dread - d0/dc/d5a/f74 zero size 2024-03-16T09:01:13.644 INFO:tasks.workunit.client.0.smithi007.stdout:5/478: dwrite d1/d7/d1c/d19/d34/d5e/f8e [0,4194304] 0 2024-03-16T09:01:13.646 INFO:tasks.workunit.client.0.smithi007.stdout:9/555: dwrite d24/d4a/d55/d56/d7c/d7f/d80/d87/d89/f9d [0,4194304] 0 2024-03-16T09:01:13.646 INFO:tasks.workunit.client.0.smithi007.stdout:1/446: write d9/dd/d1b/d28/f32 [4185688,79704] 0 2024-03-16T09:01:13.646 INFO:tasks.workunit.client.0.smithi007.stdout:1/447: write d9/dd/d1b/d28/d29/f2a [5335136,114151] 0 2024-03-16T09:01:13.651 INFO:tasks.workunit.client.0.smithi007.stdout:1/448: mknod d9/dd/d1b/d28/c8a 0 2024-03-16T09:01:13.654 INFO:tasks.workunit.client.0.smithi007.stdout:6/384: sync 2024-03-16T09:01:13.654 INFO:tasks.workunit.client.0.smithi007.stdout:8/727: sync 2024-03-16T09:01:13.654 INFO:tasks.workunit.client.0.smithi007.stdout:7/409: dread d0/d19/f26 [0,4194304] 0 2024-03-16T09:01:13.656 INFO:tasks.workunit.client.0.smithi007.stdout:8/728: rename d0/dc/d80/ldb to d0/dc/d50/d8e/dd2/dea/d18/da7/de2/l103 0 2024-03-16T09:01:13.657 INFO:tasks.workunit.client.0.smithi007.stdout:7/410: link d0/d19/l36 d0/d19/d1f/d76/l8b 0 2024-03-16T09:01:13.660 INFO:tasks.workunit.client.0.smithi007.stdout:8/729: creat d0/dc/d50/d8e/dd2/dea/f104 x:0 0 0 2024-03-16T09:01:13.660 INFO:tasks.workunit.client.0.smithi007.stdout:8/730: write d0/dc/d8a/fb6 [1264554,91025] 0 2024-03-16T09:01:13.660 INFO:tasks.workunit.client.0.smithi007.stdout:8/731: truncate d0/dc/d50/d8e/dd2/dea/d18/f48 1145044 0 2024-03-16T09:01:13.660 INFO:tasks.workunit.client.0.smithi007.stdout:8/732: dread - d0/dc/dd/d8b/d9e/fde zero size 2024-03-16T09:01:13.660 INFO:tasks.workunit.client.0.smithi007.stdout:8/733: dread - d0/dc/d50/d8e/dd2/dea/d18/da7/de2/ff9 zero size 2024-03-16T09:01:13.660 INFO:tasks.workunit.client.0.smithi007.stdout:8/734: fdatasync d0/dc/d50/d8e/dd2/dea/d18/f8d 0 2024-03-16T09:01:13.662 INFO:tasks.workunit.client.0.smithi007.stdout:8/735: rename d0/dc/d54/fd8 to d0/d26/d2d/d43/d4c/dba/f105 0 2024-03-16T09:01:13.662 INFO:tasks.workunit.client.0.smithi007.stdout:8/736: chown d0/dc/d50/d8e/dd2/dea/f104 38627 1 2024-03-16T09:01:13.663 INFO:tasks.workunit.client.0.smithi007.stdout:8/737: symlink d0/d26/d2d/d43/l106 0 2024-03-16T09:01:13.664 INFO:tasks.workunit.client.0.smithi007.stdout:5/479: dwrite d1/d7/d1c/d19/d34/d3e/d41/f9f [0,4194304] 0 2024-03-16T09:01:13.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/738: creat d0/dc/d75/dc3/f107 x:0 0 0 2024-03-16T09:01:13.668 INFO:tasks.workunit.client.1.smithi019.stdout:2/258: unlink d5/d6/f11 0 2024-03-16T09:01:13.669 INFO:tasks.workunit.client.0.smithi007.stdout:5/480: creat d1/d7/d1c/d19/d34/d5e/d6f/fb4 x:0 0 0 2024-03-16T09:01:13.670 INFO:tasks.workunit.client.0.smithi007.stdout:8/739: symlink d0/dc/d75/l108 0 2024-03-16T09:01:13.672 INFO:tasks.workunit.client.0.smithi007.stdout:8/740: creat d0/dc/d50/d8e/f109 x:0 0 0 2024-03-16T09:01:13.675 INFO:tasks.workunit.client.0.smithi007.stdout:8/741: mkdir d0/dc/d50/d8e/dd2/dea/d10a 0 2024-03-16T09:01:13.675 INFO:tasks.workunit.client.0.smithi007.stdout:5/481: write d1/d7/d1c/d88/da2/d89/db3/f47 [327440,128188] 0 2024-03-16T09:01:13.678 INFO:tasks.workunit.client.1.smithi019.stdout:2/259: unlink d5/ca 0 2024-03-16T09:01:13.680 INFO:tasks.workunit.client.0.smithi007.stdout:5/482: rename d1/d7/d1c/d19/d34/d5e/c5b to d1/d7/d1c/d19/d34/d3e/cb5 0 2024-03-16T09:01:13.680 INFO:tasks.workunit.client.0.smithi007.stdout:5/483: creat d1/d7/d1c/d88/da2/d89/fb6 x:0 0 0 2024-03-16T09:01:13.680 INFO:tasks.workunit.client.1.smithi019.stdout:8/282: rename d6/d2a/d45/d19/d28 to d6/d2a/d45/d5d 0 2024-03-16T09:01:13.682 INFO:tasks.workunit.client.1.smithi019.stdout:6/388: sync 2024-03-16T09:01:13.682 INFO:tasks.workunit.client.1.smithi019.stdout:7/209: sync 2024-03-16T09:01:13.682 INFO:tasks.workunit.client.1.smithi019.stdout:5/247: sync 2024-03-16T09:01:13.682 INFO:tasks.workunit.client.1.smithi019.stdout:7/210: chown d9/d25 6236489 1 2024-03-16T09:01:13.683 INFO:tasks.workunit.client.1.smithi019.stdout:2/260: rename d5/d6/f1d to d5/d2e/f50 0 2024-03-16T09:01:13.683 INFO:tasks.workunit.client.1.smithi019.stdout:8/283: rmdir d6/d2a/d45/d19 39 2024-03-16T09:01:13.685 INFO:tasks.workunit.client.1.smithi019.stdout:2/261: mkdir d5/d6/d2b/d2f/d35/d4b/d51 0 2024-03-16T09:01:13.697 INFO:tasks.workunit.client.0.smithi007.stdout:9/556: dwrite f7 [0,4194304] 0 2024-03-16T09:01:13.697 INFO:tasks.workunit.client.0.smithi007.stdout:9/557: read d24/f2d [514198,26507] 0 2024-03-16T09:01:13.699 INFO:tasks.workunit.client.0.smithi007.stdout:6/385: dwrite d2/d11/f79 [0,4194304] 0 2024-03-16T09:01:13.703 INFO:tasks.workunit.client.0.smithi007.stdout:6/386: unlink d2/d13/d3f/d50/d72/c39 0 2024-03-16T09:01:13.703 INFO:tasks.workunit.client.1.smithi019.stdout:3/350: dwrite d3/f3e [0,4194304] 0 2024-03-16T09:01:13.705 INFO:tasks.workunit.client.0.smithi007.stdout:6/387: symlink d2/d13/l7e 0 2024-03-16T09:01:13.705 INFO:tasks.workunit.client.0.smithi007.stdout:6/388: write d2/d13/d62/f6b [311533,70930] 0 2024-03-16T09:01:13.706 INFO:tasks.workunit.client.0.smithi007.stdout:1/449: dread d9/dd/d1b/f25 [0,4194304] 0 2024-03-16T09:01:13.706 INFO:tasks.workunit.client.0.smithi007.stdout:6/389: mknod d2/d13/d24/c7f 0 2024-03-16T09:01:13.708 INFO:tasks.workunit.client.1.smithi019.stdout:3/351: creat d3/d1b/d5f/f6b x:0 0 0 2024-03-16T09:01:13.709 INFO:tasks.workunit.client.1.smithi019.stdout:3/352: creat d3/f6c x:0 0 0 2024-03-16T09:01:13.709 INFO:tasks.workunit.client.1.smithi019.stdout:3/353: stat d3/ff 0 2024-03-16T09:01:13.710 INFO:tasks.workunit.client.1.smithi019.stdout:3/354: symlink d3/d17/d5a/l6d 0 2024-03-16T09:01:13.710 INFO:tasks.workunit.client.1.smithi019.stdout:3/355: write d3/d11/f1f [4116123,16343] 0 2024-03-16T09:01:13.711 INFO:tasks.workunit.client.1.smithi019.stdout:3/356: fdatasync f2 0 2024-03-16T09:01:13.711 INFO:tasks.workunit.client.1.smithi019.stdout:3/357: stat d3/d17/d5a/f61 0 2024-03-16T09:01:13.711 INFO:tasks.workunit.client.1.smithi019.stdout:3/358: stat d3/d17/f59 0 2024-03-16T09:01:13.717 INFO:tasks.workunit.client.0.smithi007.stdout:6/390: write d2/d10/d12/f35 [956679,85841] 0 2024-03-16T09:01:13.721 INFO:tasks.workunit.client.0.smithi007.stdout:7/411: dwrite d0/dc/fe [4194304,4194304] 0 2024-03-16T09:01:13.722 INFO:tasks.workunit.client.0.smithi007.stdout:7/412: unlink d0/d19/d1f/d42/f57 0 2024-03-16T09:01:13.723 INFO:tasks.workunit.client.0.smithi007.stdout:7/413: symlink d0/d19/d1f/d69/l8c 0 2024-03-16T09:01:13.724 INFO:tasks.workunit.client.0.smithi007.stdout:7/414: truncate d0/d6c/d25/d3a/f77 767977 0 2024-03-16T09:01:13.725 INFO:tasks.workunit.client.0.smithi007.stdout:7/415: truncate d0/d6c/d25/d73/f5c 972068 0 2024-03-16T09:01:13.726 INFO:tasks.workunit.client.0.smithi007.stdout:7/416: chown d0/dc/c79 1 1 2024-03-16T09:01:13.726 INFO:tasks.workunit.client.0.smithi007.stdout:7/417: read d0/d19/d1f/f54 [2302243,45976] 0 2024-03-16T09:01:13.726 INFO:tasks.workunit.client.0.smithi007.stdout:8/742: dwrite d0/dc/dd/d8b/d9e/fde [0,4194304] 0 2024-03-16T09:01:13.726 INFO:tasks.workunit.client.0.smithi007.stdout:8/743: fsync d0/dc/d50/d8e/dd2/dea/d18/da7/fed 0 2024-03-16T09:01:13.726 INFO:tasks.workunit.client.0.smithi007.stdout:8/744: write d0/d26/d2d/d43/ff6 [701742,74217] 0 2024-03-16T09:01:13.726 INFO:tasks.workunit.client.0.smithi007.stdout:7/418: mknod d0/d19/d1f/d42/c8d 0 2024-03-16T09:01:13.726 INFO:tasks.workunit.client.0.smithi007.stdout:7/419: dread - d0/d6c/d25/d3a/f86 zero size 2024-03-16T09:01:13.729 INFO:tasks.workunit.client.0.smithi007.stdout:7/420: creat d0/d19/d1f/d42/f8e x:0 0 0 2024-03-16T09:01:13.730 INFO:tasks.workunit.client.0.smithi007.stdout:7/421: mknod d0/d19/d1f/d5d/c8f 0 2024-03-16T09:01:13.736 INFO:tasks.workunit.client.1.smithi019.stdout:8/284: dwrite d6/d2a/d45/d5d/f5b [0,4194304] 0 2024-03-16T09:01:13.736 INFO:tasks.workunit.client.1.smithi019.stdout:5/248: dwrite d6/dd/f38 [0,4194304] 0 2024-03-16T09:01:13.736 INFO:tasks.workunit.client.1.smithi019.stdout:5/249: creat d6/dd/f49 x:0 0 0 2024-03-16T09:01:13.737 INFO:tasks.workunit.client.0.smithi007.stdout:7/422: write d0/f3 [3532129,60935] 0 2024-03-16T09:01:13.737 INFO:tasks.workunit.client.0.smithi007.stdout:9/558: dwrite d24/f33 [0,4194304] 0 2024-03-16T09:01:13.741 INFO:tasks.workunit.client.1.smithi019.stdout:8/285: symlink d6/de/d24/d30/d4a/l5e 0 2024-03-16T09:01:13.749 INFO:tasks.workunit.client.1.smithi019.stdout:0/295: sync 2024-03-16T09:01:13.749 INFO:tasks.workunit.client.1.smithi019.stdout:9/205: sync 2024-03-16T09:01:13.749 INFO:tasks.workunit.client.1.smithi019.stdout:1/215: sync 2024-03-16T09:01:13.750 INFO:tasks.workunit.client.1.smithi019.stdout:4/301: sync 2024-03-16T09:01:13.750 INFO:tasks.workunit.client.1.smithi019.stdout:4/302: truncate d4/d6/d15/d1f/d2c/f3f 876952 0 2024-03-16T09:01:13.750 INFO:tasks.workunit.client.1.smithi019.stdout:4/303: creat d4/d6/d15/f67 x:0 0 0 2024-03-16T09:01:13.750 INFO:tasks.workunit.client.1.smithi019.stdout:4/304: chown d4/d1b/d49/l4a 42 1 2024-03-16T09:01:13.750 INFO:tasks.workunit.client.1.smithi019.stdout:4/305: chown d4/lb 41710 1 2024-03-16T09:01:13.750 INFO:tasks.workunit.client.1.smithi019.stdout:2/262: dwrite d5/d6/f36 [0,4194304] 0 2024-03-16T09:01:13.751 INFO:tasks.workunit.client.0.smithi007.stdout:1/450: dwrite d9/dd/d5e/f65 [0,4194304] 0 2024-03-16T09:01:13.751 INFO:tasks.workunit.client.1.smithi019.stdout:9/206: rename d1/l2b to d1/d7/d21/l4b 0 2024-03-16T09:01:13.752 INFO:tasks.workunit.client.1.smithi019.stdout:9/207: creat d1/d7/f4c x:0 0 0 2024-03-16T09:01:13.752 INFO:tasks.workunit.client.1.smithi019.stdout:9/208: write d1/d7/d21/d27/f35 [1938345,1684] 0 2024-03-16T09:01:13.752 INFO:tasks.workunit.client.1.smithi019.stdout:9/209: chown d1/d7/de/d24/c29 4852 1 2024-03-16T09:01:13.753 INFO:tasks.workunit.client.1.smithi019.stdout:9/210: write d1/d7/de/d24/f40 [390788,41970] 0 2024-03-16T09:01:13.753 INFO:tasks.workunit.client.1.smithi019.stdout:1/216: mknod d16/d1f/c3e 0 2024-03-16T09:01:13.753 INFO:tasks.workunit.client.1.smithi019.stdout:1/217: chown d16/l25 25824 1 2024-03-16T09:01:13.753 INFO:tasks.workunit.client.1.smithi019.stdout:1/218: fdatasync d16/f20 0 2024-03-16T09:01:13.753 INFO:tasks.workunit.client.1.smithi019.stdout:1/219: fsync d16/f3d 0 2024-03-16T09:01:13.753 INFO:tasks.workunit.client.0.smithi007.stdout:7/423: read d0/d19/d1f/f2c [6988869,110371] 0 2024-03-16T09:01:13.754 INFO:tasks.workunit.client.0.smithi007.stdout:1/451: link d9/dd/d1b/f83 d9/f8b 0 2024-03-16T09:01:13.755 INFO:tasks.workunit.client.1.smithi019.stdout:9/211: mknod d1/d7/d21/c4d 0 2024-03-16T09:01:13.758 INFO:tasks.workunit.client.1.smithi019.stdout:8/286: write f4 [6977962,114944] 0 2024-03-16T09:01:13.758 INFO:tasks.workunit.client.1.smithi019.stdout:3/359: dwrite d3/d17/f50 [0,4194304] 0 2024-03-16T09:01:13.758 INFO:tasks.workunit.client.0.smithi007.stdout:1/452: creat d9/dd/d1b/d28/d4c/f8c x:0 0 0 2024-03-16T09:01:13.758 INFO:tasks.workunit.client.0.smithi007.stdout:1/453: chown d9/f35 12719956 1 2024-03-16T09:01:13.759 INFO:tasks.workunit.client.1.smithi019.stdout:1/220: rename d16/d18/l26 to d16/d31/l3f 0 2024-03-16T09:01:13.759 INFO:tasks.workunit.client.1.smithi019.stdout:2/263: mknod d5/d6/d2b/d2f/d35/c52 0 2024-03-16T09:01:13.759 INFO:tasks.workunit.client.1.smithi019.stdout:0/296: dread d7/d8/d32/d36/f3b [0,4194304] 0 2024-03-16T09:01:13.760 INFO:tasks.workunit.client.1.smithi019.stdout:8/287: mknod d6/de/d24/c5f 0 2024-03-16T09:01:13.761 INFO:tasks.workunit.client.1.smithi019.stdout:3/360: dread d3/d11/f37 [0,4194304] 0 2024-03-16T09:01:13.761 INFO:tasks.workunit.client.1.smithi019.stdout:3/361: fdatasync d3/d1b/d40/d4a/f41 0 2024-03-16T09:01:13.762 INFO:tasks.workunit.client.1.smithi019.stdout:3/362: creat d3/d1b/f6e x:0 0 0 2024-03-16T09:01:13.762 INFO:tasks.workunit.client.0.smithi007.stdout:9/559: write d24/f31 [1543670,114793] 0 2024-03-16T09:01:13.762 INFO:tasks.workunit.client.0.smithi007.stdout:9/560: chown d24/f25 1 1 2024-03-16T09:01:13.764 INFO:tasks.workunit.client.0.smithi007.stdout:8/745: dwrite d0/dc/dd/d8b/d9e/fde [0,4194304] 0 2024-03-16T09:01:13.765 INFO:tasks.workunit.client.1.smithi019.stdout:2/264: mkdir d5/d6/d2b/d53 0 2024-03-16T09:01:13.765 INFO:tasks.workunit.client.1.smithi019.stdout:2/265: stat d5/d6/f36 0 2024-03-16T09:01:13.765 INFO:tasks.workunit.client.1.smithi019.stdout:3/363: dread d3/d11/f37 [0,4194304] 0 2024-03-16T09:01:13.765 INFO:tasks.workunit.client.1.smithi019.stdout:3/364: chown d3/d1b/d5f/f38 87039 1 2024-03-16T09:01:13.765 INFO:tasks.workunit.client.1.smithi019.stdout:3/365: fdatasync d3/f30 0 2024-03-16T09:01:13.765 INFO:tasks.workunit.client.1.smithi019.stdout:0/297: getdents d7/d8/d22 0 2024-03-16T09:01:13.766 INFO:tasks.workunit.client.1.smithi019.stdout:8/288: creat d6/d2a/d45/d57/f60 x:0 0 0 2024-03-16T09:01:13.766 INFO:tasks.workunit.client.1.smithi019.stdout:8/289: chown d6/d2a/d45/d19/f2e 117950 1 2024-03-16T09:01:13.766 INFO:tasks.workunit.client.1.smithi019.stdout:8/290: creat d6/d2a/d3c/f61 x:0 0 0 2024-03-16T09:01:13.770 INFO:tasks.workunit.client.0.smithi007.stdout:1/454: dread f5 [0,4194304] 0 2024-03-16T09:01:13.770 INFO:tasks.workunit.client.0.smithi007.stdout:1/455: chown d9/dd/d1b/d2d/f44 793941 1 2024-03-16T09:01:13.770 INFO:tasks.workunit.client.0.smithi007.stdout:9/561: dread d24/f29 [0,4194304] 0 2024-03-16T09:01:13.771 INFO:tasks.workunit.client.1.smithi019.stdout:0/298: truncate d7/d8/d22/f25 2053995 0 2024-03-16T09:01:13.771 INFO:tasks.workunit.client.1.smithi019.stdout:0/299: creat d7/d1d/d28/f5a x:0 0 0 2024-03-16T09:01:13.773 INFO:tasks.workunit.client.1.smithi019.stdout:3/366: getdents d3/d1b/d40 0 2024-03-16T09:01:13.773 INFO:tasks.workunit.client.1.smithi019.stdout:3/367: chown d3/d11/f69 2894787 1 2024-03-16T09:01:13.774 INFO:tasks.workunit.client.0.smithi007.stdout:8/746: symlink d0/dc/dd/d8b/d9e/l10b 0 2024-03-16T09:01:13.776 INFO:tasks.workunit.client.1.smithi019.stdout:3/368: unlink d3/d11/f37 0 2024-03-16T09:01:13.776 INFO:tasks.workunit.client.0.smithi007.stdout:9/562: rename d24/d4a/d55/d56/d7c/d7f/d80/d87/d89/c9a to d24/d4a/d55/d56/d7c/d7f/d80/d87/ca1 0 2024-03-16T09:01:13.777 INFO:tasks.workunit.client.1.smithi019.stdout:3/369: creat d3/d11/f6f x:0 0 0 2024-03-16T09:01:13.779 INFO:tasks.workunit.client.0.smithi007.stdout:1/456: dread d9/dd/d1b/d2d/d30/d45/f5c [0,4194304] 0 2024-03-16T09:01:13.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/747: link d0/dc/d50/d8e/dd2/dea/f104 d0/d26/d2d/d43/d4c/deb/f10c 0 2024-03-16T09:01:13.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/748: read d0/dc/d50/d8e/dd2/dea/d74/fd3 [489555,67575] 0 2024-03-16T09:01:13.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/749: dread - d0/dc/d50/d8e/dd2/dea/d18/f8d zero size 2024-03-16T09:01:13.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/750: stat d0/d26/d2d/l41 0 2024-03-16T09:01:13.780 INFO:tasks.workunit.client.1.smithi019.stdout:2/266: write d5/f13 [3729719,129996] 0 2024-03-16T09:01:13.781 INFO:tasks.workunit.client.1.smithi019.stdout:3/370: creat d3/d17/d6a/f70 x:0 0 0 2024-03-16T09:01:13.781 INFO:tasks.workunit.client.1.smithi019.stdout:3/371: write d3/d1b/d5f/f38 [3008260,97573] 0 2024-03-16T09:01:13.783 INFO:tasks.workunit.client.1.smithi019.stdout:8/291: dread d6/d2a/d45/d19/f2e [4194304,4194304] 0 2024-03-16T09:01:13.783 INFO:tasks.workunit.client.1.smithi019.stdout:8/292: fdatasync d6/de/d24/d30/f4f 0 2024-03-16T09:01:13.783 INFO:tasks.workunit.client.0.smithi007.stdout:8/751: creat d0/dc/d50/d8e/dd2/dea/d10a/f10d x:0 0 0 2024-03-16T09:01:13.785 INFO:tasks.workunit.client.0.smithi007.stdout:8/752: dread d0/f7b [0,4194304] 0 2024-03-16T09:01:13.786 INFO:tasks.workunit.client.1.smithi019.stdout:3/372: mknod d3/d1b/d40/c71 0 2024-03-16T09:01:13.787 INFO:tasks.workunit.client.1.smithi019.stdout:2/267: symlink d5/d6/d2b/d2f/l54 0 2024-03-16T09:01:13.788 INFO:tasks.workunit.client.1.smithi019.stdout:3/373: write d3/d11/f3b [579389,55923] 0 2024-03-16T09:01:13.788 INFO:tasks.workunit.client.1.smithi019.stdout:3/374: truncate d3/d1b/d40/d4a/f41 2494791 0 2024-03-16T09:01:13.789 INFO:tasks.workunit.client.0.smithi007.stdout:8/753: rename d0/dc/dd/d8b/dc9/dce to d0/d26/d2d/d64/da6/d10e 0 2024-03-16T09:01:13.789 INFO:tasks.workunit.client.0.smithi007.stdout:8/754: write d0/d26/d2d/d43/d6f/f8f [1257162,118613] 0 2024-03-16T09:01:13.789 INFO:tasks.workunit.client.0.smithi007.stdout:8/755: chown d0/dc/d50/d8e/dd2/dea/d18/c1a 1 1 2024-03-16T09:01:13.790 INFO:tasks.workunit.client.1.smithi019.stdout:3/375: getdents d3/d17/d3f 0 2024-03-16T09:01:13.791 INFO:tasks.workunit.client.0.smithi007.stdout:8/756: mkdir d0/d26/d2d/d43/d70/d10f 0 2024-03-16T09:01:13.793 INFO:tasks.workunit.client.1.smithi019.stdout:3/376: link d3/d11/f1f d3/d17/f72 0 2024-03-16T09:01:13.794 INFO:tasks.workunit.client.0.smithi007.stdout:8/757: getdents d0/dc/d50/d8e/dd2/dea/d18 0 2024-03-16T09:01:13.794 INFO:tasks.workunit.client.1.smithi019.stdout:2/268: dread d5/de/fd [0,4194304] 0 2024-03-16T09:01:13.795 INFO:tasks.workunit.client.1.smithi019.stdout:2/269: write d5/d6/d2b/d2f/f46 [68051,112446] 0 2024-03-16T09:01:13.796 INFO:tasks.workunit.client.1.smithi019.stdout:2/270: truncate d5/fb 5071249 0 2024-03-16T09:01:13.796 INFO:tasks.workunit.client.1.smithi019.stdout:2/271: chown d5/d2e/d4f 0 1 2024-03-16T09:01:13.798 INFO:tasks.workunit.client.1.smithi019.stdout:2/272: rename d5/f40 to d5/de/f55 0 2024-03-16T09:01:13.798 INFO:tasks.workunit.client.1.smithi019.stdout:1/221: dwrite f12 [0,4194304] 0 2024-03-16T09:01:13.799 INFO:tasks.workunit.client.1.smithi019.stdout:2/273: fdatasync d5/d6/d2b/d2f/f3a 0 2024-03-16T09:01:13.799 INFO:tasks.workunit.client.1.smithi019.stdout:2/274: creat d5/d6/d2b/d2f/d35/f56 x:0 0 0 2024-03-16T09:01:13.802 INFO:tasks.workunit.client.0.smithi007.stdout:8/758: dread d0/f3c [0,4194304] 0 2024-03-16T09:01:13.802 INFO:tasks.workunit.client.0.smithi007.stdout:8/759: dread - d0/dc/d50/d8e/dd2/dea/d18/da7/fed zero size 2024-03-16T09:01:13.802 INFO:tasks.workunit.client.1.smithi019.stdout:1/222: link d16/d1f/c29 d16/d18/c40 0 2024-03-16T09:01:13.802 INFO:tasks.workunit.client.1.smithi019.stdout:1/223: write d16/d18/f1e [759464,126951] 0 2024-03-16T09:01:13.803 INFO:tasks.workunit.client.0.smithi007.stdout:7/424: read d0/d6c/d25/d3a/f77 [602890,97152] 0 2024-03-16T09:01:13.803 INFO:tasks.workunit.client.0.smithi007.stdout:8/760: creat d0/d26/d2d/d43/d4c/f110 x:0 0 0 2024-03-16T09:01:13.804 INFO:tasks.workunit.client.1.smithi019.stdout:2/275: creat d5/d2e/d4f/f57 x:0 0 0 2024-03-16T09:01:13.805 INFO:tasks.workunit.client.1.smithi019.stdout:1/224: creat d16/d1f/f41 x:0 0 0 2024-03-16T09:01:13.806 INFO:tasks.workunit.client.1.smithi019.stdout:3/377: dread d3/d1b/d40/f67 [0,4194304] 0 2024-03-16T09:01:13.806 INFO:tasks.workunit.client.1.smithi019.stdout:4/306: dwrite d4/f26 [0,4194304] 0 2024-03-16T09:01:13.807 INFO:tasks.workunit.client.0.smithi007.stdout:0/557: sync 2024-03-16T09:01:13.807 INFO:tasks.workunit.client.0.smithi007.stdout:2/587: sync 2024-03-16T09:01:13.807 INFO:tasks.workunit.client.0.smithi007.stdout:0/558: creat d3/d1f/fb7 x:0 0 0 2024-03-16T09:01:13.808 INFO:tasks.workunit.client.0.smithi007.stdout:2/588: creat dc/df/d24/d28/d4c/fbe x:0 0 0 2024-03-16T09:01:13.810 INFO:tasks.workunit.client.0.smithi007.stdout:0/559: creat d3/d9/db4/d2c/fb8 x:0 0 0 2024-03-16T09:01:13.810 INFO:tasks.workunit.client.0.smithi007.stdout:2/589: symlink dc/dba/lbf 0 2024-03-16T09:01:13.812 INFO:tasks.workunit.client.0.smithi007.stdout:0/560: unlink d3/d9/da/d18/d23/f75 0 2024-03-16T09:01:13.812 INFO:tasks.workunit.client.0.smithi007.stdout:0/561: readlink d3/d9/da/d18/d23/lb5 0 2024-03-16T09:01:13.812 INFO:tasks.workunit.client.1.smithi019.stdout:4/307: rename d4/d6/d15/d1f/d2c/f5b to d4/d64/f68 0 2024-03-16T09:01:13.813 INFO:tasks.workunit.client.0.smithi007.stdout:2/590: mkdir dc/db7/dc0 0 2024-03-16T09:01:13.814 INFO:tasks.workunit.client.0.smithi007.stdout:0/562: mkdir d3/d9/da/d18/d27/d34/db9 0 2024-03-16T09:01:13.815 INFO:tasks.workunit.client.0.smithi007.stdout:2/591: getdents dc/df/d24/d28/d97/d99 0 2024-03-16T09:01:13.815 INFO:tasks.workunit.client.0.smithi007.stdout:2/592: dread - dc/d27/f8f zero size 2024-03-16T09:01:13.815 INFO:tasks.workunit.client.0.smithi007.stdout:2/593: chown dc/df/d24/d28/d4c/dbc 315698 1 2024-03-16T09:01:13.815 INFO:tasks.workunit.client.0.smithi007.stdout:2/594: creat dc/df/d24/d28/d2e/d4a/fc1 x:0 0 0 2024-03-16T09:01:13.815 INFO:tasks.workunit.client.1.smithi019.stdout:1/225: dread f9 [0,4194304] 0 2024-03-16T09:01:13.816 INFO:tasks.workunit.client.1.smithi019.stdout:1/226: creat d16/f42 x:0 0 0 2024-03-16T09:01:13.816 INFO:tasks.workunit.client.1.smithi019.stdout:4/308: symlink d4/l69 0 2024-03-16T09:01:13.817 INFO:tasks.workunit.client.0.smithi007.stdout:0/563: creat d3/d9/da/d48/d25/d43/d91/fba x:0 0 0 2024-03-16T09:01:13.819 INFO:tasks.workunit.client.0.smithi007.stdout:8/761: write d0/f16 [1813170,2252] 0 2024-03-16T09:01:13.819 INFO:tasks.workunit.client.1.smithi019.stdout:4/309: dread d4/d6/f39 [0,4194304] 0 2024-03-16T09:01:13.820 INFO:tasks.workunit.client.1.smithi019.stdout:4/310: readlink d4/l11 0 2024-03-16T09:01:13.820 INFO:tasks.workunit.client.1.smithi019.stdout:0/300: dwrite d7/d1d/d28/f44 [0,4194304] 0 2024-03-16T09:01:13.820 INFO:tasks.workunit.client.1.smithi019.stdout:4/311: read - d4/d6/d15/f67 zero size 2024-03-16T09:01:13.820 INFO:tasks.workunit.client.1.smithi019.stdout:0/301: rename d7/d1d to d7/d1d/d28/d48/d5b 22 2024-03-16T09:01:13.820 INFO:tasks.workunit.client.0.smithi007.stdout:8/762: mknod d0/d26/d2d/c111 0 2024-03-16T09:01:13.821 INFO:tasks.workunit.client.1.smithi019.stdout:9/212: dwrite d1/d7/f4c [0,4194304] 0 2024-03-16T09:01:13.821 INFO:tasks.workunit.client.1.smithi019.stdout:9/213: fdatasync d1/d7/de/f2a 0 2024-03-16T09:01:13.821 INFO:tasks.workunit.client.1.smithi019.stdout:2/276: dread d5/d2e/f50 [0,4194304] 0 2024-03-16T09:01:13.824 INFO:tasks.workunit.client.1.smithi019.stdout:4/312: mknod d4/d1b/d49/c6a 0 2024-03-16T09:01:13.825 INFO:tasks.workunit.client.1.smithi019.stdout:4/313: write d4/d6/d15/d1f/d2c/f3b [294550,107492] 0 2024-03-16T09:01:13.825 INFO:tasks.workunit.client.1.smithi019.stdout:4/314: chown d4/d6/d15 1745 1 2024-03-16T09:01:13.825 INFO:tasks.workunit.client.1.smithi019.stdout:4/315: creat d4/d6/d15/d1f/d2c/f6b x:0 0 0 2024-03-16T09:01:13.826 INFO:tasks.workunit.client.0.smithi007.stdout:0/564: dread d3/d1f/f7f [0,4194304] 0 2024-03-16T09:01:13.826 INFO:tasks.workunit.client.0.smithi007.stdout:8/763: dread d0/d26/d2d/d43/ff6 [0,4194304] 0 2024-03-16T09:01:13.827 INFO:tasks.workunit.client.0.smithi007.stdout:9/563: dwrite d24/f2d [0,4194304] 0 2024-03-16T09:01:13.828 INFO:tasks.workunit.client.1.smithi019.stdout:2/277: rename f4 to d5/d6/d2b/d2f/d35/f58 0 2024-03-16T09:01:13.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/565: rmdir d3/d9/da/d48/d25/d43 39 2024-03-16T09:01:13.831 INFO:tasks.workunit.client.0.smithi007.stdout:1/457: dwrite d9/dd/d1b/d2d/d30/d45/f5b [0,4194304] 0 2024-03-16T09:01:13.831 INFO:tasks.workunit.client.0.smithi007.stdout:1/458: fdatasync d9/dd/d1b/d2d/d30/d45/f5c 0 2024-03-16T09:01:13.831 INFO:tasks.workunit.client.0.smithi007.stdout:1/459: truncate d9/dd/d1b/d28/d29/f47 4731020 0 2024-03-16T09:01:13.832 INFO:tasks.workunit.client.1.smithi019.stdout:2/278: mknod d5/d6/d2b/d2f/c59 0 2024-03-16T09:01:13.833 INFO:tasks.workunit.client.0.smithi007.stdout:8/764: mknod d0/dc/d50/d8e/dd2/dea/d18/c112 0 2024-03-16T09:01:13.834 INFO:tasks.workunit.client.1.smithi019.stdout:8/293: dwrite f5 [0,4194304] 0 2024-03-16T09:01:13.835 INFO:tasks.workunit.client.0.smithi007.stdout:0/566: rmdir d3/d9/da/d18/d27/d34 39 2024-03-16T09:01:13.835 INFO:tasks.workunit.client.1.smithi019.stdout:2/279: mkdir d5/d5a 0 2024-03-16T09:01:13.836 INFO:tasks.workunit.client.1.smithi019.stdout:2/280: dread - d5/d2e/f3b zero size 2024-03-16T09:01:13.836 INFO:tasks.workunit.client.1.smithi019.stdout:2/281: fdatasync d5/fb 0 2024-03-16T09:01:13.837 INFO:tasks.workunit.client.1.smithi019.stdout:2/282: dread d5/d6/d2b/d2f/f46 [0,4194304] 0 2024-03-16T09:01:13.838 INFO:tasks.workunit.client.1.smithi019.stdout:8/294: rename d6/d2a/d45/d5d/f5b to d6/d2a/d45/d57/f62 0 2024-03-16T09:01:13.838 INFO:tasks.workunit.client.1.smithi019.stdout:8/295: fsync d6/d2a/d45/d57/f60 0 2024-03-16T09:01:13.838 INFO:tasks.workunit.client.1.smithi019.stdout:8/296: write d6/d2a/d3c/f61 [809412,54751] 0 2024-03-16T09:01:13.839 INFO:tasks.workunit.client.0.smithi007.stdout:7/425: dwrite d0/d19/f48 [0,4194304] 0 2024-03-16T09:01:13.840 INFO:tasks.workunit.client.0.smithi007.stdout:7/426: creat d0/d6c/d25/f90 x:0 0 0 2024-03-16T09:01:13.840 INFO:tasks.workunit.client.1.smithi019.stdout:4/316: dread d4/d6/d15/d1f/d2c/f40 [0,4194304] 0 2024-03-16T09:01:13.840 INFO:tasks.workunit.client.1.smithi019.stdout:2/283: dread d5/de/f32 [0,4194304] 0 2024-03-16T09:01:13.840 INFO:tasks.workunit.client.1.smithi019.stdout:2/284: write d5/d2e/f3d [873118,128412] 0 2024-03-16T09:01:13.843 INFO:tasks.workunit.client.0.smithi007.stdout:1/460: symlink d9/d6b/l8d 0 2024-03-16T09:01:13.843 INFO:tasks.workunit.client.0.smithi007.stdout:1/461: chown d9/l20 15 1 2024-03-16T09:01:13.843 INFO:tasks.workunit.client.0.smithi007.stdout:1/462: creat d9/dd/d1b/d2d/d30/f8e x:0 0 0 2024-03-16T09:01:13.844 INFO:tasks.workunit.client.0.smithi007.stdout:1/463: write d9/dd/d1b/d2d/d30/f61 [559651,15658] 0 2024-03-16T09:01:13.845 INFO:tasks.workunit.client.0.smithi007.stdout:8/765: mkdir d0/dc/d50/d8e/dd2/dea/d18/da7/de2/d113 0 2024-03-16T09:01:13.849 INFO:tasks.workunit.client.0.smithi007.stdout:7/427: rmdir d0/d19 39 2024-03-16T09:01:13.852 INFO:tasks.workunit.client.1.smithi019.stdout:4/317: mkdir d4/d6c 0 2024-03-16T09:01:13.853 INFO:tasks.workunit.client.0.smithi007.stdout:0/567: read d3/d9/da/d18/d27/d34/f57 [3549799,59583] 0 2024-03-16T09:01:13.853 INFO:tasks.workunit.client.0.smithi007.stdout:1/464: write d9/dd/f59 [1532131,8121] 0 2024-03-16T09:01:13.854 INFO:tasks.workunit.client.0.smithi007.stdout:0/568: mkdir d3/d9/da/d48/d25/d87/dbb 0 2024-03-16T09:01:13.854 INFO:tasks.workunit.client.0.smithi007.stdout:0/569: creat d3/d9/da/d18/d27/fbc x:0 0 0 2024-03-16T09:01:13.857 INFO:tasks.workunit.client.0.smithi007.stdout:1/465: getdents d9/dd/d1b/d28/d29/d7b 0 2024-03-16T09:01:13.857 INFO:tasks.workunit.client.0.smithi007.stdout:1/466: chown d9/dd/d1b/f49 35985958 1 2024-03-16T09:01:13.858 INFO:tasks.workunit.client.1.smithi019.stdout:2/285: unlink d5/d6/d2b/l3f 0 2024-03-16T09:01:13.859 INFO:tasks.workunit.client.1.smithi019.stdout:4/318: creat d4/d6c/f6d x:0 0 0 2024-03-16T09:01:13.860 INFO:tasks.workunit.client.0.smithi007.stdout:0/570: mkdir d3/d6c/dbd 0 2024-03-16T09:01:13.860 INFO:tasks.workunit.client.0.smithi007.stdout:0/571: creat d3/d9/da/d48/fbe x:0 0 0 2024-03-16T09:01:13.860 INFO:tasks.workunit.client.0.smithi007.stdout:8/766: write d0/dc/d50/d8e/dd2/dea/d74/f96 [1296972,14642] 0 2024-03-16T09:01:13.860 INFO:tasks.workunit.client.0.smithi007.stdout:8/767: read - d0/dc/d8a/f99 zero size 2024-03-16T09:01:13.861 INFO:tasks.workunit.client.0.smithi007.stdout:1/467: symlink d9/d6b/l8f 0 2024-03-16T09:01:13.861 INFO:tasks.workunit.client.1.smithi019.stdout:2/286: rename d5/fb to d5/d6/d2b/f5b 0 2024-03-16T09:01:13.861 INFO:tasks.workunit.client.1.smithi019.stdout:9/214: dwrite d1/d7/d21/d27/f35 [0,4194304] 0 2024-03-16T09:01:13.861 INFO:tasks.workunit.client.1.smithi019.stdout:9/215: dread - d1/d7/f41 zero size 2024-03-16T09:01:13.861 INFO:tasks.workunit.client.1.smithi019.stdout:9/216: creat d1/d7/d21/d27/f4e x:0 0 0 2024-03-16T09:01:13.862 INFO:tasks.workunit.client.1.smithi019.stdout:1/227: dwrite d16/d18/f39 [0,4194304] 0 2024-03-16T09:01:13.862 INFO:tasks.workunit.client.0.smithi007.stdout:2/595: dwrite dc/d27/f65 [0,4194304] 0 2024-03-16T09:01:13.863 INFO:tasks.workunit.client.0.smithi007.stdout:0/572: mknod d3/d9/da/d18/cbf 0 2024-03-16T09:01:13.863 INFO:tasks.workunit.client.1.smithi019.stdout:9/217: link d1/d7/de/d2d/l49 d1/d7/de/l4f 0 2024-03-16T09:01:13.863 INFO:tasks.workunit.client.1.smithi019.stdout:4/319: mkdir d4/d1b/d49/d6e 0 2024-03-16T09:01:13.863 INFO:tasks.workunit.client.1.smithi019.stdout:8/297: dwrite d6/de/d24/d30/f4f [0,4194304] 0 2024-03-16T09:01:13.866 INFO:tasks.workunit.client.0.smithi007.stdout:7/428: dwrite d0/f3c [0,4194304] 0 2024-03-16T09:01:13.866 INFO:tasks.workunit.client.0.smithi007.stdout:2/596: rename dc/c16 to dc/df/d24/d28/d2e/cc2 0 2024-03-16T09:01:13.872 INFO:tasks.workunit.client.0.smithi007.stdout:0/573: write d3/d9/da/d48/f95 [941646,20473] 0 2024-03-16T09:01:13.873 INFO:tasks.workunit.client.1.smithi019.stdout:9/218: unlink d1/l28 0 2024-03-16T09:01:13.873 INFO:tasks.workunit.client.1.smithi019.stdout:9/219: creat d1/d7/f50 x:0 0 0 2024-03-16T09:01:13.873 INFO:tasks.workunit.client.1.smithi019.stdout:9/220: getdents d1/d7/d33 0 2024-03-16T09:01:13.873 INFO:tasks.workunit.client.1.smithi019.stdout:9/221: fsync d1/d7/f3e 0 2024-03-16T09:01:13.874 INFO:tasks.workunit.client.1.smithi019.stdout:8/298: mknod d6/d2a/d45/d5d/c63 0 2024-03-16T09:01:13.878 INFO:tasks.workunit.client.0.smithi007.stdout:7/429: getdents d0/d6c 0 2024-03-16T09:01:13.879 INFO:tasks.workunit.client.1.smithi019.stdout:8/299: symlink d6/d2a/d45/d57/l64 0 2024-03-16T09:01:13.879 INFO:tasks.workunit.client.1.smithi019.stdout:8/300: write d6/d2a/d3c/f44 [3141368,83681] 0 2024-03-16T09:01:13.879 INFO:tasks.workunit.client.1.smithi019.stdout:8/301: creat d6/d2a/d45/d57/f65 x:0 0 0 2024-03-16T09:01:13.882 INFO:tasks.workunit.client.0.smithi007.stdout:8/768: dwrite d0/dc/d50/d8e/dd2/dea/d18/f4e [4194304,4194304] 0 2024-03-16T09:01:13.883 INFO:tasks.workunit.client.0.smithi007.stdout:2/597: mknod dc/df/d24/d28/d97/da4/db8/cc3 0 2024-03-16T09:01:13.883 INFO:tasks.workunit.client.0.smithi007.stdout:8/769: dread d0/dc/d8a/f91 [0,4194304] 0 2024-03-16T09:01:13.885 INFO:tasks.workunit.client.0.smithi007.stdout:0/574: mknod d3/d9/da/d48/cc0 0 2024-03-16T09:01:13.885 INFO:tasks.workunit.client.0.smithi007.stdout:0/575: stat d3/d9/da/d48/d4b/c54 0 2024-03-16T09:01:13.886 INFO:tasks.workunit.client.0.smithi007.stdout:7/430: creat d0/d6c/d25/d73/f91 x:0 0 0 2024-03-16T09:01:13.889 INFO:tasks.workunit.client.0.smithi007.stdout:2/598: rename dc/df/d24/d28/f39 to dc/df/d24/d28/d97/da4/fc4 0 2024-03-16T09:01:13.890 INFO:tasks.workunit.client.0.smithi007.stdout:0/576: unlink d3/d9/da/d48/f95 0 2024-03-16T09:01:13.890 INFO:tasks.workunit.client.0.smithi007.stdout:0/577: creat d3/d9/db4/d2c/fc1 x:0 0 0 2024-03-16T09:01:13.891 INFO:tasks.workunit.client.1.smithi019.stdout:6/389: sync 2024-03-16T09:01:13.892 INFO:tasks.workunit.client.0.smithi007.stdout:7/431: mkdir d0/d6c/d25/d3a/d92 0 2024-03-16T09:01:13.913 INFO:tasks.workunit.client.1.smithi019.stdout:2/287: dwrite d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:13.913 INFO:tasks.workunit.client.0.smithi007.stdout:1/468: dwrite d9/dd/d1b/d2d/d30/d45/f5b [0,4194304] 0 2024-03-16T09:01:13.913 INFO:tasks.workunit.client.0.smithi007.stdout:1/469: chown d9/f8b 2431 1 2024-03-16T09:01:13.914 INFO:tasks.workunit.client.1.smithi019.stdout:6/390: dwrite de/d1c/d1d/d66/d72/f76 [0,4194304] 0 2024-03-16T09:01:13.914 INFO:tasks.workunit.client.1.smithi019.stdout:8/302: dwrite d6/de/f49 [0,4194304] 0 2024-03-16T09:01:13.914 INFO:tasks.workunit.client.1.smithi019.stdout:8/303: chown d6/de/f49 14948 1 2024-03-16T09:01:13.914 INFO:tasks.workunit.client.1.smithi019.stdout:8/304: read d6/d2a/d45/d19/f31 [290086,19023] 0 2024-03-16T09:01:13.915 INFO:tasks.workunit.client.1.smithi019.stdout:6/391: dread de/d26/f39 [0,4194304] 0 2024-03-16T09:01:13.918 INFO:tasks.workunit.client.1.smithi019.stdout:8/305: link d6/d2a/d45/d2c/f58 d6/d2a/d45/d2c/f66 0 2024-03-16T09:01:13.919 INFO:tasks.workunit.client.1.smithi019.stdout:8/306: mkdir d6/de/d24/d30/d4a/d67 0 2024-03-16T09:01:13.919 INFO:tasks.workunit.client.1.smithi019.stdout:8/307: truncate d6/de/ff 1318434 0 2024-03-16T09:01:13.921 INFO:tasks.workunit.client.1.smithi019.stdout:8/308: mkdir d6/d2a/d68 0 2024-03-16T09:01:13.921 INFO:tasks.workunit.client.1.smithi019.stdout:8/309: fsync d6/d2a/d45/d2c/f5c 0 2024-03-16T09:01:13.921 INFO:tasks.workunit.client.1.smithi019.stdout:8/310: fsync d6/de/f49 0 2024-03-16T09:01:13.922 INFO:tasks.workunit.client.0.smithi007.stdout:4/406: sync 2024-03-16T09:01:13.922 INFO:tasks.workunit.client.1.smithi019.stdout:2/288: write d5/d6/d2b/f1e [1912383,102989] 0 2024-03-16T09:01:13.923 INFO:tasks.workunit.client.1.smithi019.stdout:2/289: dread d5/d6/d2b/d2f/f46 [0,4194304] 0 2024-03-16T09:01:13.923 INFO:tasks.workunit.client.1.smithi019.stdout:2/290: fsync d5/f4d 0 2024-03-16T09:01:13.924 INFO:tasks.workunit.client.0.smithi007.stdout:8/770: rename d0/d26/d2d/d43/d4c/dba/cc0 to d0/d26/d2d/d43/d4c/c114 0 2024-03-16T09:01:13.924 INFO:tasks.workunit.client.0.smithi007.stdout:8/771: dread - d0/dc/d50/d8e/dd2/dea/f104 zero size 2024-03-16T09:01:13.924 INFO:tasks.workunit.client.1.smithi019.stdout:8/311: getdents d6/d2a/d3c 0 2024-03-16T09:01:13.925 INFO:tasks.workunit.client.1.smithi019.stdout:8/312: readlink d6/de/d24/d30/d4a/l53 0 2024-03-16T09:01:13.925 INFO:tasks.workunit.client.0.smithi007.stdout:2/599: mkdir dc/df/d24/d28/d2e/dc5 0 2024-03-16T09:01:13.925 INFO:tasks.workunit.client.0.smithi007.stdout:2/600: read - dc/df/d24/d28/d4c/f73 zero size 2024-03-16T09:01:13.926 INFO:tasks.workunit.client.1.smithi019.stdout:8/313: rmdir d6/de/d24 39 2024-03-16T09:01:13.926 INFO:tasks.workunit.client.1.smithi019.stdout:8/314: chown d6/d2a/d45/d19/f43 27762758 1 2024-03-16T09:01:13.927 INFO:tasks.workunit.client.1.smithi019.stdout:8/315: mknod d6/de/d51/c69 0 2024-03-16T09:01:13.927 INFO:tasks.workunit.client.1.smithi019.stdout:8/316: creat d6/de/f6a x:0 0 0 2024-03-16T09:01:13.933 INFO:tasks.workunit.client.1.smithi019.stdout:2/291: dread d5/de/f45 [0,4194304] 0 2024-03-16T09:01:13.933 INFO:tasks.workunit.client.1.smithi019.stdout:2/292: chown d5/d6/d2b/d2f 67163 1 2024-03-16T09:01:13.939 INFO:tasks.workunit.client.1.smithi019.stdout:8/317: dread f2 [0,4194304] 0 2024-03-16T09:01:13.939 INFO:tasks.workunit.client.1.smithi019.stdout:8/318: creat d6/d2a/d45/f6b x:0 0 0 2024-03-16T09:01:13.939 INFO:tasks.workunit.client.1.smithi019.stdout:8/319: truncate d6/d2a/d45/d2c/f5c 965120 0 2024-03-16T09:01:13.939 INFO:tasks.workunit.client.1.smithi019.stdout:8/320: write d6/d2a/d3c/f61 [1894865,10546] 0 2024-03-16T09:01:13.939 INFO:tasks.workunit.client.1.smithi019.stdout:8/321: write d6/d2a/d45/d57/f60 [466942,29958] 0 2024-03-16T09:01:13.939 INFO:tasks.workunit.client.1.smithi019.stdout:8/322: dread - d6/de/d24/d30/d4a/f4e zero size 2024-03-16T09:01:13.941 INFO:tasks.workunit.client.1.smithi019.stdout:8/323: rmdir d6/d2a/d45/d5d 39 2024-03-16T09:01:13.941 INFO:tasks.workunit.client.1.smithi019.stdout:8/324: creat d6/de/f6c x:0 0 0 2024-03-16T09:01:13.943 INFO:tasks.workunit.client.1.smithi019.stdout:8/325: creat d6/d2a/d45/d5d/f6d x:0 0 0 2024-03-16T09:01:13.943 INFO:tasks.workunit.client.1.smithi019.stdout:8/326: chown f2 38641 1 2024-03-16T09:01:13.945 INFO:tasks.workunit.client.1.smithi019.stdout:2/293: write d5/d6/f36 [1947491,4806] 0 2024-03-16T09:01:13.945 INFO:tasks.workunit.client.1.smithi019.stdout:8/327: creat d6/d2a/d55/f6e x:0 0 0 2024-03-16T09:01:13.947 INFO:tasks.workunit.client.1.smithi019.stdout:2/294: creat d5/d6/d2b/d2f/d35/d4b/f5c x:0 0 0 2024-03-16T09:01:13.948 INFO:tasks.workunit.client.1.smithi019.stdout:6/392: dwrite de/f45 [0,4194304] 0 2024-03-16T09:01:13.948 INFO:tasks.workunit.client.1.smithi019.stdout:6/393: readlink de/d26/l5f 0 2024-03-16T09:01:13.948 INFO:tasks.workunit.client.1.smithi019.stdout:8/328: symlink d6/d2a/d68/l6f 0 2024-03-16T09:01:13.948 INFO:tasks.workunit.client.1.smithi019.stdout:6/394: creat de/d1c/d1d/d66/d72/f82 x:0 0 0 2024-03-16T09:01:13.949 INFO:tasks.workunit.client.0.smithi007.stdout:8/772: dwrite d0/dc/d50/d8e/f109 [0,4194304] 0 2024-03-16T09:01:13.950 INFO:tasks.workunit.client.1.smithi019.stdout:2/295: creat d5/de/f5d x:0 0 0 2024-03-16T09:01:13.950 INFO:tasks.workunit.client.1.smithi019.stdout:2/296: write d5/de/f1f [926863,41383] 0 2024-03-16T09:01:13.950 INFO:tasks.workunit.client.1.smithi019.stdout:2/297: creat d5/d2e/d4f/f5e x:0 0 0 2024-03-16T09:01:13.959 INFO:tasks.workunit.client.1.smithi019.stdout:6/395: dread de/d1c/d1d/f30 [0,4194304] 0 2024-03-16T09:01:13.959 INFO:tasks.workunit.client.1.smithi019.stdout:6/396: fdatasync de/d26/f39 0 2024-03-16T09:01:13.960 INFO:tasks.workunit.client.1.smithi019.stdout:6/397: read de/d1c/f28 [2464843,51643] 0 2024-03-16T09:01:13.961 INFO:tasks.workunit.client.1.smithi019.stdout:6/398: symlink de/d1c/d1d/d2c/l83 0 2024-03-16T09:01:13.961 INFO:tasks.workunit.client.1.smithi019.stdout:6/399: read - de/d26/d73/f7e zero size 2024-03-16T09:01:13.962 INFO:tasks.workunit.client.1.smithi019.stdout:6/400: symlink de/d26/l84 0 2024-03-16T09:01:13.963 INFO:tasks.workunit.client.1.smithi019.stdout:6/401: link de/d26/d32/f37 de/d26/f85 0 2024-03-16T09:01:13.965 INFO:tasks.workunit.client.1.smithi019.stdout:6/402: rmdir de/d1c/d1d/d51 39 2024-03-16T09:01:13.965 INFO:tasks.workunit.client.1.smithi019.stdout:6/403: chown de/d26/f5c 129892 1 2024-03-16T09:01:13.968 INFO:tasks.workunit.client.1.smithi019.stdout:8/329: dwrite d6/d2a/d55/f6e [0,4194304] 0 2024-03-16T09:01:13.970 INFO:tasks.workunit.client.1.smithi019.stdout:8/330: stat d6/de/d51/c59 0 2024-03-16T09:01:13.972 INFO:tasks.workunit.client.1.smithi019.stdout:8/331: rmdir d6/de/d24/d30 39 2024-03-16T09:01:13.974 INFO:tasks.workunit.client.1.smithi019.stdout:2/298: dwrite d5/d2e/d4f/f5e [0,4194304] 0 2024-03-16T09:01:13.974 INFO:tasks.workunit.client.1.smithi019.stdout:6/404: write de/f1a [1425229,97789] 0 2024-03-16T09:01:13.974 INFO:tasks.workunit.client.1.smithi019.stdout:2/299: readlink d5/d6/d2b/d2f/l54 0 2024-03-16T09:01:13.975 INFO:tasks.workunit.client.1.smithi019.stdout:8/332: creat d6/de/d24/d30/d4a/d67/f70 x:0 0 0 2024-03-16T09:01:13.975 INFO:tasks.workunit.client.1.smithi019.stdout:8/333: fsync d6/d2a/d45/d19/f2e 0 2024-03-16T09:01:13.976 INFO:tasks.workunit.client.1.smithi019.stdout:6/405: dread de/d1c/f56 [0,4194304] 0 2024-03-16T09:01:13.976 INFO:tasks.workunit.client.1.smithi019.stdout:6/406: readlink de/d1c/d1d/d2c/l83 0 2024-03-16T09:01:13.976 INFO:tasks.workunit.client.1.smithi019.stdout:6/407: chown f7 28375 1 2024-03-16T09:01:13.976 INFO:tasks.workunit.client.1.smithi019.stdout:6/408: dread - de/d1c/d1d/d66/d72/f82 zero size 2024-03-16T09:01:13.977 INFO:tasks.workunit.client.1.smithi019.stdout:6/409: creat de/f86 x:0 0 0 2024-03-16T09:01:13.977 INFO:tasks.workunit.client.1.smithi019.stdout:7/211: sync 2024-03-16T09:01:13.978 INFO:tasks.workunit.client.1.smithi019.stdout:2/300: mkdir d5/d6/d2b/d2f/d35/d4b/d5f 0 2024-03-16T09:01:13.978 INFO:tasks.workunit.client.1.smithi019.stdout:7/212: creat d9/db/f3d x:0 0 0 2024-03-16T09:01:13.980 INFO:tasks.workunit.client.1.smithi019.stdout:8/334: getdents d6/de 0 2024-03-16T09:01:13.981 INFO:tasks.workunit.client.0.smithi007.stdout:1/470: mkdir d9/dd/d1b/d28/d29/d7b/d90 0 2024-03-16T09:01:13.982 INFO:tasks.workunit.client.1.smithi019.stdout:2/301: mknod d5/d6/d2b/d2f/d35/d4b/d51/c60 0 2024-03-16T09:01:13.982 INFO:tasks.workunit.client.1.smithi019.stdout:2/302: write d5/de/f1f [1961335,49652] 0 2024-03-16T09:01:13.983 INFO:tasks.workunit.client.1.smithi019.stdout:7/213: truncate f7 2967910 0 2024-03-16T09:01:13.984 INFO:tasks.workunit.client.0.smithi007.stdout:4/407: mkdir d4/d7/d78/d80/d84/d89 0 2024-03-16T09:01:13.985 INFO:tasks.workunit.client.1.smithi019.stdout:2/303: truncate d5/d6/d2b/f18 1952528 0 2024-03-16T09:01:13.985 INFO:tasks.workunit.client.1.smithi019.stdout:2/304: fsync d5/d6/d2b/d2f/d35/f56 0 2024-03-16T09:01:13.986 INFO:tasks.workunit.client.0.smithi007.stdout:2/601: dread - dc/df/daa/fb3 zero size 2024-03-16T09:01:13.987 INFO:tasks.workunit.client.0.smithi007.stdout:0/578: rmdir d3/d9/da/d48/d25/d87 39 2024-03-16T09:01:13.990 INFO:tasks.workunit.client.0.smithi007.stdout:8/773: mknod d0/d26/d2d/d64/da6/c115 0 2024-03-16T09:01:13.990 INFO:tasks.workunit.client.0.smithi007.stdout:8/774: creat d0/dc/d75/f116 x:0 0 0 2024-03-16T09:01:13.991 INFO:tasks.workunit.client.0.smithi007.stdout:1/471: mkdir d9/d6b/d7e/d91 0 2024-03-16T09:01:13.992 INFO:tasks.workunit.client.1.smithi019.stdout:6/410: dwrite de/d26/d32/d4e/f79 [0,4194304] 0 2024-03-16T09:01:13.992 INFO:tasks.workunit.client.1.smithi019.stdout:6/411: read de/d26/d32/d4e/f59 [158301,20614] 0 2024-03-16T09:01:13.993 INFO:tasks.workunit.client.0.smithi007.stdout:4/408: rmdir d4 39 2024-03-16T09:01:13.993 INFO:tasks.workunit.client.0.smithi007.stdout:4/409: creat d4/d7/f8a x:0 0 0 2024-03-16T09:01:13.993 INFO:tasks.workunit.client.0.smithi007.stdout:4/410: readlink d4/l2a 0 2024-03-16T09:01:13.994 INFO:tasks.workunit.client.1.smithi019.stdout:2/305: dread d5/d6/f36 [0,4194304] 0 2024-03-16T09:01:13.994 INFO:tasks.workunit.client.1.smithi019.stdout:6/412: link de/d1c/f58 de/d1c/d1d/d66/d72/f87 0 2024-03-16T09:01:13.995 INFO:tasks.workunit.client.1.smithi019.stdout:6/413: chown de/d26/l80 3256 1 2024-03-16T09:01:13.995 INFO:tasks.workunit.client.0.smithi007.stdout:0/579: creat d3/d9/da/d48/d25/d43/fc2 x:0 0 0 2024-03-16T09:01:13.996 INFO:tasks.workunit.client.0.smithi007.stdout:0/580: chown d3/d9/da/d18/c24 3431984 1 2024-03-16T09:01:13.996 INFO:tasks.workunit.client.0.smithi007.stdout:0/581: fsync d3/d9/f93 0 2024-03-16T09:01:13.997 INFO:tasks.workunit.client.1.smithi019.stdout:6/414: getdents de/d1c 0 2024-03-16T09:01:13.997 INFO:tasks.workunit.client.1.smithi019.stdout:6/415: readlink de/d1c/l3e 0 2024-03-16T09:01:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:13 smithi007 ceph-mon[29470]: pgmap v20: 65 pgs: 65 active+clean; 932 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 89 MiB/s rd, 113 MiB/s wr, 373 op/s 2024-03-16T09:01:13.998 INFO:tasks.workunit.client.1.smithi019.stdout:6/416: rename de/d1c/f47 to de/d1c/d1d/d51/d70/f88 0 2024-03-16T09:01:13.999 INFO:tasks.workunit.client.1.smithi019.stdout:6/417: chown de/d1c/d1d 150896 1 2024-03-16T09:01:13.999 INFO:tasks.workunit.client.0.smithi007.stdout:1/472: truncate d9/dd/d1b/f2c 2657864 0 2024-03-16T09:01:14.001 INFO:tasks.workunit.client.0.smithi007.stdout:3/544: sync 2024-03-16T09:01:14.001 INFO:tasks.workunit.client.0.smithi007.stdout:5/484: sync 2024-03-16T09:01:14.001 INFO:tasks.workunit.client.0.smithi007.stdout:5/485: chown d1/d7/d1c/d19/d34/d3e/d41/d48/f75 971331 1 2024-03-16T09:01:14.002 INFO:tasks.workunit.client.1.smithi019.stdout:6/418: dread de/d1c/f56 [0,4194304] 0 2024-03-16T09:01:14.002 INFO:tasks.workunit.client.0.smithi007.stdout:4/411: creat d4/d14/d64/d68/f8b x:0 0 0 2024-03-16T09:01:14.005 INFO:tasks.workunit.client.0.smithi007.stdout:2/602: link dc/df/d24/d28/d4c/f67 dc/df/fc6 0 2024-03-16T09:01:14.005 INFO:tasks.workunit.client.0.smithi007.stdout:2/603: creat dc/df/d24/d28/d2e/fc7 x:0 0 0 2024-03-16T09:01:14.006 INFO:tasks.workunit.client.0.smithi007.stdout:2/604: fdatasync dc/df/d24/d28/d4c/d5e/f4d 0 2024-03-16T09:01:14.007 INFO:tasks.workunit.client.0.smithi007.stdout:0/582: mknod d3/d9/da/d48/d25/d43/cc3 0 2024-03-16T09:01:14.007 INFO:tasks.workunit.client.0.smithi007.stdout:0/583: read - d3/faf zero size 2024-03-16T09:01:14.009 INFO:tasks.workunit.client.1.smithi019.stdout:2/306: dwrite d5/d6/d2b/f3c [0,4194304] 0 2024-03-16T09:01:14.010 INFO:tasks.workunit.client.0.smithi007.stdout:1/473: creat d9/d6b/d7e/f92 x:0 0 0 2024-03-16T09:01:14.013 INFO:tasks.workunit.client.1.smithi019.stdout:2/307: truncate d5/f23 1790354 0 2024-03-16T09:01:14.013 INFO:tasks.workunit.client.1.smithi019.stdout:2/308: chown d5/d6/c38 3510388 1 2024-03-16T09:01:14.013 INFO:tasks.workunit.client.1.smithi019.stdout:2/309: creat d5/f61 x:0 0 0 2024-03-16T09:01:14.013 INFO:tasks.workunit.client.0.smithi007.stdout:5/486: mkdir d1/d7/d1c/d19/d34/d5e/d6f/db7 0 2024-03-16T09:01:14.014 INFO:tasks.workunit.client.0.smithi007.stdout:4/412: mknod d4/d14/d1c/d5b/d67/c8c 0 2024-03-16T09:01:14.014 INFO:tasks.workunit.client.0.smithi007.stdout:4/413: read d4/f2f [6096842,92864] 0 2024-03-16T09:01:14.014 INFO:tasks.workunit.client.0.smithi007.stdout:4/414: write d4/d14/f1b [499982,118648] 0 2024-03-16T09:01:14.016 INFO:tasks.workunit.client.1.smithi019.stdout:2/310: rename d5/d6/d2b/d2f/d35/c52 to d5/d5a/c62 0 2024-03-16T09:01:14.017 INFO:tasks.workunit.client.0.smithi007.stdout:0/584: dread d3/d9/da/d18/d27/d34/f89 [0,4194304] 0 2024-03-16T09:01:14.018 INFO:tasks.workunit.client.0.smithi007.stdout:2/605: symlink dc/df/d24/d28/lc8 0 2024-03-16T09:01:14.020 INFO:tasks.workunit.client.0.smithi007.stdout:1/474: unlink d9/dd/d1b/d28/d29/f42 0 2024-03-16T09:01:14.020 INFO:tasks.workunit.client.0.smithi007.stdout:1/475: write d9/dd/d1b/d2d/f3f [1755488,105405] 0 2024-03-16T09:01:14.020 INFO:tasks.workunit.client.0.smithi007.stdout:1/476: dread - d9/dd/d1b/d2d/d30/d45/f76 zero size 2024-03-16T09:01:14.021 INFO:tasks.workunit.client.0.smithi007.stdout:5/487: link d1/d7/d1c/d88/da2/d89/db3/f31 d1/d7/d1c/d88/da2/d89/fb8 0 2024-03-16T09:01:14.021 INFO:tasks.workunit.client.0.smithi007.stdout:5/488: chown d1/d7/d1c/d19/d34/d3e/d62/c64 223 1 2024-03-16T09:01:14.021 INFO:tasks.workunit.client.0.smithi007.stdout:5/489: fsync d1/d7/d1c/d19/d34/d5e/d6f/f51 0 2024-03-16T09:01:14.021 INFO:tasks.workunit.client.0.smithi007.stdout:5/490: dread - d1/faa zero size 2024-03-16T09:01:14.023 INFO:tasks.workunit.client.0.smithi007.stdout:0/585: rename d3/fa1 to d3/d9/da/d92/d9f/fc4 0 2024-03-16T09:01:14.024 INFO:tasks.workunit.client.0.smithi007.stdout:5/491: mknod d1/d7/cb9 0 2024-03-16T09:01:14.025 INFO:tasks.workunit.client.0.smithi007.stdout:2/606: rename dc/df/d24/d28/d2e/f52 to dc/df/d6c/fc9 0 2024-03-16T09:01:14.027 INFO:tasks.workunit.client.0.smithi007.stdout:0/586: creat d3/d9/da/d48/d25/d87/dbb/fc5 x:0 0 0 2024-03-16T09:01:14.028 INFO:tasks.workunit.client.0.smithi007.stdout:8/775: dwrite d0/dc/d50/d8e/dd2/dea/fa [0,4194304] 0 2024-03-16T09:01:14.028 INFO:tasks.workunit.client.0.smithi007.stdout:8/776: write d0/dc/d50/d8e/dd2/dea/d18/da7/de2/f102 [538361,126436] 0 2024-03-16T09:01:14.030 INFO:tasks.workunit.client.1.smithi019.stdout:6/419: dwrite de/f49 [0,4194304] 0 2024-03-16T09:01:14.032 INFO:tasks.workunit.client.0.smithi007.stdout:3/545: dwrite d15/d18/d1f/d3b/d63/f93 [0,4194304] 0 2024-03-16T09:01:14.032 INFO:tasks.workunit.client.1.smithi019.stdout:2/311: getdents d5/d6/d2b/d2f/d35 0 2024-03-16T09:01:14.036 INFO:tasks.workunit.client.0.smithi007.stdout:1/477: dread d9/dd/d1b/d2d/d30/d45/f4a [0,4194304] 0 2024-03-16T09:01:14.036 INFO:tasks.workunit.client.0.smithi007.stdout:1/478: fsync d9/f21 0 2024-03-16T09:01:14.036 INFO:tasks.workunit.client.0.smithi007.stdout:1/479: stat d9/c4e 0 2024-03-16T09:01:14.036 INFO:tasks.workunit.client.1.smithi019.stdout:6/420: symlink de/l89 0 2024-03-16T09:01:14.036 INFO:tasks.workunit.client.1.smithi019.stdout:6/421: dread - de/d1c/d1d/d66/d72/f82 zero size 2024-03-16T09:01:14.037 INFO:tasks.workunit.client.0.smithi007.stdout:4/415: dwrite d4/f5 [0,4194304] 0 2024-03-16T09:01:14.039 INFO:tasks.workunit.client.1.smithi019.stdout:2/312: rename d5/d6/f36 to d5/d6/d2b/d2f/d35/f63 0 2024-03-16T09:01:14.041 INFO:tasks.workunit.client.1.smithi019.stdout:2/313: rmdir d5/de 39 2024-03-16T09:01:14.041 INFO:tasks.workunit.client.1.smithi019.stdout:2/314: getdents d5/d6/d2b/d53 0 2024-03-16T09:01:14.042 INFO:tasks.workunit.client.0.smithi007.stdout:1/480: unlink d9/d6b/c73 0 2024-03-16T09:01:14.042 INFO:tasks.workunit.client.0.smithi007.stdout:1/481: chown f5 15836 1 2024-03-16T09:01:14.047 INFO:tasks.workunit.client.0.smithi007.stdout:4/416: link d4/d14/d64/d68/f56 d4/d14/d64/d68/d82/f8d 0 2024-03-16T09:01:14.051 INFO:tasks.workunit.client.0.smithi007.stdout:4/417: read d4/ff [2759073,71669] 0 2024-03-16T09:01:14.056 INFO:tasks.workunit.client.1.smithi019.stdout:3/378: sync 2024-03-16T09:01:14.057 INFO:tasks.workunit.client.1.smithi019.stdout:5/250: sync 2024-03-16T09:01:14.059 INFO:tasks.workunit.client.0.smithi007.stdout:5/492: dwrite d1/d7/d1c/d19/d34/d3e/f69 [0,4194304] 0 2024-03-16T09:01:14.059 INFO:tasks.workunit.client.0.smithi007.stdout:5/493: chown d1/d7/d1c/d19/d34/d1d/da8 2700 1 2024-03-16T09:01:14.060 INFO:tasks.workunit.client.0.smithi007.stdout:2/607: dwrite dc/f15 [0,4194304] 0 2024-03-16T09:01:14.060 INFO:tasks.workunit.client.1.smithi019.stdout:5/251: dread d6/d19/f1b [0,4194304] 0 2024-03-16T09:01:14.061 INFO:tasks.workunit.client.1.smithi019.stdout:5/252: chown d6/dd/l12 5814 1 2024-03-16T09:01:14.061 INFO:tasks.workunit.client.1.smithi019.stdout:5/253: fsync d6/dd/d37/f3d 0 2024-03-16T09:01:14.061 INFO:tasks.workunit.client.1.smithi019.stdout:3/379: rename d3/d11/f4c to d3/d17/d5a/f73 0 2024-03-16T09:01:14.062 INFO:tasks.workunit.client.1.smithi019.stdout:3/380: rename d3/f43 to d3/d1b/d40/f74 0 2024-03-16T09:01:14.063 INFO:tasks.workunit.client.1.smithi019.stdout:3/381: read - d3/f6c zero size 2024-03-16T09:01:14.064 INFO:tasks.workunit.client.1.smithi019.stdout:3/382: mkdir d3/d17/d5a/d75 0 2024-03-16T09:01:14.065 INFO:tasks.workunit.client.0.smithi007.stdout:5/494: creat d1/d7/d1c/d19/d34/d1d/da8/fba x:0 0 0 2024-03-16T09:01:14.066 INFO:tasks.workunit.client.0.smithi007.stdout:5/495: chown d1/d7/d1c/c7b 2 1 2024-03-16T09:01:14.066 INFO:tasks.workunit.client.0.smithi007.stdout:5/496: dread - d1/d7/d1c/d88/da2/d89/fb6 zero size 2024-03-16T09:01:14.066 INFO:tasks.workunit.client.0.smithi007.stdout:8/777: dwrite d0/d26/d2d/ff2 [0,4194304] 0 2024-03-16T09:01:14.067 INFO:tasks.workunit.client.0.smithi007.stdout:5/497: symlink d1/d7/d1c/d88/da2/dae/lbb 0 2024-03-16T09:01:14.070 INFO:tasks.workunit.client.0.smithi007.stdout:2/608: rename dc/df/d24/d28/d2e/f9b to dc/df/fca 0 2024-03-16T09:01:14.070 INFO:tasks.workunit.client.0.smithi007.stdout:5/498: symlink d1/d7/d1c/d19/d34/d5e/d6f/da9/lbc 0 2024-03-16T09:01:14.072 INFO:tasks.workunit.client.1.smithi019.stdout:3/383: dread d3/d11/f3b [0,4194304] 0 2024-03-16T09:01:14.072 INFO:tasks.workunit.client.0.smithi007.stdout:5/499: rename d1/d7/d1c/c7b to d1/d7/d1c/d88/da2/d89/db3/d2a/d81/cbd 0 2024-03-16T09:01:14.074 INFO:tasks.workunit.client.0.smithi007.stdout:5/500: creat d1/d7/d1c/d88/da2/d89/db3/d2a/d79/fbe x:0 0 0 2024-03-16T09:01:14.074 INFO:tasks.workunit.client.0.smithi007.stdout:5/501: chown d1/d7/d1c/d19/d34/d4d/d99/fa3 3409774 1 2024-03-16T09:01:14.074 INFO:tasks.workunit.client.0.smithi007.stdout:5/502: chown d1/d7/f7c 1 1 2024-03-16T09:01:14.076 INFO:tasks.workunit.client.0.smithi007.stdout:5/503: link d1/d7/d16/d1b/d3c/fa7 d1/d7/d1c/d88/da2/d89/db3/d2a/d79/fbf 0 2024-03-16T09:01:14.080 INFO:tasks.workunit.client.0.smithi007.stdout:3/546: dwrite d15/d18/d1f/d29/d43/d72/f50 [4194304,4194304] 0 2024-03-16T09:01:14.084 INFO:tasks.workunit.client.0.smithi007.stdout:8/778: dread d0/dc/dd/d8b/d9e/fde [0,4194304] 0 2024-03-16T09:01:14.086 INFO:tasks.workunit.client.0.smithi007.stdout:8/779: mknod d0/dc/d50/d8e/dd2/dea/d18/da7/c117 0 2024-03-16T09:01:14.086 INFO:tasks.workunit.client.0.smithi007.stdout:8/780: dread - d0/dc/d50/d8e/dd2/dea/f104 zero size 2024-03-16T09:01:14.087 INFO:tasks.workunit.client.1.smithi019.stdout:2/315: dwrite d5/de/f1f [0,4194304] 0 2024-03-16T09:01:14.088 INFO:tasks.workunit.client.0.smithi007.stdout:6/391: sync 2024-03-16T09:01:14.088 INFO:tasks.workunit.client.0.smithi007.stdout:6/392: dread - d2/d13/d3f/d50/d72/f69 zero size 2024-03-16T09:01:14.088 INFO:tasks.workunit.client.0.smithi007.stdout:6/393: chown d2/d13/d1f/f67 0 1 2024-03-16T09:01:14.088 INFO:tasks.workunit.client.0.smithi007.stdout:2/609: dread dc/f10 [0,4194304] 0 2024-03-16T09:01:14.088 INFO:tasks.workunit.client.0.smithi007.stdout:8/781: symlink d0/dc/d75/dc3/l118 0 2024-03-16T09:01:14.088 INFO:tasks.workunit.client.0.smithi007.stdout:8/782: chown d0/dc/d50/d8e/dd2/le3 189277680 1 2024-03-16T09:01:14.088 INFO:tasks.workunit.client.0.smithi007.stdout:8/783: chown d0/d26/d2d/d43/d72/dad/ffa 1 1 2024-03-16T09:01:14.089 INFO:tasks.workunit.client.0.smithi007.stdout:6/394: creat d2/d11/d3b/d74/f80 x:0 0 0 2024-03-16T09:01:14.089 INFO:tasks.workunit.client.0.smithi007.stdout:6/395: getdents d2/d13/d3f/d50/d78 0 2024-03-16T09:01:14.090 INFO:tasks.workunit.client.1.smithi019.stdout:2/316: mkdir d5/d6/d64 0 2024-03-16T09:01:14.090 INFO:tasks.workunit.client.1.smithi019.stdout:2/317: getdents d5/d4e 0 2024-03-16T09:01:14.090 INFO:tasks.workunit.client.1.smithi019.stdout:2/318: chown d5/d6/d2b/l20 19 1 2024-03-16T09:01:14.091 INFO:tasks.workunit.client.0.smithi007.stdout:2/610: chown dc/df/l30 127813 1 2024-03-16T09:01:14.091 INFO:tasks.workunit.client.0.smithi007.stdout:2/611: chown dc/df/d24/d28/d2e/d4a/c53 527114141 1 2024-03-16T09:01:14.092 INFO:tasks.workunit.client.1.smithi019.stdout:2/319: rename d5/d6/d2b/d2f/d35/d4b/d5f to d5/d6/d2b/d2f/d35/d65 0 2024-03-16T09:01:14.092 INFO:tasks.workunit.client.0.smithi007.stdout:1/482: dwrite d9/dd/f59 [0,4194304] 0 2024-03-16T09:01:14.097 INFO:tasks.workunit.client.0.smithi007.stdout:4/418: dwrite d4/d14/d1c/f3b [0,4194304] 0 2024-03-16T09:01:14.097 INFO:tasks.workunit.client.0.smithi007.stdout:4/419: readlink d4/d14/d64/l6c 0 2024-03-16T09:01:14.102 INFO:tasks.workunit.client.0.smithi007.stdout:4/420: rmdir d4/d14/d64/d68/d82 39 2024-03-16T09:01:14.102 INFO:tasks.workunit.client.0.smithi007.stdout:4/421: fsync d4/d14/d1c/d5b/d67/f81 0 2024-03-16T09:01:14.102 INFO:tasks.workunit.client.0.smithi007.stdout:4/422: creat d4/d14/d1c/d5b/f8e x:0 0 0 2024-03-16T09:01:14.106 INFO:tasks.workunit.client.0.smithi007.stdout:8/784: dread d0/dc/d50/d8e/dd2/dea/fa [0,4194304] 0 2024-03-16T09:01:14.112 INFO:tasks.workunit.client.0.smithi007.stdout:2/612: dread dc/f68 [0,4194304] 0 2024-03-16T09:01:14.113 INFO:tasks.workunit.client.1.smithi019.stdout:5/254: dwrite d6/f8 [4194304,4194304] 0 2024-03-16T09:01:14.114 INFO:tasks.workunit.client.1.smithi019.stdout:5/255: creat d6/dd/d47/f4a x:0 0 0 2024-03-16T09:01:14.114 INFO:tasks.workunit.client.1.smithi019.stdout:5/256: chown d6/d19/d34 630104 1 2024-03-16T09:01:14.114 INFO:tasks.workunit.client.1.smithi019.stdout:5/257: chown f4 265 1 2024-03-16T09:01:14.124 INFO:tasks.workunit.client.0.smithi007.stdout:2/613: dread dc/df/d24/d28/d2e/f5c [0,4194304] 0 2024-03-16T09:01:14.124 INFO:tasks.workunit.client.0.smithi007.stdout:2/614: getdents dc/df/d24/d28/d2e/dc5 0 2024-03-16T09:01:14.124 INFO:tasks.workunit.client.0.smithi007.stdout:2/615: getdents dc/df/d24/d28/d97/d99 0 2024-03-16T09:01:14.131 INFO:tasks.workunit.client.0.smithi007.stdout:5/504: dwrite d1/d7/d1c/d88/da2/f74 [0,4194304] 0 2024-03-16T09:01:14.132 INFO:tasks.workunit.client.1.smithi019.stdout:3/384: dwrite d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:14.132 INFO:tasks.workunit.client.1.smithi019.stdout:3/385: chown d3/d17 6407 1 2024-03-16T09:01:14.133 INFO:tasks.workunit.client.1.smithi019.stdout:3/386: mknod d3/d17/d5a/d75/c76 0 2024-03-16T09:01:14.135 INFO:tasks.workunit.client.0.smithi007.stdout:5/505: symlink d1/d7/d1c/d19/d34/d5e/d6f/d72/lc0 0 2024-03-16T09:01:14.135 INFO:tasks.workunit.client.0.smithi007.stdout:5/506: dread - d1/d7/d1c/d88/da2/d89/fb6 zero size 2024-03-16T09:01:14.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/258: dread d6/f9 [0,4194304] 0 2024-03-16T09:01:14.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/259: truncate d6/dd/f44 1052208 0 2024-03-16T09:01:14.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/260: write d6/dd/f1a [39950,32395] 0 2024-03-16T09:01:14.137 INFO:tasks.workunit.client.1.smithi019.stdout:3/387: truncate d3/d17/d5a/f62 61739 0 2024-03-16T09:01:14.137 INFO:tasks.workunit.client.0.smithi007.stdout:8/785: dread d0/ff [4194304,4194304] 0 2024-03-16T09:01:14.137 INFO:tasks.workunit.client.0.smithi007.stdout:8/786: write d0/d26/d2d/d43/d4c/f110 [206026,97248] 0 2024-03-16T09:01:14.137 INFO:tasks.workunit.client.0.smithi007.stdout:8/787: write d0/dc/d50/d8e/dd2/dea/f58 [170297,115738] 0 2024-03-16T09:01:14.138 INFO:tasks.workunit.client.0.smithi007.stdout:5/507: symlink d1/d7/d1c/d19/d34/d5e/d6f/d72/lc1 0 2024-03-16T09:01:14.138 INFO:tasks.workunit.client.0.smithi007.stdout:5/508: write d1/d7/d16/f33 [1294074,81663] 0 2024-03-16T09:01:14.139 INFO:tasks.workunit.client.0.smithi007.stdout:8/788: truncate d0/d26/d2d/d64/da6/d10e/dfd/fc7 3198381 0 2024-03-16T09:01:14.140 INFO:tasks.workunit.client.0.smithi007.stdout:8/789: mkdir d0/d26/d2d/d64/da6/d10e/dfd/d119 0 2024-03-16T09:01:14.142 INFO:tasks.workunit.client.0.smithi007.stdout:8/790: mkdir d0/d26/d11a 0 2024-03-16T09:01:14.142 INFO:tasks.workunit.client.0.smithi007.stdout:8/791: chown d0/dc/dd/f2c 3008 1 2024-03-16T09:01:14.142 INFO:tasks.workunit.client.0.smithi007.stdout:8/792: creat d0/dc/d75/f11b x:0 0 0 2024-03-16T09:01:14.144 INFO:tasks.workunit.client.0.smithi007.stdout:5/509: dread d1/d7/d1c/d88/da2/d89/db3/f47 [0,4194304] 0 2024-03-16T09:01:14.144 INFO:tasks.workunit.client.0.smithi007.stdout:5/510: stat d1/d7/d1c/d88/da2/d89/db3/d2a/d79 0 2024-03-16T09:01:14.144 INFO:tasks.workunit.client.0.smithi007.stdout:5/511: chown d1/d7/d1c/d88/da2/d89/db3/d2a/d79/c83 338338 1 2024-03-16T09:01:14.146 INFO:tasks.workunit.client.0.smithi007.stdout:5/512: rmdir d1/d7/d1c/d19/d34/d5e/d6f/db7 0 2024-03-16T09:01:14.148 INFO:tasks.workunit.client.0.smithi007.stdout:5/513: mknod d1/d7/cc2 0 2024-03-16T09:01:14.148 INFO:tasks.workunit.client.0.smithi007.stdout:5/514: chown d1/d7/d1c/d19/d34/d3e/d41/d48/c98 38785482 1 2024-03-16T09:01:14.148 INFO:tasks.workunit.client.0.smithi007.stdout:5/515: creat d1/d7/d1c/d19/d34/d3e/d41/d48/fc3 x:0 0 0 2024-03-16T09:01:14.150 INFO:tasks.workunit.client.0.smithi007.stdout:5/516: symlink d1/d7/d1c/d19/d34/d3e/d41/d48/lc4 0 2024-03-16T09:01:14.150 INFO:tasks.workunit.client.0.smithi007.stdout:5/517: fdatasync d1/d7/d1c/d88/f95 0 2024-03-16T09:01:14.151 INFO:tasks.workunit.client.0.smithi007.stdout:5/518: stat d1/d7/d1c/d19/d34/d5e/d6f/d72/f85 0 2024-03-16T09:01:14.151 INFO:tasks.workunit.client.1.smithi019.stdout:5/261: read d6/d19/f29 [2487582,96112] 0 2024-03-16T09:01:14.151 INFO:tasks.workunit.client.1.smithi019.stdout:5/262: creat d6/d1f/d33/f4b x:0 0 0 2024-03-16T09:01:14.151 INFO:tasks.workunit.client.1.smithi019.stdout:5/263: write d6/f32 [725777,52268] 0 2024-03-16T09:01:14.152 INFO:tasks.workunit.client.0.smithi007.stdout:5/519: mknod d1/d7/cc5 0 2024-03-16T09:01:14.153 INFO:tasks.workunit.client.1.smithi019.stdout:5/264: mkdir d6/d1f/d33/d4c 0 2024-03-16T09:01:14.157 INFO:tasks.workunit.client.0.smithi007.stdout:3/547: dwrite d15/d18/d1f/d29/d7c/f94 [0,4194304] 0 2024-03-16T09:01:14.157 INFO:tasks.workunit.client.0.smithi007.stdout:3/548: fdatasync d15/d18/d6a/f7e 0 2024-03-16T09:01:14.158 INFO:tasks.workunit.client.0.smithi007.stdout:6/396: dwrite d2/d10/d3d/f6c [0,4194304] 0 2024-03-16T09:01:14.158 INFO:tasks.workunit.client.0.smithi007.stdout:6/397: readlink d2/d13/d62/l6e 0 2024-03-16T09:01:14.158 INFO:tasks.workunit.client.0.smithi007.stdout:6/398: read - d2/d13/d3f/d50/d72/f69 zero size 2024-03-16T09:01:14.158 INFO:tasks.workunit.client.0.smithi007.stdout:4/423: dwrite d4/d14/d1c/f3e [0,4194304] 0 2024-03-16T09:01:14.160 INFO:tasks.workunit.client.0.smithi007.stdout:1/483: dwrite d9/f13 [0,4194304] 0 2024-03-16T09:01:14.160 INFO:tasks.workunit.client.0.smithi007.stdout:2/616: dwrite dc/df/d24/d28/d4c/f58 [0,4194304] 0 2024-03-16T09:01:14.160 INFO:tasks.workunit.client.0.smithi007.stdout:2/617: fsync dc/df/d24/d28/d2e/f45 0 2024-03-16T09:01:14.160 INFO:tasks.workunit.client.0.smithi007.stdout:2/618: chown dc/df/d24/d28/d4c/d5e/c5d 828488051 1 2024-03-16T09:01:14.161 INFO:tasks.workunit.client.1.smithi019.stdout:2/320: dwrite d5/d6/d2b/f3c [4194304,4194304] 0 2024-03-16T09:01:14.162 INFO:tasks.workunit.client.1.smithi019.stdout:5/265: write d6/fb [7596143,112605] 0 2024-03-16T09:01:14.163 INFO:tasks.workunit.client.1.smithi019.stdout:5/266: read - d6/d1f/f2d zero size 2024-03-16T09:01:14.164 INFO:tasks.workunit.client.0.smithi007.stdout:5/520: dread d1/d7/d1c/d88/da2/d89/fb8 [0,4194304] 0 2024-03-16T09:01:14.164 INFO:tasks.workunit.client.0.smithi007.stdout:5/521: stat d1/d7/d1c/d19/d34/de/l50 0 2024-03-16T09:01:14.165 INFO:tasks.workunit.client.0.smithi007.stdout:6/399: mkdir d2/d13/d3f/d50/d78/d81 0 2024-03-16T09:01:14.166 INFO:tasks.workunit.client.0.smithi007.stdout:4/424: mkdir d4/d14/d1c/d5b/d67/d8f 0 2024-03-16T09:01:14.168 INFO:tasks.workunit.client.1.smithi019.stdout:2/321: mknod d5/d5a/c66 0 2024-03-16T09:01:14.171 INFO:tasks.workunit.client.1.smithi019.stdout:5/267: unlink d6/dd/f49 0 2024-03-16T09:01:14.173 INFO:tasks.workunit.client.0.smithi007.stdout:8/793: dwrite d0/dc/d50/d8e/dd2/dea/d18/f3f [0,4194304] 0 2024-03-16T09:01:14.173 INFO:tasks.workunit.client.0.smithi007.stdout:8/794: write d0/dc/d50/d8e/db8/ffb [178247,125617] 0 2024-03-16T09:01:14.173 INFO:tasks.workunit.client.0.smithi007.stdout:8/795: fsync d0/dc/d8a/f91 0 2024-03-16T09:01:14.173 INFO:tasks.workunit.client.0.smithi007.stdout:8/796: fsync d0/dc/dd/d8b/fd0 0 2024-03-16T09:01:14.174 INFO:tasks.workunit.client.1.smithi019.stdout:2/322: creat d5/d6/d2b/d2f/f67 x:0 0 0 2024-03-16T09:01:14.174 INFO:tasks.workunit.client.0.smithi007.stdout:1/484: rename d9/c38 to d9/dd/d1b/d28/d29/d7b/d90/c93 0 2024-03-16T09:01:14.174 INFO:tasks.workunit.client.1.smithi019.stdout:2/323: dread d5/d6/d2b/d2f/f46 [0,4194304] 0 2024-03-16T09:01:14.174 INFO:tasks.workunit.client.1.smithi019.stdout:2/324: write d5/d6/d2b/d2f/d35/f56 [214443,58506] 0 2024-03-16T09:01:14.175 INFO:tasks.workunit.client.1.smithi019.stdout:5/268: symlink d6/d19/d34/d3e/l4d 0 2024-03-16T09:01:14.177 INFO:tasks.workunit.client.0.smithi007.stdout:2/619: mknod dc/df/d24/d28/d2e/d35/ccb 0 2024-03-16T09:01:14.178 INFO:tasks.workunit.client.1.smithi019.stdout:5/269: link d6/dd/d37/f3d d6/d19/f4e 0 2024-03-16T09:01:14.178 INFO:tasks.workunit.client.1.smithi019.stdout:5/270: write d6/d1f/d33/f46 [268259,67742] 0 2024-03-16T09:01:14.178 INFO:tasks.workunit.client.0.smithi007.stdout:3/549: getdents d15/d18/d1f/d3b/d87 0 2024-03-16T09:01:14.179 INFO:tasks.workunit.client.1.smithi019.stdout:2/325: write d5/de/fd [323546,12977] 0 2024-03-16T09:01:14.180 INFO:tasks.workunit.client.0.smithi007.stdout:5/522: getdents d1/d7/d1c/d88/da2/d89/db3/d7d 0 2024-03-16T09:01:14.180 INFO:tasks.workunit.client.0.smithi007.stdout:4/425: dread d4/d14/d1c/f60 [0,4194304] 0 2024-03-16T09:01:14.181 INFO:tasks.workunit.client.0.smithi007.stdout:4/426: chown d4/d14/d1c 62755118 1 2024-03-16T09:01:14.181 INFO:tasks.workunit.client.0.smithi007.stdout:4/427: chown d4/d7/l4e 28 1 2024-03-16T09:01:14.182 INFO:tasks.workunit.client.1.smithi019.stdout:2/326: symlink d5/d6/d2b/l68 0 2024-03-16T09:01:14.182 INFO:tasks.workunit.client.0.smithi007.stdout:6/400: mkdir d2/d82 0 2024-03-16T09:01:14.183 INFO:tasks.workunit.client.1.smithi019.stdout:2/327: symlink d5/d2e/d4f/l69 0 2024-03-16T09:01:14.184 INFO:tasks.workunit.client.1.smithi019.stdout:0/302: sync 2024-03-16T09:01:14.185 INFO:tasks.workunit.client.1.smithi019.stdout:2/328: mknod d5/d5a/c6a 0 2024-03-16T09:01:14.185 INFO:tasks.workunit.client.1.smithi019.stdout:2/329: stat d5/de/f55 0 2024-03-16T09:01:14.186 INFO:tasks.workunit.client.0.smithi007.stdout:8/797: chown d0/dc/d50/d8e/dd2/dea/c5 144 1 2024-03-16T09:01:14.186 INFO:tasks.workunit.client.0.smithi007.stdout:8/798: creat d0/dc/f11c x:0 0 0 2024-03-16T09:01:14.186 INFO:tasks.workunit.client.0.smithi007.stdout:8/799: dread - d0/dc/d50/d8e/dd2/dea/f104 zero size 2024-03-16T09:01:14.187 INFO:tasks.workunit.client.1.smithi019.stdout:0/303: truncate d7/d8/f12 1823696 0 2024-03-16T09:01:14.187 INFO:tasks.workunit.client.1.smithi019.stdout:0/304: stat d7/d29/f41 0 2024-03-16T09:01:14.187 INFO:tasks.workunit.client.0.smithi007.stdout:1/485: creat d9/dd/d1b/f94 x:0 0 0 2024-03-16T09:01:14.187 INFO:tasks.workunit.client.0.smithi007.stdout:1/486: rename d9/dd/d1b/d28/d29/d7b to d9/dd/d1b/d28/d29/d7b/d95 22 2024-03-16T09:01:14.188 INFO:tasks.workunit.client.0.smithi007.stdout:1/487: readlink d9/d6b/l74 0 2024-03-16T09:01:14.188 INFO:tasks.workunit.client.1.smithi019.stdout:2/330: mkdir d5/d2e/d6b 0 2024-03-16T09:01:14.188 INFO:tasks.workunit.client.0.smithi007.stdout:3/550: rmdir d15/d18/d6a/d78 39 2024-03-16T09:01:14.188 INFO:tasks.workunit.client.0.smithi007.stdout:3/551: write f5 [4935149,119096] 0 2024-03-16T09:01:14.190 INFO:tasks.workunit.client.1.smithi019.stdout:2/331: mknod d5/d6/c6c 0 2024-03-16T09:01:14.190 INFO:tasks.workunit.client.1.smithi019.stdout:0/305: truncate d7/f2a 1346144 0 2024-03-16T09:01:14.192 INFO:tasks.workunit.client.0.smithi007.stdout:2/620: dwrite dc/df/d24/d28/d2e/d35/f98 [0,4194304] 0 2024-03-16T09:01:14.192 INFO:tasks.workunit.client.0.smithi007.stdout:2/621: chown dc/df/d24/d28/d97/da4 1767 1 2024-03-16T09:01:14.192 INFO:tasks.workunit.client.0.smithi007.stdout:5/523: unlink d1/fc 0 2024-03-16T09:01:14.194 INFO:tasks.workunit.client.0.smithi007.stdout:4/428: mkdir d4/d14/d1c/d5b/d67/d8f/d90 0 2024-03-16T09:01:14.195 INFO:tasks.workunit.client.1.smithi019.stdout:2/332: getdents d5/d6/d2b 0 2024-03-16T09:01:14.195 INFO:tasks.workunit.client.1.smithi019.stdout:2/333: dread - d5/d6/d2b/d2f/d35/d4b/f5c zero size 2024-03-16T09:01:14.196 INFO:tasks.workunit.client.0.smithi007.stdout:6/401: mknod d2/d13/d3f/d50/d78/c83 0 2024-03-16T09:01:14.196 INFO:tasks.workunit.client.1.smithi019.stdout:2/334: truncate d5/ff 2905238 0 2024-03-16T09:01:14.196 INFO:tasks.workunit.client.1.smithi019.stdout:2/335: write d5/f4d [39434,81909] 0 2024-03-16T09:01:14.198 INFO:tasks.workunit.client.1.smithi019.stdout:2/336: creat d5/d6/d2b/f6d x:0 0 0 2024-03-16T09:01:14.199 INFO:tasks.workunit.client.0.smithi007.stdout:1/488: creat d9/dd/d1b/f96 x:0 0 0 2024-03-16T09:01:14.200 INFO:tasks.workunit.client.1.smithi019.stdout:2/337: symlink d5/d6/d2b/d2f/d35/d4b/d51/l6e 0 2024-03-16T09:01:14.201 INFO:tasks.workunit.client.0.smithi007.stdout:3/552: creat d15/d18/d1f/d29/d3e/d85/fac x:0 0 0 2024-03-16T09:01:14.201 INFO:tasks.workunit.client.0.smithi007.stdout:3/553: write d15/d18/d6a/d78/f9e [509905,109567] 0 2024-03-16T09:01:14.201 INFO:tasks.workunit.client.0.smithi007.stdout:3/554: stat d15/f2c 0 2024-03-16T09:01:14.202 INFO:tasks.workunit.client.1.smithi019.stdout:2/338: mkdir d5/d6/d64/d6f 0 2024-03-16T09:01:14.202 INFO:tasks.workunit.client.1.smithi019.stdout:2/339: readlink d5/d6/d2b/d2f/l54 0 2024-03-16T09:01:14.202 INFO:tasks.workunit.client.1.smithi019.stdout:0/306: dread d7/d1d/d28/d48/f4e [0,4194304] 0 2024-03-16T09:01:14.209 INFO:tasks.workunit.client.0.smithi007.stdout:9/564: sync 2024-03-16T09:01:14.209 INFO:tasks.workunit.client.0.smithi007.stdout:7/432: sync 2024-03-16T09:01:14.212 INFO:tasks.workunit.client.0.smithi007.stdout:2/622: mkdir dc/df/d24/d28/d2e/dcc 0 2024-03-16T09:01:14.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:13 smithi019 ceph-mon[36407]: pgmap v20: 65 pgs: 65 active+clean; 932 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 89 MiB/s rd, 113 MiB/s wr, 373 op/s 2024-03-16T09:01:14.215 INFO:tasks.workunit.client.0.smithi007.stdout:2/623: write dc/df/d24/d28/d2e/d6e/f9a [18607,126896] 0 2024-03-16T09:01:14.215 INFO:tasks.workunit.client.0.smithi007.stdout:2/624: stat dc/df/d24/d28/d2e/d35/l93 0 2024-03-16T09:01:14.216 INFO:tasks.workunit.client.0.smithi007.stdout:4/429: creat d4/d14/d1c/d5b/d67/d8f/d90/f91 x:0 0 0 2024-03-16T09:01:14.217 INFO:tasks.workunit.client.0.smithi007.stdout:9/565: write f8 [5704466,51938] 0 2024-03-16T09:01:14.217 INFO:tasks.workunit.client.0.smithi007.stdout:0/587: sync 2024-03-16T09:01:14.220 INFO:tasks.workunit.client.0.smithi007.stdout:6/402: mkdir d2/d13/d1f/d49/d84 0 2024-03-16T09:01:14.220 INFO:tasks.workunit.client.0.smithi007.stdout:7/433: mkdir d0/d19/d1f/d5d/d8a/d93 0 2024-03-16T09:01:14.220 INFO:tasks.workunit.client.0.smithi007.stdout:1/489: mknod d9/dd/d1b/d2d/d30/c97 0 2024-03-16T09:01:14.221 INFO:tasks.workunit.client.0.smithi007.stdout:7/434: write d0/dc/d5a/f74 [623203,8847] 0 2024-03-16T09:01:14.221 INFO:tasks.workunit.client.0.smithi007.stdout:6/403: write d2/d10/d12/f35 [3343645,34640] 0 2024-03-16T09:01:14.221 INFO:tasks.workunit.client.0.smithi007.stdout:5/524: rmdir d1/d7/d1c/d88 39 2024-03-16T09:01:14.221 INFO:tasks.workunit.client.0.smithi007.stdout:5/525: write d1/d7/d1c/d19/d34/d3e/f6a [1958888,25734] 0 2024-03-16T09:01:14.221 INFO:tasks.workunit.client.0.smithi007.stdout:5/526: truncate d1/d7/d1c/d19/d34/d3e/d41/d48/fc3 212147 0 2024-03-16T09:01:14.222 INFO:tasks.workunit.client.0.smithi007.stdout:2/625: truncate dc/df/f5a 326785 0 2024-03-16T09:01:14.222 INFO:tasks.workunit.client.0.smithi007.stdout:2/626: dread - dc/df/d24/d28/d2e/f78 zero size 2024-03-16T09:01:14.223 INFO:tasks.workunit.client.0.smithi007.stdout:1/490: unlink d9/dd/d1b/d28/d29/c2e 0 2024-03-16T09:01:14.226 INFO:tasks.workunit.client.0.smithi007.stdout:6/404: rename d2/d11/f26 to d2/d11/d1e/f85 0 2024-03-16T09:01:14.226 INFO:tasks.workunit.client.0.smithi007.stdout:6/405: write d2/d13/d3f/d50/d72/f4c [5234941,72512] 0 2024-03-16T09:01:14.230 INFO:tasks.workunit.client.0.smithi007.stdout:6/406: mkdir d2/d13/d3f/d86 0 2024-03-16T09:01:14.230 INFO:tasks.workunit.client.0.smithi007.stdout:5/527: dread d1/d7/d1c/d88/da2/d89/db3/f6e [0,4194304] 0 2024-03-16T09:01:14.230 INFO:tasks.workunit.client.0.smithi007.stdout:5/528: truncate d1/d7/d1c/f15 4269400 0 2024-03-16T09:01:14.230 INFO:tasks.workunit.client.0.smithi007.stdout:6/407: write d2/d10/f70 [284800,123571] 0 2024-03-16T09:01:14.230 INFO:tasks.workunit.client.0.smithi007.stdout:6/408: creat d2/d11/d3b/f87 x:0 0 0 2024-03-16T09:01:14.235 INFO:tasks.workunit.client.0.smithi007.stdout:2/627: write dc/df/d24/d28/d2e/d35/f5f [3141072,97480] 0 2024-03-16T09:01:14.236 INFO:tasks.workunit.client.0.smithi007.stdout:2/628: creat dc/df/d6c/fcd x:0 0 0 2024-03-16T09:01:14.238 INFO:tasks.workunit.client.0.smithi007.stdout:9/566: dread d24/f39 [0,4194304] 0 2024-03-16T09:01:14.238 INFO:tasks.workunit.client.0.smithi007.stdout:9/567: dread - d24/d4a/d55/d56/d7c/d95/f6f zero size 2024-03-16T09:01:14.239 INFO:tasks.workunit.client.1.smithi019.stdout:5/271: dwrite d6/d1f/d33/f46 [0,4194304] 0 2024-03-16T09:01:14.239 INFO:tasks.workunit.client.1.smithi019.stdout:5/272: truncate d6/d19/f29 5196071 0 2024-03-16T09:01:14.243 INFO:tasks.workunit.client.0.smithi007.stdout:2/629: rename dc/df/l30 to dc/df/d24/d28/d4c/d5e/lce 0 2024-03-16T09:01:14.243 INFO:tasks.workunit.client.0.smithi007.stdout:9/568: mkdir d24/d4a/da2 0 2024-03-16T09:01:14.243 INFO:tasks.workunit.client.1.smithi019.stdout:5/273: link d6/d19/d34/d3e/f48 d6/d1f/d33/f4f 0 2024-03-16T09:01:14.245 INFO:tasks.workunit.client.0.smithi007.stdout:8/800: dwrite d0/dc/d50/d8e/dd2/dea/d5f/fc2 [0,4194304] 0 2024-03-16T09:01:14.245 INFO:tasks.workunit.client.0.smithi007.stdout:6/409: dread d2/d11/d1e/d54/f63 [0,4194304] 0 2024-03-16T09:01:14.245 INFO:tasks.workunit.client.0.smithi007.stdout:6/410: stat d2/d11/l32 0 2024-03-16T09:01:14.245 INFO:tasks.workunit.client.0.smithi007.stdout:6/411: read d2/f46 [3147937,44479] 0 2024-03-16T09:01:14.247 INFO:tasks.workunit.client.0.smithi007.stdout:9/569: creat d24/d4a/d55/d56/fa3 x:0 0 0 2024-03-16T09:01:14.247 INFO:tasks.workunit.client.0.smithi007.stdout:9/570: write d24/d4c/f4e [894139,109312] 0 2024-03-16T09:01:14.247 INFO:tasks.workunit.client.0.smithi007.stdout:6/412: mknod d2/d29/c88 0 2024-03-16T09:01:14.258 INFO:tasks.workunit.client.0.smithi007.stdout:9/571: write f1e [439722,49379] 0 2024-03-16T09:01:14.258 INFO:tasks.workunit.client.0.smithi007.stdout:9/572: readlink d24/l70 0 2024-03-16T09:01:14.270 INFO:tasks.workunit.client.1.smithi019.stdout:5/274: dwrite d6/f2f [0,4194304] 0 2024-03-16T09:01:14.270 INFO:tasks.workunit.client.1.smithi019.stdout:5/275: chown l5 23995 1 2024-03-16T09:01:14.271 INFO:tasks.workunit.client.1.smithi019.stdout:5/276: unlink d6/l2b 0 2024-03-16T09:01:14.271 INFO:tasks.workunit.client.1.smithi019.stdout:0/307: dwrite d7/d8/d22/f2c [0,4194304] 0 2024-03-16T09:01:14.275 INFO:tasks.workunit.client.1.smithi019.stdout:2/340: dwrite d5/de/f45 [0,4194304] 0 2024-03-16T09:01:14.275 INFO:tasks.workunit.client.1.smithi019.stdout:2/341: write d5/d2e/f50 [3842746,97154] 0 2024-03-16T09:01:14.278 INFO:tasks.workunit.client.0.smithi007.stdout:4/430: dwrite d4/d7/d6b/f6d [4194304,4194304] 0 2024-03-16T09:01:14.278 INFO:tasks.workunit.client.0.smithi007.stdout:4/431: write d4/d14/d64/f7d [220333,121736] 0 2024-03-16T09:01:14.278 INFO:tasks.workunit.client.0.smithi007.stdout:4/432: chown d4/d7/c46 691 1 2024-03-16T09:01:14.281 INFO:tasks.workunit.client.1.smithi019.stdout:5/277: write d6/f17 [1182272,50730] 0 2024-03-16T09:01:14.283 INFO:tasks.workunit.client.1.smithi019.stdout:5/278: symlink d6/dd/d37/l50 0 2024-03-16T09:01:14.284 INFO:tasks.workunit.client.1.smithi019.stdout:5/279: truncate d6/f8 225533 0 2024-03-16T09:01:14.285 INFO:tasks.workunit.client.1.smithi019.stdout:5/280: mknod d6/d19/d34/c51 0 2024-03-16T09:01:14.285 INFO:tasks.workunit.client.1.smithi019.stdout:5/281: truncate d6/d1f/d33/f4b 254346 0 2024-03-16T09:01:14.285 INFO:tasks.workunit.client.1.smithi019.stdout:5/282: stat d6/d19/l2a 0 2024-03-16T09:01:14.285 INFO:tasks.workunit.client.0.smithi007.stdout:4/433: dread d4/f2f [0,4194304] 0 2024-03-16T09:01:14.289 INFO:tasks.workunit.client.0.smithi007.stdout:0/588: dwrite d3/d9/da/d18/d23/f63 [0,4194304] 0 2024-03-16T09:01:14.289 INFO:tasks.workunit.client.0.smithi007.stdout:5/529: dwrite d1/d7/d1c/d19/f26 [0,4194304] 0 2024-03-16T09:01:14.289 INFO:tasks.workunit.client.0.smithi007.stdout:0/589: stat d3/d9/da/f4c 0 2024-03-16T09:01:14.290 INFO:tasks.workunit.client.0.smithi007.stdout:5/530: creat d1/d7/d16/d1b/fc6 x:0 0 0 2024-03-16T09:01:14.293 INFO:tasks.workunit.client.0.smithi007.stdout:0/590: unlink d3/d9/da/d18/d27/d34/d3a/f4f 0 2024-03-16T09:01:14.293 INFO:tasks.workunit.client.0.smithi007.stdout:0/591: chown d3/d9/da/d92/d99 9575531 1 2024-03-16T09:01:14.294 INFO:tasks.workunit.client.0.smithi007.stdout:5/531: link d1/d7/d1c/d19/d34/d1d/f5a d1/d7/d1c/d19/d34/d1d/da8/fc7 0 2024-03-16T09:01:14.297 INFO:tasks.workunit.client.0.smithi007.stdout:5/532: unlink d1/d7/d1c/d88/da2/d89/db3/c39 0 2024-03-16T09:01:14.298 INFO:tasks.workunit.client.0.smithi007.stdout:2/630: dwrite dc/df/d24/d28/d4c/f59 [0,4194304] 0 2024-03-16T09:01:14.298 INFO:tasks.workunit.client.0.smithi007.stdout:2/631: chown dc/df/d24/d28/l31 0 1 2024-03-16T09:01:14.298 INFO:tasks.workunit.client.0.smithi007.stdout:2/632: fsync dc/fab 0 2024-03-16T09:01:14.298 INFO:tasks.workunit.client.0.smithi007.stdout:7/435: dwrite d0/d19/d1f/d76/f88 [0,4194304] 0 2024-03-16T09:01:14.298 INFO:tasks.workunit.client.0.smithi007.stdout:7/436: write d0/d6c/d25/d73/f58 [147631,55292] 0 2024-03-16T09:01:14.299 INFO:tasks.workunit.client.0.smithi007.stdout:2/633: symlink dc/df/d24/d28/lcf 0 2024-03-16T09:01:14.300 INFO:tasks.workunit.client.0.smithi007.stdout:6/413: dwrite d2/d10/d12/d15/f28 [0,4194304] 0 2024-03-16T09:01:14.300 INFO:tasks.workunit.client.0.smithi007.stdout:6/414: write d2/d10/f70 [1032399,29367] 0 2024-03-16T09:01:14.301 INFO:tasks.workunit.client.0.smithi007.stdout:7/437: mknod d0/d19/d1f/d76/c94 0 2024-03-16T09:01:14.302 INFO:tasks.workunit.client.0.smithi007.stdout:0/592: write d3/d9/da/d18/d23/d36/f86 [1416963,9739] 0 2024-03-16T09:01:14.302 INFO:tasks.workunit.client.0.smithi007.stdout:0/593: write d3/d9/da/d18/d23/f78 [114744,48933] 0 2024-03-16T09:01:14.302 INFO:tasks.workunit.client.0.smithi007.stdout:8/801: dwrite d0/d26/d2d/d64/f7f [0,4194304] 0 2024-03-16T09:01:14.305 INFO:tasks.workunit.client.0.smithi007.stdout:0/594: mknod d3/d9/db4/d2c/d5b/cc6 0 2024-03-16T09:01:14.305 INFO:tasks.workunit.client.0.smithi007.stdout:0/595: write d3/d9/da/d48/fa9 [813386,41197] 0 2024-03-16T09:01:14.305 INFO:tasks.workunit.client.0.smithi007.stdout:0/596: fdatasync d3/d9/f93 0 2024-03-16T09:01:14.306 INFO:tasks.workunit.client.0.smithi007.stdout:5/533: dread d1/f2 [0,4194304] 0 2024-03-16T09:01:14.306 INFO:tasks.workunit.client.0.smithi007.stdout:5/534: readlink d1/d7/d1c/l97 0 2024-03-16T09:01:14.306 INFO:tasks.workunit.client.0.smithi007.stdout:5/535: dread - d1/d7/d1c/d19/d34/d1d/da8/fba zero size 2024-03-16T09:01:14.307 INFO:tasks.workunit.client.0.smithi007.stdout:5/536: truncate d1/d7/d1c/d19/d34/d4d/f86 183615 0 2024-03-16T09:01:14.307 INFO:tasks.workunit.client.0.smithi007.stdout:8/802: rename d0/d26/d2d/d43/d4c/dba/lcb to d0/dc/d50/d8e/dd2/dea/d18/da7/de2/l11d 0 2024-03-16T09:01:14.307 INFO:tasks.workunit.client.0.smithi007.stdout:7/438: dread d0/d19/f6a [0,4194304] 0 2024-03-16T09:01:14.307 INFO:tasks.workunit.client.0.smithi007.stdout:8/803: dread - d0/dc/d50/d8e/dd2/dea/d18/da7/de2/ff9 zero size 2024-03-16T09:01:14.307 INFO:tasks.workunit.client.0.smithi007.stdout:0/597: unlink d3/d9/db4/d1c/c85 0 2024-03-16T09:01:14.307 INFO:tasks.workunit.client.0.smithi007.stdout:0/598: chown d3/d9/da/d48/d25/d87/dbb 0 1 2024-03-16T09:01:14.308 INFO:tasks.workunit.client.0.smithi007.stdout:5/537: mknod d1/d7/d1c/d88/da2/d89/cc8 0 2024-03-16T09:01:14.308 INFO:tasks.workunit.client.0.smithi007.stdout:5/538: chown d1/d7/d1c/d19/d34/d5e/d6f/da9 2705 1 2024-03-16T09:01:14.309 INFO:tasks.workunit.client.0.smithi007.stdout:8/804: mkdir d0/dc/d75/d11e 0 2024-03-16T09:01:14.309 INFO:tasks.workunit.client.0.smithi007.stdout:2/634: write dc/df/f29 [2195145,77208] 0 2024-03-16T09:01:14.309 INFO:tasks.workunit.client.0.smithi007.stdout:0/599: link d3/f22 d3/d9/da/d92/d9f/fc7 0 2024-03-16T09:01:14.310 INFO:tasks.workunit.client.0.smithi007.stdout:5/539: creat d1/d7/d1c/d19/d34/d3e/d41/da6/fc9 x:0 0 0 2024-03-16T09:01:14.310 INFO:tasks.workunit.client.0.smithi007.stdout:5/540: readlink d1/d7/d16/d1b/d3c/l78 0 2024-03-16T09:01:14.311 INFO:tasks.workunit.client.0.smithi007.stdout:8/805: symlink d0/d26/dfe/l11f 0 2024-03-16T09:01:14.311 INFO:tasks.workunit.client.0.smithi007.stdout:2/635: chown dc/df/l1d 25780724 1 2024-03-16T09:01:14.312 INFO:tasks.workunit.client.0.smithi007.stdout:6/415: dread d2/d13/d1f/d4e/f52 [0,4194304] 0 2024-03-16T09:01:14.312 INFO:tasks.workunit.client.0.smithi007.stdout:6/416: dread - d2/d10/d12/f60 zero size 2024-03-16T09:01:14.312 INFO:tasks.workunit.client.0.smithi007.stdout:5/541: mknod d1/d7/d1c/d88/da2/d89/d9b/cca 0 2024-03-16T09:01:14.313 INFO:tasks.workunit.client.1.smithi019.stdout:2/342: dwrite d5/d6/d2b/d2f/f67 [0,4194304] 0 2024-03-16T09:01:14.313 INFO:tasks.workunit.client.0.smithi007.stdout:9/573: dwrite d24/d4a/d55/d56/d7c/f98 [0,4194304] 0 2024-03-16T09:01:14.313 INFO:tasks.workunit.client.0.smithi007.stdout:9/574: write d24/d4a/d55/d56/d7c/f88 [1047253,32088] 0 2024-03-16T09:01:14.314 INFO:tasks.workunit.client.0.smithi007.stdout:8/806: rename d0/dc/d50/d8e/dd2/dea/d18/f48 to d0/d26/d2d/f120 0 2024-03-16T09:01:14.315 INFO:tasks.workunit.client.0.smithi007.stdout:8/807: fsync d0/d26/f5b 0 2024-03-16T09:01:14.315 INFO:tasks.workunit.client.0.smithi007.stdout:8/808: chown d0/d26/d95 241 1 2024-03-16T09:01:14.315 INFO:tasks.workunit.client.0.smithi007.stdout:8/809: write d0/fff [682301,71535] 0 2024-03-16T09:01:14.315 INFO:tasks.workunit.client.0.smithi007.stdout:7/439: dread d0/d19/f26 [4194304,4194304] 0 2024-03-16T09:01:14.315 INFO:tasks.workunit.client.0.smithi007.stdout:7/440: chown d0/d19/d1f/d5d/d8a 7729292 1 2024-03-16T09:01:14.315 INFO:tasks.workunit.client.0.smithi007.stdout:2/636: mknod dc/df/d24/d28/d4c/dbc/cd0 0 2024-03-16T09:01:14.318 INFO:tasks.workunit.client.0.smithi007.stdout:4/434: dwrite d4/d14/f19 [0,4194304] 0 2024-03-16T09:01:14.318 INFO:tasks.workunit.client.0.smithi007.stdout:4/435: fsync d4/d14/d1c/f6a 0 2024-03-16T09:01:14.318 INFO:tasks.workunit.client.0.smithi007.stdout:6/417: creat d2/d13/d3f/d50/f89 x:0 0 0 2024-03-16T09:01:14.319 INFO:tasks.workunit.client.0.smithi007.stdout:5/542: link d1/d7/d16/d1b/l3f d1/d7/d1c/d88/da2/d89/db3/lcb 0 2024-03-16T09:01:14.319 INFO:tasks.workunit.client.0.smithi007.stdout:5/543: fdatasync d1/d7/d1c/d88/da2/d89/db3/f6e 0 2024-03-16T09:01:14.319 INFO:tasks.workunit.client.0.smithi007.stdout:9/575: fdatasync f1e 0 2024-03-16T09:01:14.319 INFO:tasks.workunit.client.0.smithi007.stdout:0/600: write d3/d9/da/d48/d25/d43/d1a/d58/d61/f8f [735671,104761] 0 2024-03-16T09:01:14.320 INFO:tasks.workunit.client.1.smithi019.stdout:5/283: dwrite d6/d19/d34/d3e/f45 [0,4194304] 0 2024-03-16T09:01:14.320 INFO:tasks.workunit.client.1.smithi019.stdout:5/284: creat d6/d1f/d33/f52 x:0 0 0 2024-03-16T09:01:14.320 INFO:tasks.workunit.client.1.smithi019.stdout:5/285: chown d6/d19/l28 20727 1 2024-03-16T09:01:14.320 INFO:tasks.workunit.client.1.smithi019.stdout:1/228: sync 2024-03-16T09:01:14.320 INFO:tasks.workunit.client.0.smithi007.stdout:7/441: dread d0/d19/d1f/d42/f45 [0,4194304] 0 2024-03-16T09:01:14.324 INFO:tasks.workunit.client.0.smithi007.stdout:8/810: rename d0/d26/f5b to d0/d26/d2d/f121 0 2024-03-16T09:01:14.325 INFO:tasks.workunit.client.1.smithi019.stdout:5/286: rename d6/d1f/d33/f46 to d6/dd/d37/f53 0 2024-03-16T09:01:14.325 INFO:tasks.workunit.client.1.smithi019.stdout:2/343: dread d5/d2e/d4f/f5e [0,4194304] 0 2024-03-16T09:01:14.327 INFO:tasks.workunit.client.1.smithi019.stdout:1/229: mkdir d16/d1f/d43 0 2024-03-16T09:01:14.327 INFO:tasks.workunit.client.1.smithi019.stdout:8/335: sync 2024-03-16T09:01:14.327 INFO:tasks.workunit.client.1.smithi019.stdout:6/422: sync 2024-03-16T09:01:14.327 INFO:tasks.workunit.client.1.smithi019.stdout:7/214: sync 2024-03-16T09:01:14.328 INFO:tasks.workunit.client.1.smithi019.stdout:6/423: chown de/l69 7 1 2024-03-16T09:01:14.328 INFO:tasks.workunit.client.1.smithi019.stdout:9/222: sync 2024-03-16T09:01:14.328 INFO:tasks.workunit.client.1.smithi019.stdout:9/223: stat d1/d7/de/d24/f39 0 2024-03-16T09:01:14.328 INFO:tasks.workunit.client.1.smithi019.stdout:4/320: sync 2024-03-16T09:01:14.328 INFO:tasks.workunit.client.1.smithi019.stdout:2/344: symlink d5/d6/d2b/l70 0 2024-03-16T09:01:14.328 INFO:tasks.workunit.client.0.smithi007.stdout:5/544: mkdir d1/d7/d1c/d19/dcc 0 2024-03-16T09:01:14.329 INFO:tasks.workunit.client.0.smithi007.stdout:9/576: rmdir d24/d4a/d55/d56/d7c/d7f/d80/d87 39 2024-03-16T09:01:14.329 INFO:tasks.workunit.client.0.smithi007.stdout:9/577: fsync d24/d4c/f7a 0 2024-03-16T09:01:14.330 INFO:tasks.workunit.client.0.smithi007.stdout:9/578: chown d24/d4a/d55/d56/fa3 0 1 2024-03-16T09:01:14.330 INFO:tasks.workunit.client.1.smithi019.stdout:7/215: unlink d9/db/d12/f20 0 2024-03-16T09:01:14.331 INFO:tasks.workunit.client.1.smithi019.stdout:6/424: creat de/d26/d32/f8a x:0 0 0 2024-03-16T09:01:14.331 INFO:tasks.workunit.client.1.smithi019.stdout:6/425: dread - de/d1c/d1d/d66/f74 zero size 2024-03-16T09:01:14.332 INFO:tasks.workunit.client.1.smithi019.stdout:6/426: stat de/d26/d32/f8a 0 2024-03-16T09:01:14.334 INFO:tasks.workunit.client.0.smithi007.stdout:0/601: symlink d3/d9/da/d18/d23/lc8 0 2024-03-16T09:01:14.335 INFO:tasks.workunit.client.1.smithi019.stdout:4/321: symlink d4/l6f 0 2024-03-16T09:01:14.335 INFO:tasks.workunit.client.1.smithi019.stdout:2/345: creat d5/d6/d2b/d53/f71 x:0 0 0 2024-03-16T09:01:14.335 INFO:tasks.workunit.client.1.smithi019.stdout:2/346: dread - d5/d6/d2b/d2f/f3a zero size 2024-03-16T09:01:14.335 INFO:tasks.workunit.client.1.smithi019.stdout:2/347: write d5/d6/d2b/d2f/d35/d4b/f5c [811261,128515] 0 2024-03-16T09:01:14.336 INFO:tasks.workunit.client.1.smithi019.stdout:6/427: mknod de/d1c/d1d/d51/c8b 0 2024-03-16T09:01:14.337 INFO:tasks.workunit.client.1.smithi019.stdout:2/348: rmdir d5/d5a 39 2024-03-16T09:01:14.338 INFO:tasks.workunit.client.0.smithi007.stdout:7/442: symlink d0/d19/d1f/l95 0 2024-03-16T09:01:14.339 INFO:tasks.workunit.client.1.smithi019.stdout:6/428: mknod de/d26/d32/d4e/d57/c8c 0 2024-03-16T09:01:14.339 INFO:tasks.workunit.client.1.smithi019.stdout:6/429: chown de/d1c/d1d/l3f 3069 1 2024-03-16T09:01:14.339 INFO:tasks.workunit.client.0.smithi007.stdout:7/443: creat d0/d6c/d25/d3a/f96 x:0 0 0 2024-03-16T09:01:14.340 INFO:tasks.workunit.client.1.smithi019.stdout:2/349: link d5/d6/d2b/f6d d5/d6/d2b/f72 0 2024-03-16T09:01:14.341 INFO:tasks.workunit.client.1.smithi019.stdout:6/430: creat de/d1c/f8d x:0 0 0 2024-03-16T09:01:14.342 INFO:tasks.workunit.client.1.smithi019.stdout:2/350: read d5/d6/d2b/d2f/d35/f56 [191640,39514] 0 2024-03-16T09:01:14.342 INFO:tasks.workunit.client.1.smithi019.stdout:2/351: write d5/d6/d2b/f72 [969816,36452] 0 2024-03-16T09:01:14.342 INFO:tasks.workunit.client.1.smithi019.stdout:2/352: readlink d5/d6/l26 0 2024-03-16T09:01:14.342 INFO:tasks.workunit.client.1.smithi019.stdout:2/353: creat d5/d6/d2b/d53/f73 x:0 0 0 2024-03-16T09:01:14.342 INFO:tasks.workunit.client.1.smithi019.stdout:2/354: dread - d5/de/f5d zero size 2024-03-16T09:01:14.343 INFO:tasks.workunit.client.1.smithi019.stdout:6/431: mknod de/d1c/d1d/d66/d72/c8e 0 2024-03-16T09:01:14.344 INFO:tasks.workunit.client.0.smithi007.stdout:5/545: link d1/d7/d1c/d19/d34/d4d/f77 d1/d7/d1c/d88/da2/fcd 0 2024-03-16T09:01:14.344 INFO:tasks.workunit.client.1.smithi019.stdout:8/336: dread d6/d2a/d55/f6e [0,4194304] 0 2024-03-16T09:01:14.347 INFO:tasks.workunit.client.1.smithi019.stdout:6/432: rename de/d1c/l3e to de/d1c/d1d/d66/d72/d7f/l8f 0 2024-03-16T09:01:14.348 INFO:tasks.workunit.client.0.smithi007.stdout:8/811: link d0/dc/d50/d8e/dd2/dea/f5d d0/dc/d50/d8e/dd2/dea/f122 0 2024-03-16T09:01:14.348 INFO:tasks.workunit.client.1.smithi019.stdout:8/337: symlink d6/d2a/d45/d19/l71 0 2024-03-16T09:01:14.350 INFO:tasks.workunit.client.0.smithi007.stdout:6/418: dread d2/d13/d1f/d4e/f5b [0,4194304] 0 2024-03-16T09:01:14.350 INFO:tasks.workunit.client.0.smithi007.stdout:8/812: mkdir d0/d26/d2d/d64/d123 0 2024-03-16T09:01:14.350 INFO:tasks.workunit.client.0.smithi007.stdout:8/813: fsync d0/dc/d50/d8e/dd2/dea/d18/f52 0 2024-03-16T09:01:14.350 INFO:tasks.workunit.client.0.smithi007.stdout:7/444: dread d0/d19/d1f/d42/f45 [0,4194304] 0 2024-03-16T09:01:14.352 INFO:tasks.workunit.client.1.smithi019.stdout:6/433: link de/d1c/d1d/f46 de/d1c/d1d/d66/f90 0 2024-03-16T09:01:14.353 INFO:tasks.workunit.client.0.smithi007.stdout:6/419: rename d2/d11/d1e/c42 to d2/d13/d1f/c8a 0 2024-03-16T09:01:14.353 INFO:tasks.workunit.client.1.smithi019.stdout:7/216: dread f4 [0,4194304] 0 2024-03-16T09:01:14.353 INFO:tasks.workunit.client.1.smithi019.stdout:8/338: mkdir d6/d2a/d45/d57/d72 0 2024-03-16T09:01:14.353 INFO:tasks.workunit.client.1.smithi019.stdout:8/339: stat d6/d2a/d45/d2c/l3e 0 2024-03-16T09:01:14.354 INFO:tasks.workunit.client.0.smithi007.stdout:5/546: dread d1/d7/d1c/d19/d34/d3e/d41/d48/f75 [0,4194304] 0 2024-03-16T09:01:14.354 INFO:tasks.workunit.client.0.smithi007.stdout:5/547: write d1/d7/d1c/d19/d34/d5e/d6f/f51 [129070,116828] 0 2024-03-16T09:01:14.354 INFO:tasks.workunit.client.0.smithi007.stdout:5/548: chown d1/d7/d1c/d88/da2/dae 164493875 1 2024-03-16T09:01:14.354 INFO:tasks.workunit.client.0.smithi007.stdout:2/637: dwrite dc/d27/f8e [0,4194304] 0 2024-03-16T09:01:14.354 INFO:tasks.workunit.client.0.smithi007.stdout:2/638: creat dc/d27/fd1 x:0 0 0 2024-03-16T09:01:14.355 INFO:tasks.workunit.client.1.smithi019.stdout:6/434: creat de/d1c/d1d/d51/d70/f91 x:0 0 0 2024-03-16T09:01:14.355 INFO:tasks.workunit.client.0.smithi007.stdout:7/445: rmdir d0/d19/d1f/d76 39 2024-03-16T09:01:14.355 INFO:tasks.workunit.client.0.smithi007.stdout:7/446: rename d0/d6c/d25 to d0/d6c/d25/d97 22 2024-03-16T09:01:14.356 INFO:tasks.workunit.client.0.smithi007.stdout:7/447: chown d0/dc/c29 9 1 2024-03-16T09:01:14.356 INFO:tasks.workunit.client.0.smithi007.stdout:7/448: write d0/dc/d5a/f74 [1123863,23904] 0 2024-03-16T09:01:14.356 INFO:tasks.workunit.client.1.smithi019.stdout:7/217: mkdir d9/da/d13/d18/d30/d3e 0 2024-03-16T09:01:14.356 INFO:tasks.workunit.client.1.smithi019.stdout:8/340: mknod d6/d2a/d55/c73 0 2024-03-16T09:01:14.356 INFO:tasks.workunit.client.1.smithi019.stdout:8/341: dread - d6/d2a/d3c/f56 zero size 2024-03-16T09:01:14.357 INFO:tasks.workunit.client.0.smithi007.stdout:6/420: symlink d2/d13/d3f/d50/d78/d81/l8b 0 2024-03-16T09:01:14.357 INFO:tasks.workunit.client.0.smithi007.stdout:6/421: truncate d2/d10/d12/f60 157949 0 2024-03-16T09:01:14.357 INFO:tasks.workunit.client.0.smithi007.stdout:6/422: write d2/d13/d24/f38 [328319,58202] 0 2024-03-16T09:01:14.358 INFO:tasks.workunit.client.1.smithi019.stdout:8/342: dread d6/d2a/d3c/f46 [0,4194304] 0 2024-03-16T09:01:14.358 INFO:tasks.workunit.client.1.smithi019.stdout:6/435: truncate de/f3c 871417 0 2024-03-16T09:01:14.358 INFO:tasks.workunit.client.1.smithi019.stdout:6/436: chown de/d26/f5c 1689 1 2024-03-16T09:01:14.358 INFO:tasks.workunit.client.1.smithi019.stdout:7/218: link d9/fc d9/f3f 0 2024-03-16T09:01:14.359 INFO:tasks.workunit.client.1.smithi019.stdout:7/219: fdatasync d9/da/d13/d18/d1a/f2b 0 2024-03-16T09:01:14.359 INFO:tasks.workunit.client.1.smithi019.stdout:7/220: write d9/d25/d28/f34 [391689,93955] 0 2024-03-16T09:01:14.359 INFO:tasks.workunit.client.1.smithi019.stdout:7/221: symlink d9/da/d13/d18/l40 0 2024-03-16T09:01:14.360 INFO:tasks.workunit.client.0.smithi007.stdout:5/549: truncate f0 167205 0 2024-03-16T09:01:14.363 INFO:tasks.workunit.client.0.smithi007.stdout:2/639: truncate dc/df/d24/d28/d97/da4/fc4 1191420 0 2024-03-16T09:01:14.363 INFO:tasks.workunit.client.0.smithi007.stdout:2/640: fsync dc/f7e 0 2024-03-16T09:01:14.363 INFO:tasks.workunit.client.0.smithi007.stdout:2/641: truncate dc/df/d24/d28/d2e/d4a/f9c 628700 0 2024-03-16T09:01:14.363 INFO:tasks.workunit.client.0.smithi007.stdout:2/642: read dc/df/d6c/fa9 [1744954,62060] 0 2024-03-16T09:01:14.364 INFO:tasks.workunit.client.0.smithi007.stdout:7/449: creat d0/d19/d1f/d42/f98 x:0 0 0 2024-03-16T09:01:14.365 INFO:tasks.workunit.client.0.smithi007.stdout:6/423: link d2/d13/d3f/d50/f89 d2/d13/d3f/d50/d78/d81/f8c 0 2024-03-16T09:01:14.365 INFO:tasks.workunit.client.0.smithi007.stdout:6/424: stat d2/d10/d12/d15/f28 0 2024-03-16T09:01:14.365 INFO:tasks.workunit.client.0.smithi007.stdout:6/425: chown d2/d13/d24/c7f 550428 1 2024-03-16T09:01:14.366 INFO:tasks.workunit.client.0.smithi007.stdout:5/550: symlink d1/d7/d1c/d19/lce 0 2024-03-16T09:01:14.366 INFO:tasks.workunit.client.0.smithi007.stdout:5/551: write d1/d7/d1c/d19/d34/de/f49 [223984,79159] 0 2024-03-16T09:01:14.367 INFO:tasks.workunit.client.0.smithi007.stdout:0/602: fdatasync d3/d9/da/d18/d23/f78 0 2024-03-16T09:01:14.367 INFO:tasks.workunit.client.0.smithi007.stdout:0/603: readlink d3/d9/da/d48/d25/d87/l3b 0 2024-03-16T09:01:14.368 INFO:tasks.workunit.client.0.smithi007.stdout:0/604: truncate d3/d1f/fb7 891976 0 2024-03-16T09:01:14.368 INFO:tasks.workunit.client.0.smithi007.stdout:7/450: mknod d0/dc/d5a/c99 0 2024-03-16T09:01:14.368 INFO:tasks.workunit.client.0.smithi007.stdout:7/451: write d0/dc/d5a/f89 [1005441,102267] 0 2024-03-16T09:01:14.368 INFO:tasks.workunit.client.0.smithi007.stdout:6/426: symlink d2/d13/d3f/l8d 0 2024-03-16T09:01:14.370 INFO:tasks.workunit.client.0.smithi007.stdout:5/552: unlink d1/d7/d1c/d88/da2/d89/db3/la1 0 2024-03-16T09:01:14.370 INFO:tasks.workunit.client.0.smithi007.stdout:5/553: fdatasync d1/d7/d1c/d19/d34/d3e/f6a 0 2024-03-16T09:01:14.373 INFO:tasks.workunit.client.0.smithi007.stdout:7/452: mkdir d0/d6c/d25/d7a/d9a 0 2024-03-16T09:01:14.375 INFO:tasks.workunit.client.1.smithi019.stdout:7/222: read f6 [2848222,122219] 0 2024-03-16T09:01:14.376 INFO:tasks.workunit.client.0.smithi007.stdout:0/605: read d3/d9/da/d18/d23/f45 [549854,65021] 0 2024-03-16T09:01:14.377 INFO:tasks.workunit.client.0.smithi007.stdout:5/554: symlink d1/d7/d1c/d88/da2/d89/db3/d2a/d79/lcf 0 2024-03-16T09:01:14.377 INFO:tasks.workunit.client.1.smithi019.stdout:7/223: creat d9/da/d13/d18/d1a/d32/f41 x:0 0 0 2024-03-16T09:01:14.378 INFO:tasks.workunit.client.1.smithi019.stdout:7/224: creat d9/d25/f42 x:0 0 0 2024-03-16T09:01:14.378 INFO:tasks.workunit.client.1.smithi019.stdout:1/230: dwrite d16/d1f/f30 [0,4194304] 0 2024-03-16T09:01:14.379 INFO:tasks.workunit.client.1.smithi019.stdout:1/231: chown d16/l2c 184769263 1 2024-03-16T09:01:14.379 INFO:tasks.workunit.client.1.smithi019.stdout:1/232: stat d16/d31/l3f 0 2024-03-16T09:01:14.380 INFO:tasks.workunit.client.0.smithi007.stdout:0/606: symlink d3/d9/da/d18/d27/lc9 0 2024-03-16T09:01:14.381 INFO:tasks.workunit.client.1.smithi019.stdout:4/322: dwrite d4/d6c/f6d [0,4194304] 0 2024-03-16T09:01:14.381 INFO:tasks.workunit.client.0.smithi007.stdout:5/555: truncate d1/d7/d1c/d88/da2/d89/db3/f3d 3868099 0 2024-03-16T09:01:14.382 INFO:tasks.workunit.client.1.smithi019.stdout:5/287: dwrite d6/dd/d37/f53 [0,4194304] 0 2024-03-16T09:01:14.382 INFO:tasks.workunit.client.1.smithi019.stdout:6/437: dwrite de/d26/d32/f4f [0,4194304] 0 2024-03-16T09:01:14.383 INFO:tasks.workunit.client.0.smithi007.stdout:5/556: write d1/d7/d1c/d88/da2/f45 [813814,126589] 0 2024-03-16T09:01:14.384 INFO:tasks.workunit.client.0.smithi007.stdout:5/557: read - d1/d7/d1c/d19/d34/fa4 zero size 2024-03-16T09:01:14.385 INFO:tasks.workunit.client.1.smithi019.stdout:4/323: link d4/d6/d15/f33 d4/d6/f70 0 2024-03-16T09:01:14.385 INFO:tasks.workunit.client.1.smithi019.stdout:4/324: creat d4/d6/d15/d1f/d2c/f71 x:0 0 0 2024-03-16T09:01:14.386 INFO:tasks.workunit.client.0.smithi007.stdout:0/607: mkdir d3/d1f/dca 0 2024-03-16T09:01:14.386 INFO:tasks.workunit.client.1.smithi019.stdout:4/325: write d4/d6/d15/f60 [5805087,65632] 0 2024-03-16T09:01:14.386 INFO:tasks.workunit.client.1.smithi019.stdout:4/326: creat d4/d1b/d49/f72 x:0 0 0 2024-03-16T09:01:14.387 INFO:tasks.workunit.client.0.smithi007.stdout:4/436: dwrite d4/d14/d64/d68/f56 [0,4194304] 0 2024-03-16T09:01:14.389 INFO:tasks.workunit.client.1.smithi019.stdout:6/438: unlink de/d1c/f58 0 2024-03-16T09:01:14.389 INFO:tasks.workunit.client.1.smithi019.stdout:6/439: chown de/d26/d32/d4e/d75 1138806 1 2024-03-16T09:01:14.393 INFO:tasks.workunit.client.0.smithi007.stdout:4/437: dread d4/d14/f1b [0,4194304] 0 2024-03-16T09:01:14.397 INFO:tasks.workunit.client.0.smithi007.stdout:5/558: mknod d1/d7/d1c/d88/da2/d89/db3/d2a/cd0 0 2024-03-16T09:01:14.401 INFO:tasks.workunit.client.0.smithi007.stdout:0/608: symlink d3/d9/db4/d2c/d5b/lcb 0 2024-03-16T09:01:14.406 INFO:tasks.workunit.client.1.smithi019.stdout:4/327: mkdir d4/d73 0 2024-03-16T09:01:14.409 INFO:tasks.workunit.client.1.smithi019.stdout:6/440: unlink de/d1c/d1d/d51/f7d 0 2024-03-16T09:01:14.411 INFO:tasks.workunit.client.1.smithi019.stdout:6/441: mknod de/d26/d32/c92 0 2024-03-16T09:01:14.411 INFO:tasks.workunit.client.1.smithi019.stdout:2/355: truncate d5/d6/d2b/f6d 818305 0 2024-03-16T09:01:14.413 INFO:tasks.workunit.client.1.smithi019.stdout:6/442: dread de/d26/f85 [0,4194304] 0 2024-03-16T09:01:14.413 INFO:tasks.workunit.client.1.smithi019.stdout:2/356: mkdir d5/d4e/d74 0 2024-03-16T09:01:14.413 INFO:tasks.workunit.client.1.smithi019.stdout:2/357: creat d5/d2e/f75 x:0 0 0 2024-03-16T09:01:14.413 INFO:tasks.workunit.client.1.smithi019.stdout:2/358: write d5/d6/d2b/d53/f71 [224012,43849] 0 2024-03-16T09:01:14.414 INFO:tasks.workunit.client.0.smithi007.stdout:9/579: dwrite d24/d4a/d55/d56/d7c/f88 [0,4194304] 0 2024-03-16T09:01:14.416 INFO:tasks.workunit.client.1.smithi019.stdout:2/359: mknod d5/d4e/c76 0 2024-03-16T09:01:14.420 INFO:tasks.workunit.client.1.smithi019.stdout:1/233: dwrite d16/d18/f24 [0,4194304] 0 2024-03-16T09:01:14.420 INFO:tasks.workunit.client.1.smithi019.stdout:5/288: dwrite d6/dd/f38 [0,4194304] 0 2024-03-16T09:01:14.423 INFO:tasks.workunit.client.1.smithi019.stdout:1/234: mkdir d16/d1f/d43/d44 0 2024-03-16T09:01:14.425 INFO:tasks.workunit.client.1.smithi019.stdout:2/360: dread d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:14.425 INFO:tasks.workunit.client.1.smithi019.stdout:1/235: creat d16/d31/f45 x:0 0 0 2024-03-16T09:01:14.428 INFO:tasks.workunit.client.1.smithi019.stdout:1/236: unlink d16/d31/f33 0 2024-03-16T09:01:14.428 INFO:tasks.workunit.client.1.smithi019.stdout:1/237: truncate d16/d31/f45 255565 0 2024-03-16T09:01:14.428 INFO:tasks.workunit.client.1.smithi019.stdout:1/238: readlink d16/l21 0 2024-03-16T09:01:14.428 INFO:tasks.workunit.client.1.smithi019.stdout:1/239: chown d16/d18/c2d 30311430 1 2024-03-16T09:01:14.428 INFO:tasks.workunit.client.1.smithi019.stdout:1/240: dread - d16/f20 zero size 2024-03-16T09:01:14.428 INFO:tasks.workunit.client.1.smithi019.stdout:2/361: dread d5/de/f32 [0,4194304] 0 2024-03-16T09:01:14.429 INFO:tasks.workunit.client.1.smithi019.stdout:1/241: mknod d16/d18/c46 0 2024-03-16T09:01:14.429 INFO:tasks.workunit.client.1.smithi019.stdout:1/242: chown d16/c1d 328033413 1 2024-03-16T09:01:14.429 INFO:tasks.workunit.client.0.smithi007.stdout:4/438: symlink d4/d7/d6b/l92 0 2024-03-16T09:01:14.430 INFO:tasks.workunit.client.1.smithi019.stdout:1/243: creat d16/d1f/d43/f47 x:0 0 0 2024-03-16T09:01:14.431 INFO:tasks.workunit.client.0.smithi007.stdout:5/559: dwrite d1/d7/d1c/d19/d34/d3e/f43 [0,4194304] 0 2024-03-16T09:01:14.432 INFO:tasks.workunit.client.1.smithi019.stdout:1/244: rmdir d16/d1f/d43/d44 0 2024-03-16T09:01:14.432 INFO:tasks.workunit.client.1.smithi019.stdout:1/245: fdatasync f12 0 2024-03-16T09:01:14.432 INFO:tasks.workunit.client.1.smithi019.stdout:1/246: fsync d16/f3d 0 2024-03-16T09:01:14.432 INFO:tasks.workunit.client.0.smithi007.stdout:6/427: rename d2/d13/d1f to d2/d10/d12/d8e 0 2024-03-16T09:01:14.433 INFO:tasks.workunit.client.1.smithi019.stdout:5/289: dread d6/d19/f2e [0,4194304] 0 2024-03-16T09:01:14.434 INFO:tasks.workunit.client.1.smithi019.stdout:1/247: creat d16/d1f/d3c/f48 x:0 0 0 2024-03-16T09:01:14.434 INFO:tasks.workunit.client.1.smithi019.stdout:1/248: fsync d16/d18/f23 0 2024-03-16T09:01:14.434 INFO:tasks.workunit.client.0.smithi007.stdout:0/609: dwrite d3/d9/da/d48/d25/d43/d1a/d58/d61/f7a [0,4194304] 0 2024-03-16T09:01:14.435 INFO:tasks.workunit.client.1.smithi019.stdout:4/328: dwrite d4/d1b/d49/f56 [4194304,4194304] 0 2024-03-16T09:01:14.435 INFO:tasks.workunit.client.1.smithi019.stdout:5/290: mkdir d6/dd/d37/d54 0 2024-03-16T09:01:14.435 INFO:tasks.workunit.client.1.smithi019.stdout:5/291: creat d6/d1f/f55 x:0 0 0 2024-03-16T09:01:14.435 INFO:tasks.workunit.client.1.smithi019.stdout:5/292: chown d6/d19/l2a 31148 1 2024-03-16T09:01:14.435 INFO:tasks.workunit.client.1.smithi019.stdout:5/293: fdatasync d6/d19/d34/d3e/f48 0 2024-03-16T09:01:14.436 INFO:tasks.workunit.client.1.smithi019.stdout:5/294: read - d6/d1f/f55 zero size 2024-03-16T09:01:14.436 INFO:tasks.workunit.client.0.smithi007.stdout:9/580: symlink d24/d4c/la4 0 2024-03-16T09:01:14.438 INFO:tasks.workunit.client.0.smithi007.stdout:6/428: truncate d2/d11/d1e/d54/f63 2445015 0 2024-03-16T09:01:14.440 INFO:tasks.workunit.client.0.smithi007.stdout:4/439: dread d4/f22 [4194304,4194304] 0 2024-03-16T09:01:14.440 INFO:tasks.workunit.client.0.smithi007.stdout:2/643: dread dc/df/d24/d28/d97/da4/fc4 [0,4194304] 0 2024-03-16T09:01:14.440 INFO:tasks.workunit.client.0.smithi007.stdout:2/644: write dc/fa5 [577978,100230] 0 2024-03-16T09:01:14.441 INFO:tasks.workunit.client.1.smithi019.stdout:1/249: unlink d16/f19 0 2024-03-16T09:01:14.444 INFO:tasks.workunit.client.1.smithi019.stdout:5/295: mkdir d6/dd/d47/d56 0 2024-03-16T09:01:14.444 INFO:tasks.workunit.client.0.smithi007.stdout:0/610: symlink d3/d9/da/d48/d25/d43/lcc 0 2024-03-16T09:01:14.445 INFO:tasks.workunit.client.0.smithi007.stdout:9/581: creat d24/d4c/d91/fa5 x:0 0 0 2024-03-16T09:01:14.447 INFO:tasks.workunit.client.1.smithi019.stdout:1/250: unlink d16/d18/c46 0 2024-03-16T09:01:14.448 INFO:tasks.workunit.client.0.smithi007.stdout:6/429: unlink d2/d10/d12/d15/f7d 0 2024-03-16T09:01:14.449 INFO:tasks.workunit.client.0.smithi007.stdout:6/430: truncate d2/d11/d1e/f2f 4340810 0 2024-03-16T09:01:14.449 INFO:tasks.workunit.client.0.smithi007.stdout:6/431: fdatasync d2/d13/d3f/d50/d72/f69 0 2024-03-16T09:01:14.449 INFO:tasks.workunit.client.0.smithi007.stdout:6/432: write d2/d10/d12/f60 [1131900,111351] 0 2024-03-16T09:01:14.449 INFO:tasks.workunit.client.1.smithi019.stdout:6/443: getdents de/d26/d32 0 2024-03-16T09:01:14.452 INFO:tasks.workunit.client.1.smithi019.stdout:6/444: creat de/d1c/d1d/f93 x:0 0 0 2024-03-16T09:01:14.453 INFO:tasks.workunit.client.0.smithi007.stdout:9/582: mkdir d24/d4a/d55/d56/da6 0 2024-03-16T09:01:14.453 INFO:tasks.workunit.client.0.smithi007.stdout:9/583: creat d24/d8e/fa7 x:0 0 0 2024-03-16T09:01:14.455 INFO:tasks.workunit.client.1.smithi019.stdout:4/329: dread d4/d1b/d2a/f4e [0,4194304] 0 2024-03-16T09:01:14.455 INFO:tasks.workunit.client.1.smithi019.stdout:4/330: chown d4/d1b/d2a/d3d/f46 3 1 2024-03-16T09:01:14.456 INFO:tasks.workunit.client.1.smithi019.stdout:6/445: rename de/d26/d32/f37 to de/d1c/d1d/d66/f94 0 2024-03-16T09:01:14.456 INFO:tasks.workunit.client.0.smithi007.stdout:4/440: dread d4/f5 [0,4194304] 0 2024-03-16T09:01:14.456 INFO:tasks.workunit.client.0.smithi007.stdout:2/645: dread dc/df/d24/d28/d4c/f59 [0,4194304] 0 2024-03-16T09:01:14.456 INFO:tasks.workunit.client.0.smithi007.stdout:0/611: rmdir d3/d9/da/d48/d25/d43/d1a 39 2024-03-16T09:01:14.457 INFO:tasks.workunit.client.0.smithi007.stdout:6/433: mkdir d2/d11/d3b/d74/d8f 0 2024-03-16T09:01:14.457 INFO:tasks.workunit.client.0.smithi007.stdout:0/612: creat d3/d1f/d70/fcd x:0 0 0 2024-03-16T09:01:14.457 INFO:tasks.workunit.client.0.smithi007.stdout:0/613: truncate d3/d9/da/d48/d4b/f51 489203 0 2024-03-16T09:01:14.457 INFO:tasks.workunit.client.0.smithi007.stdout:0/614: truncate d3/d9/da/d18/d27/f9a 570180 0 2024-03-16T09:01:14.458 INFO:tasks.workunit.client.1.smithi019.stdout:6/446: rename de/d26/d32/d4e/d57/l60 to de/d26/d32/d4e/d75/l95 0 2024-03-16T09:01:14.458 INFO:tasks.workunit.client.1.smithi019.stdout:6/447: readlink de/d26/d32/l34 0 2024-03-16T09:01:14.458 INFO:tasks.workunit.client.0.smithi007.stdout:6/434: dread d2/d13/d62/f6b [0,4194304] 0 2024-03-16T09:01:14.459 INFO:tasks.workunit.client.0.smithi007.stdout:4/441: rmdir d4/d14/d1c/d1e 39 2024-03-16T09:01:14.459 INFO:tasks.workunit.client.0.smithi007.stdout:5/560: dwrite d1/d7/d1c/d88/da2/fcd [0,4194304] 0 2024-03-16T09:01:14.460 INFO:tasks.workunit.client.1.smithi019.stdout:2/362: dwrite d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:14.460 INFO:tasks.workunit.client.1.smithi019.stdout:5/296: write d6/dd/d37/f53 [2061787,87692] 0 2024-03-16T09:01:14.461 INFO:tasks.workunit.client.1.smithi019.stdout:6/448: unlink de/d1c/d1d/d66/d72/f76 0 2024-03-16T09:01:14.462 INFO:tasks.workunit.client.0.smithi007.stdout:5/561: dread d1/d7/d16/d1b/f60 [0,4194304] 0 2024-03-16T09:01:14.463 INFO:tasks.workunit.client.0.smithi007.stdout:0/615: creat d3/d9/da/d92/fce x:0 0 0 2024-03-16T09:01:14.466 INFO:tasks.workunit.client.1.smithi019.stdout:2/363: symlink d5/d4e/l77 0 2024-03-16T09:01:14.466 INFO:tasks.workunit.client.1.smithi019.stdout:2/364: fsync d5/d2e/f3d 0 2024-03-16T09:01:14.466 INFO:tasks.workunit.client.1.smithi019.stdout:2/365: write d5/f4d [598797,63213] 0 2024-03-16T09:01:14.469 INFO:tasks.workunit.client.1.smithi019.stdout:6/449: symlink de/d26/d32/d4e/l96 0 2024-03-16T09:01:14.469 INFO:tasks.workunit.client.1.smithi019.stdout:6/450: read - de/d1c/d1d/f93 zero size 2024-03-16T09:01:14.473 INFO:tasks.workunit.client.0.smithi007.stdout:2/646: rename dc/df/l9f to dc/df/d24/d28/d2e/ld2 0 2024-03-16T09:01:14.473 INFO:tasks.workunit.client.0.smithi007.stdout:6/435: dread d2/d13/d3f/d50/f53 [0,4194304] 0 2024-03-16T09:01:14.475 INFO:tasks.workunit.client.0.smithi007.stdout:5/562: creat d1/d7/d1c/d19/d34/d5e/d6f/da9/fd1 x:0 0 0 2024-03-16T09:01:14.478 INFO:tasks.workunit.client.0.smithi007.stdout:0/616: creat d3/d9/db4/fcf x:0 0 0 2024-03-16T09:01:14.483 INFO:tasks.workunit.client.0.smithi007.stdout:2/647: rename dc/df/d24/d28/d2e/d35/c46 to dc/df/d24/d28/d2e/d4a/cd3 0 2024-03-16T09:01:14.486 INFO:tasks.workunit.client.0.smithi007.stdout:0/617: mkdir d3/d9/da/d48/d25/d87/dd0 0 2024-03-16T09:01:14.486 INFO:tasks.workunit.client.0.smithi007.stdout:0/618: fdatasync d3/d9/da/d48/f47 0 2024-03-16T09:01:14.487 INFO:tasks.workunit.client.0.smithi007.stdout:2/648: mknod dc/db7/cd4 0 2024-03-16T09:01:14.489 INFO:tasks.workunit.client.0.smithi007.stdout:5/563: getdents d1/d7/d1c/d19/d34/d3e/d41/d48 0 2024-03-16T09:01:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:2/649: creat dc/df/d24/d28/d2e/dc5/fd5 x:0 0 0 2024-03-16T09:01:14.491 INFO:tasks.workunit.client.0.smithi007.stdout:2/650: readlink dc/df/d24/d28/d2e/d6e/lae 0 2024-03-16T09:01:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:5/564: mknod d1/d7/d1c/d19/d34/d5e/d6f/da9/cd2 0 2024-03-16T09:01:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:5/565: creat d1/d7/d1c/d19/d34/d5e/d6f/d72/fd3 x:0 0 0 2024-03-16T09:01:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:4/442: truncate d4/d7/d6b/f6d 791472 0 2024-03-16T09:01:14.493 INFO:tasks.workunit.client.0.smithi007.stdout:4/443: stat d4/d14/d1c/d5b/f8e 0 2024-03-16T09:01:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:2/651: mknod dc/df/d24/d28/d4c/cd6 0 2024-03-16T09:01:14.494 INFO:tasks.workunit.client.0.smithi007.stdout:2/652: write dc/f17 [791744,4417] 0 2024-03-16T09:01:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:2/653: write dc/d27/f65 [4991399,55652] 0 2024-03-16T09:01:14.495 INFO:tasks.workunit.client.0.smithi007.stdout:5/566: rmdir d1/d7/d1c 39 2024-03-16T09:01:14.496 INFO:tasks.workunit.client.0.smithi007.stdout:5/567: write d1/d7/d1c/d19/d34/d4d/f86 [777415,38576] 0 2024-03-16T09:01:14.497 INFO:tasks.workunit.client.0.smithi007.stdout:4/444: mkdir d4/d7/d78/d80/d84/d93 0 2024-03-16T09:01:14.499 INFO:tasks.workunit.client.0.smithi007.stdout:5/568: unlink d1/d7/d1c/d19/d34/d3e/d41/f9f 0 2024-03-16T09:01:14.506 INFO:tasks.workunit.client.0.smithi007.stdout:2/654: dread dc/df/d6c/fc9 [0,4194304] 0 2024-03-16T09:01:14.508 INFO:tasks.workunit.client.0.smithi007.stdout:2/655: truncate dc/f11 82035 0 2024-03-16T09:01:14.508 INFO:tasks.workunit.client.0.smithi007.stdout:2/656: truncate dc/d27/f4f 3871113 0 2024-03-16T09:01:14.511 INFO:tasks.workunit.client.0.smithi007.stdout:2/657: rmdir dc/df/d24/d28/d2e/d35/da7 0 2024-03-16T09:01:14.511 INFO:tasks.workunit.client.0.smithi007.stdout:2/658: write dc/df/d24/d28/d2e/fc7 [948347,67744] 0 2024-03-16T09:01:14.511 INFO:tasks.workunit.client.0.smithi007.stdout:2/659: fdatasync dc/df/d24/d28/d4c/d5e/f4d 0 2024-03-16T09:01:14.512 INFO:tasks.workunit.client.0.smithi007.stdout:2/660: rmdir dc/db7/dc0 0 2024-03-16T09:01:14.514 INFO:tasks.workunit.client.0.smithi007.stdout:2/661: creat dc/df/d24/d28/d97/da4/db8/fd7 x:0 0 0 2024-03-16T09:01:14.514 INFO:tasks.workunit.client.0.smithi007.stdout:2/662: creat dc/df/d24/d28/d4c/d5e/fd8 x:0 0 0 2024-03-16T09:01:14.514 INFO:tasks.workunit.client.0.smithi007.stdout:9/584: dwrite d24/d4a/d55/d56/d7c/f98 [0,4194304] 0 2024-03-16T09:01:14.514 INFO:tasks.workunit.client.0.smithi007.stdout:9/585: readlink d24/d4c/l51 0 2024-03-16T09:01:14.517 INFO:tasks.workunit.client.0.smithi007.stdout:0/619: dread d3/d9/da/d48/d25/d43/d1a/d58/d61/f7a [0,4194304] 0 2024-03-16T09:01:14.517 INFO:tasks.workunit.client.0.smithi007.stdout:0/620: dread - d3/faf zero size 2024-03-16T09:01:14.517 INFO:tasks.workunit.client.0.smithi007.stdout:0/621: readlink d3/d9/da/d18/d23/lb5 0 2024-03-16T09:01:14.517 INFO:tasks.workunit.client.0.smithi007.stdout:0/622: write d3/d9/db4/fcf [961170,28159] 0 2024-03-16T09:01:14.518 INFO:tasks.workunit.client.0.smithi007.stdout:0/623: write d3/d9/da/d48/d25/d43/fc2 [109426,36203] 0 2024-03-16T09:01:14.519 INFO:tasks.workunit.client.0.smithi007.stdout:0/624: read d3/d9/db4/d1c/d72/f82 [448444,30717] 0 2024-03-16T09:01:14.525 INFO:tasks.workunit.client.1.smithi019.stdout:1/251: dwrite d16/f22 [0,4194304] 0 2024-03-16T09:01:14.525 INFO:tasks.workunit.client.1.smithi019.stdout:1/252: write d16/d1f/d3c/f48 [794644,55879] 0 2024-03-16T09:01:14.525 INFO:tasks.workunit.client.1.smithi019.stdout:5/297: dwrite d6/dd/f38 [0,4194304] 0 2024-03-16T09:01:14.525 INFO:tasks.workunit.client.0.smithi007.stdout:4/445: dwrite d4/d14/d64/d68/f8b [0,4194304] 0 2024-03-16T09:01:14.526 INFO:tasks.workunit.client.0.smithi007.stdout:6/436: dwrite d2/d13/d62/f7b [0,4194304] 0 2024-03-16T09:01:14.526 INFO:tasks.workunit.client.0.smithi007.stdout:6/437: creat d2/f90 x:0 0 0 2024-03-16T09:01:14.530 INFO:tasks.workunit.client.1.smithi019.stdout:6/451: dwrite de/d1c/d1d/f5a [0,4194304] 0 2024-03-16T09:01:14.530 INFO:tasks.workunit.client.1.smithi019.stdout:2/366: dwrite d5/d6/d2b/f6d [0,4194304] 0 2024-03-16T09:01:14.530 INFO:tasks.workunit.client.1.smithi019.stdout:2/367: chown d5/d6/d2b/d2f/c59 11 1 2024-03-16T09:01:14.531 INFO:tasks.workunit.client.1.smithi019.stdout:2/368: chown d5/d6/d2b/f5b 12459 1 2024-03-16T09:01:14.531 INFO:tasks.workunit.client.0.smithi007.stdout:4/446: rename d4/d14/d1c/d5b/d67/c8c to d4/d7/d78/d80/d84/c94 0 2024-03-16T09:01:14.531 INFO:tasks.workunit.client.0.smithi007.stdout:5/569: dwrite d1/d7/d1c/d19/d34/d4d/d99/fa3 [0,4194304] 0 2024-03-16T09:01:14.531 INFO:tasks.workunit.client.1.smithi019.stdout:5/298: mknod d6/dd/d47/d56/c57 0 2024-03-16T09:01:14.531 INFO:tasks.workunit.client.1.smithi019.stdout:5/299: write d6/d19/f1b [645056,995] 0 2024-03-16T09:01:14.532 INFO:tasks.workunit.client.1.smithi019.stdout:5/300: fsync d6/d1f/d33/f4b 0 2024-03-16T09:01:14.534 INFO:tasks.workunit.client.0.smithi007.stdout:6/438: getdents d2/d10/d12 0 2024-03-16T09:01:14.537 INFO:tasks.workunit.client.1.smithi019.stdout:3/388: sync 2024-03-16T09:01:14.539 INFO:tasks.workunit.client.0.smithi007.stdout:4/447: getdents d4/d14/d64 0 2024-03-16T09:01:14.539 INFO:tasks.workunit.client.0.smithi007.stdout:2/663: dwrite dc/df/d24/f63 [0,4194304] 0 2024-03-16T09:01:14.540 INFO:tasks.workunit.client.0.smithi007.stdout:2/664: creat dc/df/d24/fd9 x:0 0 0 2024-03-16T09:01:14.541 INFO:tasks.workunit.client.0.smithi007.stdout:5/570: dread d1/d7/d16/d1b/f8d [0,4194304] 0 2024-03-16T09:01:14.541 INFO:tasks.workunit.client.0.smithi007.stdout:9/586: dwrite f16 [0,4194304] 0 2024-03-16T09:01:14.541 INFO:tasks.workunit.client.0.smithi007.stdout:5/571: dread - d1/d7/d1c/d88/da2/d89/db3/f94 zero size 2024-03-16T09:01:14.542 INFO:tasks.workunit.client.0.smithi007.stdout:5/572: chown d1/d7/d1c/d88/da2/d89/db3/d2a/d79 50749707 1 2024-03-16T09:01:14.542 INFO:tasks.workunit.client.1.smithi019.stdout:2/369: mkdir d5/d4e/d78 0 2024-03-16T09:01:14.542 INFO:tasks.workunit.client.0.smithi007.stdout:4/448: creat d4/d14/f95 x:0 0 0 2024-03-16T09:01:14.542 INFO:tasks.workunit.client.0.smithi007.stdout:4/449: fdatasync d4/d14/d1c/f30 0 2024-03-16T09:01:14.543 INFO:tasks.workunit.client.1.smithi019.stdout:5/301: creat d6/dd/d37/f58 x:0 0 0 2024-03-16T09:01:14.549 INFO:tasks.workunit.client.0.smithi007.stdout:2/665: mknod dc/df/d24/d28/d4c/dbc/cda 0 2024-03-16T09:01:14.549 INFO:tasks.workunit.client.0.smithi007.stdout:2/666: chown dc/df/d24/d28/d2e/d4a/l90 114556 1 2024-03-16T09:01:14.549 INFO:tasks.workunit.client.0.smithi007.stdout:2/667: fdatasync dc/df/d24/d28/d2e/d35/f91 0 2024-03-16T09:01:14.550 INFO:tasks.workunit.client.0.smithi007.stdout:6/439: dread d2/d11/d1e/f2f [0,4194304] 0 2024-03-16T09:01:14.550 INFO:tasks.workunit.client.0.smithi007.stdout:6/440: chown d2/d10/d12/d8e/c8a 721 1 2024-03-16T09:01:14.551 INFO:tasks.workunit.client.1.smithi019.stdout:1/253: dwrite d16/d1f/d43/f47 [0,4194304] 0 2024-03-16T09:01:14.551 INFO:tasks.workunit.client.1.smithi019.stdout:1/254: write d16/d1f/f2e [1235309,40715] 0 2024-03-16T09:01:14.551 INFO:tasks.workunit.client.0.smithi007.stdout:9/587: rmdir d24/d4a/d55/d56/d7c/d7f/d80 39 2024-03-16T09:01:14.552 INFO:tasks.workunit.client.0.smithi007.stdout:9/588: fsync d24/d4c/f7a 0 2024-03-16T09:01:14.552 INFO:tasks.workunit.client.0.smithi007.stdout:9/589: chown d24/c47 48122 1 2024-03-16T09:01:14.553 INFO:tasks.workunit.client.1.smithi019.stdout:2/370: link d5/d6/d2b/c2d d5/d6/d2b/d2f/d35/d4b/c79 0 2024-03-16T09:01:14.554 INFO:tasks.workunit.client.1.smithi019.stdout:5/302: link d6/dd/f44 d6/d19/f59 0 2024-03-16T09:01:14.563 INFO:tasks.workunit.client.1.smithi019.stdout:6/452: rename de/d1c/d1d to de/d26/d32/d97 0 2024-03-16T09:01:14.563 INFO:tasks.workunit.client.1.smithi019.stdout:6/453: fsync de/f67 0 2024-03-16T09:01:14.563 INFO:tasks.workunit.client.1.smithi019.stdout:2/371: stat d5/c16 0 2024-03-16T09:01:14.564 INFO:tasks.workunit.client.1.smithi019.stdout:1/255: getdents d16/d31 0 2024-03-16T09:01:14.564 INFO:tasks.workunit.client.1.smithi019.stdout:1/256: creat d16/f49 x:0 0 0 2024-03-16T09:01:14.564 INFO:tasks.workunit.client.1.smithi019.stdout:1/257: chown f14 517 1 2024-03-16T09:01:14.564 INFO:tasks.workunit.client.1.smithi019.stdout:1/258: stat d16/d1f/d43 0 2024-03-16T09:01:14.564 INFO:tasks.workunit.client.1.smithi019.stdout:1/259: readlink l13 0 2024-03-16T09:01:14.565 INFO:tasks.workunit.client.0.smithi007.stdout:9/590: getdents d24/d4a/d55/d56/d7c/d7f 0 2024-03-16T09:01:14.565 INFO:tasks.workunit.client.1.smithi019.stdout:2/372: truncate d5/de/f4a 1593716 0 2024-03-16T09:01:14.566 INFO:tasks.workunit.client.0.smithi007.stdout:5/573: getdents d1/d7/d1c/d19/d34/d3e 0 2024-03-16T09:01:14.566 INFO:tasks.workunit.client.0.smithi007.stdout:6/441: write d2/d10/d12/d15/f28 [378619,45114] 0 2024-03-16T09:01:14.567 INFO:tasks.workunit.client.1.smithi019.stdout:1/260: symlink d16/d18/l4a 0 2024-03-16T09:01:14.567 INFO:tasks.workunit.client.0.smithi007.stdout:2/668: getdents dc/df/daa 0 2024-03-16T09:01:14.567 INFO:tasks.workunit.client.0.smithi007.stdout:2/669: chown dc/df/d24/d28/d2e/d35/f40 14056 1 2024-03-16T09:01:14.568 INFO:tasks.workunit.client.1.smithi019.stdout:1/261: mknod d16/d1f/d3c/c4b 0 2024-03-16T09:01:14.568 INFO:tasks.workunit.client.1.smithi019.stdout:1/262: read - d16/f42 zero size 2024-03-16T09:01:14.568 INFO:tasks.workunit.client.1.smithi019.stdout:1/263: creat d16/d1f/d43/f4c x:0 0 0 2024-03-16T09:01:14.569 INFO:tasks.workunit.client.1.smithi019.stdout:1/264: mknod d16/d1f/d43/c4d 0 2024-03-16T09:01:14.569 INFO:tasks.workunit.client.1.smithi019.stdout:1/265: chown d16/d31/f45 61872168 1 2024-03-16T09:01:14.570 INFO:tasks.workunit.client.0.smithi007.stdout:6/442: rename d2/d10/d12/d8e/d49/d58 to d2/d13/d62/d91 0 2024-03-16T09:01:14.570 INFO:tasks.workunit.client.1.smithi019.stdout:1/266: truncate f10 694819 0 2024-03-16T09:01:14.570 INFO:tasks.workunit.client.1.smithi019.stdout:1/267: dread - d16/f37 zero size 2024-03-16T09:01:14.572 INFO:tasks.workunit.client.0.smithi007.stdout:5/574: read d1/d7/d1c/d19/d34/d5e/f8e [207103,12380] 0 2024-03-16T09:01:14.573 INFO:tasks.workunit.client.0.smithi007.stdout:5/575: symlink d1/d7/d1c/d19/d34/d3e/d62/ld4 0 2024-03-16T09:01:14.575 INFO:tasks.workunit.client.0.smithi007.stdout:0/625: dwrite d3/d9/da/d48/d25/d43/fc2 [0,4194304] 0 2024-03-16T09:01:14.575 INFO:tasks.workunit.client.0.smithi007.stdout:0/626: dread - d3/d9/da/d48/fbe zero size 2024-03-16T09:01:14.575 INFO:tasks.workunit.client.1.smithi019.stdout:3/389: dwrite d3/f51 [0,4194304] 0 2024-03-16T09:01:14.577 INFO:tasks.workunit.client.0.smithi007.stdout:5/576: link d1/d7/f12 d1/d7/d1c/d88/da2/d89/fd5 0 2024-03-16T09:01:14.577 INFO:tasks.workunit.client.1.smithi019.stdout:5/303: dwrite d6/dd/f38 [0,4194304] 0 2024-03-16T09:01:14.578 INFO:tasks.workunit.client.1.smithi019.stdout:5/304: creat d6/d1f/f5a x:0 0 0 2024-03-16T09:01:14.578 INFO:tasks.workunit.client.1.smithi019.stdout:6/454: write de/d26/d32/f4f [3959115,53881] 0 2024-03-16T09:01:14.578 INFO:tasks.workunit.client.1.smithi019.stdout:6/455: chown de/d26/d32/d97/d2c 17545744 1 2024-03-16T09:01:14.579 INFO:tasks.workunit.client.0.smithi007.stdout:6/443: read d2/d10/d12/d8e/f4d [713348,87886] 0 2024-03-16T09:01:14.579 INFO:tasks.workunit.client.0.smithi007.stdout:6/444: creat d2/d10/d12/d15/f92 x:0 0 0 2024-03-16T09:01:14.581 INFO:tasks.workunit.client.0.smithi007.stdout:4/450: dwrite d4/d14/d1c/d5b/f62 [0,4194304] 0 2024-03-16T09:01:14.581 INFO:tasks.workunit.client.0.smithi007.stdout:4/451: truncate d4/d14/d1c/d1e/f59 141054 0 2024-03-16T09:01:14.581 INFO:tasks.workunit.client.0.smithi007.stdout:4/452: creat d4/d14/f96 x:0 0 0 2024-03-16T09:01:14.581 INFO:tasks.workunit.client.0.smithi007.stdout:4/453: readlink d4/d14/l4f 0 2024-03-16T09:01:14.581 INFO:tasks.workunit.client.0.smithi007.stdout:4/454: readlink d4/d7/l7f 0 2024-03-16T09:01:14.581 INFO:tasks.workunit.client.1.smithi019.stdout:6/456: dread de/d26/f85 [0,4194304] 0 2024-03-16T09:01:14.581 INFO:tasks.workunit.client.1.smithi019.stdout:6/457: chown de/d26/c63 4151 1 2024-03-16T09:01:14.582 INFO:tasks.workunit.client.0.smithi007.stdout:0/627: creat d3/d9/db4/d1c/d72/fd1 x:0 0 0 2024-03-16T09:01:14.582 INFO:tasks.workunit.client.0.smithi007.stdout:0/628: truncate d3/d9/da/d48/d25/d87/f98 25763 0 2024-03-16T09:01:14.592 INFO:tasks.workunit.client.1.smithi019.stdout:2/373: write d5/de/f4a [1812281,121325] 0 2024-03-16T09:01:14.592 INFO:tasks.workunit.client.1.smithi019.stdout:2/374: dread - d5/d2e/d4f/f57 zero size 2024-03-16T09:01:14.593 INFO:tasks.workunit.client.1.smithi019.stdout:2/375: mknod d5/d4e/d74/c7a 0 2024-03-16T09:01:14.593 INFO:tasks.workunit.client.1.smithi019.stdout:2/376: write d5/d6/d2b/f72 [4545724,93578] 0 2024-03-16T09:01:14.595 INFO:tasks.workunit.client.1.smithi019.stdout:2/377: mknod d5/d6/d2b/d2f/d35/d4b/d51/c7b 0 2024-03-16T09:01:14.596 INFO:tasks.workunit.client.1.smithi019.stdout:1/268: dwrite d16/f2b [0,4194304] 0 2024-03-16T09:01:14.596 INFO:tasks.workunit.client.0.smithi007.stdout:0/629: dread d3/d9/da/d18/d23/f63 [0,4194304] 0 2024-03-16T09:01:14.597 INFO:tasks.workunit.client.0.smithi007.stdout:0/630: stat d3/d9/da/d18/f7d 0 2024-03-16T09:01:14.597 INFO:tasks.workunit.client.0.smithi007.stdout:4/455: dread d4/f22 [0,4194304] 0 2024-03-16T09:01:14.602 INFO:tasks.workunit.client.1.smithi019.stdout:2/378: link d5/d6/d2b/d2f/c41 d5/d6/d2b/d2f/d35/d65/c7c 0 2024-03-16T09:01:14.604 INFO:tasks.workunit.client.1.smithi019.stdout:2/379: mkdir d5/d6/d2b/d2f/d35/d4b/d51/d7d 0 2024-03-16T09:01:14.606 INFO:tasks.workunit.client.1.smithi019.stdout:2/380: creat d5/d6/d2b/d2f/d35/f7e x:0 0 0 2024-03-16T09:01:14.612 INFO:tasks.workunit.client.1.smithi019.stdout:0/308: sync 2024-03-16T09:01:14.612 INFO:tasks.workunit.client.1.smithi019.stdout:0/309: truncate d7/d8/f12 2713438 0 2024-03-16T09:01:14.613 INFO:tasks.workunit.client.1.smithi019.stdout:0/310: mknod d7/d1d/c5c 0 2024-03-16T09:01:14.615 INFO:tasks.workunit.client.1.smithi019.stdout:0/311: creat d7/f5d x:0 0 0 2024-03-16T09:01:14.615 INFO:tasks.workunit.client.1.smithi019.stdout:0/312: write d7/f5d [1033070,88437] 0 2024-03-16T09:01:14.618 INFO:tasks.workunit.client.0.smithi007.stdout:6/445: rename d2/d10/d12 to d2/d11/d3b/d93 0 2024-03-16T09:01:14.620 INFO:tasks.workunit.client.0.smithi007.stdout:6/446: mkdir d2/d13/d4b/d94 0 2024-03-16T09:01:14.620 INFO:tasks.workunit.client.0.smithi007.stdout:6/447: read d2/d11/f21 [3881876,112048] 0 2024-03-16T09:01:14.632 INFO:tasks.workunit.client.1.smithi019.stdout:0/313: write d7/d1d/d28/f44 [1720611,68646] 0 2024-03-16T09:01:14.645 INFO:tasks.workunit.client.0.smithi007.stdout:5/577: dwrite d1/d7/d1c/d88/da2/d89/db3/d2a/d81/f90 [0,4194304] 0 2024-03-16T09:01:14.645 INFO:tasks.workunit.client.1.smithi019.stdout:3/390: dwrite d3/d1b/d40/f67 [0,4194304] 0 2024-03-16T09:01:14.646 INFO:tasks.workunit.client.1.smithi019.stdout:6/458: dwrite f7 [0,4194304] 0 2024-03-16T09:01:14.646 INFO:tasks.workunit.client.1.smithi019.stdout:0/314: write d7/d8/d22/f2c [3912754,113255] 0 2024-03-16T09:01:14.646 INFO:tasks.workunit.client.1.smithi019.stdout:5/305: dwrite d6/d1f/f55 [0,4194304] 0 2024-03-16T09:01:14.646 INFO:tasks.workunit.client.0.smithi007.stdout:4/456: dwrite d4/d14/d1c/f37 [0,4194304] 0 2024-03-16T09:01:14.646 INFO:tasks.workunit.client.0.smithi007.stdout:0/631: dwrite d3/d9/da/d18/f6a [0,4194304] 0 2024-03-16T09:01:14.647 INFO:tasks.workunit.client.0.smithi007.stdout:0/632: write d3/d9/da/d48/d25/d43/d1a/f20 [2262454,96472] 0 2024-03-16T09:01:14.647 INFO:tasks.workunit.client.0.smithi007.stdout:6/448: dwrite d2/d13/d24/f38 [0,4194304] 0 2024-03-16T09:01:14.648 INFO:tasks.workunit.client.1.smithi019.stdout:3/391: creat d3/d17/d3f/f77 x:0 0 0 2024-03-16T09:01:14.649 INFO:tasks.workunit.client.1.smithi019.stdout:6/459: unlink de/f6f 0 2024-03-16T09:01:14.651 INFO:tasks.workunit.client.0.smithi007.stdout:0/633: mknod d3/d9/da/d92/d99/cd2 0 2024-03-16T09:01:14.653 INFO:tasks.workunit.client.1.smithi019.stdout:0/315: mknod d7/d29/c5e 0 2024-03-16T09:01:14.653 INFO:tasks.workunit.client.1.smithi019.stdout:0/316: read d7/d8/f12 [682414,74710] 0 2024-03-16T09:01:14.654 INFO:tasks.workunit.client.0.smithi007.stdout:4/457: write d4/f5f [3958480,17867] 0 2024-03-16T09:01:14.655 INFO:tasks.workunit.client.0.smithi007.stdout:6/449: creat d2/d13/d62/d91/f95 x:0 0 0 2024-03-16T09:01:14.655 INFO:tasks.workunit.client.0.smithi007.stdout:6/450: read - d2/d13/d4b/f55 zero size 2024-03-16T09:01:14.656 INFO:tasks.workunit.client.1.smithi019.stdout:3/392: dread d3/d1b/d40/d4a/f41 [0,4194304] 0 2024-03-16T09:01:14.656 INFO:tasks.workunit.client.0.smithi007.stdout:0/634: symlink d3/d9/da/d48/d25/d87/dbb/ld3 0 2024-03-16T09:01:14.656 INFO:tasks.workunit.client.0.smithi007.stdout:0/635: readlink d3/d9/da/d48/d25/d43/l2e 0 2024-03-16T09:01:14.657 INFO:tasks.workunit.client.0.smithi007.stdout:0/636: read d3/d1f/fb7 [572473,60421] 0 2024-03-16T09:01:14.658 INFO:tasks.workunit.client.1.smithi019.stdout:3/393: dread d3/d1b/d40/d4a/f41 [0,4194304] 0 2024-03-16T09:01:14.658 INFO:tasks.workunit.client.1.smithi019.stdout:3/394: write d3/d11/f3b [708857,69352] 0 2024-03-16T09:01:14.660 INFO:tasks.workunit.client.0.smithi007.stdout:0/637: mkdir d3/d1f/d70/d7c/dd4 0 2024-03-16T09:01:14.661 INFO:tasks.workunit.client.0.smithi007.stdout:4/458: getdents d4/d7/d78/d80 0 2024-03-16T09:01:14.661 INFO:tasks.workunit.client.0.smithi007.stdout:4/459: chown d4/d7/c6e 50155249 1 2024-03-16T09:01:14.662 INFO:tasks.workunit.client.0.smithi007.stdout:4/460: getdents d4/d7/d78/d80 0 2024-03-16T09:01:14.662 INFO:tasks.workunit.client.0.smithi007.stdout:4/461: chown d4/l2a 5736 1 2024-03-16T09:01:14.662 INFO:tasks.workunit.client.1.smithi019.stdout:5/306: mknod d6/d19/d34/c5b 0 2024-03-16T09:01:14.662 INFO:tasks.workunit.client.1.smithi019.stdout:5/307: truncate d6/d1f/d33/f4f 1009501 0 2024-03-16T09:01:14.663 INFO:tasks.workunit.client.1.smithi019.stdout:6/460: truncate de/f45 267113 0 2024-03-16T09:01:14.663 INFO:tasks.workunit.client.1.smithi019.stdout:5/308: write d6/d19/f59 [941720,4577] 0 2024-03-16T09:01:14.663 INFO:tasks.workunit.client.1.smithi019.stdout:5/309: creat d6/dd/f5c x:0 0 0 2024-03-16T09:01:14.664 INFO:tasks.workunit.client.1.smithi019.stdout:0/317: creat d7/d8/d32/d36/f5f x:0 0 0 2024-03-16T09:01:14.665 INFO:tasks.workunit.client.1.smithi019.stdout:3/395: fsync d3/d11/f49 0 2024-03-16T09:01:14.667 INFO:tasks.workunit.client.0.smithi007.stdout:6/451: read d2/d11/d3b/d93/d8e/f4f [3362666,8865] 0 2024-03-16T09:01:14.667 INFO:tasks.workunit.client.0.smithi007.stdout:6/452: fsync d2/d11/d3b/d93/d8e/d4e/f52 0 2024-03-16T09:01:14.667 INFO:tasks.workunit.client.0.smithi007.stdout:0/638: truncate d3/d9/da/d18/d23/f32 2891919 0 2024-03-16T09:01:14.667 INFO:tasks.workunit.client.0.smithi007.stdout:4/462: symlink d4/d7/d78/l97 0 2024-03-16T09:01:14.669 INFO:tasks.workunit.client.0.smithi007.stdout:5/578: dwrite d1/d7/d1c/d19/d34/d5e/d6f/d72/fab [4194304,4194304] 0 2024-03-16T09:01:14.669 INFO:tasks.workunit.client.0.smithi007.stdout:4/463: mknod d4/d14/d1c/c98 0 2024-03-16T09:01:14.669 INFO:tasks.workunit.client.0.smithi007.stdout:4/464: dread - d4/d14/d1c/d5b/f8e zero size 2024-03-16T09:01:14.669 INFO:tasks.workunit.client.0.smithi007.stdout:4/465: stat d4/l3c 0 2024-03-16T09:01:14.669 INFO:tasks.workunit.client.1.smithi019.stdout:5/310: mknod d6/d1f/d33/d4c/c5d 0 2024-03-16T09:01:14.670 INFO:tasks.workunit.client.1.smithi019.stdout:3/396: read - d3/d17/d5a/f61 zero size 2024-03-16T09:01:14.671 INFO:tasks.workunit.client.0.smithi007.stdout:4/466: mkdir d4/d99 0 2024-03-16T09:01:14.672 INFO:tasks.workunit.client.1.smithi019.stdout:3/397: symlink d3/d17/d6a/l78 0 2024-03-16T09:01:14.672 INFO:tasks.workunit.client.1.smithi019.stdout:3/398: write d3/d11/f69 [220253,118218] 0 2024-03-16T09:01:14.673 INFO:tasks.workunit.client.0.smithi007.stdout:6/453: write d2/d13/d3f/d50/f53 [4030087,126945] 0 2024-03-16T09:01:14.673 INFO:tasks.workunit.client.1.smithi019.stdout:6/461: write de/d1c/f5d [2894013,18987] 0 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:3/399: link d3/d17/d3f/l54 d3/d1b/l79 0 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:3/400: stat d3/c4f 0 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:3/401: dread - d3/d17/d5a/f73 zero size 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:8/343: sync 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:4/331: sync 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:9/224: sync 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:7/225: sync 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:7/226: chown d9/f16 191702072 1 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:4/332: dread - d4/d6/d15/f67 zero size 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:4/333: chown d4/d6/l4c 12206081 1 2024-03-16T09:01:14.675 INFO:tasks.workunit.client.1.smithi019.stdout:4/334: creat d4/d1b/d2a/f74 x:0 0 0 2024-03-16T09:01:14.678 INFO:tasks.workunit.client.1.smithi019.stdout:3/402: mkdir d3/d17/d5a/d75/d7a 0 2024-03-16T09:01:14.678 INFO:tasks.workunit.client.1.smithi019.stdout:3/403: dread - d3/d17/d5a/f73 zero size 2024-03-16T09:01:14.678 INFO:tasks.workunit.client.1.smithi019.stdout:3/404: write d3/d17/f59 [662845,110688] 0 2024-03-16T09:01:14.678 INFO:tasks.workunit.client.1.smithi019.stdout:3/405: stat d3/l28 0 2024-03-16T09:01:14.678 INFO:tasks.workunit.client.0.smithi007.stdout:4/467: write d4/f51 [705682,69313] 0 2024-03-16T09:01:14.679 INFO:tasks.workunit.client.1.smithi019.stdout:3/406: creat d3/d17/d6a/f7b x:0 0 0 2024-03-16T09:01:14.679 INFO:tasks.workunit.client.1.smithi019.stdout:5/311: dread d6/f2f [0,4194304] 0 2024-03-16T09:01:14.680 INFO:tasks.workunit.client.0.smithi007.stdout:5/579: dread d1/d7/d16/f33 [0,4194304] 0 2024-03-16T09:01:14.680 INFO:tasks.workunit.client.0.smithi007.stdout:5/580: fsync d1/f2 0 2024-03-16T09:01:14.680 INFO:tasks.workunit.client.0.smithi007.stdout:5/581: read - d1/d7/d1c/d88/da2/d89/db3/d2a/d79/fbe zero size 2024-03-16T09:01:14.680 INFO:tasks.workunit.client.0.smithi007.stdout:5/582: chown d1/d7/d1c/d88/da2/d89/db3/d2a/f57 16 1 2024-03-16T09:01:14.681 INFO:tasks.workunit.client.1.smithi019.stdout:8/344: mkdir d6/d2a/d45/d19/d74 0 2024-03-16T09:01:14.683 INFO:tasks.workunit.client.0.smithi007.stdout:5/583: rename d1/d7/f7c to d1/d7/d1c/d88/fd6 0 2024-03-16T09:01:14.683 INFO:tasks.workunit.client.0.smithi007.stdout:6/454: dread d2/f7 [0,4194304] 0 2024-03-16T09:01:14.685 INFO:tasks.workunit.client.1.smithi019.stdout:4/335: link d4/f26 d4/d1b/d49/f75 0 2024-03-16T09:01:14.686 INFO:tasks.workunit.client.0.smithi007.stdout:6/455: getdents d2/d11/d3b/d74 0 2024-03-16T09:01:14.689 INFO:tasks.workunit.client.1.smithi019.stdout:3/407: unlink d3/f3e 0 2024-03-16T09:01:14.689 INFO:tasks.workunit.client.1.smithi019.stdout:3/408: creat d3/d17/d3f/f7c x:0 0 0 2024-03-16T09:01:14.689 INFO:tasks.workunit.client.0.smithi007.stdout:6/456: rename d2/d11/d3b/d93/d8e/f67 to d2/d13/d3f/d86/f96 0 2024-03-16T09:01:14.691 INFO:tasks.workunit.client.1.smithi019.stdout:5/312: link d6/d19/f29 d6/d1f/f5e 0 2024-03-16T09:01:14.691 INFO:tasks.workunit.client.0.smithi007.stdout:4/468: dread d4/d14/d1c/f60 [0,4194304] 0 2024-03-16T09:01:14.693 INFO:tasks.workunit.client.0.smithi007.stdout:4/469: rename d4/d14/d64/d68/d77 to d4/d14/d1c/d1e/d9a 0 2024-03-16T09:01:14.693 INFO:tasks.workunit.client.0.smithi007.stdout:3/555: sync 2024-03-16T09:01:14.693 INFO:tasks.workunit.client.0.smithi007.stdout:1/491: sync 2024-03-16T09:01:14.694 INFO:tasks.workunit.client.1.smithi019.stdout:8/345: rename d6/f7 to d6/d2a/d45/f75 0 2024-03-16T09:01:14.694 INFO:tasks.workunit.client.1.smithi019.stdout:8/346: chown d6/d2a/d3c/f44 495982 1 2024-03-16T09:01:14.694 INFO:tasks.workunit.client.0.smithi007.stdout:3/556: rmdir d15/d18/d1f/d29/d43 39 2024-03-16T09:01:14.694 INFO:tasks.workunit.client.0.smithi007.stdout:3/557: write d15/d18/d6a/f7e [1013326,115049] 0 2024-03-16T09:01:14.694 INFO:tasks.workunit.client.0.smithi007.stdout:3/558: chown d15/d18/d1f/d29/f5b 9722 1 2024-03-16T09:01:14.695 INFO:tasks.workunit.client.1.smithi019.stdout:4/336: mknod d4/d1b/d2a/c76 0 2024-03-16T09:01:14.695 INFO:tasks.workunit.client.0.smithi007.stdout:3/559: symlink d15/d18/d1f/d29/d43/lad 0 2024-03-16T09:01:14.696 INFO:tasks.workunit.client.0.smithi007.stdout:3/560: mkdir d15/d18/d1f/d3b/dae 0 2024-03-16T09:01:14.697 INFO:tasks.workunit.client.1.smithi019.stdout:5/313: creat d6/dd/d37/d54/f5f x:0 0 0 2024-03-16T09:01:14.697 INFO:tasks.workunit.client.1.smithi019.stdout:5/314: stat d6/d1f/d33/f4b 0 2024-03-16T09:01:14.697 INFO:tasks.workunit.client.1.smithi019.stdout:8/347: creat d6/de/f76 x:0 0 0 2024-03-16T09:01:14.697 INFO:tasks.workunit.client.1.smithi019.stdout:5/315: truncate d6/d1f/f3f 1593052 0 2024-03-16T09:01:14.698 INFO:tasks.workunit.client.0.smithi007.stdout:0/639: dwrite d3/d9/da/d48/fa5 [0,4194304] 0 2024-03-16T09:01:14.698 INFO:tasks.workunit.client.0.smithi007.stdout:3/561: mkdir d15/d18/d1f/d29/d3e/d85/daf 0 2024-03-16T09:01:14.698 INFO:tasks.workunit.client.0.smithi007.stdout:3/562: chown d15/d18/d6a/c74 23 1 2024-03-16T09:01:14.698 INFO:tasks.workunit.client.0.smithi007.stdout:3/563: creat d15/d18/d1f/d29/d7c/fb0 x:0 0 0 2024-03-16T09:01:14.699 INFO:tasks.workunit.client.1.smithi019.stdout:8/348: creat d6/f77 x:0 0 0 2024-03-16T09:01:14.699 INFO:tasks.workunit.client.1.smithi019.stdout:8/349: getdents d6/d2a/d45/d57/d72 0 2024-03-16T09:01:14.704 INFO:tasks.workunit.client.0.smithi007.stdout:6/457: write d2/d11/f79 [764920,80875] 0 2024-03-16T09:01:14.704 INFO:tasks.workunit.client.0.smithi007.stdout:6/458: dread - d2/f59 zero size 2024-03-16T09:01:14.705 INFO:tasks.workunit.client.0.smithi007.stdout:3/564: rename d15/d18/d1f/d77/d9d to d15/d18/d6a/db1 0 2024-03-16T09:01:14.705 INFO:tasks.workunit.client.0.smithi007.stdout:3/565: fdatasync d15/d18/d1f/d56/f8a 0 2024-03-16T09:01:14.705 INFO:tasks.workunit.client.0.smithi007.stdout:3/566: creat d15/d18/d1f/d56/fb2 x:0 0 0 2024-03-16T09:01:14.706 INFO:tasks.workunit.client.1.smithi019.stdout:9/225: dwrite d1/d7/d21/d27/f3c [0,4194304] 0 2024-03-16T09:01:14.707 INFO:tasks.workunit.client.0.smithi007.stdout:6/459: creat d2/d11/d1e/d54/d7a/f97 x:0 0 0 2024-03-16T09:01:14.707 INFO:tasks.workunit.client.0.smithi007.stdout:6/460: readlink d2/d11/d3b/d93/l31 0 2024-03-16T09:01:14.707 INFO:tasks.workunit.client.0.smithi007.stdout:6/461: fsync d2/d11/d1e/f2f 0 2024-03-16T09:01:14.707 INFO:tasks.workunit.client.0.smithi007.stdout:1/492: dread d9/dd/d1b/d2d/d30/d45/f5b [0,4194304] 0 2024-03-16T09:01:14.709 INFO:tasks.workunit.client.0.smithi007.stdout:3/567: truncate d15/d18/d1f/d29/d3e/d85/fa6 612419 0 2024-03-16T09:01:14.711 INFO:tasks.workunit.client.1.smithi019.stdout:9/226: rename d1/d7/f41 to d1/d7/f51 0 2024-03-16T09:01:14.720 INFO:tasks.workunit.client.1.smithi019.stdout:3/409: dwrite d3/d17/f59 [0,4194304] 0 2024-03-16T09:01:14.730 INFO:tasks.workunit.client.1.smithi019.stdout:9/227: dwrite d1/d7/de/d24/f47 [0,4194304] 0 2024-03-16T09:01:14.730 INFO:tasks.workunit.client.1.smithi019.stdout:9/228: getdents d1/d7/de/d24/d44 0 2024-03-16T09:01:14.732 INFO:tasks.workunit.client.1.smithi019.stdout:9/229: creat d1/d7/de/d2d/f52 x:0 0 0 2024-03-16T09:01:14.732 INFO:tasks.workunit.client.1.smithi019.stdout:3/410: read d3/f30 [3241268,5155] 0 2024-03-16T09:01:14.732 INFO:tasks.workunit.client.1.smithi019.stdout:3/411: fsync d3/d17/d5a/f5c 0 2024-03-16T09:01:14.735 INFO:tasks.workunit.client.1.smithi019.stdout:3/412: rename d3/d1b/d5f to d3/d11/d7d 0 2024-03-16T09:01:14.735 INFO:tasks.workunit.client.1.smithi019.stdout:3/413: creat d3/d17/d6a/f7e x:0 0 0 2024-03-16T09:01:14.736 INFO:tasks.workunit.client.1.smithi019.stdout:3/414: symlink d3/d17/d6a/l7f 0 2024-03-16T09:01:14.737 INFO:tasks.workunit.client.1.smithi019.stdout:3/415: mkdir d3/d17/d5a/d80 0 2024-03-16T09:01:14.738 INFO:tasks.workunit.client.1.smithi019.stdout:3/416: symlink d3/d11/d7d/l81 0 2024-03-16T09:01:14.738 INFO:tasks.workunit.client.1.smithi019.stdout:3/417: mknod d3/d1b/d40/c82 0 2024-03-16T09:01:14.745 INFO:tasks.workunit.client.1.smithi019.stdout:7/227: dwrite d9/da/d13/d18/d1a/f1f [0,4194304] 0 2024-03-16T09:01:14.746 INFO:tasks.workunit.client.1.smithi019.stdout:7/228: creat d9/da/d13/d18/d1a/d32/d33/f43 x:0 0 0 2024-03-16T09:01:14.746 INFO:tasks.workunit.client.1.smithi019.stdout:7/229: read d9/db/d12/f2f [389626,13993] 0 2024-03-16T09:01:14.747 INFO:tasks.workunit.client.1.smithi019.stdout:7/230: write d9/db/f3d [757288,64806] 0 2024-03-16T09:01:14.747 INFO:tasks.workunit.client.1.smithi019.stdout:7/231: dread - d9/d25/f42 zero size 2024-03-16T09:01:14.747 INFO:tasks.workunit.client.1.smithi019.stdout:5/316: dread d6/f9 [0,4194304] 0 2024-03-16T09:01:14.747 INFO:tasks.workunit.client.1.smithi019.stdout:7/232: link d9/db/l3b d9/da/l44 0 2024-03-16T09:01:14.748 INFO:tasks.workunit.client.1.smithi019.stdout:7/233: fdatasync d9/d25/d28/f34 0 2024-03-16T09:01:14.748 INFO:tasks.workunit.client.1.smithi019.stdout:3/418: dread f2 [0,4194304] 0 2024-03-16T09:01:14.748 INFO:tasks.workunit.client.1.smithi019.stdout:9/230: dread d1/d7/d21/d27/f13 [0,4194304] 0 2024-03-16T09:01:14.748 INFO:tasks.workunit.client.1.smithi019.stdout:4/337: dwrite d4/d6/d15/f66 [0,4194304] 0 2024-03-16T09:01:14.753 INFO:tasks.workunit.client.1.smithi019.stdout:5/317: dread d6/d1f/f20 [0,4194304] 0 2024-03-16T09:01:14.753 INFO:tasks.workunit.client.1.smithi019.stdout:5/318: creat d6/dd/d37/f60 x:0 0 0 2024-03-16T09:01:14.753 INFO:tasks.workunit.client.1.smithi019.stdout:5/319: fsync d6/d1f/f2d 0 2024-03-16T09:01:14.755 INFO:tasks.workunit.client.1.smithi019.stdout:5/320: mkdir d6/dd/d61 0 2024-03-16T09:01:14.756 INFO:tasks.workunit.client.1.smithi019.stdout:7/234: dread f8 [0,4194304] 0 2024-03-16T09:01:14.756 INFO:tasks.workunit.client.1.smithi019.stdout:7/235: fsync d9/da/d13/f17 0 2024-03-16T09:01:14.756 INFO:tasks.workunit.client.1.smithi019.stdout:7/236: chown d9/da/d13/d18/d1a/d32/l35 3 1 2024-03-16T09:01:14.757 INFO:tasks.workunit.client.1.smithi019.stdout:7/237: mknod d9/db/c45 0 2024-03-16T09:01:14.758 INFO:tasks.workunit.client.1.smithi019.stdout:9/231: dread d1/d7/de/d24/f39 [0,4194304] 0 2024-03-16T09:01:14.758 INFO:tasks.workunit.client.1.smithi019.stdout:9/232: chown d1/d7/de/c19 98 1 2024-03-16T09:01:14.760 INFO:tasks.workunit.client.1.smithi019.stdout:9/233: creat d1/d7/de/d24/d44/f53 x:0 0 0 2024-03-16T09:01:14.760 INFO:tasks.workunit.client.1.smithi019.stdout:9/234: dread - d1/d7/de/d2d/f52 zero size 2024-03-16T09:01:14.761 INFO:tasks.workunit.client.1.smithi019.stdout:4/338: dread d4/d1b/d2a/f4e [0,4194304] 0 2024-03-16T09:01:14.763 INFO:tasks.workunit.client.1.smithi019.stdout:9/235: rmdir d1/d7/de/d43 0 2024-03-16T09:01:14.764 INFO:tasks.workunit.client.0.smithi007.stdout:6/462: creat d2/d11/d3b/d93/d15/f98 x:0 0 0 2024-03-16T09:01:14.768 INFO:tasks.workunit.client.1.smithi019.stdout:7/238: dread f6 [0,4194304] 0 2024-03-16T09:01:14.768 INFO:tasks.workunit.client.1.smithi019.stdout:7/239: chown d9/db/c45 26450492 1 2024-03-16T09:01:14.770 INFO:tasks.workunit.client.0.smithi007.stdout:1/493: dwrite d9/dd/d1b/d28/d29/f80 [0,4194304] 0 2024-03-16T09:01:14.771 INFO:tasks.workunit.client.1.smithi019.stdout:7/240: mkdir d9/da/d13/d18/d46 0 2024-03-16T09:01:14.776 INFO:tasks.workunit.client.1.smithi019.stdout:3/419: rmdir d3/d17/d6a 39 2024-03-16T09:01:14.776 INFO:tasks.workunit.client.1.smithi019.stdout:3/420: write d3/d11/f6f [650150,48557] 0 2024-03-16T09:01:14.777 INFO:tasks.workunit.client.1.smithi019.stdout:3/421: getdents d3/d17/d6a 0 2024-03-16T09:01:14.779 INFO:tasks.workunit.client.0.smithi007.stdout:3/568: rmdir d15 39 2024-03-16T09:01:14.782 INFO:tasks.workunit.client.0.smithi007.stdout:1/494: symlink d9/l98 0 2024-03-16T09:01:14.783 INFO:tasks.workunit.client.1.smithi019.stdout:7/241: dread d9/da/d13/f17 [0,4194304] 0 2024-03-16T09:01:14.783 INFO:tasks.workunit.client.1.smithi019.stdout:7/242: fdatasync f8 0 2024-03-16T09:01:14.784 INFO:tasks.workunit.client.0.smithi007.stdout:1/495: creat d9/dd/d1b/d2d/f99 x:0 0 0 2024-03-16T09:01:14.784 INFO:tasks.workunit.client.0.smithi007.stdout:3/569: read d15/d18/d1f/f54 [190359,50778] 0 2024-03-16T09:01:14.784 INFO:tasks.workunit.client.0.smithi007.stdout:3/570: truncate d15/d18/d1f/d56/f95 904416 0 2024-03-16T09:01:14.787 INFO:tasks.workunit.client.0.smithi007.stdout:3/571: creat d15/d18/d1f/d29/d7c/fb3 x:0 0 0 2024-03-16T09:01:14.787 INFO:tasks.workunit.client.0.smithi007.stdout:1/496: write d9/dd/d1b/d28/d29/d7b/f88 [866580,60878] 0 2024-03-16T09:01:14.788 INFO:tasks.workunit.client.0.smithi007.stdout:1/497: creat d9/d6b/d7e/f9a x:0 0 0 2024-03-16T09:01:14.789 INFO:tasks.workunit.client.1.smithi019.stdout:7/243: dread d9/db/f3d [0,4194304] 0 2024-03-16T09:01:14.789 INFO:tasks.workunit.client.0.smithi007.stdout:1/498: creat d9/dd/d1b/d28/d29/d7b/f9b x:0 0 0 2024-03-16T09:01:14.792 INFO:tasks.workunit.client.1.smithi019.stdout:7/244: unlink d9/fc 0 2024-03-16T09:01:14.792 INFO:tasks.workunit.client.1.smithi019.stdout:7/245: truncate d9/da/d13/d18/d1a/d32/f41 530441 0 2024-03-16T09:01:14.794 INFO:tasks.workunit.client.1.smithi019.stdout:7/246: read d9/f3f [1464382,113128] 0 2024-03-16T09:01:14.795 INFO:tasks.workunit.client.1.smithi019.stdout:7/247: link d9/f16 d9/db/f47 0 2024-03-16T09:01:14.796 INFO:tasks.workunit.client.1.smithi019.stdout:7/248: mkdir d9/d25/d28/d48 0 2024-03-16T09:01:14.796 INFO:tasks.workunit.client.1.smithi019.stdout:7/249: dread - d9/da/d13/d18/d30/f37 zero size 2024-03-16T09:01:14.797 INFO:tasks.workunit.client.1.smithi019.stdout:7/250: dread d9/db/d12/f2f [0,4194304] 0 2024-03-16T09:01:14.798 INFO:tasks.workunit.client.1.smithi019.stdout:7/251: symlink d9/d25/d28/l49 0 2024-03-16T09:01:14.798 INFO:tasks.workunit.client.1.smithi019.stdout:7/252: chown d9/db/d12/f2f 9 1 2024-03-16T09:01:14.799 INFO:tasks.workunit.client.1.smithi019.stdout:7/253: creat d9/da/d13/d18/d30/f4a x:0 0 0 2024-03-16T09:01:14.799 INFO:tasks.workunit.client.1.smithi019.stdout:7/254: creat d9/d25/d28/d48/f4b x:0 0 0 2024-03-16T09:01:14.804 INFO:tasks.workunit.client.1.smithi019.stdout:9/236: dwrite d1/d7/f51 [0,4194304] 0 2024-03-16T09:01:14.804 INFO:tasks.workunit.client.0.smithi007.stdout:6/463: dwrite d2/d11/d1e/f85 [0,4194304] 0 2024-03-16T09:01:14.806 INFO:tasks.workunit.client.0.smithi007.stdout:6/464: symlink d2/d11/d3b/d93/d15/l99 0 2024-03-16T09:01:14.806 INFO:tasks.workunit.client.1.smithi019.stdout:9/237: truncate d1/d7/d21/d27/f35 732791 0 2024-03-16T09:01:14.808 INFO:tasks.workunit.client.1.smithi019.stdout:9/238: mknod d1/d7/de/c54 0 2024-03-16T09:01:14.808 INFO:tasks.workunit.client.1.smithi019.stdout:9/239: creat d1/d7/d21/d27/f55 x:0 0 0 2024-03-16T09:01:14.808 INFO:tasks.workunit.client.1.smithi019.stdout:9/240: write d1/d7/de/d2d/f52 [311054,83354] 0 2024-03-16T09:01:14.808 INFO:tasks.workunit.client.1.smithi019.stdout:9/241: write d1/d7/de/f3a [939865,125684] 0 2024-03-16T09:01:14.808 INFO:tasks.workunit.client.1.smithi019.stdout:9/242: fdatasync d1/d7/f51 0 2024-03-16T09:01:14.809 INFO:tasks.workunit.client.0.smithi007.stdout:1/499: rmdir d9 39 2024-03-16T09:01:14.810 INFO:tasks.workunit.client.0.smithi007.stdout:1/500: symlink d9/l9c 0 2024-03-16T09:01:14.810 INFO:tasks.workunit.client.0.smithi007.stdout:1/501: mknod d9/d6b/d7e/d91/c9d 0 2024-03-16T09:01:14.812 INFO:tasks.workunit.client.0.smithi007.stdout:1/502: symlink d9/dd/d1b/d2d/d30/l9e 0 2024-03-16T09:01:14.812 INFO:tasks.workunit.client.0.smithi007.stdout:1/503: chown d9/dd/d1b/f4d 257387 1 2024-03-16T09:01:14.814 INFO:tasks.workunit.client.1.smithi019.stdout:4/339: dwrite d4/d1b/d2a/d3d/f46 [0,4194304] 0 2024-03-16T09:01:14.817 INFO:tasks.workunit.client.0.smithi007.stdout:6/465: read d2/d10/d3d/f6c [1396796,114818] 0 2024-03-16T09:01:14.818 INFO:tasks.workunit.client.1.smithi019.stdout:5/321: dwrite d6/fb [4194304,4194304] 0 2024-03-16T09:01:14.818 INFO:tasks.workunit.client.1.smithi019.stdout:5/322: truncate d6/dd/d37/d54/f5f 322871 0 2024-03-16T09:01:14.819 INFO:tasks.workunit.client.1.smithi019.stdout:5/323: fdatasync d6/dd/d37/f58 0 2024-03-16T09:01:14.819 INFO:tasks.workunit.client.1.smithi019.stdout:4/340: write d4/d6/f24 [461062,7224] 0 2024-03-16T09:01:14.819 INFO:tasks.workunit.client.1.smithi019.stdout:5/324: truncate d6/d1f/f20 2922 0 2024-03-16T09:01:14.820 INFO:tasks.workunit.client.1.smithi019.stdout:3/422: dwrite d3/d1b/f1c [0,4194304] 0 2024-03-16T09:01:14.820 INFO:tasks.workunit.client.0.smithi007.stdout:3/572: dwrite d15/d18/d1f/d3b/d63/f93 [4194304,4194304] 0 2024-03-16T09:01:14.821 INFO:tasks.workunit.client.0.smithi007.stdout:1/504: write d9/f22 [4142234,35081] 0 2024-03-16T09:01:14.822 INFO:tasks.workunit.client.1.smithi019.stdout:5/325: dread d6/d19/f59 [0,4194304] 0 2024-03-16T09:01:14.824 INFO:tasks.workunit.client.1.smithi019.stdout:4/341: symlink d4/d6c/l77 0 2024-03-16T09:01:14.824 INFO:tasks.workunit.client.1.smithi019.stdout:4/342: chown d4/d1b/d49/f72 86 1 2024-03-16T09:01:14.825 INFO:tasks.workunit.client.0.smithi007.stdout:1/505: creat d9/dd/d5e/f9f x:0 0 0 2024-03-16T09:01:14.828 INFO:tasks.workunit.client.0.smithi007.stdout:6/466: dread d2/d13/d24/f38 [0,4194304] 0 2024-03-16T09:01:14.829 INFO:tasks.workunit.client.1.smithi019.stdout:7/255: getdents d9/d25/d28/d48 0 2024-03-16T09:01:14.829 INFO:tasks.workunit.client.1.smithi019.stdout:7/256: chown f6 13589 1 2024-03-16T09:01:14.830 INFO:tasks.workunit.client.0.smithi007.stdout:6/467: rmdir d2/d13/d62/d91 39 2024-03-16T09:01:14.830 INFO:tasks.workunit.client.0.smithi007.stdout:6/468: fdatasync d2/d11/d3b/d93/d8e/f4d 0 2024-03-16T09:01:14.831 INFO:tasks.workunit.client.0.smithi007.stdout:6/469: readlink d2/d11/d3b/d93/l31 0 2024-03-16T09:01:14.832 INFO:tasks.workunit.client.0.smithi007.stdout:6/470: mkdir d2/d11/d1e/d54/d7a/d9a 0 2024-03-16T09:01:14.833 INFO:tasks.workunit.client.0.smithi007.stdout:6/471: link d2/d11/c43 d2/d13/d3f/d86/c9b 0 2024-03-16T09:01:14.838 INFO:tasks.workunit.client.0.smithi007.stdout:3/573: dwrite d15/d18/d1f/d29/d3e/d85/fac [0,4194304] 0 2024-03-16T09:01:14.840 INFO:tasks.workunit.client.1.smithi019.stdout:9/243: getdents d1/d7/d21/d27 0 2024-03-16T09:01:14.840 INFO:tasks.workunit.client.0.smithi007.stdout:3/574: dread d15/d18/d6a/f7e [0,4194304] 0 2024-03-16T09:01:14.841 INFO:tasks.workunit.client.1.smithi019.stdout:3/423: dwrite d3/d17/d5a/f73 [0,4194304] 0 2024-03-16T09:01:14.841 INFO:tasks.workunit.client.1.smithi019.stdout:3/424: chown d3/d11/d7d/l68 25663912 1 2024-03-16T09:01:14.842 INFO:tasks.workunit.client.0.smithi007.stdout:3/575: mkdir d15/d18/d6a/db1/db4 0 2024-03-16T09:01:14.842 INFO:tasks.workunit.client.0.smithi007.stdout:3/576: stat d15/c76 0 2024-03-16T09:01:14.843 INFO:tasks.workunit.client.0.smithi007.stdout:3/577: mknod d15/d18/d1f/d56/cb5 0 2024-03-16T09:01:14.844 INFO:tasks.workunit.client.1.smithi019.stdout:9/244: dread d1/d7/de/f1a [0,4194304] 0 2024-03-16T09:01:14.844 INFO:tasks.workunit.client.1.smithi019.stdout:5/326: creat d6/d1f/d33/d4c/f62 x:0 0 0 2024-03-16T09:01:14.846 INFO:tasks.workunit.client.1.smithi019.stdout:4/343: symlink d4/d1b/d2a/d3d/l78 0 2024-03-16T09:01:14.847 INFO:tasks.workunit.client.0.smithi007.stdout:3/578: rename f13 to d15/d18/d1f/d29/d43/fb6 0 2024-03-16T09:01:14.847 INFO:tasks.workunit.client.0.smithi007.stdout:3/579: dread - d15/d18/d1f/d29/f5b zero size 2024-03-16T09:01:14.847 INFO:tasks.workunit.client.0.smithi007.stdout:3/580: chown d15/d18/d1f/d3b/d87 23240 1 2024-03-16T09:01:14.852 INFO:tasks.workunit.client.1.smithi019.stdout:3/425: symlink d3/d17/d6a/l83 0 2024-03-16T09:01:14.855 INFO:tasks.workunit.client.1.smithi019.stdout:5/327: mknod d6/d1f/d33/c63 0 2024-03-16T09:01:14.855 INFO:tasks.workunit.client.1.smithi019.stdout:5/328: write d6/d1f/d33/d4c/f62 [14186,123829] 0 2024-03-16T09:01:14.855 INFO:tasks.workunit.client.1.smithi019.stdout:5/329: creat d6/dd/d47/f64 x:0 0 0 2024-03-16T09:01:14.856 INFO:tasks.workunit.client.1.smithi019.stdout:0/318: sync 2024-03-16T09:01:14.856 INFO:tasks.workunit.client.1.smithi019.stdout:1/269: sync 2024-03-16T09:01:14.856 INFO:tasks.workunit.client.1.smithi019.stdout:2/381: sync 2024-03-16T09:01:14.856 INFO:tasks.workunit.client.1.smithi019.stdout:1/270: dread - d16/f37 zero size 2024-03-16T09:01:14.857 INFO:tasks.workunit.client.1.smithi019.stdout:4/344: mkdir d4/d6c/d79 0 2024-03-16T09:01:14.858 INFO:tasks.workunit.client.1.smithi019.stdout:7/257: mkdir d9/db/d4c 0 2024-03-16T09:01:14.858 INFO:tasks.workunit.client.1.smithi019.stdout:6/462: sync 2024-03-16T09:01:14.858 INFO:tasks.workunit.client.1.smithi019.stdout:6/463: fdatasync de/d1c/f28 0 2024-03-16T09:01:14.858 INFO:tasks.workunit.client.1.smithi019.stdout:6/464: stat de/d26/d32/d97/d66 0 2024-03-16T09:01:14.859 INFO:tasks.workunit.client.1.smithi019.stdout:3/426: rename d3/d1b/d40/d4a/l3d to d3/d17/l84 0 2024-03-16T09:01:14.859 INFO:tasks.workunit.client.1.smithi019.stdout:3/427: write d3/d11/f6f [793694,94592] 0 2024-03-16T09:01:14.861 INFO:tasks.workunit.client.1.smithi019.stdout:5/330: symlink d6/l65 0 2024-03-16T09:01:14.861 INFO:tasks.workunit.client.1.smithi019.stdout:9/245: dwrite d1/d7/de/d24/f42 [0,4194304] 0 2024-03-16T09:01:14.862 INFO:tasks.workunit.client.0.smithi007.stdout:6/472: dwrite d2/d11/d1e/d54/d7a/f97 [0,4194304] 0 2024-03-16T09:01:14.864 INFO:tasks.workunit.client.0.smithi007.stdout:3/581: fsync d15/d18/d1f/d29/d43/fb6 0 2024-03-16T09:01:14.865 INFO:tasks.workunit.client.0.smithi007.stdout:3/582: chown d15/d18/d1f/d3b/d63/l9f 9 1 2024-03-16T09:01:14.866 INFO:tasks.workunit.client.1.smithi019.stdout:6/465: mknod de/d26/d32/d97/d2c/c98 0 2024-03-16T09:01:14.868 INFO:tasks.workunit.client.0.smithi007.stdout:6/473: mkdir d2/d11/d1e/d54/d9c 0 2024-03-16T09:01:14.868 INFO:tasks.workunit.client.0.smithi007.stdout:6/474: chown d2/d11/d3b/d93/d8e/d49 23 1 2024-03-16T09:01:14.869 INFO:tasks.workunit.client.1.smithi019.stdout:7/258: dread d9/d25/d28/f34 [0,4194304] 0 2024-03-16T09:01:14.869 INFO:tasks.workunit.client.1.smithi019.stdout:8/350: sync 2024-03-16T09:01:14.869 INFO:tasks.workunit.client.1.smithi019.stdout:8/351: fsync d6/d2a/d45/d57/f60 0 2024-03-16T09:01:14.869 INFO:tasks.workunit.client.1.smithi019.stdout:8/352: readlink d6/d2a/d45/d57/l64 0 2024-03-16T09:01:14.871 INFO:tasks.workunit.client.0.smithi007.stdout:3/583: rename d15/d18/d1f/d3b/d87/f8e to d15/d18/d1f/d3b/d63/fb7 0 2024-03-16T09:01:14.871 INFO:tasks.workunit.client.0.smithi007.stdout:3/584: dread - d15/f59 zero size 2024-03-16T09:01:14.871 INFO:tasks.workunit.client.0.smithi007.stdout:6/475: readlink d2/d11/l18 0 2024-03-16T09:01:14.872 INFO:tasks.workunit.client.1.smithi019.stdout:0/319: symlink d7/l60 0 2024-03-16T09:01:14.872 INFO:tasks.workunit.client.1.smithi019.stdout:2/382: getdents d5/d2e/d4f 0 2024-03-16T09:01:14.873 INFO:tasks.workunit.client.0.smithi007.stdout:3/585: creat d15/d18/d1f/d3b/d87/fb8 x:0 0 0 2024-03-16T09:01:14.874 INFO:tasks.workunit.client.1.smithi019.stdout:3/428: rename d3/d11/d7d/f6b to d3/d17/d5a/f85 0 2024-03-16T09:01:14.874 INFO:tasks.workunit.client.0.smithi007.stdout:6/476: rename d2/d11/d3b/d93/d8e/l2a to d2/d11/d3b/d93/d15/l9d 0 2024-03-16T09:01:14.875 INFO:tasks.workunit.client.1.smithi019.stdout:8/353: symlink d6/l78 0 2024-03-16T09:01:14.875 INFO:tasks.workunit.client.1.smithi019.stdout:8/354: write d6/d2a/d45/d57/f60 [467949,61199] 0 2024-03-16T09:01:14.875 INFO:tasks.workunit.client.1.smithi019.stdout:8/355: dread - d6/de/f76 zero size 2024-03-16T09:01:14.875 INFO:tasks.workunit.client.1.smithi019.stdout:8/356: stat d6/de/c13 0 2024-03-16T09:01:14.876 INFO:tasks.workunit.client.0.smithi007.stdout:3/586: symlink d15/d18/d1f/d29/d3e/d85/lb9 0 2024-03-16T09:01:14.876 INFO:tasks.workunit.client.0.smithi007.stdout:6/477: link d2/d11/d3b/d93/d8e/f5c d2/d11/d3b/d93/d8e/d49/d84/f9e 0 2024-03-16T09:01:14.877 INFO:tasks.workunit.client.0.smithi007.stdout:6/478: chown d2/d11/d3b/d93/d8e/l65 53 1 2024-03-16T09:01:14.877 INFO:tasks.workunit.client.0.smithi007.stdout:6/479: fsync d2/d10/f3a 0 2024-03-16T09:01:14.877 INFO:tasks.workunit.client.0.smithi007.stdout:6/480: rmdir d2/d13/d3f/d50/d78/d81 39 2024-03-16T09:01:14.878 INFO:tasks.workunit.client.0.smithi007.stdout:6/481: creat d2/d11/d3b/f9f x:0 0 0 2024-03-16T09:01:14.878 INFO:tasks.workunit.client.0.smithi007.stdout:6/482: dread - d2/d11/d3b/f76 zero size 2024-03-16T09:01:14.878 INFO:tasks.workunit.client.0.smithi007.stdout:9/591: sync 2024-03-16T09:01:14.878 INFO:tasks.workunit.client.0.smithi007.stdout:8/814: sync 2024-03-16T09:01:14.878 INFO:tasks.workunit.client.0.smithi007.stdout:8/815: stat d0/dc/dd/d8b/d9e/fa0 0 2024-03-16T09:01:14.881 INFO:tasks.workunit.client.0.smithi007.stdout:8/816: symlink d0/d26/l124 0 2024-03-16T09:01:14.881 INFO:tasks.workunit.client.0.smithi007.stdout:8/817: creat d0/dc/d50/d8e/dd2/dea/d18/da7/f125 x:0 0 0 2024-03-16T09:01:14.882 INFO:tasks.workunit.client.0.smithi007.stdout:8/818: truncate d0/dc/d75/dc3/f107 394994 0 2024-03-16T09:01:14.884 INFO:tasks.workunit.client.1.smithi019.stdout:1/271: dwrite f9 [0,4194304] 0 2024-03-16T09:01:14.884 INFO:tasks.workunit.client.0.smithi007.stdout:8/819: dread d0/dc/d8a/fb6 [0,4194304] 0 2024-03-16T09:01:14.887 INFO:tasks.workunit.client.1.smithi019.stdout:0/320: write d7/d8/fc [4012996,7861] 0 2024-03-16T09:01:14.887 INFO:tasks.workunit.client.1.smithi019.stdout:0/321: write d7/d1d/d28/f5a [146774,76928] 0 2024-03-16T09:01:14.887 INFO:tasks.workunit.client.1.smithi019.stdout:0/322: chown d7/d8/f12 4006 1 2024-03-16T09:01:14.889 INFO:tasks.workunit.client.0.smithi007.stdout:8/820: rename d0/dc/d50/d8e/dd2/dea/d18/l6e to d0/dc/dd/d8b/dc9/l126 0 2024-03-16T09:01:14.889 INFO:tasks.workunit.client.0.smithi007.stdout:7/453: sync 2024-03-16T09:01:14.889 INFO:tasks.workunit.client.0.smithi007.stdout:8/821: chown d0/dc/d50/d8e/dd2/dea/d74/f96 5136671 1 2024-03-16T09:01:14.889 INFO:tasks.workunit.client.0.smithi007.stdout:8/822: chown d0/dc/d50/d8e/dd2/dea/d18/c4a 918625909 1 2024-03-16T09:01:14.890 INFO:tasks.workunit.client.1.smithi019.stdout:4/345: dwrite d4/f12 [0,4194304] 0 2024-03-16T09:01:14.892 INFO:tasks.workunit.client.0.smithi007.stdout:7/454: link d0/d19/l35 d0/l9b 0 2024-03-16T09:01:14.892 INFO:tasks.workunit.client.0.smithi007.stdout:7/455: write d0/d6c/d25/d7a/f71 [415398,117723] 0 2024-03-16T09:01:14.894 INFO:tasks.workunit.client.0.smithi007.stdout:7/456: mkdir d0/d6c/d25/d7a/d9a/d9c 0 2024-03-16T09:01:14.894 INFO:tasks.workunit.client.1.smithi019.stdout:2/383: link d5/d6/d2b/d2f/c41 d5/d6/d2b/d2f/d35/c7f 0 2024-03-16T09:01:14.895 INFO:tasks.workunit.client.0.smithi007.stdout:3/587: dwrite d15/d18/d1f/d29/d3e/d85/fac [0,4194304] 0 2024-03-16T09:01:14.897 INFO:tasks.workunit.client.0.smithi007.stdout:6/483: dread d2/d11/d1e/d54/f63 [0,4194304] 0 2024-03-16T09:01:14.897 INFO:tasks.workunit.client.0.smithi007.stdout:6/484: chown d2/d11/d1e/d54/d7a 608 1 2024-03-16T09:01:14.898 INFO:tasks.workunit.client.0.smithi007.stdout:8/823: rmdir d0/d26/d2d 39 2024-03-16T09:01:14.898 INFO:tasks.workunit.client.0.smithi007.stdout:8/824: fsync d0/d26/d2d/d43/d72/dad/ffa 0 2024-03-16T09:01:14.898 INFO:tasks.workunit.client.0.smithi007.stdout:8/825: fdatasync d0/dc/f11c 0 2024-03-16T09:01:14.898 INFO:tasks.workunit.client.0.smithi007.stdout:8/826: creat d0/dc/d50/f127 x:0 0 0 2024-03-16T09:01:14.899 INFO:tasks.workunit.client.0.smithi007.stdout:8/827: write d0/dc/d75/dc3/f107 [285657,75102] 0 2024-03-16T09:01:14.900 INFO:tasks.workunit.client.0.smithi007.stdout:3/588: rename d15/d18/f4f to d15/d18/d1f/d29/d7c/fba 0 2024-03-16T09:01:14.901 INFO:tasks.workunit.client.1.smithi019.stdout:3/429: mknod d3/d17/d5a/d75/d7a/c86 0 2024-03-16T09:01:14.901 INFO:tasks.workunit.client.0.smithi007.stdout:8/828: dread d0/d26/d95/fb9 [0,4194304] 0 2024-03-16T09:01:14.902 INFO:tasks.workunit.client.0.smithi007.stdout:6/485: creat d2/d82/fa0 x:0 0 0 2024-03-16T09:01:14.902 INFO:tasks.workunit.client.0.smithi007.stdout:6/486: chown d2/c3 361430631 1 2024-03-16T09:01:14.902 INFO:tasks.workunit.client.0.smithi007.stdout:6/487: write d2/d13/d3f/f44 [735959,81045] 0 2024-03-16T09:01:14.902 INFO:tasks.workunit.client.0.smithi007.stdout:6/488: read - d2/d11/d3b/f87 zero size 2024-03-16T09:01:14.905 INFO:tasks.workunit.client.0.smithi007.stdout:3/589: mknod d15/d18/d1f/d56/cbb 0 2024-03-16T09:01:14.906 INFO:tasks.workunit.client.1.smithi019.stdout:8/357: mkdir d6/d2a/d45/d57/d72/d79 0 2024-03-16T09:01:14.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/590: symlink d15/d18/d6a/d78/lbc 0 2024-03-16T09:01:14.916 INFO:tasks.workunit.client.0.smithi007.stdout:3/591: chown d15/d18/d6a/db1/db4 9 1 2024-03-16T09:01:14.919 INFO:tasks.workunit.client.0.smithi007.stdout:8/829: rmdir d0/dc/dd/d8b 39 2024-03-16T09:01:14.920 INFO:tasks.workunit.client.0.smithi007.stdout:3/592: getdents d15/d18/d1f/d29/d3e/d4c 0 2024-03-16T09:01:14.920 INFO:tasks.workunit.client.0.smithi007.stdout:3/593: fdatasync f10 0 2024-03-16T09:01:14.922 INFO:tasks.workunit.client.0.smithi007.stdout:8/830: truncate d0/d26/d2d/d43/d4c/dba/fd5 1529401 0 2024-03-16T09:01:14.922 INFO:tasks.workunit.client.0.smithi007.stdout:8/831: fsync d0/dc/d50/ffc 0 2024-03-16T09:01:14.922 INFO:tasks.workunit.client.0.smithi007.stdout:3/594: symlink d15/d18/d1f/d29/d3e/d85/daf/lbd 0 2024-03-16T09:01:14.923 INFO:tasks.workunit.client.1.smithi019.stdout:1/272: unlink d16/l25 0 2024-03-16T09:01:14.923 INFO:tasks.workunit.client.0.smithi007.stdout:8/832: truncate d0/d26/f39 1090161 0 2024-03-16T09:01:14.924 INFO:tasks.workunit.client.0.smithi007.stdout:3/595: rmdir d15/d18/d1f/d56 39 2024-03-16T09:01:14.924 INFO:tasks.workunit.client.0.smithi007.stdout:3/596: readlink d15/d18/d1f/d3b/l71 0 2024-03-16T09:01:14.925 INFO:tasks.workunit.client.0.smithi007.stdout:8/833: mkdir d0/d26/d2d/d43/d4c/deb/d128 0 2024-03-16T09:01:14.926 INFO:tasks.workunit.client.0.smithi007.stdout:3/597: mknod d15/d18/d1f/d3b/dae/cbe 0 2024-03-16T09:01:14.927 INFO:tasks.workunit.client.0.smithi007.stdout:8/834: symlink d0/dc/dd/l129 0 2024-03-16T09:01:14.927 INFO:tasks.workunit.client.0.smithi007.stdout:8/835: truncate d0/d26/d2d/d43/d4c/dba/f105 306510 0 2024-03-16T09:01:14.928 INFO:tasks.workunit.client.0.smithi007.stdout:8/836: write d0/dc/dd/fda [522950,128240] 0 2024-03-16T09:01:14.928 INFO:tasks.workunit.client.0.smithi007.stdout:8/837: readlink d0/dc/d50/d8e/dd2/dea/le7 0 2024-03-16T09:01:14.928 INFO:tasks.workunit.client.0.smithi007.stdout:3/598: rmdir d15/d18/d6a/db1/db4 0 2024-03-16T09:01:14.935 INFO:tasks.workunit.client.1.smithi019.stdout:0/323: dwrite d7/d1d/f4b [0,4194304] 0 2024-03-16T09:01:14.937 INFO:tasks.workunit.client.0.smithi007.stdout:3/599: dread d15/d18/f22 [0,4194304] 0 2024-03-16T09:01:14.938 INFO:tasks.workunit.client.0.smithi007.stdout:3/600: symlink d15/d18/d1f/d29/d7c/lbf 0 2024-03-16T09:01:14.940 INFO:tasks.workunit.client.0.smithi007.stdout:7/457: dwrite d0/dc/fd [0,4194304] 0 2024-03-16T09:01:14.940 INFO:tasks.workunit.client.0.smithi007.stdout:7/458: fsync d0/f11 0 2024-03-16T09:01:14.941 INFO:tasks.workunit.client.0.smithi007.stdout:3/601: rename d15/d18/d1f/d3b/d63/d8f to d15/d18/d1f/d29/d3e/dc0 0 2024-03-16T09:01:14.941 INFO:tasks.workunit.client.0.smithi007.stdout:3/602: write d15/d18/d6a/f7e [2055015,97037] 0 2024-03-16T09:01:14.943 INFO:tasks.workunit.client.1.smithi019.stdout:4/346: dwrite d4/f35 [0,4194304] 0 2024-03-16T09:01:14.943 INFO:tasks.workunit.client.1.smithi019.stdout:4/347: getdents d4/d6c/d79 0 2024-03-16T09:01:14.943 INFO:tasks.workunit.client.0.smithi007.stdout:8/838: dwrite d0/d26/d2d/d43/d4c/dba/f105 [0,4194304] 0 2024-03-16T09:01:14.944 INFO:tasks.workunit.client.0.smithi007.stdout:6/489: dwrite d2/d11/d3b/d93/d8e/d4e/f5b [0,4194304] 0 2024-03-16T09:01:14.944 INFO:tasks.workunit.client.0.smithi007.stdout:6/490: chown d2/d11/d1e/d54/d7a/d9a 708709 1 2024-03-16T09:01:14.948 INFO:tasks.workunit.client.0.smithi007.stdout:3/603: creat d15/d18/d6a/db1/fc1 x:0 0 0 2024-03-16T09:01:14.948 INFO:tasks.workunit.client.0.smithi007.stdout:3/604: write d15/d18/d1f/d3b/d87/f97 [686701,9723] 0 2024-03-16T09:01:14.953 INFO:tasks.workunit.client.0.smithi007.stdout:8/839: mknod d0/dc/d50/d8e/dd2/dea/d18/da7/c12a 0 2024-03-16T09:01:14.953 INFO:tasks.workunit.client.0.smithi007.stdout:8/840: fsync d0/d26/d95/fb9 0 2024-03-16T09:01:14.954 INFO:tasks.workunit.client.0.smithi007.stdout:6/491: dread d2/d11/f25 [0,4194304] 0 2024-03-16T09:01:14.957 INFO:tasks.workunit.client.0.smithi007.stdout:7/459: dwrite d0/d6c/d25/d73/f91 [0,4194304] 0 2024-03-16T09:01:14.959 INFO:tasks.workunit.client.1.smithi019.stdout:2/384: mkdir d5/d5a/d80 0 2024-03-16T09:01:14.960 INFO:tasks.workunit.client.1.smithi019.stdout:3/430: creat d3/d17/d5a/d75/d7a/f87 x:0 0 0 2024-03-16T09:01:14.961 INFO:tasks.workunit.client.1.smithi019.stdout:1/273: symlink d16/d18/l4e 0 2024-03-16T09:01:14.961 INFO:tasks.workunit.client.1.smithi019.stdout:1/274: chown d16/c38 15254095 1 2024-03-16T09:01:14.963 INFO:tasks.workunit.client.0.smithi007.stdout:7/460: dread d0/d6c/d25/d73/f5c [0,4194304] 0 2024-03-16T09:01:14.965 INFO:tasks.workunit.client.1.smithi019.stdout:0/324: truncate d7/d8/d22/f25 524364 0 2024-03-16T09:01:14.965 INFO:tasks.workunit.client.0.smithi007.stdout:3/605: rename d15/d18/d1f/d29/d3e/d4c/l70 to d15/lc2 0 2024-03-16T09:01:14.966 INFO:tasks.workunit.client.0.smithi007.stdout:3/606: stat d15/d18/d1f/d3b/d63/fb7 0 2024-03-16T09:01:14.968 INFO:tasks.workunit.client.0.smithi007.stdout:8/841: link d0/dc/d50/d8e/db8/lec d0/dc/dd/d8b/d9e/l12b 0 2024-03-16T09:01:14.971 INFO:tasks.workunit.client.1.smithi019.stdout:3/431: mkdir d3/d1b/d88 0 2024-03-16T09:01:14.971 INFO:tasks.workunit.client.1.smithi019.stdout:5/331: rename d6/d19/d34/d3e to d6/d1f/d33/d66 0 2024-03-16T09:01:14.971 INFO:tasks.workunit.client.1.smithi019.stdout:5/332: readlink d6/dd/l31 0 2024-03-16T09:01:14.972 INFO:tasks.workunit.client.1.smithi019.stdout:5/333: creat d6/dd/f67 x:0 0 0 2024-03-16T09:01:14.973 INFO:tasks.workunit.client.1.smithi019.stdout:3/432: write d3/d17/d5a/f5c [283982,86793] 0 2024-03-16T09:01:14.973 INFO:tasks.workunit.client.1.smithi019.stdout:3/433: write d3/f6c [655602,60897] 0 2024-03-16T09:01:14.973 INFO:tasks.workunit.client.1.smithi019.stdout:3/434: fdatasync d3/d17/d3f/f7c 0 2024-03-16T09:01:14.973 INFO:tasks.workunit.client.1.smithi019.stdout:3/435: chown f2 3125 1 2024-03-16T09:01:14.973 INFO:tasks.workunit.client.1.smithi019.stdout:3/436: readlink d3/d11/d7d/l81 0 2024-03-16T09:01:14.974 INFO:tasks.workunit.client.1.smithi019.stdout:1/275: mkdir d16/d1f/d4f 0 2024-03-16T09:01:14.974 INFO:tasks.workunit.client.1.smithi019.stdout:1/276: fdatasync d16/f2b 0 2024-03-16T09:01:14.976 INFO:tasks.workunit.client.0.smithi007.stdout:6/492: rename d2/d13/d3f/d50/f89 to d2/d10/d3d/d6d/fa1 0 2024-03-16T09:01:14.976 INFO:tasks.workunit.client.0.smithi007.stdout:6/493: fsync d2/d13/d3f/f44 0 2024-03-16T09:01:14.976 INFO:tasks.workunit.client.0.smithi007.stdout:6/494: chown d2/f7 0 1 2024-03-16T09:01:14.976 INFO:tasks.workunit.client.0.smithi007.stdout:6/495: stat d2/d11/d3b/d93/f35 0 2024-03-16T09:01:14.982 INFO:tasks.workunit.client.0.smithi007.stdout:3/607: truncate d15/d18/d1f/d29/d43/f55 3902656 0 2024-03-16T09:01:14.983 INFO:tasks.workunit.client.1.smithi019.stdout:3/437: rmdir d3/d11/d7d 39 2024-03-16T09:01:14.985 INFO:tasks.workunit.client.0.smithi007.stdout:7/461: rename d0/d6c/d25/d3a/d92 to d0/d19/d1f/d5d/d9d 0 2024-03-16T09:01:14.985 INFO:tasks.workunit.client.0.smithi007.stdout:7/462: chown d0/dc/c56 0 1 2024-03-16T09:01:14.985 INFO:tasks.workunit.client.1.smithi019.stdout:1/277: creat d16/d1f/d4f/f50 x:0 0 0 2024-03-16T09:01:14.988 INFO:tasks.workunit.client.1.smithi019.stdout:1/278: rmdir d16/d1f/d43 39 2024-03-16T09:01:14.990 INFO:tasks.workunit.client.1.smithi019.stdout:2/385: dwrite d5/d6/d2b/d2f/f67 [0,4194304] 0 2024-03-16T09:01:14.990 INFO:tasks.workunit.client.1.smithi019.stdout:2/386: write d5/f34 [1491354,6899] 0 2024-03-16T09:01:14.991 INFO:tasks.workunit.client.0.smithi007.stdout:3/608: write d15/d18/f23 [7507213,27218] 0 2024-03-16T09:01:14.991 INFO:tasks.workunit.client.0.smithi007.stdout:3/609: chown d15/d18/d1f/f8d 55531 1 2024-03-16T09:01:14.991 INFO:tasks.workunit.client.0.smithi007.stdout:3/610: chown d15/d18/d6a/c80 9592 1 2024-03-16T09:01:14.991 INFO:tasks.workunit.client.1.smithi019.stdout:1/279: link f10 d16/d1f/f51 0 2024-03-16T09:01:14.992 INFO:tasks.workunit.client.1.smithi019.stdout:1/280: chown d16/d31/f45 6027 1 2024-03-16T09:01:14.992 INFO:tasks.workunit.client.1.smithi019.stdout:1/281: fdatasync f10 0 2024-03-16T09:01:14.992 INFO:tasks.workunit.client.1.smithi019.stdout:1/282: fsync d16/f49 0 2024-03-16T09:01:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:8/842: dwrite d0/d26/d2d/d43/d6f/da5/fc8 [0,4194304] 0 2024-03-16T09:01:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:3/611: dread d15/d18/d6a/d78/fa8 [4194304,4194304] 0 2024-03-16T09:01:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:8/843: chown d0/dc/d50/d8e/dd2/dea/lca 738378 1 2024-03-16T09:01:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:8/844: stat d0/dc/l3d 0 2024-03-16T09:01:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:3/612: dread - d15/d18/d1f/d29/d7c/fb3 zero size 2024-03-16T09:01:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:8/845: stat d0/dc/d50/d8e/dd2/dea/f104 0 2024-03-16T09:01:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:3/613: chown d15/d18/d1f/d3b/d87/l99 115827847 1 2024-03-16T09:01:14.993 INFO:tasks.workunit.client.0.smithi007.stdout:3/614: write d15/fa4 [774373,58592] 0 2024-03-16T09:01:14.994 INFO:tasks.workunit.client.0.smithi007.stdout:3/615: truncate d15/d18/d1f/d56/fb2 414744 0 2024-03-16T09:01:14.994 INFO:tasks.workunit.client.1.smithi019.stdout:2/387: link d5/d6/d2b/d2f/c59 d5/d6/d64/c81 0 2024-03-16T09:01:14.994 INFO:tasks.workunit.client.1.smithi019.stdout:2/388: chown d5/d6/d2b/d2f/f46 183701 1 2024-03-16T09:01:14.996 INFO:tasks.workunit.client.1.smithi019.stdout:2/389: stat d5/d6/d2b/d2f/d35/d65/c7c 0 2024-03-16T09:01:14.997 INFO:tasks.workunit.client.0.smithi007.stdout:8/846: read d0/d26/f39 [415467,30853] 0 2024-03-16T09:01:14.999 INFO:tasks.workunit.client.0.smithi007.stdout:3/616: getdents d15/d18/d1f/d29/d3e/d85 0 2024-03-16T09:01:15.001 INFO:tasks.workunit.client.0.smithi007.stdout:6/496: dwrite d2/d10/f3a [4194304,4194304] 0 2024-03-16T09:01:15.002 INFO:tasks.workunit.client.1.smithi019.stdout:1/283: read d16/f22 [119222,88813] 0 2024-03-16T09:01:15.003 INFO:tasks.workunit.client.0.smithi007.stdout:5/584: sync 2024-03-16T09:01:15.003 INFO:tasks.workunit.client.0.smithi007.stdout:5/585: readlink d1/d7/d1c/d19/d34/de/l24 0 2024-03-16T09:01:15.003 INFO:tasks.workunit.client.0.smithi007.stdout:2/670: sync 2024-03-16T09:01:15.003 INFO:tasks.workunit.client.0.smithi007.stdout:2/671: stat dc/df/d24/d28/d2e/f78 0 2024-03-16T09:01:15.003 INFO:tasks.workunit.client.0.smithi007.stdout:2/672: chown dc/l42 0 1 2024-03-16T09:01:15.007 INFO:tasks.workunit.client.0.smithi007.stdout:3/617: dread f5 [4194304,4194304] 0 2024-03-16T09:01:15.007 INFO:tasks.workunit.client.1.smithi019.stdout:1/284: symlink d16/d1f/d43/l52 0 2024-03-16T09:01:15.008 INFO:tasks.workunit.client.1.smithi019.stdout:1/285: truncate d16/d1f/d4f/f50 162041 0 2024-03-16T09:01:15.008 INFO:tasks.workunit.client.1.smithi019.stdout:1/286: chown d16/l3b 112040 1 2024-03-16T09:01:15.009 INFO:tasks.workunit.client.0.smithi007.stdout:8/847: rmdir d0/dc 39 2024-03-16T09:01:15.010 INFO:tasks.workunit.client.0.smithi007.stdout:7/463: dwrite d0/d19/f26 [4194304,4194304] 0 2024-03-16T09:01:15.013 INFO:tasks.workunit.client.1.smithi019.stdout:4/348: mknod d4/d6/c7a 0 2024-03-16T09:01:15.013 INFO:tasks.workunit.client.1.smithi019.stdout:4/349: dread - d4/d6/f5f zero size 2024-03-16T09:01:15.014 INFO:tasks.workunit.client.0.smithi007.stdout:5/586: read d1/d7/d1c/d19/d34/d4d/d99/fa3 [379006,50940] 0 2024-03-16T09:01:15.014 INFO:tasks.workunit.client.0.smithi007.stdout:6/497: unlink d2/d11/d3b/d93/d8e/f4f 0 2024-03-16T09:01:15.014 INFO:tasks.workunit.client.0.smithi007.stdout:6/498: creat d2/d82/fa2 x:0 0 0 2024-03-16T09:01:15.014 INFO:tasks.workunit.client.0.smithi007.stdout:6/499: write d2/d13/d4b/f7c [297392,78636] 0 2024-03-16T09:01:15.015 INFO:tasks.workunit.client.0.smithi007.stdout:6/500: write d2/d11/d1e/f2f [5371328,7135] 0 2024-03-16T09:01:15.015 INFO:tasks.workunit.client.1.smithi019.stdout:4/350: truncate d4/d1b/f28 3301695 0 2024-03-16T09:01:15.015 INFO:tasks.workunit.client.0.smithi007.stdout:2/673: rename dc/d27/f92 to dc/d27/da3/fdb 0 2024-03-16T09:01:15.015 INFO:tasks.workunit.client.0.smithi007.stdout:6/501: rename d2/d11 to d2/d11/d3b/d93/d15/da3 22 2024-03-16T09:01:15.017 INFO:tasks.workunit.client.0.smithi007.stdout:3/618: truncate f5 3195105 0 2024-03-16T09:01:15.017 INFO:tasks.workunit.client.0.smithi007.stdout:3/619: readlink d15/d18/d1f/d3b/d63/l67 0 2024-03-16T09:01:15.017 INFO:tasks.workunit.client.0.smithi007.stdout:3/620: write d15/d18/f22 [3062116,60198] 0 2024-03-16T09:01:15.017 INFO:tasks.workunit.client.1.smithi019.stdout:4/351: unlink d4/d6/d15/c2d 0 2024-03-16T09:01:15.019 INFO:tasks.workunit.client.0.smithi007.stdout:8/848: dread d0/d26/d2d/ff2 [0,4194304] 0 2024-03-16T09:01:15.020 INFO:tasks.workunit.client.0.smithi007.stdout:7/464: getdents d0/d6c/d25/d7a/d9a 0 2024-03-16T09:01:15.020 INFO:tasks.workunit.client.0.smithi007.stdout:5/587: link d1/d7/d16/d1b/f60 d1/d7/d1c/d19/d34/d4d/d99/fd7 0 2024-03-16T09:01:15.020 INFO:tasks.workunit.client.0.smithi007.stdout:5/588: read - d1/d7/d1c/d19/d34/d5e/d6f/fb4 zero size 2024-03-16T09:01:15.021 INFO:tasks.workunit.client.1.smithi019.stdout:2/390: dwrite d5/d6/d2b/d2f/f46 [0,4194304] 0 2024-03-16T09:01:15.021 INFO:tasks.workunit.client.1.smithi019.stdout:2/391: write d5/d6/d2b/d2f/d35/f7e [913153,13928] 0 2024-03-16T09:01:15.021 INFO:tasks.workunit.client.1.smithi019.stdout:2/392: write d5/d6/d2b/d2f/f3a [9767,95271] 0 2024-03-16T09:01:15.021 INFO:tasks.workunit.client.1.smithi019.stdout:2/393: write d5/d2e/f3d [1947584,49367] 0 2024-03-16T09:01:15.027 INFO:tasks.workunit.client.0.smithi007.stdout:8/849: mkdir d0/dc/d50/d8e/dd2/dea/d18/da7/de2/d113/d12c 0 2024-03-16T09:01:15.027 INFO:tasks.workunit.client.0.smithi007.stdout:7/465: rmdir d0/d6c/d25/d7a/d9a 39 2024-03-16T09:01:15.031 INFO:tasks.workunit.client.1.smithi019.stdout:2/394: dread d5/d6/d2b/f72 [0,4194304] 0 2024-03-16T09:01:15.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/674: getdents dc/df/d24/d28/d2e 0 2024-03-16T09:01:15.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/675: chown dc/df/d6c/fa9 117 1 2024-03-16T09:01:15.032 INFO:tasks.workunit.client.0.smithi007.stdout:2/676: creat dc/df/d24/d28/d4c/d5e/fdc x:0 0 0 2024-03-16T09:01:15.034 INFO:tasks.workunit.client.0.smithi007.stdout:8/850: unlink d0/l1b 0 2024-03-16T09:01:15.036 INFO:tasks.workunit.client.0.smithi007.stdout:7/466: creat d0/d19/d1f/d5d/d8a/d93/f9e x:0 0 0 2024-03-16T09:01:15.036 INFO:tasks.workunit.client.0.smithi007.stdout:7/467: truncate d0/d19/d1f/d42/f45 1415950 0 2024-03-16T09:01:15.038 INFO:tasks.workunit.client.0.smithi007.stdout:2/677: creat dc/df/d24/d28/d2e/fdd x:0 0 0 2024-03-16T09:01:15.038 INFO:tasks.workunit.client.0.smithi007.stdout:2/678: write dc/df/d24/d28/d4c/d5e/f4d [110207,1524] 0 2024-03-16T09:01:15.038 INFO:tasks.workunit.client.0.smithi007.stdout:2/679: chown dc/f3f 26983418 1 2024-03-16T09:01:15.039 INFO:tasks.workunit.client.0.smithi007.stdout:7/468: symlink d0/d6c/d25/l9f 0 2024-03-16T09:01:15.062 INFO:tasks.workunit.client.0.smithi007.stdout:4/470: sync 2024-03-16T09:01:15.064 INFO:tasks.workunit.client.0.smithi007.stdout:4/471: rename d4/d14/d1c/d1e/f59 to d4/d7/d78/d80/d84/d89/f9b 0 2024-03-16T09:01:15.066 INFO:tasks.workunit.client.0.smithi007.stdout:4/472: mknod d4/d14/d1c/d5b/d67/d8f/c9c 0 2024-03-16T09:01:15.076 INFO:tasks.workunit.client.0.smithi007.stdout:7/469: dread d0/d19/f48 [0,4194304] 0 2024-03-16T09:01:15.077 INFO:tasks.workunit.client.0.smithi007.stdout:7/470: write d0/f49 [2957623,59748] 0 2024-03-16T09:01:15.078 INFO:tasks.workunit.client.0.smithi007.stdout:6/502: dwrite d2/d13/d3f/d50/d72/f69 [0,4194304] 0 2024-03-16T09:01:15.078 INFO:tasks.workunit.client.0.smithi007.stdout:6/503: dread - d2/d11/d3b/d93/d8e/d4e/f64 zero size 2024-03-16T09:01:15.078 INFO:tasks.workunit.client.0.smithi007.stdout:6/504: write d2/d11/d1e/f2f [5876650,108702] 0 2024-03-16T09:01:15.079 INFO:tasks.workunit.client.0.smithi007.stdout:7/471: link d0/c59 d0/ca0 0 2024-03-16T09:01:15.079 INFO:tasks.workunit.client.0.smithi007.stdout:8/851: dwrite d0/d26/f79 [0,4194304] 0 2024-03-16T09:01:15.080 INFO:tasks.workunit.client.1.smithi019.stdout:2/395: dwrite d5/de/f5d [0,4194304] 0 2024-03-16T09:01:15.080 INFO:tasks.workunit.client.1.smithi019.stdout:2/396: creat d5/d6/d2b/f82 x:0 0 0 2024-03-16T09:01:15.082 INFO:tasks.workunit.client.0.smithi007.stdout:2/680: dwrite dc/df/d24/d28/d2e/d35/fb4 [0,4194304] 0 2024-03-16T09:01:15.082 INFO:tasks.workunit.client.0.smithi007.stdout:2/681: stat dc/df/f18 0 2024-03-16T09:01:15.082 INFO:tasks.workunit.client.0.smithi007.stdout:8/852: write d0/dc/dd/d8b/d9e/fa0 [618369,59366] 0 2024-03-16T09:01:15.082 INFO:tasks.workunit.client.0.smithi007.stdout:8/853: write d0/dc/dd/d8b/fd0 [1712069,29796] 0 2024-03-16T09:01:15.082 INFO:tasks.workunit.client.0.smithi007.stdout:6/505: unlink d2/d11/d3b/d93/d8e/f4d 0 2024-03-16T09:01:15.083 INFO:tasks.workunit.client.0.smithi007.stdout:2/682: dread dc/df/fca [0,4194304] 0 2024-03-16T09:01:15.083 INFO:tasks.workunit.client.1.smithi019.stdout:4/352: dwrite d4/d6/d15/d1f/d2c/f3f [0,4194304] 0 2024-03-16T09:01:15.083 INFO:tasks.workunit.client.1.smithi019.stdout:4/353: creat d4/d6/d15/d1f/d2c/f7b x:0 0 0 2024-03-16T09:01:15.083 INFO:tasks.workunit.client.1.smithi019.stdout:4/354: write d4/d6/d15/f2e [1087043,20490] 0 2024-03-16T09:01:15.084 INFO:tasks.workunit.client.1.smithi019.stdout:4/355: write d4/d1b/d49/f72 [145114,70450] 0 2024-03-16T09:01:15.085 INFO:tasks.workunit.client.1.smithi019.stdout:2/397: mkdir d5/d6/d2b/d2f/d35/d4b/d51/d7d/d83 0 2024-03-16T09:01:15.085 INFO:tasks.workunit.client.0.smithi007.stdout:7/472: mkdir d0/d6c/d25/da1 0 2024-03-16T09:01:15.089 INFO:tasks.workunit.client.0.smithi007.stdout:8/854: link d0/dc/dd/d8b/fd0 d0/d26/d2d/d64/d123/f12d 0 2024-03-16T09:01:15.089 INFO:tasks.workunit.client.1.smithi019.stdout:2/398: creat d5/d5a/f84 x:0 0 0 2024-03-16T09:01:15.091 INFO:tasks.workunit.client.1.smithi019.stdout:2/399: truncate d5/de/f4a 1031170 0 2024-03-16T09:01:15.091 INFO:tasks.workunit.client.1.smithi019.stdout:2/400: write d5/d2e/f75 [839910,8996] 0 2024-03-16T09:01:15.091 INFO:tasks.workunit.client.1.smithi019.stdout:2/401: truncate d5/d6/d2b/f82 587481 0 2024-03-16T09:01:15.091 INFO:tasks.workunit.client.1.smithi019.stdout:2/402: chown d5/d6/d2b/d2f/d35/f58 10108598 1 2024-03-16T09:01:15.091 INFO:tasks.workunit.client.1.smithi019.stdout:2/403: chown d5/d6/d2b/f5b 2925 1 2024-03-16T09:01:15.092 INFO:tasks.workunit.client.1.smithi019.stdout:5/334: rmdir d6/d1f/d33/d66 39 2024-03-16T09:01:15.092 INFO:tasks.workunit.client.1.smithi019.stdout:5/335: chown d6/dd/c1e 105106582 1 2024-03-16T09:01:15.092 INFO:tasks.workunit.client.1.smithi019.stdout:5/336: fdatasync d6/dd/f1d 0 2024-03-16T09:01:15.092 INFO:tasks.workunit.client.0.smithi007.stdout:6/506: creat d2/d13/d3f/d50/d72/fa4 x:0 0 0 2024-03-16T09:01:15.092 INFO:tasks.workunit.client.0.smithi007.stdout:6/507: creat d2/d13/d3f/d50/d72/fa5 x:0 0 0 2024-03-16T09:01:15.093 INFO:tasks.workunit.client.0.smithi007.stdout:4/473: dwrite d4/d14/f95 [0,4194304] 0 2024-03-16T09:01:15.094 INFO:tasks.workunit.client.0.smithi007.stdout:7/473: creat d0/d6c/d25/d7a/d9a/fa2 x:0 0 0 2024-03-16T09:01:15.094 INFO:tasks.workunit.client.1.smithi019.stdout:2/404: mknod d5/d6/d2b/d2f/d35/d4b/d51/d7d/c85 0 2024-03-16T09:01:15.096 INFO:tasks.workunit.client.1.smithi019.stdout:5/337: link d6/dd/l15 d6/dd/d37/d54/l68 0 2024-03-16T09:01:15.099 INFO:tasks.workunit.client.0.smithi007.stdout:8/855: read d0/dc/f3a [3909721,25235] 0 2024-03-16T09:01:15.099 INFO:tasks.workunit.client.0.smithi007.stdout:8/856: stat d0/d26/d2d/d64/d123/f12d 0 2024-03-16T09:01:15.099 INFO:tasks.workunit.client.0.smithi007.stdout:6/508: dread d2/f20 [0,4194304] 0 2024-03-16T09:01:15.099 INFO:tasks.workunit.client.0.smithi007.stdout:2/683: getdents dc/df/d24 0 2024-03-16T09:01:15.099 INFO:tasks.workunit.client.0.smithi007.stdout:6/509: creat d2/d13/d4b/fa6 x:0 0 0 2024-03-16T09:01:15.102 INFO:tasks.workunit.client.0.smithi007.stdout:7/474: rename d0/d6c/f3b to d0/d19/d1f/d76/fa3 0 2024-03-16T09:01:15.102 INFO:tasks.workunit.client.0.smithi007.stdout:4/474: write d4/f51 [5575086,22512] 0 2024-03-16T09:01:15.103 INFO:tasks.workunit.client.1.smithi019.stdout:1/287: getdents d16/d1f/d43 0 2024-03-16T09:01:15.105 INFO:tasks.workunit.client.1.smithi019.stdout:1/288: write d16/d31/f45 [177261,17025] 0 2024-03-16T09:01:15.105 INFO:tasks.workunit.client.1.smithi019.stdout:2/405: write d5/d6/d2b/f3c [2697956,113316] 0 2024-03-16T09:01:15.105 INFO:tasks.workunit.client.1.smithi019.stdout:1/289: fsync d16/d1f/f30 0 2024-03-16T09:01:15.105 INFO:tasks.workunit.client.1.smithi019.stdout:2/406: fdatasync d5/de/f55 0 2024-03-16T09:01:15.106 INFO:tasks.workunit.client.1.smithi019.stdout:4/356: dwrite d4/d6/f70 [0,4194304] 0 2024-03-16T09:01:15.108 INFO:tasks.workunit.client.1.smithi019.stdout:9/246: rename d1/f25 to d1/d7/f56 0 2024-03-16T09:01:15.108 INFO:tasks.workunit.client.1.smithi019.stdout:9/247: dread - d1/d7/de/d24/d44/f53 zero size 2024-03-16T09:01:15.108 INFO:tasks.workunit.client.0.smithi007.stdout:2/684: truncate dc/df/f14 581347 0 2024-03-16T09:01:15.109 INFO:tasks.workunit.client.0.smithi007.stdout:2/685: write dc/df/d6c/fcd [636238,83213] 0 2024-03-16T09:01:15.110 INFO:tasks.workunit.client.0.smithi007.stdout:6/510: creat d2/d11/d1e/d54/d7a/d9a/fa7 x:0 0 0 2024-03-16T09:01:15.110 INFO:tasks.workunit.client.1.smithi019.stdout:1/290: mknod d16/d31/c53 0 2024-03-16T09:01:15.113 INFO:tasks.workunit.client.1.smithi019.stdout:4/357: truncate d4/d1b/d2a/d3d/f46 1477745 0 2024-03-16T09:01:15.113 INFO:tasks.workunit.client.1.smithi019.stdout:4/358: read - d4/d1b/d2a/f4d zero size 2024-03-16T09:01:15.113 INFO:tasks.workunit.client.1.smithi019.stdout:4/359: chown d4/d6/d15/d1f/d2c/f6b 36 1 2024-03-16T09:01:15.113 INFO:tasks.workunit.client.1.smithi019.stdout:4/360: getdents d4/d73 0 2024-03-16T09:01:15.113 INFO:tasks.workunit.client.0.smithi007.stdout:7/475: creat d0/d19/d1f/d5d/d9d/fa4 x:0 0 0 2024-03-16T09:01:15.114 INFO:tasks.workunit.client.0.smithi007.stdout:7/476: write d0/d6c/d25/d7a/d9a/fa2 [673856,58867] 0 2024-03-16T09:01:15.114 INFO:tasks.workunit.client.0.smithi007.stdout:7/477: creat d0/fa5 x:0 0 0 2024-03-16T09:01:15.115 INFO:tasks.workunit.client.0.smithi007.stdout:4/475: mkdir d4/d7/d9d 0 2024-03-16T09:01:15.115 INFO:tasks.workunit.client.0.smithi007.stdout:4/476: write d4/d14/f35 [1595409,100148] 0 2024-03-16T09:01:15.115 INFO:tasks.workunit.client.1.smithi019.stdout:7/259: rename d9/d25/d28/l49 to d9/d25/l4d 0 2024-03-16T09:01:15.116 INFO:tasks.workunit.client.0.smithi007.stdout:6/511: symlink d2/d11/d3b/la8 0 2024-03-16T09:01:15.118 INFO:tasks.workunit.client.1.smithi019.stdout:9/248: link d1/d7/f34 d1/d7/d21/d27/f57 0 2024-03-16T09:01:15.119 INFO:tasks.workunit.client.0.smithi007.stdout:7/478: symlink d0/d6c/la6 0 2024-03-16T09:01:15.120 INFO:tasks.workunit.client.1.smithi019.stdout:8/358: rename d6/de/c13 to d6/de/d24/c7a 0 2024-03-16T09:01:15.121 INFO:tasks.workunit.client.1.smithi019.stdout:5/338: dwrite d6/d19/f4e [4194304,4194304] 0 2024-03-16T09:01:15.121 INFO:tasks.workunit.client.0.smithi007.stdout:7/479: read d0/d19/f2d [371608,113752] 0 2024-03-16T09:01:15.122 INFO:tasks.workunit.client.0.smithi007.stdout:4/477: symlink d4/d7/d78/d80/d84/l9e 0 2024-03-16T09:01:15.123 INFO:tasks.workunit.client.1.smithi019.stdout:9/249: creat d1/d7/d21/d27/f58 x:0 0 0 2024-03-16T09:01:15.124 INFO:tasks.workunit.client.1.smithi019.stdout:9/250: dread d1/d7/d21/d27/f13 [0,4194304] 0 2024-03-16T09:01:15.125 INFO:tasks.workunit.client.0.smithi007.stdout:8/857: dwrite d0/dc/d50/d8e/dd2/dea/f67 [0,4194304] 0 2024-03-16T09:01:15.125 INFO:tasks.workunit.client.1.smithi019.stdout:0/325: rename d7/d8/d22 to d7/d8/d32/d4a/d61 0 2024-03-16T09:01:15.128 INFO:tasks.workunit.client.0.smithi007.stdout:7/480: creat d0/d19/d1f/d5d/fa7 x:0 0 0 2024-03-16T09:01:15.131 INFO:tasks.workunit.client.1.smithi019.stdout:3/438: rename d3/d11/d7d to d3/d89 0 2024-03-16T09:01:15.133 INFO:tasks.workunit.client.1.smithi019.stdout:5/339: getdents d6/d1f/d33/d4c 0 2024-03-16T09:01:15.133 INFO:tasks.workunit.client.1.smithi019.stdout:5/340: stat d6/d1f/d33/d4c/f62 0 2024-03-16T09:01:15.133 INFO:tasks.workunit.client.1.smithi019.stdout:5/341: creat d6/dd/d47/f69 x:0 0 0 2024-03-16T09:01:15.133 INFO:tasks.workunit.client.1.smithi019.stdout:5/342: write d6/dd/d37/f60 [884203,33794] 0 2024-03-16T09:01:15.134 INFO:tasks.workunit.client.1.smithi019.stdout:7/260: write d9/da/d13/d18/d1a/f1f [2833840,21388] 0 2024-03-16T09:01:15.134 INFO:tasks.workunit.client.1.smithi019.stdout:7/261: read - d9/da/d13/d18/d1a/d32/d33/f43 zero size 2024-03-16T09:01:15.134 INFO:tasks.workunit.client.1.smithi019.stdout:7/262: stat d9/db/d12/c1e 0 2024-03-16T09:01:15.134 INFO:tasks.workunit.client.1.smithi019.stdout:7/263: stat d9/d25/d28/d48 0 2024-03-16T09:01:15.135 INFO:tasks.workunit.client.0.smithi007.stdout:4/478: dread d4/d14/d64/d68/f56 [0,4194304] 0 2024-03-16T09:01:15.135 INFO:tasks.workunit.client.1.smithi019.stdout:1/291: rename d16/d1f/c29 to d16/d31/c54 0 2024-03-16T09:01:15.136 INFO:tasks.workunit.client.1.smithi019.stdout:1/292: stat d16/d1f/d3c 0 2024-03-16T09:01:15.136 INFO:tasks.workunit.client.1.smithi019.stdout:1/293: read d16/d18/f1a [1235251,113545] 0 2024-03-16T09:01:15.136 INFO:tasks.workunit.client.0.smithi007.stdout:7/481: dread d0/d19/f48 [0,4194304] 0 2024-03-16T09:01:15.138 INFO:tasks.workunit.client.1.smithi019.stdout:3/439: symlink d3/d17/l8a 0 2024-03-16T09:01:15.140 INFO:tasks.workunit.client.1.smithi019.stdout:9/251: write d1/d7/d21/d27/f22 [5828961,95952] 0 2024-03-16T09:01:15.140 INFO:tasks.workunit.client.1.smithi019.stdout:9/252: fsync d1/d7/de/d2d/f30 0 2024-03-16T09:01:15.140 INFO:tasks.workunit.client.1.smithi019.stdout:5/343: dread d6/d19/f1b [0,4194304] 0 2024-03-16T09:01:15.140 INFO:tasks.workunit.client.1.smithi019.stdout:5/344: fsync d6/d1f/d33/f4f 0 2024-03-16T09:01:15.140 INFO:tasks.workunit.client.0.smithi007.stdout:8/858: creat d0/d26/d2d/d43/d4c/dba/f12e x:0 0 0 2024-03-16T09:01:15.141 INFO:tasks.workunit.client.0.smithi007.stdout:8/859: fdatasync d0/dc/dd/d8b/fd0 0 2024-03-16T09:01:15.141 INFO:tasks.workunit.client.0.smithi007.stdout:8/860: chown d0/d26/d2d/d43/d6f/f8f 50378 1 2024-03-16T09:01:15.141 INFO:tasks.workunit.client.1.smithi019.stdout:7/264: rmdir d9/db/d4c 0 2024-03-16T09:01:15.141 INFO:tasks.workunit.client.1.smithi019.stdout:0/326: dread d7/d8/d32/d36/f3b [0,4194304] 0 2024-03-16T09:01:15.142 INFO:tasks.workunit.client.0.smithi007.stdout:4/479: unlink d4/f51 0 2024-03-16T09:01:15.143 INFO:tasks.workunit.client.1.smithi019.stdout:4/361: rename d4/d64/f68 to d4/d6c/f7c 0 2024-03-16T09:01:15.143 INFO:tasks.workunit.client.1.smithi019.stdout:4/362: creat d4/d1b/d49/f7d x:0 0 0 2024-03-16T09:01:15.143 INFO:tasks.workunit.client.1.smithi019.stdout:8/359: dwrite d6/de/f76 [0,4194304] 0 2024-03-16T09:01:15.144 INFO:tasks.workunit.client.0.smithi007.stdout:7/482: creat d0/d6c/d25/d7a/fa8 x:0 0 0 2024-03-16T09:01:15.145 INFO:tasks.workunit.client.0.smithi007.stdout:4/480: rename d4/d14/d1c/d1e to d4/d14/d64/d9f 0 2024-03-16T09:01:15.145 INFO:tasks.workunit.client.0.smithi007.stdout:4/481: creat d4/d14/d1c/d5b/fa0 x:0 0 0 2024-03-16T09:01:15.147 INFO:tasks.workunit.client.0.smithi007.stdout:7/483: rename d0/d6c/la6 to d0/d19/d1f/d42/la9 0 2024-03-16T09:01:15.147 INFO:tasks.workunit.client.0.smithi007.stdout:7/484: write d0/d6c/d25/d3a/f96 [188433,122284] 0 2024-03-16T09:01:15.147 INFO:tasks.workunit.client.1.smithi019.stdout:2/407: unlink d5/l49 0 2024-03-16T09:01:15.149 INFO:tasks.workunit.client.1.smithi019.stdout:9/253: mkdir d1/d7/d33/d59 0 2024-03-16T09:01:15.149 INFO:tasks.workunit.client.1.smithi019.stdout:9/254: fsync d1/d7/de/d24/f40 0 2024-03-16T09:01:15.151 INFO:tasks.workunit.client.1.smithi019.stdout:5/345: symlink d6/d1f/d33/d4c/l6a 0 2024-03-16T09:01:15.152 INFO:tasks.workunit.client.0.smithi007.stdout:4/482: rename d4/fa to d4/d14/d1c/d5b/d67/d8f/d90/fa1 0 2024-03-16T09:01:15.153 INFO:tasks.workunit.client.0.smithi007.stdout:0/640: sync 2024-03-16T09:01:15.153 INFO:tasks.workunit.client.0.smithi007.stdout:9/592: sync 2024-03-16T09:01:15.154 INFO:tasks.workunit.client.0.smithi007.stdout:1/506: sync 2024-03-16T09:01:15.154 INFO:tasks.workunit.client.0.smithi007.stdout:1/507: truncate d9/d6b/d7e/f9a 310665 0 2024-03-16T09:01:15.154 INFO:tasks.workunit.client.0.smithi007.stdout:9/593: readlink d24/d4a/d55/d56/d7c/d7f/d8f/l97 0 2024-03-16T09:01:15.154 INFO:tasks.workunit.client.0.smithi007.stdout:9/594: write d24/d4a/d55/f5d [99246,52906] 0 2024-03-16T09:01:15.154 INFO:tasks.workunit.client.0.smithi007.stdout:9/595: chown d24/d4a/d55/l5f 216421 1 2024-03-16T09:01:15.154 INFO:tasks.workunit.client.0.smithi007.stdout:9/596: truncate fd 4986116 0 2024-03-16T09:01:15.154 INFO:tasks.workunit.client.1.smithi019.stdout:0/327: symlink d7/d29/l62 0 2024-03-16T09:01:15.154 INFO:tasks.workunit.client.1.smithi019.stdout:0/328: fsync d7/f19 0 2024-03-16T09:01:15.156 INFO:tasks.workunit.client.0.smithi007.stdout:0/641: mknod d3/d6c/dbd/cd5 0 2024-03-16T09:01:15.157 INFO:tasks.workunit.client.1.smithi019.stdout:4/363: creat d4/d1b/f7e x:0 0 0 2024-03-16T09:01:15.158 INFO:tasks.workunit.client.0.smithi007.stdout:1/508: rename d9/dd/f69 to d9/d6b/fa0 0 2024-03-16T09:01:15.160 INFO:tasks.workunit.client.0.smithi007.stdout:9/597: creat d24/d4a/d96/fa8 x:0 0 0 2024-03-16T09:01:15.160 INFO:tasks.workunit.client.0.smithi007.stdout:9/598: chown d24/d4c/l57 89931128 1 2024-03-16T09:01:15.160 INFO:tasks.workunit.client.1.smithi019.stdout:8/360: mkdir d6/d2a/d45/d57/d72/d79/d7b 0 2024-03-16T09:01:15.160 INFO:tasks.workunit.client.0.smithi007.stdout:7/485: read d0/d19/d1f/f2c [6142146,32345] 0 2024-03-16T09:01:15.161 INFO:tasks.workunit.client.0.smithi007.stdout:4/483: write d4/d14/d64/d68/f56 [1864463,110619] 0 2024-03-16T09:01:15.161 INFO:tasks.workunit.client.1.smithi019.stdout:2/408: getdents d5/d6/d64 0 2024-03-16T09:01:15.161 INFO:tasks.workunit.client.1.smithi019.stdout:2/409: chown d5/d6/d2b/d2f/d35/f58 47 1 2024-03-16T09:01:15.161 INFO:tasks.workunit.client.1.smithi019.stdout:2/410: truncate d5/d6/d2b/f18 2265247 0 2024-03-16T09:01:15.162 INFO:tasks.workunit.client.0.smithi007.stdout:1/509: mknod d9/dd/d1b/ca1 0 2024-03-16T09:01:15.163 INFO:tasks.workunit.client.0.smithi007.stdout:7/486: symlink d0/d19/d1f/d69/laa 0 2024-03-16T09:01:15.163 INFO:tasks.workunit.client.1.smithi019.stdout:5/346: unlink d6/f32 0 2024-03-16T09:01:15.163 INFO:tasks.workunit.client.1.smithi019.stdout:5/347: write d6/dd/f44 [1780592,69127] 0 2024-03-16T09:01:15.163 INFO:tasks.workunit.client.1.smithi019.stdout:5/348: creat d6/dd/d37/f6b x:0 0 0 2024-03-16T09:01:15.164 INFO:tasks.workunit.client.1.smithi019.stdout:7/265: unlink d9/f16 0 2024-03-16T09:01:15.166 INFO:tasks.workunit.client.0.smithi007.stdout:7/487: rename d0/d6c/d25/d3a/f86 to d0/d6c/d25/da1/fab 0 2024-03-16T09:01:15.166 INFO:tasks.workunit.client.0.smithi007.stdout:7/488: write d0/d19/d1f/d42/f45 [2322634,12603] 0 2024-03-16T09:01:15.166 INFO:tasks.workunit.client.0.smithi007.stdout:7/489: fsync d0/d6c/d25/f90 0 2024-03-16T09:01:15.166 INFO:tasks.workunit.client.0.smithi007.stdout:7/490: fsync d0/d19/d1f/d69/f87 0 2024-03-16T09:01:15.166 INFO:tasks.workunit.client.1.smithi019.stdout:4/364: mknod d4/d64/c7f 0 2024-03-16T09:01:15.168 INFO:tasks.workunit.client.0.smithi007.stdout:4/484: dread d4/d14/f1b [0,4194304] 0 2024-03-16T09:01:15.168 INFO:tasks.workunit.client.1.smithi019.stdout:8/361: unlink d6/fa 0 2024-03-16T09:01:15.169 INFO:tasks.workunit.client.1.smithi019.stdout:2/411: creat d5/d2e/d4f/f86 x:0 0 0 2024-03-16T09:01:15.169 INFO:tasks.workunit.client.1.smithi019.stdout:2/412: chown d5/d6/d64/c81 27 1 2024-03-16T09:01:15.170 INFO:tasks.workunit.client.0.smithi007.stdout:7/491: truncate d0/dc/f3d 3295283 0 2024-03-16T09:01:15.171 INFO:tasks.workunit.client.1.smithi019.stdout:5/349: unlink d6/dd/f38 0 2024-03-16T09:01:15.171 INFO:tasks.workunit.client.1.smithi019.stdout:5/350: stat d6/dd/d37/f6b 0 2024-03-16T09:01:15.171 INFO:tasks.workunit.client.1.smithi019.stdout:7/266: symlink d9/da/d13/d18/d1a/d32/l4e 0 2024-03-16T09:01:15.172 INFO:tasks.workunit.client.1.smithi019.stdout:2/413: mkdir d5/d6/d64/d6f/d87 0 2024-03-16T09:01:15.174 INFO:tasks.workunit.client.1.smithi019.stdout:5/351: creat d6/d1f/d33/d4c/f6c x:0 0 0 2024-03-16T09:01:15.174 INFO:tasks.workunit.client.1.smithi019.stdout:5/352: dread - d6/dd/d37/f6b zero size 2024-03-16T09:01:15.176 INFO:tasks.workunit.client.1.smithi019.stdout:6/466: sync 2024-03-16T09:01:15.191 INFO:tasks.workunit.client.1.smithi019.stdout:6/467: write de/f15 [2096559,54203] 0 2024-03-16T09:01:15.191 INFO:tasks.workunit.client.1.smithi019.stdout:6/468: write de/d26/d32/d97/f30 [5035187,60554] 0 2024-03-16T09:01:15.196 INFO:tasks.workunit.client.0.smithi007.stdout:9/599: dread d24/d4a/d55/d56/f5a [0,4194304] 0 2024-03-16T09:01:15.198 INFO:tasks.workunit.client.0.smithi007.stdout:9/600: mknod d24/d4a/d96/ca9 0 2024-03-16T09:01:15.198 INFO:tasks.workunit.client.0.smithi007.stdout:9/601: write d24/d4a/d55/d56/d7c/d7f/d80/f8c [1862542,52147] 0 2024-03-16T09:01:15.200 INFO:tasks.workunit.client.0.smithi007.stdout:0/642: dwrite d3/d9/da/d92/fce [0,4194304] 0 2024-03-16T09:01:15.202 INFO:tasks.workunit.client.0.smithi007.stdout:1/510: dread d9/f22 [0,4194304] 0 2024-03-16T09:01:15.204 INFO:tasks.workunit.client.0.smithi007.stdout:7/492: dread d0/f3c [0,4194304] 0 2024-03-16T09:01:15.205 INFO:tasks.workunit.client.0.smithi007.stdout:9/602: rename d24/d4a/d55/d56/d7c to d24/d4c/d86/daa 0 2024-03-16T09:01:15.206 INFO:tasks.workunit.client.1.smithi019.stdout:3/440: dwrite d3/d17/f72 [0,4194304] 0 2024-03-16T09:01:15.206 INFO:tasks.workunit.client.1.smithi019.stdout:3/441: chown d3/d17/d3f/l54 1494661 1 2024-03-16T09:01:15.210 INFO:tasks.workunit.client.0.smithi007.stdout:1/511: symlink d9/la2 0 2024-03-16T09:01:15.211 INFO:tasks.workunit.client.1.smithi019.stdout:8/362: dwrite d6/d2a/d3c/f61 [0,4194304] 0 2024-03-16T09:01:15.215 INFO:tasks.workunit.client.1.smithi019.stdout:8/363: write d6/de/ff [951200,79863] 0 2024-03-16T09:01:15.215 INFO:tasks.workunit.client.1.smithi019.stdout:8/364: truncate d6/d2a/d45/f6b 756229 0 2024-03-16T09:01:15.215 INFO:tasks.workunit.client.1.smithi019.stdout:8/365: write d6/de/f49 [4791884,59781] 0 2024-03-16T09:01:15.215 INFO:tasks.workunit.client.0.smithi007.stdout:9/603: write d24/d4a/f74 [1561318,101986] 0 2024-03-16T09:01:15.215 INFO:tasks.workunit.client.0.smithi007.stdout:9/604: chown lc 4 1 2024-03-16T09:01:15.216 INFO:tasks.workunit.client.1.smithi019.stdout:3/442: dread d3/d89/f38 [0,4194304] 0 2024-03-16T09:01:15.218 INFO:tasks.workunit.client.1.smithi019.stdout:8/366: creat d6/de/f7c x:0 0 0 2024-03-16T09:01:15.218 INFO:tasks.workunit.client.1.smithi019.stdout:8/367: dread - d6/de/f6a zero size 2024-03-16T09:01:15.218 INFO:tasks.workunit.client.0.smithi007.stdout:9/605: creat d24/d4c/d86/daa/d7f/d8f/fab x:0 0 0 2024-03-16T09:01:15.218 INFO:tasks.workunit.client.0.smithi007.stdout:1/512: dread d9/f19 [0,4194304] 0 2024-03-16T09:01:15.223 INFO:tasks.workunit.client.0.smithi007.stdout:0/643: dwrite d3/f22 [0,4194304] 0 2024-03-16T09:01:15.226 INFO:tasks.workunit.client.0.smithi007.stdout:1/513: creat d9/d6b/d7e/d91/fa3 x:0 0 0 2024-03-16T09:01:15.227 INFO:tasks.workunit.client.0.smithi007.stdout:9/606: rename d24/d4a/f53 to d24/d4c/fac 0 2024-03-16T09:01:15.228 INFO:tasks.workunit.client.0.smithi007.stdout:0/644: truncate d3/f49 584320 0 2024-03-16T09:01:15.230 INFO:tasks.workunit.client.0.smithi007.stdout:1/514: creat d9/dd/d1b/d28/d4c/fa4 x:0 0 0 2024-03-16T09:01:15.233 INFO:tasks.workunit.client.0.smithi007.stdout:0/645: unlink d3/d9/da/d48/d4b/c54 0 2024-03-16T09:01:15.234 INFO:tasks.workunit.client.0.smithi007.stdout:1/515: mknod d9/d6b/ca5 0 2024-03-16T09:01:15.234 INFO:tasks.workunit.client.1.smithi019.stdout:8/368: dwrite d6/f27 [0,4194304] 0 2024-03-16T09:01:15.236 INFO:tasks.workunit.client.1.smithi019.stdout:8/369: creat d6/d2a/d3f/f7d x:0 0 0 2024-03-16T09:01:15.238 INFO:tasks.workunit.client.1.smithi019.stdout:8/370: rmdir d6/de/d24/d30/d4a/d67 39 2024-03-16T09:01:15.249 INFO:tasks.workunit.client.1.smithi019.stdout:2/414: read d5/de/f32 [1371454,123791] 0 2024-03-16T09:01:15.253 INFO:tasks.workunit.client.1.smithi019.stdout:2/415: mknod d5/d6/d2b/d2f/d35/d4b/d51/d7d/c88 0 2024-03-16T09:01:15.253 INFO:tasks.workunit.client.1.smithi019.stdout:2/416: dread - d5/d6/d2b/d53/f73 zero size 2024-03-16T09:01:15.253 INFO:tasks.workunit.client.1.smithi019.stdout:2/417: readlink d5/d6/d2b/l70 0 2024-03-16T09:01:15.253 INFO:tasks.workunit.client.1.smithi019.stdout:5/353: dwrite d6/d1f/d33/d66/f48 [0,4194304] 0 2024-03-16T09:01:15.253 INFO:tasks.workunit.client.1.smithi019.stdout:5/354: write d6/f35 [824786,131017] 0 2024-03-16T09:01:15.254 INFO:tasks.workunit.client.0.smithi007.stdout:9/607: truncate d24/d4a/f78 195180 0 2024-03-16T09:01:15.254 INFO:tasks.workunit.client.1.smithi019.stdout:2/418: unlink d5/f34 0 2024-03-16T09:01:15.257 INFO:tasks.workunit.client.1.smithi019.stdout:3/443: dread d3/d17/d5a/f5c [0,4194304] 0 2024-03-16T09:01:15.257 INFO:tasks.workunit.client.1.smithi019.stdout:0/329: write d7/d1d/f4b [2575470,79284] 0 2024-03-16T09:01:15.257 INFO:tasks.workunit.client.1.smithi019.stdout:1/294: dwrite f9 [0,4194304] 0 2024-03-16T09:01:15.257 INFO:tasks.workunit.client.1.smithi019.stdout:5/355: mknod d6/c6d 0 2024-03-16T09:01:15.258 INFO:tasks.workunit.client.1.smithi019.stdout:5/356: chown d6/dd/d37/l41 4863 1 2024-03-16T09:01:15.258 INFO:tasks.workunit.client.1.smithi019.stdout:5/357: stat d6/d1f/f2d 0 2024-03-16T09:01:15.258 INFO:tasks.workunit.client.1.smithi019.stdout:5/358: chown d6/d1f/d33/d66/f48 7632 1 2024-03-16T09:01:15.258 INFO:tasks.workunit.client.0.smithi007.stdout:0/646: dwrite d3/d9/da/d48/d4b/f51 [0,4194304] 0 2024-03-16T09:01:15.259 INFO:tasks.workunit.client.0.smithi007.stdout:0/647: fsync d3/d9/db4/fcf 0 2024-03-16T09:01:15.259 INFO:tasks.workunit.client.1.smithi019.stdout:0/330: mkdir d7/d1d/d63 0 2024-03-16T09:01:15.259 INFO:tasks.workunit.client.1.smithi019.stdout:8/371: dwrite d6/d2a/d45/d57/f65 [0,4194304] 0 2024-03-16T09:01:15.260 INFO:tasks.workunit.client.0.smithi007.stdout:4/485: dwrite d4/d14/d64/d68/f56 [4194304,4194304] 0 2024-03-16T09:01:15.260 INFO:tasks.workunit.client.1.smithi019.stdout:5/359: mknod d6/dd/c6e 0 2024-03-16T09:01:15.262 INFO:tasks.workunit.client.1.smithi019.stdout:1/295: getdents d16/d1f 0 2024-03-16T09:01:15.262 INFO:tasks.workunit.client.1.smithi019.stdout:1/296: truncate d16/d1f/d3c/f48 1170301 0 2024-03-16T09:01:15.263 INFO:tasks.workunit.client.0.smithi007.stdout:9/608: creat d24/d4a/d55/d56/da6/fad x:0 0 0 2024-03-16T09:01:15.264 INFO:tasks.workunit.client.1.smithi019.stdout:8/372: truncate d6/d2a/d45/d57/f62 2338310 0 2024-03-16T09:01:15.264 INFO:tasks.workunit.client.0.smithi007.stdout:1/516: write d9/f35 [1628938,126250] 0 2024-03-16T09:01:15.264 INFO:tasks.workunit.client.0.smithi007.stdout:8/861: dwrite d0/dc/d8a/f99 [0,4194304] 0 2024-03-16T09:01:15.265 INFO:tasks.workunit.client.1.smithi019.stdout:1/297: truncate d16/d18/f1e 693390 0 2024-03-16T09:01:15.266 INFO:tasks.workunit.client.0.smithi007.stdout:0/648: rmdir d3/d1f/d70/d7c/dd4 0 2024-03-16T09:01:15.266 INFO:tasks.workunit.client.1.smithi019.stdout:8/373: link d6/d2a/d55/f6e d6/d2a/d45/d19/d74/f7e 0 2024-03-16T09:01:15.268 INFO:tasks.workunit.client.1.smithi019.stdout:1/298: mkdir d16/d1f/d43/d55 0 2024-03-16T09:01:15.269 INFO:tasks.workunit.client.0.smithi007.stdout:9/609: link d24/f31 d24/d4c/d86/daa/d7f/d80/d87/d89/fae 0 2024-03-16T09:01:15.270 INFO:tasks.workunit.client.1.smithi019.stdout:6/469: dwrite de/d26/d32/d4e/f59 [0,4194304] 0 2024-03-16T09:01:15.270 INFO:tasks.workunit.client.1.smithi019.stdout:3/444: dread d3/d17/d5a/f73 [0,4194304] 0 2024-03-16T09:01:15.271 INFO:tasks.workunit.client.0.smithi007.stdout:1/517: dread d9/dd/d1b/d28/f32 [0,4194304] 0 2024-03-16T09:01:15.272 INFO:tasks.workunit.client.0.smithi007.stdout:0/649: creat d3/d9/da/d92/d9f/fd6 x:0 0 0 2024-03-16T09:01:15.272 INFO:tasks.workunit.client.0.smithi007.stdout:0/650: dread - d3/faf zero size 2024-03-16T09:01:15.273 INFO:tasks.workunit.client.1.smithi019.stdout:2/419: rename d5/d6/d2b/d2f/d35/d4b/d51 to d5/d6/d2b/d89 0 2024-03-16T09:01:15.273 INFO:tasks.workunit.client.1.smithi019.stdout:2/420: stat d5/d6/d2b/c37 0 2024-03-16T09:01:15.273 INFO:tasks.workunit.client.1.smithi019.stdout:9/255: dread d1/d7/d21/d27/f22 [0,4194304] 0 2024-03-16T09:01:15.273 INFO:tasks.workunit.client.1.smithi019.stdout:9/256: readlink d1/d7/de/d2d/l49 0 2024-03-16T09:01:15.275 INFO:tasks.workunit.client.0.smithi007.stdout:4/486: creat d4/d7/d78/d80/d84/d89/fa2 x:0 0 0 2024-03-16T09:01:15.276 INFO:tasks.workunit.client.1.smithi019.stdout:0/331: rename d7/d8/d32/d36/f3b to d7/d1d/d28/f64 0 2024-03-16T09:01:15.276 INFO:tasks.workunit.client.1.smithi019.stdout:0/332: write d7/d1d/d28/d48/f4e [4932744,106325] 0 2024-03-16T09:01:15.277 INFO:tasks.workunit.client.1.smithi019.stdout:0/333: fsync d7/d8/f50 0 2024-03-16T09:01:15.277 INFO:tasks.workunit.client.1.smithi019.stdout:2/421: creat d5/d6/d2b/d89/d7d/d83/f8a x:0 0 0 2024-03-16T09:01:15.278 INFO:tasks.workunit.client.0.smithi007.stdout:9/610: creat d24/d4a/d55/faf x:0 0 0 2024-03-16T09:01:15.279 INFO:tasks.workunit.client.0.smithi007.stdout:9/611: getdents d24/d4a/d9e 0 2024-03-16T09:01:15.280 INFO:tasks.workunit.client.1.smithi019.stdout:1/299: rename d16/l2c to d16/d1f/d43/l56 0 2024-03-16T09:01:15.280 INFO:tasks.workunit.client.1.smithi019.stdout:1/300: fsync d16/d1f/d43/f47 0 2024-03-16T09:01:15.283 INFO:tasks.workunit.client.1.smithi019.stdout:1/301: unlink d16/d18/l4e 0 2024-03-16T09:01:15.283 INFO:tasks.workunit.client.1.smithi019.stdout:1/302: readlink d16/d18/l4a 0 2024-03-16T09:01:15.283 INFO:tasks.workunit.client.1.smithi019.stdout:1/303: write d16/f20 [693715,49853] 0 2024-03-16T09:01:15.283 INFO:tasks.workunit.client.1.smithi019.stdout:1/304: creat d16/d1f/d4f/f57 x:0 0 0 2024-03-16T09:01:15.284 INFO:tasks.workunit.client.1.smithi019.stdout:5/360: dwrite d6/dd/f67 [0,4194304] 0 2024-03-16T09:01:15.284 INFO:tasks.workunit.client.1.smithi019.stdout:5/361: chown d6/d1f/f2d 170835996 1 2024-03-16T09:01:15.284 INFO:tasks.workunit.client.1.smithi019.stdout:5/362: chown d6/d19/f3c 62 1 2024-03-16T09:01:15.284 INFO:tasks.workunit.client.1.smithi019.stdout:6/470: dread de/d26/d32/d4e/f79 [0,4194304] 0 2024-03-16T09:01:15.285 INFO:tasks.workunit.client.1.smithi019.stdout:2/422: truncate d5/de/fd 372791 0 2024-03-16T09:01:15.285 INFO:tasks.workunit.client.0.smithi007.stdout:9/612: mknod d24/d8e/cb0 0 2024-03-16T09:01:15.286 INFO:tasks.workunit.client.1.smithi019.stdout:1/305: truncate d16/d1f/f2e 159728 0 2024-03-16T09:01:15.286 INFO:tasks.workunit.client.1.smithi019.stdout:1/306: dread - d16/d1f/f34 zero size 2024-03-16T09:01:15.287 INFO:tasks.workunit.client.0.smithi007.stdout:1/518: mknod d9/dd/d1b/d2d/ca6 0 2024-03-16T09:01:15.287 INFO:tasks.workunit.client.0.smithi007.stdout:1/519: chown d9/dd/d1b/f96 314 1 2024-03-16T09:01:15.288 INFO:tasks.workunit.client.1.smithi019.stdout:9/257: dread d1/d7/f51 [0,4194304] 0 2024-03-16T09:01:15.291 INFO:tasks.workunit.client.1.smithi019.stdout:8/374: dwrite d6/d2a/d45/d2c/f58 [0,4194304] 0 2024-03-16T09:01:15.291 INFO:tasks.workunit.client.1.smithi019.stdout:8/375: chown d6/de/d51/c69 9 1 2024-03-16T09:01:15.292 INFO:tasks.workunit.client.1.smithi019.stdout:2/423: rename d5/d6/d2b/d2f/d35/f63 to d5/d6/d2b/d89/d7d/f8b 0 2024-03-16T09:01:15.292 INFO:tasks.workunit.client.1.smithi019.stdout:2/424: write d5/d2e/f3d [2562870,33504] 0 2024-03-16T09:01:15.293 INFO:tasks.workunit.client.0.smithi007.stdout:4/487: dread d4/d14/d64/d9f/f42 [0,4194304] 0 2024-03-16T09:01:15.294 INFO:tasks.workunit.client.1.smithi019.stdout:6/471: rmdir de/d26/d32/d4e 39 2024-03-16T09:01:15.297 INFO:tasks.workunit.client.0.smithi007.stdout:0/651: rename d3/d1f/d70 to d3/d6c/dbd/dd7 0 2024-03-16T09:01:15.297 INFO:tasks.workunit.client.0.smithi007.stdout:0/652: write d3/d9/da/d18/d27/d34/f89 [1755010,98082] 0 2024-03-16T09:01:15.298 INFO:tasks.workunit.client.1.smithi019.stdout:6/472: getdents de/d26/d32/d4e/d75 0 2024-03-16T09:01:15.300 INFO:tasks.workunit.client.0.smithi007.stdout:1/520: rename d9/f21 to d9/dd/d1b/d58/fa7 0 2024-03-16T09:01:15.301 INFO:tasks.workunit.client.1.smithi019.stdout:6/473: symlink de/d26/l99 0 2024-03-16T09:01:15.302 INFO:tasks.workunit.client.0.smithi007.stdout:0/653: mknod d3/d9/db4/cd8 0 2024-03-16T09:01:15.305 INFO:tasks.workunit.client.0.smithi007.stdout:1/521: rename d9/dd/d1b/d28/l6e to d9/dd/d1b/d2d/d30/d45/la8 0 2024-03-16T09:01:15.307 INFO:tasks.workunit.client.0.smithi007.stdout:0/654: mknod d3/d9/da/d48/d4b/cd9 0 2024-03-16T09:01:15.309 INFO:tasks.workunit.client.1.smithi019.stdout:8/376: dread f5 [0,4194304] 0 2024-03-16T09:01:15.309 INFO:tasks.workunit.client.0.smithi007.stdout:1/522: symlink d9/dd/d1b/d58/la9 0 2024-03-16T09:01:15.310 INFO:tasks.workunit.client.0.smithi007.stdout:8/862: dwrite d0/dc/f11c [0,4194304] 0 2024-03-16T09:01:15.311 INFO:tasks.workunit.client.0.smithi007.stdout:0/655: symlink d3/d9/da/d48/d9e/lda 0 2024-03-16T09:01:15.312 INFO:tasks.workunit.client.0.smithi007.stdout:1/523: mknod d9/d6b/d7e/d91/caa 0 2024-03-16T09:01:15.312 INFO:tasks.workunit.client.0.smithi007.stdout:1/524: fdatasync d9/dd/f59 0 2024-03-16T09:01:15.314 INFO:tasks.workunit.client.0.smithi007.stdout:0/656: mknod d3/d6c/dbd/dd7/d7b/cdb 0 2024-03-16T09:01:15.318 INFO:tasks.workunit.client.1.smithi019.stdout:2/425: rmdir d5/d6/d2b/d89 39 2024-03-16T09:01:15.318 INFO:tasks.workunit.client.1.smithi019.stdout:2/426: write d5/d6/d2b/d2f/d35/f7e [911337,95801] 0 2024-03-16T09:01:15.320 INFO:tasks.workunit.client.1.smithi019.stdout:2/427: rename d5/d6/d2b/d89/c60 to d5/d6/d2b/d2f/d35/d4b/c8c 0 2024-03-16T09:01:15.323 INFO:tasks.workunit.client.1.smithi019.stdout:8/377: write d6/d2a/d45/d19/d74/f7e [472149,85568] 0 2024-03-16T09:01:15.324 INFO:tasks.workunit.client.1.smithi019.stdout:8/378: truncate d6/de/d24/d30/d4a/d67/f70 178089 0 2024-03-16T09:01:15.326 INFO:tasks.workunit.client.1.smithi019.stdout:9/258: dwrite d1/d7/de/d24/f39 [4194304,4194304] 0 2024-03-16T09:01:15.327 INFO:tasks.workunit.client.1.smithi019.stdout:9/259: chown d1/f3 1504346113 1 2024-03-16T09:01:15.327 INFO:tasks.workunit.client.1.smithi019.stdout:9/260: dread - d1/d7/d21/d27/f55 zero size 2024-03-16T09:01:15.327 INFO:tasks.workunit.client.1.smithi019.stdout:9/261: write d1/d7/de/d24/f2c [525112,39345] 0 2024-03-16T09:01:15.327 INFO:tasks.workunit.client.0.smithi007.stdout:9/613: dwrite d24/f2d [0,4194304] 0 2024-03-16T09:01:15.328 INFO:tasks.workunit.client.0.smithi007.stdout:9/614: write d24/f34 [5189327,44569] 0 2024-03-16T09:01:15.328 INFO:tasks.workunit.client.1.smithi019.stdout:9/262: write d1/d7/de/f1a [586632,128228] 0 2024-03-16T09:01:15.329 INFO:tasks.workunit.client.1.smithi019.stdout:2/428: dread d5/d6/d2b/d2f/d35/f58 [0,4194304] 0 2024-03-16T09:01:15.330 INFO:tasks.workunit.client.1.smithi019.stdout:5/363: dwrite d6/dd/d37/f26 [0,4194304] 0 2024-03-16T09:01:15.330 INFO:tasks.workunit.client.1.smithi019.stdout:5/364: chown d6/dd/d37 25 1 2024-03-16T09:01:15.330 INFO:tasks.workunit.client.1.smithi019.stdout:9/263: mkdir d1/d7/d33/d5a 0 2024-03-16T09:01:15.330 INFO:tasks.workunit.client.0.smithi007.stdout:9/615: mkdir d24/d4a/d55/d56/db1 0 2024-03-16T09:01:15.331 INFO:tasks.workunit.client.0.smithi007.stdout:9/616: dread - d24/d4c/d86/daa/d95/f68 zero size 2024-03-16T09:01:15.331 INFO:tasks.workunit.client.0.smithi007.stdout:9/617: rename c14 to d24/d4c/d91/cb2 0 2024-03-16T09:01:15.332 INFO:tasks.workunit.client.1.smithi019.stdout:5/365: unlink d6/d1f/l43 0 2024-03-16T09:01:15.333 INFO:tasks.workunit.client.1.smithi019.stdout:9/264: mkdir d1/d7/de/d24/d5b 0 2024-03-16T09:01:15.333 INFO:tasks.workunit.client.1.smithi019.stdout:5/366: mknod d6/dd/d47/d56/c6f 0 2024-03-16T09:01:15.333 INFO:tasks.workunit.client.1.smithi019.stdout:5/367: write d6/d1f/d33/d4c/f62 [864516,123651] 0 2024-03-16T09:01:15.334 INFO:tasks.workunit.client.1.smithi019.stdout:5/368: mkdir d6/dd/d47/d70 0 2024-03-16T09:01:15.334 INFO:tasks.workunit.client.1.smithi019.stdout:5/369: truncate d6/d1f/f2d 995564 0 2024-03-16T09:01:15.336 INFO:tasks.workunit.client.1.smithi019.stdout:5/370: mknod d6/d1f/d33/d66/c71 0 2024-03-16T09:01:15.336 INFO:tasks.workunit.client.1.smithi019.stdout:2/429: write d5/d6/d2b/f18 [2038287,37051] 0 2024-03-16T09:01:15.339 INFO:tasks.workunit.client.1.smithi019.stdout:2/430: mknod d5/c8d 0 2024-03-16T09:01:15.339 INFO:tasks.workunit.client.1.smithi019.stdout:2/431: fsync d5/d6/d2b/d2f/d35/f58 0 2024-03-16T09:01:15.339 INFO:tasks.workunit.client.1.smithi019.stdout:2/432: write d5/f13 [4041512,41200] 0 2024-03-16T09:01:15.339 INFO:tasks.workunit.client.1.smithi019.stdout:5/371: rmdir d6/dd/d37/d54 39 2024-03-16T09:01:15.341 INFO:tasks.workunit.client.1.smithi019.stdout:1/307: dread d16/d1f/f2e [0,4194304] 0 2024-03-16T09:01:15.341 INFO:tasks.workunit.client.0.smithi007.stdout:4/488: dwrite d4/d14/d64/d68/d82/f8d [8388608,4194304] 0 2024-03-16T09:01:15.342 INFO:tasks.workunit.client.1.smithi019.stdout:6/474: dwrite de/d26/f39 [0,4194304] 0 2024-03-16T09:01:15.342 INFO:tasks.workunit.client.0.smithi007.stdout:8/863: dwrite d0/dc/dd/fda [0,4194304] 0 2024-03-16T09:01:15.342 INFO:tasks.workunit.client.0.smithi007.stdout:8/864: dread - d0/dc/d75/f116 zero size 2024-03-16T09:01:15.345 INFO:tasks.workunit.client.0.smithi007.stdout:1/525: dread d9/f35 [0,4194304] 0 2024-03-16T09:01:15.347 INFO:tasks.workunit.client.0.smithi007.stdout:0/657: dwrite d3/d9/da/d48/d25/d43/fc2 [0,4194304] 0 2024-03-16T09:01:15.347 INFO:tasks.workunit.client.0.smithi007.stdout:0/658: chown d3/d9/da/d48/d25/d43/d91 2 1 2024-03-16T09:01:15.347 INFO:tasks.workunit.client.0.smithi007.stdout:0/659: truncate d3/d9/da/d18/d23/f4d 1050684 0 2024-03-16T09:01:15.347 INFO:tasks.workunit.client.0.smithi007.stdout:0/660: creat d3/d9/da/d48/d25/d43/fdc x:0 0 0 2024-03-16T09:01:15.347 INFO:tasks.workunit.client.0.smithi007.stdout:0/661: fsync d3/d9/da/d92/fce 0 2024-03-16T09:01:15.348 INFO:tasks.workunit.client.1.smithi019.stdout:6/475: mkdir de/d1c/d9a 0 2024-03-16T09:01:15.348 INFO:tasks.workunit.client.1.smithi019.stdout:6/476: read de/f1a [3370867,99370] 0 2024-03-16T09:01:15.348 INFO:tasks.workunit.client.1.smithi019.stdout:6/477: chown de/d1c/f20 168944836 1 2024-03-16T09:01:15.348 INFO:tasks.workunit.client.1.smithi019.stdout:6/478: readlink de/d1c/l4b 0 2024-03-16T09:01:15.351 INFO:tasks.workunit.client.1.smithi019.stdout:2/433: write d5/de/f1f [850098,59185] 0 2024-03-16T09:01:15.351 INFO:tasks.workunit.client.1.smithi019.stdout:8/379: dread d6/d2a/d45/d19/d74/f7e [0,4194304] 0 2024-03-16T09:01:15.351 INFO:tasks.workunit.client.1.smithi019.stdout:8/380: readlink d6/de/d24/d30/d4a/l5e 0 2024-03-16T09:01:15.351 INFO:tasks.workunit.client.1.smithi019.stdout:6/479: creat de/d26/d32/d4e/f9b x:0 0 0 2024-03-16T09:01:15.352 INFO:tasks.workunit.client.0.smithi007.stdout:8/865: symlink d0/dc/d50/d8e/dd2/l12f 0 2024-03-16T09:01:15.352 INFO:tasks.workunit.client.0.smithi007.stdout:8/866: write d0/dc/d75/f11b [902503,48502] 0 2024-03-16T09:01:15.353 INFO:tasks.workunit.client.1.smithi019.stdout:2/434: symlink d5/d6/d2b/d2f/d35/l8e 0 2024-03-16T09:01:15.354 INFO:tasks.workunit.client.1.smithi019.stdout:6/480: rmdir de/d26/d32/d97/d51 39 2024-03-16T09:01:15.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/489: rename d4/d7/d78/d80/f83 to d4/fa3 0 2024-03-16T09:01:15.357 INFO:tasks.workunit.client.0.smithi007.stdout:4/490: chown d4/d14/d1c/f60 0 1 2024-03-16T09:01:15.357 INFO:tasks.workunit.client.1.smithi019.stdout:2/435: rmdir d5/d2e/d6b 0 2024-03-16T09:01:15.358 INFO:tasks.workunit.client.1.smithi019.stdout:6/481: creat de/d26/d32/f9c x:0 0 0 2024-03-16T09:01:15.358 INFO:tasks.workunit.client.0.smithi007.stdout:8/867: mkdir d0/dc/d50/d8e/dd2/dea/d18/da7/dc5/d130 0 2024-03-16T09:01:15.359 INFO:tasks.workunit.client.0.smithi007.stdout:4/491: symlink d4/d14/d1c/d5b/d67/d8f/la4 0 2024-03-16T09:01:15.360 INFO:tasks.workunit.client.1.smithi019.stdout:2/436: symlink d5/d2e/l8f 0 2024-03-16T09:01:15.360 INFO:tasks.workunit.client.1.smithi019.stdout:6/482: symlink de/d26/d32/l9d 0 2024-03-16T09:01:15.361 INFO:tasks.workunit.client.0.smithi007.stdout:8/868: mknod d0/dc/d50/d8e/dd2/dea/d18/da7/de2/d113/d12c/c131 0 2024-03-16T09:01:15.361 INFO:tasks.workunit.client.0.smithi007.stdout:8/869: chown d0/dc/dd/d8b/fcf 322543 1 2024-03-16T09:01:15.362 INFO:tasks.workunit.client.1.smithi019.stdout:2/437: creat d5/d4e/d74/f90 x:0 0 0 2024-03-16T09:01:15.362 INFO:tasks.workunit.client.0.smithi007.stdout:1/526: dread d9/dd/d1b/f25 [4194304,4194304] 0 2024-03-16T09:01:15.362 INFO:tasks.workunit.client.0.smithi007.stdout:9/618: dwrite d24/d4c/d86/daa/d7f/d80/f8c [0,4194304] 0 2024-03-16T09:01:15.363 INFO:tasks.workunit.client.0.smithi007.stdout:4/492: rename d4/d14/d1c/d5b/d67/d8f/d90/fa1 to d4/d7/d78/fa5 0 2024-03-16T09:01:15.365 INFO:tasks.workunit.client.0.smithi007.stdout:1/527: creat d9/dd/d1b/d28/d29/fab x:0 0 0 2024-03-16T09:01:15.368 INFO:tasks.workunit.client.0.smithi007.stdout:0/662: dread d3/d9/db4/d1c/f2a [0,4194304] 0 2024-03-16T09:01:15.369 INFO:tasks.workunit.client.0.smithi007.stdout:0/663: creat d3/d9/da/d92/d9f/fdd x:0 0 0 2024-03-16T09:01:15.372 INFO:tasks.workunit.client.0.smithi007.stdout:0/664: mkdir d3/d9/da/d48/d25/d87/dbb/dde 0 2024-03-16T09:01:15.372 INFO:tasks.workunit.client.0.smithi007.stdout:0/665: readlink d3/d9/db4/d60/lb6 0 2024-03-16T09:01:15.373 INFO:tasks.workunit.client.0.smithi007.stdout:0/666: rmdir d3/d9/da/d48/d25/d43/d1a/d58 39 2024-03-16T09:01:15.374 INFO:tasks.workunit.client.0.smithi007.stdout:0/667: symlink d3/d9/db4/d2c/d5b/ldf 0 2024-03-16T09:01:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:0/668: creat d3/d9/da/d48/d25/d43/d1a/d58/d61/fe0 x:0 0 0 2024-03-16T09:01:15.377 INFO:tasks.workunit.client.0.smithi007.stdout:0/669: dread - d3/d9/db4/d1c/d72/fd1 zero size 2024-03-16T09:01:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:1/528: dread d9/dd/d1b/d28/d29/f2a [4194304,4194304] 0 2024-03-16T09:01:15.379 INFO:tasks.workunit.client.0.smithi007.stdout:1/529: chown d9/dd/d1b/d2d/d30/c97 417 1 2024-03-16T09:01:15.382 INFO:tasks.workunit.client.0.smithi007.stdout:1/530: mkdir d9/d6b/d7e/dac 0 2024-03-16T09:01:15.385 INFO:tasks.workunit.client.1.smithi019.stdout:8/381: dwrite d6/de/ff [0,4194304] 0 2024-03-16T09:01:15.386 INFO:tasks.workunit.client.1.smithi019.stdout:8/382: readlink d6/d2a/d45/d57/l64 0 2024-03-16T09:01:15.389 INFO:tasks.workunit.client.1.smithi019.stdout:8/383: creat d6/d2a/d45/d19/d74/f7f x:0 0 0 2024-03-16T09:01:15.389 INFO:tasks.workunit.client.1.smithi019.stdout:8/384: chown d6 809376 1 2024-03-16T09:01:15.394 INFO:tasks.workunit.client.1.smithi019.stdout:1/308: dwrite d16/d18/f24 [4194304,4194304] 0 2024-03-16T09:01:15.394 INFO:tasks.workunit.client.1.smithi019.stdout:1/309: stat d16/d1f/d3c 0 2024-03-16T09:01:15.395 INFO:tasks.workunit.client.1.smithi019.stdout:1/310: creat d16/d1f/d4f/f58 x:0 0 0 2024-03-16T09:01:15.395 INFO:tasks.workunit.client.0.smithi007.stdout:1/531: dread d9/f19 [0,4194304] 0 2024-03-16T09:01:15.395 INFO:tasks.workunit.client.0.smithi007.stdout:1/532: write d9/dd/d1b/d2d/d30/f6d [138087,71640] 0 2024-03-16T09:01:15.395 INFO:tasks.workunit.client.0.smithi007.stdout:1/533: write d9/d6b/d7e/d91/fa3 [707690,19150] 0 2024-03-16T09:01:15.395 INFO:tasks.workunit.client.0.smithi007.stdout:0/670: dread d3/d9/da/d18/d27/d34/f57 [0,4194304] 0 2024-03-16T09:01:15.396 INFO:tasks.workunit.client.1.smithi019.stdout:1/311: symlink d16/l59 0 2024-03-16T09:01:15.396 INFO:tasks.workunit.client.0.smithi007.stdout:1/534: rename d9/la2 to d9/dd/d1b/d28/d29/d7b/lad 0 2024-03-16T09:01:15.404 INFO:tasks.workunit.client.1.smithi019.stdout:1/312: dread d16/d1f/d43/f47 [0,4194304] 0 2024-03-16T09:01:15.406 INFO:tasks.workunit.client.1.smithi019.stdout:1/313: truncate d16/d18/f1e 264813 0 2024-03-16T09:01:15.409 INFO:tasks.workunit.client.1.smithi019.stdout:1/314: rmdir d16/d1f/d3c 39 2024-03-16T09:01:15.410 INFO:tasks.workunit.client.1.smithi019.stdout:1/315: rmdir d16/d31 39 2024-03-16T09:01:15.411 INFO:tasks.workunit.client.1.smithi019.stdout:1/316: stat d16/c17 0 2024-03-16T09:01:15.411 INFO:tasks.workunit.client.1.smithi019.stdout:1/317: readlink l13 0 2024-03-16T09:01:15.411 INFO:tasks.workunit.client.0.smithi007.stdout:0/671: write d3/d9/da/d92/d9f/fc7 [3658276,25486] 0 2024-03-16T09:01:15.411 INFO:tasks.workunit.client.0.smithi007.stdout:0/672: stat d3/d9/da/d18/l9d 0 2024-03-16T09:01:15.412 INFO:tasks.workunit.client.0.smithi007.stdout:0/673: getdents d3/d1f 0 2024-03-16T09:01:15.412 INFO:tasks.workunit.client.0.smithi007.stdout:0/674: write d3/f49 [1168888,18701] 0 2024-03-16T09:01:15.412 INFO:tasks.workunit.client.1.smithi019.stdout:1/318: symlink d16/d1f/l5a 0 2024-03-16T09:01:15.412 INFO:tasks.workunit.client.1.smithi019.stdout:1/319: truncate d16/d18/f1b 635252 0 2024-03-16T09:01:15.413 INFO:tasks.workunit.client.1.smithi019.stdout:1/320: stat d16/d1f/d43/d55 0 2024-03-16T09:01:15.415 INFO:tasks.workunit.client.0.smithi007.stdout:0/675: creat d3/d9/da/d48/d9e/fe1 x:0 0 0 2024-03-16T09:01:15.415 INFO:tasks.workunit.client.0.smithi007.stdout:0/676: truncate d3/d9/db4/d1c/d72/f82 1075447 0 2024-03-16T09:01:15.415 INFO:tasks.workunit.client.0.smithi007.stdout:0/677: read - d3/d9/da/d48/d25/d87/dbb/fc5 zero size 2024-03-16T09:01:15.415 INFO:tasks.workunit.client.0.smithi007.stdout:0/678: getdents d3/d9/da/d48/d25/d87/dbb/dde 0 2024-03-16T09:01:15.415 INFO:tasks.workunit.client.0.smithi007.stdout:0/679: dread - d3/d9/da/d18/f7d zero size 2024-03-16T09:01:15.415 INFO:tasks.workunit.client.1.smithi019.stdout:1/321: getdents d16/d1f/d3c 0 2024-03-16T09:01:15.416 INFO:tasks.workunit.client.1.smithi019.stdout:1/322: write d16/d18/f23 [4478704,3077] 0 2024-03-16T09:01:15.416 INFO:tasks.workunit.client.1.smithi019.stdout:1/323: chown d16/l21 1 1 2024-03-16T09:01:15.416 INFO:tasks.workunit.client.1.smithi019.stdout:1/324: fdatasync d16/f20 0 2024-03-16T09:01:15.416 INFO:tasks.workunit.client.1.smithi019.stdout:1/325: chown l13 58101152 1 2024-03-16T09:01:15.416 INFO:tasks.workunit.client.1.smithi019.stdout:1/326: symlink d16/d18/l5b 0 2024-03-16T09:01:15.417 INFO:tasks.workunit.client.0.smithi007.stdout:4/493: dwrite d4/d14/f35 [0,4194304] 0 2024-03-16T09:01:15.417 INFO:tasks.workunit.client.0.smithi007.stdout:4/494: stat d4/d14/d1c/l7c 0 2024-03-16T09:01:15.418 INFO:tasks.workunit.client.0.smithi007.stdout:8/870: dwrite d0/dc/d50/d8e/dd2/dea/f66 [0,4194304] 0 2024-03-16T09:01:15.418 INFO:tasks.workunit.client.0.smithi007.stdout:8/871: write d0/dc/d75/f11b [1610231,60961] 0 2024-03-16T09:01:15.418 INFO:tasks.workunit.client.1.smithi019.stdout:2/438: dwrite d5/d6/d2b/f3c [8388608,4194304] 0 2024-03-16T09:01:15.419 INFO:tasks.workunit.client.1.smithi019.stdout:2/439: getdents d5/d6/d2b/d89/d7d 0 2024-03-16T09:01:15.419 INFO:tasks.workunit.client.1.smithi019.stdout:2/440: fsync d5/d6/d2b/d2f/f67 0 2024-03-16T09:01:15.419 INFO:tasks.workunit.client.1.smithi019.stdout:2/441: fsync d5/ff 0 2024-03-16T09:01:15.420 INFO:tasks.workunit.client.1.smithi019.stdout:6/483: dwrite de/d26/d32/d97/f36 [0,4194304] 0 2024-03-16T09:01:15.420 INFO:tasks.workunit.client.1.smithi019.stdout:8/385: dwrite d6/d2a/d45/d19/d74/f7e [0,4194304] 0 2024-03-16T09:01:15.420 INFO:tasks.workunit.client.0.smithi007.stdout:9/619: dwrite d24/d4c/d86/daa/d7f/d8f/fab [0,4194304] 0 2024-03-16T09:01:15.421 INFO:tasks.workunit.client.0.smithi007.stdout:9/620: chown d24/d4a/d55/l5f 15 1 2024-03-16T09:01:15.424 INFO:tasks.workunit.client.1.smithi019.stdout:6/484: unlink de/f67 0 2024-03-16T09:01:15.425 INFO:tasks.workunit.client.1.smithi019.stdout:8/386: truncate d6/d2a/d45/f26 3978505 0 2024-03-16T09:01:15.426 INFO:tasks.workunit.client.1.smithi019.stdout:8/387: dread - d6/d2a/d3f/f7d zero size 2024-03-16T09:01:15.426 INFO:tasks.workunit.client.0.smithi007.stdout:4/495: write d4/d14/f95 [1918313,12447] 0 2024-03-16T09:01:15.426 INFO:tasks.workunit.client.0.smithi007.stdout:4/496: chown d4/d14/d1c/f3b 3473 1 2024-03-16T09:01:15.427 INFO:tasks.workunit.client.1.smithi019.stdout:6/485: mkdir de/d1c/d9a/d9e 0 2024-03-16T09:01:15.429 INFO:tasks.workunit.client.1.smithi019.stdout:8/388: creat d6/d2a/f80 x:0 0 0 2024-03-16T09:01:15.431 INFO:tasks.workunit.client.1.smithi019.stdout:6/486: unlink de/d26/d32/d97/d66/d72/f87 0 2024-03-16T09:01:15.432 INFO:tasks.workunit.client.0.smithi007.stdout:8/872: dread d0/dc/d80/fe5 [0,4194304] 0 2024-03-16T09:01:15.438 INFO:tasks.workunit.client.1.smithi019.stdout:2/442: write d5/d6/d2b/d2f/f67 [649742,123358] 0 2024-03-16T09:01:15.438 INFO:tasks.workunit.client.1.smithi019.stdout:2/443: creat d5/d6/d2b/f91 x:0 0 0 2024-03-16T09:01:15.440 INFO:tasks.workunit.client.1.smithi019.stdout:2/444: rename d5/d6/d2b/c2d to d5/d5a/c92 0 2024-03-16T09:01:15.440 INFO:tasks.workunit.client.1.smithi019.stdout:2/445: fsync d5/d6/d2b/d2f/f3a 0 2024-03-16T09:01:15.441 INFO:tasks.workunit.client.1.smithi019.stdout:2/446: creat d5/d6/d2b/d2f/d35/d4b/f93 x:0 0 0 2024-03-16T09:01:15.447 INFO:tasks.workunit.client.1.smithi019.stdout:8/389: write d6/d2a/d45/d19/f1a [2056919,123584] 0 2024-03-16T09:01:15.449 INFO:tasks.workunit.client.1.smithi019.stdout:8/390: creat d6/de/d24/d30/d4a/f81 x:0 0 0 2024-03-16T09:01:15.457 INFO:tasks.workunit.client.1.smithi019.stdout:8/391: dread d6/d2a/d55/f6e [0,4194304] 0 2024-03-16T09:01:15.459 INFO:tasks.workunit.client.1.smithi019.stdout:8/392: symlink d6/d2a/d45/d19/d74/l82 0 2024-03-16T09:01:15.460 INFO:tasks.workunit.client.1.smithi019.stdout:8/393: mknod d6/de/c83 0 2024-03-16T09:01:15.461 INFO:tasks.workunit.client.1.smithi019.stdout:8/394: mknod d6/d2a/d45/d2c/c84 0 2024-03-16T09:01:15.461 INFO:tasks.workunit.client.1.smithi019.stdout:8/395: fdatasync d6/de/f6c 0 2024-03-16T09:01:15.464 INFO:tasks.workunit.client.1.smithi019.stdout:1/327: dwrite d16/d18/f39 [0,4194304] 0 2024-03-16T09:01:15.464 INFO:tasks.workunit.client.1.smithi019.stdout:1/328: fsync d16/d18/f1a 0 2024-03-16T09:01:15.464 INFO:tasks.workunit.client.1.smithi019.stdout:1/329: write d16/f3d [413884,116490] 0 2024-03-16T09:01:15.464 INFO:tasks.workunit.client.1.smithi019.stdout:1/330: readlink d16/l21 0 2024-03-16T09:01:15.465 INFO:tasks.workunit.client.0.smithi007.stdout:8/873: dwrite d0/d26/d2d/d43/d4c/f77 [0,4194304] 0 2024-03-16T09:01:15.465 INFO:tasks.workunit.client.0.smithi007.stdout:0/680: dwrite d3/d9/da/d92/fce [0,4194304] 0 2024-03-16T09:01:15.465 INFO:tasks.workunit.client.1.smithi019.stdout:6/487: dwrite de/d26/d32/d97/f30 [4194304,4194304] 0 2024-03-16T09:01:15.465 INFO:tasks.workunit.client.1.smithi019.stdout:6/488: fdatasync de/d1c/f8d 0 2024-03-16T09:01:15.466 INFO:tasks.workunit.client.1.smithi019.stdout:2/447: dwrite d5/d6/d2b/d89/d7d/d83/f8a [0,4194304] 0 2024-03-16T09:01:15.471 INFO:tasks.workunit.client.0.smithi007.stdout:8/874: rename d0/d26/d2d/c45 to d0/dc/d50/d8e/dd2/c132 0 2024-03-16T09:01:15.475 INFO:tasks.workunit.client.1.smithi019.stdout:1/331: link d16/f20 d16/d1f/d4f/f5c 0 2024-03-16T09:01:15.476 INFO:tasks.workunit.client.0.smithi007.stdout:8/875: creat d0/d26/d2d/d43/d6f/da5/f133 x:0 0 0 2024-03-16T09:01:15.478 INFO:tasks.workunit.client.1.smithi019.stdout:6/489: mkdir de/d26/d32/d4e/d57/d9f 0 2024-03-16T09:01:15.480 INFO:tasks.workunit.client.0.smithi007.stdout:3/621: sync 2024-03-16T09:01:15.480 INFO:tasks.workunit.client.0.smithi007.stdout:0/681: write d3/f8b [3923961,109695] 0 2024-03-16T09:01:15.480 INFO:tasks.workunit.client.0.smithi007.stdout:0/682: dread - d3/d9/da/d18/d27/fbc zero size 2024-03-16T09:01:15.483 INFO:tasks.workunit.client.0.smithi007.stdout:3/622: mkdir d15/d18/d1f/d29/dc3 0 2024-03-16T09:01:15.483 INFO:tasks.workunit.client.1.smithi019.stdout:8/396: dwrite d6/d2a/d45/d2c/f5c [0,4194304] 0 2024-03-16T09:01:15.484 INFO:tasks.workunit.client.1.smithi019.stdout:8/397: creat d6/d2a/f85 x:0 0 0 2024-03-16T09:01:15.487 INFO:tasks.workunit.client.1.smithi019.stdout:8/398: unlink d6/d2a/d3c/f61 0 2024-03-16T09:01:15.487 INFO:tasks.workunit.client.1.smithi019.stdout:8/399: fsync d6/d2a/d3c/f44 0 2024-03-16T09:01:15.490 INFO:tasks.workunit.client.0.smithi007.stdout:0/683: read d3/d9/da/d18/d23/f63 [3611575,124813] 0 2024-03-16T09:01:15.495 INFO:tasks.workunit.client.0.smithi007.stdout:8/876: write d0/ff [7067886,102199] 0 2024-03-16T09:01:15.498 INFO:tasks.workunit.client.0.smithi007.stdout:8/877: unlink d0/dc/d50/f6b 0 2024-03-16T09:01:15.502 INFO:tasks.workunit.client.1.smithi019.stdout:2/448: dwrite d5/d6/d2b/f91 [0,4194304] 0 2024-03-16T09:01:15.503 INFO:tasks.workunit.client.1.smithi019.stdout:8/400: dread d6/de/f76 [0,4194304] 0 2024-03-16T09:01:15.505 INFO:tasks.workunit.client.1.smithi019.stdout:6/490: dwrite de/d26/d32/d97/f30 [4194304,4194304] 0 2024-03-16T09:01:15.505 INFO:tasks.workunit.client.1.smithi019.stdout:2/449: symlink d5/d5a/d80/l94 0 2024-03-16T09:01:15.506 INFO:tasks.workunit.client.0.smithi007.stdout:8/878: dread d0/d26/d2d/ff2 [0,4194304] 0 2024-03-16T09:01:15.506 INFO:tasks.workunit.client.0.smithi007.stdout:3/623: dwrite d15/f16 [0,4194304] 0 2024-03-16T09:01:15.507 INFO:tasks.workunit.client.1.smithi019.stdout:4/365: sync 2024-03-16T09:01:15.507 INFO:tasks.workunit.client.1.smithi019.stdout:8/401: creat d6/d2a/d68/f86 x:0 0 0 2024-03-16T09:01:15.507 INFO:tasks.workunit.client.1.smithi019.stdout:2/450: dread d5/d6/d2b/f72 [4194304,4194304] 0 2024-03-16T09:01:15.507 INFO:tasks.workunit.client.1.smithi019.stdout:1/332: dwrite d16/d1f/d4f/f50 [0,4194304] 0 2024-03-16T09:01:15.508 INFO:tasks.workunit.client.0.smithi007.stdout:8/879: creat d0/dc/d75/d11e/f134 x:0 0 0 2024-03-16T09:01:15.508 INFO:tasks.workunit.client.0.smithi007.stdout:8/880: symlink d0/dc/d54/l135 0 2024-03-16T09:01:15.509 INFO:tasks.workunit.client.1.smithi019.stdout:6/491: symlink de/d26/d32/d97/d66/la0 0 2024-03-16T09:01:15.510 INFO:tasks.workunit.client.0.smithi007.stdout:8/881: creat d0/dc/d80/f136 x:0 0 0 2024-03-16T09:01:15.510 INFO:tasks.workunit.client.1.smithi019.stdout:4/366: mkdir d4/d1b/d49/d58/d80 0 2024-03-16T09:01:15.510 INFO:tasks.workunit.client.1.smithi019.stdout:4/367: fsync d4/d6/d15/f67 0 2024-03-16T09:01:15.511 INFO:tasks.workunit.client.1.smithi019.stdout:4/368: fsync d4/d6/f24 0 2024-03-16T09:01:15.511 INFO:tasks.workunit.client.0.smithi007.stdout:8/882: truncate d0/d26/d2d/d43/d70/fcd 142233 0 2024-03-16T09:01:15.512 INFO:tasks.workunit.client.0.smithi007.stdout:8/883: symlink d0/dbd/l137 0 2024-03-16T09:01:15.512 INFO:tasks.workunit.client.0.smithi007.stdout:8/884: getdents d0/dc/d50/d8e/dd2/dea/d74/df0 0 2024-03-16T09:01:15.513 INFO:tasks.workunit.client.0.smithi007.stdout:0/684: dwrite d3/d9/db4/d2c/d5b/f96 [0,4194304] 0 2024-03-16T09:01:15.513 INFO:tasks.workunit.client.0.smithi007.stdout:0/685: truncate d3/d9/da/d18/d27/f9a 1489449 0 2024-03-16T09:01:15.513 INFO:tasks.workunit.client.0.smithi007.stdout:0/686: truncate d3/f49 2025019 0 2024-03-16T09:01:15.513 INFO:tasks.workunit.client.1.smithi019.stdout:2/451: unlink d5/d6/l26 0 2024-03-16T09:01:15.514 INFO:tasks.workunit.client.1.smithi019.stdout:9/265: sync 2024-03-16T09:01:15.514 INFO:tasks.workunit.client.1.smithi019.stdout:0/334: sync 2024-03-16T09:01:15.514 INFO:tasks.workunit.client.1.smithi019.stdout:9/266: fdatasync d1/d7/de/d24/f40 0 2024-03-16T09:01:15.514 INFO:tasks.workunit.client.1.smithi019.stdout:7/267: sync 2024-03-16T09:01:15.514 INFO:tasks.workunit.client.1.smithi019.stdout:5/372: sync 2024-03-16T09:01:15.514 INFO:tasks.workunit.client.1.smithi019.stdout:5/373: chown d6/d1f/d33/d66/l4d 109 1 2024-03-16T09:01:15.514 INFO:tasks.workunit.client.1.smithi019.stdout:1/333: dread d16/f3d [0,4194304] 0 2024-03-16T09:01:15.514 INFO:tasks.workunit.client.1.smithi019.stdout:1/334: creat d16/f5d x:0 0 0 2024-03-16T09:01:15.515 INFO:tasks.workunit.client.1.smithi019.stdout:3/445: sync 2024-03-16T09:01:15.515 INFO:tasks.workunit.client.1.smithi019.stdout:3/446: chown d3/d17/d6a/f7b 9583 1 2024-03-16T09:01:15.517 INFO:tasks.workunit.client.1.smithi019.stdout:6/492: mkdir de/d26/d32/d97/da1 0 2024-03-16T09:01:15.519 INFO:tasks.workunit.client.1.smithi019.stdout:8/402: getdents d6/de 0 2024-03-16T09:01:15.519 INFO:tasks.workunit.client.1.smithi019.stdout:8/403: fsync d6/de/f6c 0 2024-03-16T09:01:15.523 INFO:tasks.workunit.client.1.smithi019.stdout:2/452: rename d5/de/f1f to d5/d4e/d74/f95 0 2024-03-16T09:01:15.523 INFO:tasks.workunit.client.1.smithi019.stdout:2/453: write d5/d6/d2b/d2f/f67 [4620357,115258] 0 2024-03-16T09:01:15.524 INFO:tasks.workunit.client.1.smithi019.stdout:2/454: dread - d5/d6/d2b/d2f/d35/f39 zero size 2024-03-16T09:01:15.524 INFO:tasks.workunit.client.1.smithi019.stdout:5/374: write d6/dd/f16 [203043,73229] 0 2024-03-16T09:01:15.525 INFO:tasks.workunit.client.1.smithi019.stdout:7/268: unlink d9/d25/c36 0 2024-03-16T09:01:15.526 INFO:tasks.workunit.client.1.smithi019.stdout:7/269: chown d9/da/d13 6484427 1 2024-03-16T09:01:15.526 INFO:tasks.workunit.client.1.smithi019.stdout:7/270: write f4 [4485186,16006] 0 2024-03-16T09:01:15.529 INFO:tasks.workunit.client.0.smithi007.stdout:0/687: dread d3/d9/da/d48/f13 [0,4194304] 0 2024-03-16T09:01:15.531 INFO:tasks.workunit.client.1.smithi019.stdout:3/447: mkdir d3/d8b 0 2024-03-16T09:01:15.531 INFO:tasks.workunit.client.1.smithi019.stdout:3/448: stat d3/d1b/c2a 0 2024-03-16T09:01:15.531 INFO:tasks.workunit.client.1.smithi019.stdout:9/267: dread d1/f3 [0,4194304] 0 2024-03-16T09:01:15.532 INFO:tasks.workunit.client.0.smithi007.stdout:0/688: mknod d3/ce2 0 2024-03-16T09:01:15.532 INFO:tasks.workunit.client.1.smithi019.stdout:6/493: creat de/d26/d32/d4e/d75/fa2 x:0 0 0 2024-03-16T09:01:15.533 INFO:tasks.workunit.client.1.smithi019.stdout:8/404: mkdir d6/de/d24/d30/d4a/d87 0 2024-03-16T09:01:15.534 INFO:tasks.workunit.client.1.smithi019.stdout:0/335: rename d7/d1d/d2e/f56 to d7/d1d/d28/d48/f65 0 2024-03-16T09:01:15.534 INFO:tasks.workunit.client.0.smithi007.stdout:0/689: unlink d3/d9/da/d18/cbf 0 2024-03-16T09:01:15.535 INFO:tasks.workunit.client.1.smithi019.stdout:2/455: mkdir d5/d5a/d96 0 2024-03-16T09:01:15.535 INFO:tasks.workunit.client.1.smithi019.stdout:2/456: stat d5/d2e/f75 0 2024-03-16T09:01:15.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/690: mkdir d3/d9/db4/de3 0 2024-03-16T09:01:15.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/691: write d3/d9/da/d92/d9f/fc4 [928430,61235] 0 2024-03-16T09:01:15.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/692: chown d3/d9/da/d48/d25/d43/d1a/d58 0 1 2024-03-16T09:01:15.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/693: truncate d3/d6c/dbd/dd7/fa3 586087 0 2024-03-16T09:01:15.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/694: rename d3/d9/da to d3/d9/da/d48/d25/d43/d1a/d58/de4 22 2024-03-16T09:01:15.537 INFO:tasks.workunit.client.0.smithi007.stdout:8/885: dwrite d0/dc/f23 [4194304,4194304] 0 2024-03-16T09:01:15.538 INFO:tasks.workunit.client.0.smithi007.stdout:3/624: dwrite d15/d18/d1f/d29/d43/d72/f50 [4194304,4194304] 0 2024-03-16T09:01:15.538 INFO:tasks.workunit.client.0.smithi007.stdout:3/625: fsync d15/d18/d1f/d56/f88 0 2024-03-16T09:01:15.539 INFO:tasks.workunit.client.1.smithi019.stdout:6/494: read de/d26/d32/d97/f46 [148254,72133] 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:6/495: fdatasync de/d26/d32/d97/d66/f94 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:7/271: symlink d9/da/d13/l4f 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:4/369: dwrite d4/d1b/f1c [0,4194304] 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:4/370: fsync d4/f35 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:7/272: fdatasync d9/da/d13/d18/d1a/f2b 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:7/273: fdatasync d9/da/d13/d18/d30/f4a 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:4/371: write d4/d1b/d2a/f4d [90984,53948] 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:4/372: write d4/d1b/d49/f75 [4940738,53539] 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:4/373: chown d4/d6/d15/d1f/d2c/f3f 97940 1 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:3/449: unlink d3/d17/d3f/l54 0 2024-03-16T09:01:15.540 INFO:tasks.workunit.client.1.smithi019.stdout:3/450: read d3/d17/d5a/f5c [274402,13845] 0 2024-03-16T09:01:15.541 INFO:tasks.workunit.client.0.smithi007.stdout:8/886: rename d0/d26/d2d/l41 to d0/dc/d50/d8e/l138 0 2024-03-16T09:01:15.541 INFO:tasks.workunit.client.0.smithi007.stdout:8/887: dread - d0/d26/d2d/d43/d4c/dba/f12e zero size 2024-03-16T09:01:15.541 INFO:tasks.workunit.client.0.smithi007.stdout:8/888: fsync d0/d26/fa1 0 2024-03-16T09:01:15.541 INFO:tasks.workunit.client.0.smithi007.stdout:8/889: write d0/dc/d8a/fb6 [2195200,19470] 0 2024-03-16T09:01:15.541 INFO:tasks.workunit.client.1.smithi019.stdout:5/375: rename d6/dd/c6e to d6/dd/d47/d70/c72 0 2024-03-16T09:01:15.541 INFO:tasks.workunit.client.0.smithi007.stdout:3/626: symlink d15/d18/d1f/d29/d43/d72/lc4 0 2024-03-16T09:01:15.542 INFO:tasks.workunit.client.0.smithi007.stdout:0/695: dread d3/d9/da/d48/d25/d43/d1a/f30 [0,4194304] 0 2024-03-16T09:01:15.542 INFO:tasks.workunit.client.1.smithi019.stdout:0/336: mkdir d7/d8/d32/d66 0 2024-03-16T09:01:15.542 INFO:tasks.workunit.client.1.smithi019.stdout:5/376: dread d6/d1f/f2d [0,4194304] 0 2024-03-16T09:01:15.543 INFO:tasks.workunit.client.1.smithi019.stdout:5/377: readlink d6/dd/l31 0 2024-03-16T09:01:15.543 INFO:tasks.workunit.client.1.smithi019.stdout:5/378: chown d6/d1f/f2d 921 1 2024-03-16T09:01:15.543 INFO:tasks.workunit.client.1.smithi019.stdout:2/457: link d5/d2e/l47 d5/de/l97 0 2024-03-16T09:01:15.544 INFO:tasks.workunit.client.0.smithi007.stdout:0/696: unlink d3/d6c/dbd/dd7/ca4 0 2024-03-16T09:01:15.546 INFO:tasks.workunit.client.1.smithi019.stdout:6/496: unlink de/d26/d32/f4a 0 2024-03-16T09:01:15.546 INFO:tasks.workunit.client.1.smithi019.stdout:6/497: fsync de/d26/d32/f62 0 2024-03-16T09:01:15.546 INFO:tasks.workunit.client.1.smithi019.stdout:7/274: chown d9/db/d12/c19 3481047 1 2024-03-16T09:01:15.548 INFO:tasks.workunit.client.1.smithi019.stdout:4/374: creat d4/d6/f81 x:0 0 0 2024-03-16T09:01:15.549 INFO:tasks.workunit.client.1.smithi019.stdout:3/451: rmdir d3/d1b/d40 39 2024-03-16T09:01:15.550 INFO:tasks.workunit.client.1.smithi019.stdout:3/452: chown d3/d17/d6a/f70 1017380 1 2024-03-16T09:01:15.550 INFO:tasks.workunit.client.1.smithi019.stdout:8/405: write d6/d2a/d45/d2c/f58 [1934160,108756] 0 2024-03-16T09:01:15.550 INFO:tasks.workunit.client.1.smithi019.stdout:8/406: fsync d6/d2a/d45/d19/f31 0 2024-03-16T09:01:15.551 INFO:tasks.workunit.client.1.smithi019.stdout:5/379: symlink d6/dd/d61/l73 0 2024-03-16T09:01:15.552 INFO:tasks.workunit.client.1.smithi019.stdout:2/458: rmdir d5/d6/d64 39 2024-03-16T09:01:15.552 INFO:tasks.workunit.client.1.smithi019.stdout:2/459: creat d5/d6/d2b/f98 x:0 0 0 2024-03-16T09:01:15.553 INFO:tasks.workunit.client.1.smithi019.stdout:6/498: symlink de/d1c/la3 0 2024-03-16T09:01:15.553 INFO:tasks.workunit.client.1.smithi019.stdout:6/499: fdatasync de/d26/d32/d97/f46 0 2024-03-16T09:01:15.553 INFO:tasks.workunit.client.1.smithi019.stdout:7/275: creat d9/da/d13/d18/d46/f50 x:0 0 0 2024-03-16T09:01:15.554 INFO:tasks.workunit.client.0.smithi007.stdout:8/890: write d0/dc/d80/fe5 [3907825,25226] 0 2024-03-16T09:01:15.557 INFO:tasks.workunit.client.1.smithi019.stdout:3/453: truncate d3/d1b/d40/f67 1389227 0 2024-03-16T09:01:15.557 INFO:tasks.workunit.client.1.smithi019.stdout:1/335: dwrite d16/f20 [0,4194304] 0 2024-03-16T09:01:15.557 INFO:tasks.workunit.client.1.smithi019.stdout:0/337: dread d7/d1d/f4b [0,4194304] 0 2024-03-16T09:01:15.557 INFO:tasks.workunit.client.1.smithi019.stdout:3/454: chown d3/d17/d5a/f62 218140 1 2024-03-16T09:01:15.558 INFO:tasks.workunit.client.1.smithi019.stdout:0/338: chown d7/d1d/d28/f64 817950 1 2024-03-16T09:01:15.558 INFO:tasks.workunit.client.1.smithi019.stdout:0/339: readlink d7/d1d/l30 0 2024-03-16T09:01:15.558 INFO:tasks.workunit.client.1.smithi019.stdout:0/340: creat d7/d1d/f67 x:0 0 0 2024-03-16T09:01:15.558 INFO:tasks.workunit.client.1.smithi019.stdout:0/341: fsync d7/d29/f41 0 2024-03-16T09:01:15.559 INFO:tasks.workunit.client.1.smithi019.stdout:5/380: rename d6/d1f/f2d to d6/f74 0 2024-03-16T09:01:15.561 INFO:tasks.workunit.client.1.smithi019.stdout:4/375: getdents d4/d6c 0 2024-03-16T09:01:15.562 INFO:tasks.workunit.client.1.smithi019.stdout:6/500: truncate de/d26/d32/d4e/f59 621644 0 2024-03-16T09:01:15.562 INFO:tasks.workunit.client.1.smithi019.stdout:6/501: readlink de/l31 0 2024-03-16T09:01:15.562 INFO:tasks.workunit.client.1.smithi019.stdout:0/342: mkdir d7/d8/d32/d4a/d61/d68 0 2024-03-16T09:01:15.563 INFO:tasks.workunit.client.1.smithi019.stdout:6/502: truncate de/d26/d32/f5b 1464056 0 2024-03-16T09:01:15.564 INFO:tasks.workunit.client.1.smithi019.stdout:0/343: fsync d7/fd 0 2024-03-16T09:01:15.564 INFO:tasks.workunit.client.1.smithi019.stdout:5/381: dread d6/dd/f44 [0,4194304] 0 2024-03-16T09:01:15.565 INFO:tasks.workunit.client.1.smithi019.stdout:8/407: write d6/d2a/d45/d57/f62 [1889911,104941] 0 2024-03-16T09:01:15.565 INFO:tasks.workunit.client.1.smithi019.stdout:8/408: fsync d6/de/f6a 0 2024-03-16T09:01:15.565 INFO:tasks.workunit.client.1.smithi019.stdout:8/409: write d6/de/d24/d30/f3b [167221,21878] 0 2024-03-16T09:01:15.565 INFO:tasks.workunit.client.1.smithi019.stdout:8/410: stat d6/de/d24/d30/d4a/d87 0 2024-03-16T09:01:15.566 INFO:tasks.workunit.client.1.smithi019.stdout:6/503: mkdir de/d26/d32/d97/d66/da4 0 2024-03-16T09:01:15.567 INFO:tasks.workunit.client.1.smithi019.stdout:0/344: rmdir d7/d8/d32/d4a/d61 39 2024-03-16T09:01:15.567 INFO:tasks.workunit.client.1.smithi019.stdout:0/345: dread - d7/d8/f50 zero size 2024-03-16T09:01:15.568 INFO:tasks.workunit.client.1.smithi019.stdout:5/382: rename d6/dd/l12 to d6/d1f/d33/d66/l75 0 2024-03-16T09:01:15.568 INFO:tasks.workunit.client.1.smithi019.stdout:5/383: chown d6/d19/f4e 33 1 2024-03-16T09:01:15.570 INFO:tasks.workunit.client.1.smithi019.stdout:2/460: dread d5/d6/d2b/d89/d7d/f8b [0,4194304] 0 2024-03-16T09:01:15.570 INFO:tasks.workunit.client.1.smithi019.stdout:8/411: unlink d6/d2a/d3c/f56 0 2024-03-16T09:01:15.571 INFO:tasks.workunit.client.1.smithi019.stdout:9/268: dwrite d1/d7/de/d24/f39 [4194304,4194304] 0 2024-03-16T09:01:15.573 INFO:tasks.workunit.client.1.smithi019.stdout:5/384: mkdir d6/dd/d37/d76 0 2024-03-16T09:01:15.576 INFO:tasks.workunit.client.1.smithi019.stdout:2/461: mknod d5/d6/d64/c99 0 2024-03-16T09:01:15.576 INFO:tasks.workunit.client.1.smithi019.stdout:2/462: fdatasync d5/de/f32 0 2024-03-16T09:01:15.576 INFO:tasks.workunit.client.1.smithi019.stdout:8/412: rename d6/d2a/d55/f6e to d6/d2a/f88 0 2024-03-16T09:01:15.577 INFO:tasks.workunit.client.1.smithi019.stdout:9/269: symlink d1/d7/de/l5c 0 2024-03-16T09:01:15.577 INFO:tasks.workunit.client.1.smithi019.stdout:9/270: stat d1/d7/f45 0 2024-03-16T09:01:15.579 INFO:tasks.workunit.client.1.smithi019.stdout:2/463: dread d5/d6/d2b/d2f/d35/d4b/f5c [0,4194304] 0 2024-03-16T09:01:15.580 INFO:tasks.workunit.client.1.smithi019.stdout:6/504: dread de/d26/d32/d97/f30 [4194304,4194304] 0 2024-03-16T09:01:15.580 INFO:tasks.workunit.client.1.smithi019.stdout:6/505: creat de/d26/d32/d4e/fa5 x:0 0 0 2024-03-16T09:01:15.582 INFO:tasks.workunit.client.1.smithi019.stdout:2/464: unlink d5/de/c43 0 2024-03-16T09:01:15.582 INFO:tasks.workunit.client.1.smithi019.stdout:2/465: creat d5/d2e/f9a x:0 0 0 2024-03-16T09:01:15.587 INFO:tasks.workunit.client.1.smithi019.stdout:4/376: fsync d4/f26 0 2024-03-16T09:01:15.587 INFO:tasks.workunit.client.1.smithi019.stdout:2/466: getdents d5/d6 0 2024-03-16T09:01:15.589 INFO:tasks.workunit.client.1.smithi019.stdout:4/377: creat d4/d1b/d2a/d4f/f82 x:0 0 0 2024-03-16T09:01:15.589 INFO:tasks.workunit.client.1.smithi019.stdout:2/467: symlink d5/d6/d64/d6f/d87/l9b 0 2024-03-16T09:01:15.591 INFO:tasks.workunit.client.1.smithi019.stdout:4/378: unlink d4/l1d 0 2024-03-16T09:01:15.592 INFO:tasks.workunit.client.1.smithi019.stdout:2/468: symlink d5/d6/d2b/l9c 0 2024-03-16T09:01:15.592 INFO:tasks.workunit.client.1.smithi019.stdout:2/469: creat d5/d5a/f9d x:0 0 0 2024-03-16T09:01:15.597 INFO:tasks.workunit.client.1.smithi019.stdout:1/336: dwrite d16/f37 [0,4194304] 0 2024-03-16T09:01:15.597 INFO:tasks.workunit.client.1.smithi019.stdout:1/337: chown d16/d1f/d43/l52 11 1 2024-03-16T09:01:15.598 INFO:tasks.workunit.client.1.smithi019.stdout:2/470: rmdir d5/d6/d2b/d2f 39 2024-03-16T09:01:15.598 INFO:tasks.workunit.client.1.smithi019.stdout:2/471: getdents d5/d5a/d96 0 2024-03-16T09:01:15.598 INFO:tasks.workunit.client.1.smithi019.stdout:1/338: read d16/d18/f1b [541632,63193] 0 2024-03-16T09:01:15.600 INFO:tasks.workunit.client.1.smithi019.stdout:2/472: rename d5/d6/d2b/l9c to d5/d6/d2b/d2f/d35/l9e 0 2024-03-16T09:01:15.600 INFO:tasks.workunit.client.1.smithi019.stdout:6/506: write de/d1c/f20 [2981504,105468] 0 2024-03-16T09:01:15.600 INFO:tasks.workunit.client.1.smithi019.stdout:6/507: fdatasync de/d26/d32/d97/d66/d72/d7f/f81 0 2024-03-16T09:01:15.600 INFO:tasks.workunit.client.1.smithi019.stdout:1/339: link d16/d1f/d43/f4c d16/d1f/d43/d55/f5e 0 2024-03-16T09:01:15.600 INFO:tasks.workunit.client.1.smithi019.stdout:1/340: chown d16/d1f/f51 944 1 2024-03-16T09:01:15.602 INFO:tasks.workunit.client.1.smithi019.stdout:6/508: rename de/d26/d32/d97/f93 to de/d26/d32/d97/da1/fa6 0 2024-03-16T09:01:15.603 INFO:tasks.workunit.client.1.smithi019.stdout:1/341: mknod d16/d31/c5f 0 2024-03-16T09:01:15.603 INFO:tasks.workunit.client.1.smithi019.stdout:1/342: truncate d16/d1f/d4f/f57 663113 0 2024-03-16T09:01:15.603 INFO:tasks.workunit.client.1.smithi019.stdout:6/509: symlink de/d26/d32/d97/d66/la7 0 2024-03-16T09:01:15.603 INFO:tasks.workunit.client.1.smithi019.stdout:6/510: fdatasync de/f14 0 2024-03-16T09:01:15.604 INFO:tasks.workunit.client.0.smithi007.stdout:3/627: dwrite d15/d18/d1f/d29/d43/f7a [0,4194304] 0 2024-03-16T09:01:15.608 INFO:tasks.workunit.client.1.smithi019.stdout:7/276: dwrite d9/da/d13/f17 [0,4194304] 0 2024-03-16T09:01:15.610 INFO:tasks.workunit.client.1.smithi019.stdout:7/277: mknod d9/d25/d28/c51 0 2024-03-16T09:01:15.614 INFO:tasks.workunit.client.0.smithi007.stdout:3/628: write d15/d18/d1f/d29/d43/f7a [1766665,73323] 0 2024-03-16T09:01:15.616 INFO:tasks.workunit.client.1.smithi019.stdout:7/278: truncate d9/db/f47 3366674 0 2024-03-16T09:01:15.617 INFO:tasks.workunit.client.1.smithi019.stdout:7/279: symlink d9/da/l52 0 2024-03-16T09:01:15.617 INFO:tasks.workunit.client.1.smithi019.stdout:7/280: truncate d9/d25/d28/f39 414148 0 2024-03-16T09:01:15.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/281: read d9/db/d12/f2f [77765,13096] 0 2024-03-16T09:01:15.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/282: truncate d9/da/d13/d18/d46/f50 233152 0 2024-03-16T09:01:15.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/283: write d9/da/d13/d18/d30/f4a [550769,71414] 0 2024-03-16T09:01:15.619 INFO:tasks.workunit.client.1.smithi019.stdout:7/284: unlink d9/da/d13/d18/d1a/d32/f41 0 2024-03-16T09:01:15.620 INFO:tasks.workunit.client.1.smithi019.stdout:7/285: rename d9/da to d9/da/d53 22 2024-03-16T09:01:15.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/286: unlink f6 0 2024-03-16T09:01:15.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/287: dread - d9/da/d13/d18/d1a/f2b zero size 2024-03-16T09:01:15.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/288: fdatasync d9/d25/d28/f34 0 2024-03-16T09:01:15.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/289: write d9/f23 [1163099,105089] 0 2024-03-16T09:01:15.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/290: creat d9/d25/f54 x:0 0 0 2024-03-16T09:01:15.623 INFO:tasks.workunit.client.0.smithi007.stdout:0/697: dwrite d3/d9/da/d18/f59 [0,4194304] 0 2024-03-16T09:01:15.623 INFO:tasks.workunit.client.0.smithi007.stdout:0/698: write d3/d1f/fb7 [1552925,47077] 0 2024-03-16T09:01:15.624 INFO:tasks.workunit.client.1.smithi019.stdout:7/291: truncate d9/d25/d28/f34 363588 0 2024-03-16T09:01:15.624 INFO:tasks.workunit.client.1.smithi019.stdout:7/292: creat d9/db/d12/f55 x:0 0 0 2024-03-16T09:01:15.625 INFO:tasks.workunit.client.1.smithi019.stdout:7/293: symlink d9/da/d13/l56 0 2024-03-16T09:01:15.625 INFO:tasks.workunit.client.1.smithi019.stdout:7/294: stat d9/db/d12/f2f 0 2024-03-16T09:01:15.625 INFO:tasks.workunit.client.0.smithi007.stdout:0/699: rename d3/d9/da/f1e to d3/d9/db4/d60/fe5 0 2024-03-16T09:01:15.626 INFO:tasks.workunit.client.1.smithi019.stdout:7/295: getdents d9/da/d13/d18/d1a/d32/d33 0 2024-03-16T09:01:15.626 INFO:tasks.workunit.client.0.smithi007.stdout:0/700: rmdir d3/d9/da/d92/d99 39 2024-03-16T09:01:15.626 INFO:tasks.workunit.client.0.smithi007.stdout:0/701: creat d3/d9/da/d48/d9e/fe6 x:0 0 0 2024-03-16T09:01:15.627 INFO:tasks.workunit.client.1.smithi019.stdout:7/296: getdents d9/db 0 2024-03-16T09:01:15.627 INFO:tasks.workunit.client.1.smithi019.stdout:7/297: creat d9/d25/d28/f57 x:0 0 0 2024-03-16T09:01:15.628 INFO:tasks.workunit.client.0.smithi007.stdout:0/702: mknod d3/d9/db4/d2c/ce7 0 2024-03-16T09:01:15.628 INFO:tasks.workunit.client.0.smithi007.stdout:0/703: fdatasync d3/d9/da/d18/f59 0 2024-03-16T09:01:15.629 INFO:tasks.workunit.client.1.smithi019.stdout:3/455: dwrite d3/d89/f47 [0,4194304] 0 2024-03-16T09:01:15.629 INFO:tasks.workunit.client.1.smithi019.stdout:5/385: dwrite d6/f8 [0,4194304] 0 2024-03-16T09:01:15.632 INFO:tasks.workunit.client.1.smithi019.stdout:5/386: truncate f4 2456179 0 2024-03-16T09:01:15.632 INFO:tasks.workunit.client.0.smithi007.stdout:0/704: rmdir d3/d9/da/d18/d27 39 2024-03-16T09:01:15.633 INFO:tasks.workunit.client.1.smithi019.stdout:3/456: rename d3/d17/d5a/f85 to d3/d17/d5a/f8c 0 2024-03-16T09:01:15.633 INFO:tasks.workunit.client.1.smithi019.stdout:3/457: fsync d3/d17/d5a/f5c 0 2024-03-16T09:01:15.633 INFO:tasks.workunit.client.1.smithi019.stdout:3/458: stat d3/ce 0 2024-03-16T09:01:15.633 INFO:tasks.workunit.client.1.smithi019.stdout:3/459: stat d3/d17/l8a 0 2024-03-16T09:01:15.635 INFO:tasks.workunit.client.1.smithi019.stdout:3/460: rename d3/d89/c53 to d3/d1b/d40/c8d 0 2024-03-16T09:01:15.635 INFO:tasks.workunit.client.1.smithi019.stdout:3/461: creat d3/d1b/f8e x:0 0 0 2024-03-16T09:01:15.635 INFO:tasks.workunit.client.1.smithi019.stdout:3/462: getdents d3/d17/d5a/d80 0 2024-03-16T09:01:15.638 INFO:tasks.workunit.client.0.smithi007.stdout:0/705: write d3/d9/da/d18/d23/f32 [2159490,109028] 0 2024-03-16T09:01:15.642 INFO:tasks.workunit.client.0.smithi007.stdout:0/706: mkdir d3/d9/da/d92/d9f/de8 0 2024-03-16T09:01:15.647 INFO:tasks.workunit.client.0.smithi007.stdout:0/707: dread d3/d9/da/d18/d23/f63 [0,4194304] 0 2024-03-16T09:01:15.647 INFO:tasks.workunit.client.0.smithi007.stdout:0/708: fdatasync d3/d9/da/d48/d25/f68 0 2024-03-16T09:01:15.647 INFO:tasks.workunit.client.0.smithi007.stdout:0/709: write d3/d9/da/d18/d23/f81 [296194,2012] 0 2024-03-16T09:01:15.651 INFO:tasks.workunit.client.1.smithi019.stdout:0/346: dwrite d7/f2a [0,4194304] 0 2024-03-16T09:01:15.657 INFO:tasks.workunit.client.0.smithi007.stdout:8/891: dwrite d0/dc/d50/d8e/dd2/dea/d18/f4e [4194304,4194304] 0 2024-03-16T09:01:15.657 INFO:tasks.workunit.client.0.smithi007.stdout:8/892: write d0/d26/f79 [4847388,130144] 0 2024-03-16T09:01:15.657 INFO:tasks.workunit.client.0.smithi007.stdout:8/893: write d0/dc/d50/d8e/dd2/dea/d10a/f10d [804268,119839] 0 2024-03-16T09:01:15.658 INFO:tasks.workunit.client.0.smithi007.stdout:8/894: getdents d0/d26/d2d/d64/da6/d10e/dfd 0 2024-03-16T09:01:15.662 INFO:tasks.workunit.client.1.smithi019.stdout:8/413: fsync d6/d2a/d45/d19/d74/f7e 0 2024-03-16T09:01:15.662 INFO:tasks.workunit.client.1.smithi019.stdout:8/414: truncate d6/f77 238335 0 2024-03-16T09:01:15.663 INFO:tasks.workunit.client.1.smithi019.stdout:8/415: rmdir d6/de/d24/d30/d4a 39 2024-03-16T09:01:15.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/895: write d0/d26/d2d/ff2 [1603679,40706] 0 2024-03-16T09:01:15.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/896: read d0/dc/d50/d8e/dd2/dea/d10a/f10d [719924,103841] 0 2024-03-16T09:01:15.666 INFO:tasks.workunit.client.0.smithi007.stdout:8/897: write d0/dc/d54/f98 [1299960,13404] 0 2024-03-16T09:01:15.668 INFO:tasks.workunit.client.0.smithi007.stdout:8/898: rename d0/dc/d50/d8e/dd2/dea/d18/da7/de2/d113/d12c to d0/dc/dd/d139 0 2024-03-16T09:01:15.669 INFO:tasks.workunit.client.0.smithi007.stdout:8/899: creat d0/dc/dd/f13a x:0 0 0 2024-03-16T09:01:15.669 INFO:tasks.workunit.client.0.smithi007.stdout:8/900: fdatasync d0/dc/d50/d8e/dd2/dea/d74/fd3 0 2024-03-16T09:01:15.670 INFO:tasks.workunit.client.0.smithi007.stdout:8/901: unlink d0/dc/dd/d139/c131 0 2024-03-16T09:01:15.671 INFO:tasks.workunit.client.0.smithi007.stdout:8/902: symlink d0/d26/d2d/d64/da6/l13b 0 2024-03-16T09:01:15.672 INFO:tasks.workunit.client.0.smithi007.stdout:8/903: mknod d0/c13c 0 2024-03-16T09:01:15.673 INFO:tasks.workunit.client.0.smithi007.stdout:8/904: creat d0/dc/d50/d8e/dd2/dea/d18/da7/dc5/f13d x:0 0 0 2024-03-16T09:01:15.673 INFO:tasks.workunit.client.0.smithi007.stdout:8/905: chown d0/d26/d2d/d64/l100 3427 1 2024-03-16T09:01:15.674 INFO:tasks.workunit.client.0.smithi007.stdout:8/906: creat d0/dc/d50/d8e/dd2/dea/d18/da7/dc5/def/f13e x:0 0 0 2024-03-16T09:01:15.674 INFO:tasks.workunit.client.0.smithi007.stdout:8/907: readlink d0/dc/d50/d8e/dd2/dea/l90 0 2024-03-16T09:01:15.678 INFO:tasks.workunit.client.1.smithi019.stdout:1/343: dwrite d16/d18/f39 [4194304,4194304] 0 2024-03-16T09:01:15.678 INFO:tasks.workunit.client.1.smithi019.stdout:1/344: chown d16/d1f/d43/d55/f5e 52 1 2024-03-16T09:01:15.681 INFO:tasks.workunit.client.1.smithi019.stdout:8/416: write d6/d2a/d45/d19/f43 [1178625,76141] 0 2024-03-16T09:01:15.681 INFO:tasks.workunit.client.1.smithi019.stdout:2/473: dwrite d5/d6/d2b/d89/d7d/f8b [0,4194304] 0 2024-03-16T09:01:15.682 INFO:tasks.workunit.client.1.smithi019.stdout:4/379: dwrite d4/d1b/f1c [4194304,4194304] 0 2024-03-16T09:01:15.682 INFO:tasks.workunit.client.1.smithi019.stdout:4/380: fsync d4/d6/f70 0 2024-03-16T09:01:15.683 INFO:tasks.workunit.client.0.smithi007.stdout:3/629: dwrite d15/d18/d1f/d3b/f47 [0,4194304] 0 2024-03-16T09:01:15.683 INFO:tasks.workunit.client.0.smithi007.stdout:0/710: dwrite d3/d6c/dbd/dd7/fa3 [0,4194304] 0 2024-03-16T09:01:15.683 INFO:tasks.workunit.client.1.smithi019.stdout:6/511: dwrite de/d26/d32/d97/f5a [0,4194304] 0 2024-03-16T09:01:15.683 INFO:tasks.workunit.client.1.smithi019.stdout:6/512: write de/f45 [744951,58171] 0 2024-03-16T09:01:15.684 INFO:tasks.workunit.client.1.smithi019.stdout:3/463: dwrite d3/d1b/f8e [0,4194304] 0 2024-03-16T09:01:15.686 INFO:tasks.workunit.client.1.smithi019.stdout:0/347: dwrite d7/f19 [0,4194304] 0 2024-03-16T09:01:15.686 INFO:tasks.workunit.client.1.smithi019.stdout:0/348: stat d7/d8/d32/d4a/d61 0 2024-03-16T09:01:15.686 INFO:tasks.workunit.client.1.smithi019.stdout:0/349: creat d7/d8/d32/d36/f69 x:0 0 0 2024-03-16T09:01:15.691 INFO:tasks.workunit.client.0.smithi007.stdout:3/630: getdents d15/d18/d1f/d29/d43/d72 0 2024-03-16T09:01:15.691 INFO:tasks.workunit.client.0.smithi007.stdout:3/631: chown d15/d18/d1f/d29/d7c/fb0 578655795 1 2024-03-16T09:01:15.691 INFO:tasks.workunit.client.0.smithi007.stdout:3/632: readlink d15/d18/d1f/d29/d43/d72/l9b 0 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:2/474: dread d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:2/475: dread - d5/d2e/f9a zero size 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:2/476: readlink d5/d5a/d80/l94 0 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:6/513: read de/d26/d32/f5b [272720,35931] 0 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:5/387: dwrite d6/d1f/d33/d4c/f62 [0,4194304] 0 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:8/417: symlink d6/d2a/d45/d2c/l89 0 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:8/418: creat d6/d2a/d45/d19/d74/f8a x:0 0 0 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:8/419: write f4 [6107798,4904] 0 2024-03-16T09:01:15.692 INFO:tasks.workunit.client.1.smithi019.stdout:8/420: fsync d6/d2a/d45/d57/f60 0 2024-03-16T09:01:15.694 INFO:tasks.workunit.client.1.smithi019.stdout:2/477: dread d5/d6/f28 [0,4194304] 0 2024-03-16T09:01:15.694 INFO:tasks.workunit.client.1.smithi019.stdout:4/381: truncate d4/d6/f24 279572 0 2024-03-16T09:01:15.695 INFO:tasks.workunit.client.0.smithi007.stdout:0/711: creat d3/d9/db4/de3/fe9 x:0 0 0 2024-03-16T09:01:15.695 INFO:tasks.workunit.client.0.smithi007.stdout:0/712: write d3/d9/da/d48/d25/d87/f98 [410527,54680] 0 2024-03-16T09:01:15.695 INFO:tasks.workunit.client.0.smithi007.stdout:0/713: dread - d3/d9/da/d18/f7d zero size 2024-03-16T09:01:15.696 INFO:tasks.workunit.client.0.smithi007.stdout:3/633: symlink d15/d18/d1f/d29/d43/d72/lc5 0 2024-03-16T09:01:15.697 INFO:tasks.workunit.client.1.smithi019.stdout:3/464: truncate f2 3880107 0 2024-03-16T09:01:15.699 INFO:tasks.workunit.client.0.smithi007.stdout:3/634: unlink d15/d18/d1f/d29/d43/d72/lc5 0 2024-03-16T09:01:15.699 INFO:tasks.workunit.client.0.smithi007.stdout:3/635: chown d15/d18/d6a/d78/lbc 149012687 1 2024-03-16T09:01:15.699 INFO:tasks.workunit.client.0.smithi007.stdout:0/714: dread d3/d9/da/d48/fa9 [0,4194304] 0 2024-03-16T09:01:15.700 INFO:tasks.workunit.client.0.smithi007.stdout:3/636: dread d15/d18/d1f/d29/d3e/d85/fa6 [0,4194304] 0 2024-03-16T09:01:15.701 INFO:tasks.workunit.client.1.smithi019.stdout:0/350: creat d7/f6a x:0 0 0 2024-03-16T09:01:15.705 INFO:tasks.workunit.client.1.smithi019.stdout:5/388: unlink d6/dd/d37/f26 0 2024-03-16T09:01:15.705 INFO:tasks.workunit.client.1.smithi019.stdout:5/389: chown d6 120459 1 2024-03-16T09:01:15.706 INFO:tasks.workunit.client.1.smithi019.stdout:7/298: dwrite d9/f23 [0,4194304] 0 2024-03-16T09:01:15.706 INFO:tasks.workunit.client.1.smithi019.stdout:1/345: dwrite d16/d1f/f27 [0,4194304] 0 2024-03-16T09:01:15.706 INFO:tasks.workunit.client.1.smithi019.stdout:6/514: link de/d1c/c78 de/d1c/d9a/d9e/ca8 0 2024-03-16T09:01:15.706 INFO:tasks.workunit.client.1.smithi019.stdout:6/515: getdents de/d26/d32/d4e/d57/d9f 0 2024-03-16T09:01:15.707 INFO:tasks.workunit.client.1.smithi019.stdout:6/516: creat de/d26/d73/fa9 x:0 0 0 2024-03-16T09:01:15.708 INFO:tasks.workunit.client.0.smithi007.stdout:3/637: dread d15/d18/f33 [0,4194304] 0 2024-03-16T09:01:15.709 INFO:tasks.workunit.client.1.smithi019.stdout:8/421: symlink d6/de/d24/l8b 0 2024-03-16T09:01:15.709 INFO:tasks.workunit.client.1.smithi019.stdout:8/422: fsync d6/de/f6c 0 2024-03-16T09:01:15.709 INFO:tasks.workunit.client.1.smithi019.stdout:8/423: truncate d6/d2a/d45/d19/d74/f8a 444231 0 2024-03-16T09:01:15.710 INFO:tasks.workunit.client.1.smithi019.stdout:2/478: symlink d5/d6/d64/d6f/d87/l9f 0 2024-03-16T09:01:15.710 INFO:tasks.workunit.client.0.smithi007.stdout:3/638: link d15/c76 d15/d18/d1f/d29/d3e/d4c/cc6 0 2024-03-16T09:01:15.712 INFO:tasks.workunit.client.1.smithi019.stdout:4/382: unlink d4/d1b/d2a/d3d/l61 0 2024-03-16T09:01:15.713 INFO:tasks.workunit.client.0.smithi007.stdout:3/639: unlink d15/f1e 0 2024-03-16T09:01:15.714 INFO:tasks.workunit.client.1.smithi019.stdout:5/390: mkdir d6/d1f/d77 0 2024-03-16T09:01:15.714 INFO:tasks.workunit.client.1.smithi019.stdout:5/391: fdatasync d6/d1f/f5e 0 2024-03-16T09:01:15.714 INFO:tasks.workunit.client.1.smithi019.stdout:5/392: chown d6/d1f/d33 0 1 2024-03-16T09:01:15.715 INFO:tasks.workunit.client.0.smithi007.stdout:3/640: rmdir d15/d18/d1f/d29/d3e/d4c 39 2024-03-16T09:01:15.717 INFO:tasks.workunit.client.1.smithi019.stdout:7/299: truncate d9/da/d13/d18/d1a/f1f 2345495 0 2024-03-16T09:01:15.717 INFO:tasks.workunit.client.1.smithi019.stdout:7/300: fsync f4 0 2024-03-16T09:01:15.717 INFO:tasks.workunit.client.0.smithi007.stdout:3/641: unlink d15/d18/d1f/f39 0 2024-03-16T09:01:15.717 INFO:tasks.workunit.client.0.smithi007.stdout:3/642: write d15/d18/d1f/d29/d43/f69 [10535,54973] 0 2024-03-16T09:01:15.717 INFO:tasks.workunit.client.0.smithi007.stdout:3/643: chown d15/d18/d1f/d29/d3e/dc0 1 1 2024-03-16T09:01:15.720 INFO:tasks.workunit.client.0.smithi007.stdout:3/644: symlink d15/d18/d1f/d29/d43/lc7 0 2024-03-16T09:01:15.721 INFO:tasks.workunit.client.0.smithi007.stdout:3/645: stat d15/d18/d1f/d3b/c6e 0 2024-03-16T09:01:15.721 INFO:tasks.workunit.client.0.smithi007.stdout:3/646: fdatasync d15/d18/d1f/d29/d3e/d4c/f64 0 2024-03-16T09:01:15.722 INFO:tasks.workunit.client.1.smithi019.stdout:0/351: dwrite d7/d8/f12 [0,4194304] 0 2024-03-16T09:01:15.723 INFO:tasks.workunit.client.1.smithi019.stdout:0/352: dread d7/d29/f47 [0,4194304] 0 2024-03-16T09:01:15.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/647: creat d15/d18/d1f/d29/d3e/fc8 x:0 0 0 2024-03-16T09:01:15.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/648: fsync d15/d18/d1f/d29/f31 0 2024-03-16T09:01:15.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/346: dread d16/d18/f24 [0,4194304] 0 2024-03-16T09:01:15.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/347: creat d16/d1f/d4f/f60 x:0 0 0 2024-03-16T09:01:15.727 INFO:tasks.workunit.client.1.smithi019.stdout:6/517: link de/d1c/c78 de/d26/d32/d97/d66/d72/d7f/caa 0 2024-03-16T09:01:15.727 INFO:tasks.workunit.client.1.smithi019.stdout:6/518: chown de/d26/d32/d97/f46 444776000 1 2024-03-16T09:01:15.729 INFO:tasks.workunit.client.1.smithi019.stdout:8/424: link d6/d2a/d45/d2c/l89 d6/d2a/d68/l8c 0 2024-03-16T09:01:15.730 INFO:tasks.workunit.client.1.smithi019.stdout:2/479: mknod d5/d4e/d74/ca0 0 2024-03-16T09:01:15.732 INFO:tasks.workunit.client.1.smithi019.stdout:5/393: mkdir d6/d1f/d78 0 2024-03-16T09:01:15.732 INFO:tasks.workunit.client.1.smithi019.stdout:5/394: fsync d6/d1f/d33/d66/f45 0 2024-03-16T09:01:15.732 INFO:tasks.workunit.client.1.smithi019.stdout:5/395: write d6/dd/d37/d54/f5f [367854,11109] 0 2024-03-16T09:01:15.733 INFO:tasks.workunit.client.0.smithi007.stdout:3/649: dread d15/d18/d1f/d3b/d63/f66 [0,4194304] 0 2024-03-16T09:01:15.733 INFO:tasks.workunit.client.0.smithi007.stdout:3/650: fsync d15/d18/d1f/d29/d43/f55 0 2024-03-16T09:01:15.733 INFO:tasks.workunit.client.1.smithi019.stdout:7/301: chown d9/f3f 5939250 1 2024-03-16T09:01:15.733 INFO:tasks.workunit.client.1.smithi019.stdout:7/302: truncate d9/da/d13/d18/d1a/d32/d33/f43 880941 0 2024-03-16T09:01:15.733 INFO:tasks.workunit.client.1.smithi019.stdout:7/303: stat d9/db/c45 0 2024-03-16T09:01:15.735 INFO:tasks.workunit.client.0.smithi007.stdout:3/651: dread d15/d18/d1f/d56/f65 [0,4194304] 0 2024-03-16T09:01:15.735 INFO:tasks.workunit.client.1.smithi019.stdout:0/353: mknod d7/d8/d32/d4a/d61/d68/c6b 0 2024-03-16T09:01:15.736 INFO:tasks.workunit.client.1.smithi019.stdout:4/383: dwrite f2 [0,4194304] 0 2024-03-16T09:01:15.738 INFO:tasks.workunit.client.1.smithi019.stdout:1/348: mkdir d16/d1f/d4f/d61 0 2024-03-16T09:01:15.738 INFO:tasks.workunit.client.1.smithi019.stdout:1/349: write fb [1883585,91761] 0 2024-03-16T09:01:15.739 INFO:tasks.workunit.client.1.smithi019.stdout:1/350: creat d16/d1f/d4f/f62 x:0 0 0 2024-03-16T09:01:15.739 INFO:tasks.workunit.client.1.smithi019.stdout:1/351: write f10 [1631568,45239] 0 2024-03-16T09:01:15.739 INFO:tasks.workunit.client.1.smithi019.stdout:1/352: readlink d16/l59 0 2024-03-16T09:01:15.739 INFO:tasks.workunit.client.1.smithi019.stdout:1/353: chown d16/d1f/d4f/f62 42 1 2024-03-16T09:01:15.740 INFO:tasks.workunit.client.1.smithi019.stdout:2/480: dread d5/f7 [0,4194304] 0 2024-03-16T09:01:15.741 INFO:tasks.workunit.client.1.smithi019.stdout:6/519: unlink de/d26/d32/d97/d2c/l55 0 2024-03-16T09:01:15.741 INFO:tasks.workunit.client.1.smithi019.stdout:6/520: fsync de/d26/d32/d4e/d75/fa2 0 2024-03-16T09:01:15.743 INFO:tasks.workunit.client.1.smithi019.stdout:8/425: unlink d6/d2a/d45/d19/f2e 0 2024-03-16T09:01:15.743 INFO:tasks.workunit.client.1.smithi019.stdout:8/426: truncate d6/de/f6c 887959 0 2024-03-16T09:01:15.743 INFO:tasks.workunit.client.1.smithi019.stdout:5/396: mknod d6/d1f/d77/c79 0 2024-03-16T09:01:15.743 INFO:tasks.workunit.client.1.smithi019.stdout:5/397: dread - d6/dd/f40 zero size 2024-03-16T09:01:15.743 INFO:tasks.workunit.client.1.smithi019.stdout:5/398: write d6/dd/d47/f64 [478593,40343] 0 2024-03-16T09:01:15.745 INFO:tasks.workunit.client.1.smithi019.stdout:7/304: creat d9/da/d13/d18/d1a/d32/d33/f58 x:0 0 0 2024-03-16T09:01:15.746 INFO:tasks.workunit.client.1.smithi019.stdout:0/354: mkdir d7/d1d/d28/d6c 0 2024-03-16T09:01:15.748 INFO:tasks.workunit.client.1.smithi019.stdout:4/384: dread d4/f35 [0,4194304] 0 2024-03-16T09:01:15.749 INFO:tasks.workunit.client.1.smithi019.stdout:8/427: rename d6/d2a/d45/d2c/f66 to d6/d2a/f8d 0 2024-03-16T09:01:15.750 INFO:tasks.workunit.client.1.smithi019.stdout:5/399: mkdir d6/dd/d47/d7a 0 2024-03-16T09:01:15.752 INFO:tasks.workunit.client.1.smithi019.stdout:7/305: creat d9/da/f59 x:0 0 0 2024-03-16T09:01:15.753 INFO:tasks.workunit.client.1.smithi019.stdout:0/355: getdents d7/d8/d32 0 2024-03-16T09:01:15.754 INFO:tasks.workunit.client.1.smithi019.stdout:5/400: mknod d6/d1f/d33/c7b 0 2024-03-16T09:01:15.755 INFO:tasks.workunit.client.1.smithi019.stdout:8/428: symlink d6/de/d24/d30/d4a/d87/l8e 0 2024-03-16T09:01:15.755 INFO:tasks.workunit.client.1.smithi019.stdout:8/429: stat d6/d2a/d45/d2c/l3e 0 2024-03-16T09:01:15.755 INFO:tasks.workunit.client.1.smithi019.stdout:8/430: chown d6/de/d24/d30/d4a/d67/f70 57431543 1 2024-03-16T09:01:15.756 INFO:tasks.workunit.client.1.smithi019.stdout:0/356: mkdir d7/d8/d32/d66/d6d 0 2024-03-16T09:01:15.757 INFO:tasks.workunit.client.1.smithi019.stdout:5/401: mkdir d6/dd/d37/d54/d7c 0 2024-03-16T09:01:15.757 INFO:tasks.workunit.client.1.smithi019.stdout:5/402: fdatasync d6/dd/d37/f60 0 2024-03-16T09:01:15.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/385: write d4/d1b/d2a/d3d/f46 [830602,35632] 0 2024-03-16T09:01:15.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/386: stat d4/d1b/d49/f53 0 2024-03-16T09:01:15.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/387: chown d4/d6/d15/f33 22357 1 2024-03-16T09:01:15.758 INFO:tasks.workunit.client.1.smithi019.stdout:4/388: truncate d4/d1b/d49/f72 494985 0 2024-03-16T09:01:15.758 INFO:tasks.workunit.client.1.smithi019.stdout:4/389: creat d4/d6/f83 x:0 0 0 2024-03-16T09:01:15.759 INFO:tasks.workunit.client.1.smithi019.stdout:0/357: mkdir d7/d1d/d63/d6e 0 2024-03-16T09:01:15.760 INFO:tasks.workunit.client.1.smithi019.stdout:0/358: creat d7/d29/f6f x:0 0 0 2024-03-16T09:01:15.760 INFO:tasks.workunit.client.1.smithi019.stdout:5/403: dread d6/dd/f44 [0,4194304] 0 2024-03-16T09:01:15.760 INFO:tasks.workunit.client.1.smithi019.stdout:5/404: dread - d6/d1f/f5a zero size 2024-03-16T09:01:15.764 INFO:tasks.workunit.client.1.smithi019.stdout:8/431: truncate d6/de/d24/f40 2008295 0 2024-03-16T09:01:15.764 INFO:tasks.workunit.client.1.smithi019.stdout:8/432: readlink d6/d2a/d45/d19/l48 0 2024-03-16T09:01:15.766 INFO:tasks.workunit.client.1.smithi019.stdout:1/354: dread d16/d1f/f51 [0,4194304] 0 2024-03-16T09:01:15.766 INFO:tasks.workunit.client.1.smithi019.stdout:1/355: chown d16/d18/f1a 118 1 2024-03-16T09:01:15.767 INFO:tasks.workunit.client.1.smithi019.stdout:0/359: rename d7/d1d/d28/c20 to d7/d8/d32/d4a/d61/d68/c70 0 2024-03-16T09:01:15.767 INFO:tasks.workunit.client.1.smithi019.stdout:0/360: creat d7/d29/f71 x:0 0 0 2024-03-16T09:01:15.771 INFO:tasks.workunit.client.1.smithi019.stdout:5/405: mkdir d6/d1f/d33/d7d 0 2024-03-16T09:01:15.771 INFO:tasks.workunit.client.1.smithi019.stdout:5/406: chown d6 117594 1 2024-03-16T09:01:15.771 INFO:tasks.workunit.client.1.smithi019.stdout:5/407: fsync d6/dd/d37/f3d 0 2024-03-16T09:01:15.772 INFO:tasks.workunit.client.1.smithi019.stdout:8/433: symlink d6/d2a/d55/l8f 0 2024-03-16T09:01:15.773 INFO:tasks.workunit.client.0.smithi007.stdout:5/589: sync 2024-03-16T09:01:15.773 INFO:tasks.workunit.client.0.smithi007.stdout:7/493: sync 2024-03-16T09:01:15.773 INFO:tasks.workunit.client.0.smithi007.stdout:1/535: sync 2024-03-16T09:01:15.773 INFO:tasks.workunit.client.0.smithi007.stdout:6/512: sync 2024-03-16T09:01:15.773 INFO:tasks.workunit.client.0.smithi007.stdout:2/686: sync 2024-03-16T09:01:15.773 INFO:tasks.workunit.client.0.smithi007.stdout:5/590: creat d1/d7/d1c/d19/d34/d4d/fd8 x:0 0 0 2024-03-16T09:01:15.774 INFO:tasks.workunit.client.1.smithi019.stdout:1/356: symlink d16/d1f/d4f/l63 0 2024-03-16T09:01:15.777 INFO:tasks.workunit.client.1.smithi019.stdout:8/434: creat d6/de/d24/f90 x:0 0 0 2024-03-16T09:01:15.777 INFO:tasks.workunit.client.1.smithi019.stdout:8/435: dread - d6/d2a/d3f/f7d zero size 2024-03-16T09:01:15.777 INFO:tasks.workunit.client.1.smithi019.stdout:8/436: dread - d6/d2a/d45/d19/d74/f7f zero size 2024-03-16T09:01:15.777 INFO:tasks.workunit.client.1.smithi019.stdout:8/437: write d6/de/d24/d30/d4a/f4e [937057,89499] 0 2024-03-16T09:01:15.781 INFO:tasks.workunit.client.1.smithi019.stdout:9/271: sync 2024-03-16T09:01:15.785 INFO:tasks.workunit.client.1.smithi019.stdout:2/481: dwrite d5/d6/d2b/d2f/d35/f58 [0,4194304] 0 2024-03-16T09:01:15.789 INFO:tasks.workunit.client.0.smithi007.stdout:3/652: symlink d15/d18/d1f/d29/d3e/d4c/lc9 0 2024-03-16T09:01:15.789 INFO:tasks.workunit.client.1.smithi019.stdout:6/521: dwrite de/d26/d32/d4e/d75/fa2 [0,4194304] 0 2024-03-16T09:01:15.792 INFO:tasks.workunit.client.1.smithi019.stdout:3/465: dwrite f2 [0,4194304] 0 2024-03-16T09:01:15.793 INFO:tasks.workunit.client.1.smithi019.stdout:7/306: dwrite d9/db/f47 [0,4194304] 0 2024-03-16T09:01:15.809 INFO:tasks.workunit.client.1.smithi019.stdout:3/466: dread d3/d17/f59 [0,4194304] 0 2024-03-16T09:01:15.819 INFO:tasks.workunit.client.1.smithi019.stdout:8/438: unlink d6/d2a/d45/f75 0 2024-03-16T09:01:15.821 INFO:tasks.workunit.client.0.smithi007.stdout:1/536: dwrite d9/dd/d1b/d28/f60 [0,4194304] 0 2024-03-16T09:01:15.821 INFO:tasks.workunit.client.0.smithi007.stdout:1/537: dread - d9/dd/d1b/d2d/d30/f8e zero size 2024-03-16T09:01:15.823 INFO:tasks.workunit.client.1.smithi019.stdout:9/272: symlink d1/d7/de/d2d/l5d 0 2024-03-16T09:01:15.823 INFO:tasks.workunit.client.1.smithi019.stdout:9/273: chown d1/d7/de/d24/f39 5609 1 2024-03-16T09:01:15.827 INFO:tasks.workunit.client.1.smithi019.stdout:2/482: rename d5/d6/d2b/d2f/d35/f7e to d5/d6/d2b/d89/d7d/fa1 0 2024-03-16T09:01:15.836 INFO:tasks.workunit.client.0.smithi007.stdout:2/687: chown dc/df/d24/d28/d2e/d6e/la8 183192917 1 2024-03-16T09:01:15.840 INFO:tasks.workunit.client.1.smithi019.stdout:7/307: creat d9/da/d13/d18/f5a x:0 0 0 2024-03-16T09:01:15.840 INFO:tasks.workunit.client.1.smithi019.stdout:7/308: fdatasync d9/db/d12/f2f 0 2024-03-16T09:01:15.840 INFO:tasks.workunit.client.1.smithi019.stdout:7/309: fdatasync d9/da/d13/d18/d1a/f1f 0 2024-03-16T09:01:15.840 INFO:tasks.workunit.client.1.smithi019.stdout:7/310: fdatasync f7 0 2024-03-16T09:01:15.840 INFO:tasks.workunit.client.1.smithi019.stdout:7/311: truncate d9/d25/f31 4688931 0 2024-03-16T09:01:15.840 INFO:tasks.workunit.client.1.smithi019.stdout:2/483: dread d5/de/f45 [0,4194304] 0 2024-03-16T09:01:15.840 INFO:tasks.workunit.client.0.smithi007.stdout:7/494: mknod d0/d6c/d25/d73/cac 0 2024-03-16T09:01:15.843 INFO:tasks.workunit.client.1.smithi019.stdout:5/408: dwrite d6/d1f/d33/f4b [0,4194304] 0 2024-03-16T09:01:15.844 INFO:tasks.workunit.client.1.smithi019.stdout:9/274: rmdir d1/d7/d33/d5a 0 2024-03-16T09:01:15.845 INFO:tasks.workunit.client.0.smithi007.stdout:3/653: write d15/d18/d1f/d29/d43/fb6 [1185478,109878] 0 2024-03-16T09:01:15.852 INFO:tasks.workunit.client.1.smithi019.stdout:6/522: dwrite de/d26/f85 [0,4194304] 0 2024-03-16T09:01:15.854 INFO:tasks.workunit.client.1.smithi019.stdout:8/439: dwrite d6/d2a/d45/f26 [0,4194304] 0 2024-03-16T09:01:15.854 INFO:tasks.workunit.client.1.smithi019.stdout:3/467: truncate d3/d1b/d40/f67 1339750 0 2024-03-16T09:01:15.854 INFO:tasks.workunit.client.1.smithi019.stdout:3/468: fdatasync d3/f30 0 2024-03-16T09:01:15.855 INFO:tasks.workunit.client.1.smithi019.stdout:1/357: rmdir d16/d1f/d4f 39 2024-03-16T09:01:15.856 INFO:tasks.workunit.client.1.smithi019.stdout:0/361: fsync d7/d29/f71 0 2024-03-16T09:01:15.856 INFO:tasks.workunit.client.0.smithi007.stdout:2/688: truncate dc/f3a 5011915 0 2024-03-16T09:01:15.859 INFO:tasks.workunit.client.1.smithi019.stdout:9/275: symlink d1/d7/de/d24/d5b/l5e 0 2024-03-16T09:01:15.859 INFO:tasks.workunit.client.1.smithi019.stdout:9/276: creat d1/d7/de/d2d/f5f x:0 0 0 2024-03-16T09:01:15.860 INFO:tasks.workunit.client.0.smithi007.stdout:6/513: rename d2/d11/d3b/d93/c17 to d2/d11/ca9 0 2024-03-16T09:01:15.860 INFO:tasks.workunit.client.0.smithi007.stdout:6/514: write d2/f46 [81279,93683] 0 2024-03-16T09:01:15.865 INFO:tasks.workunit.client.1.smithi019.stdout:6/523: mknod de/d26/d32/d4e/d75/cab 0 2024-03-16T09:01:15.867 INFO:tasks.workunit.client.0.smithi007.stdout:6/515: mknod d2/d13/d4b/d94/caa 0 2024-03-16T09:01:15.867 INFO:tasks.workunit.client.0.smithi007.stdout:6/516: write d2/fd [2195598,64618] 0 2024-03-16T09:01:15.867 INFO:tasks.workunit.client.1.smithi019.stdout:8/440: symlink d6/d2a/d45/d19/d74/l91 0 2024-03-16T09:01:15.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/654: creat d15/d18/d1f/d3b/fca x:0 0 0 2024-03-16T09:01:15.872 INFO:tasks.workunit.client.0.smithi007.stdout:6/517: creat d2/d29/fab x:0 0 0 2024-03-16T09:01:15.873 INFO:tasks.workunit.client.1.smithi019.stdout:0/362: dwrite d7/d8/f50 [0,4194304] 0 2024-03-16T09:01:15.875 INFO:tasks.workunit.client.0.smithi007.stdout:6/518: rename d2/d11/d3b/f9f to d2/d13/d4b/d94/fac 0 2024-03-16T09:01:15.876 INFO:tasks.workunit.client.1.smithi019.stdout:7/312: rename d9/da/d13/d18/d1a/d32 to d9/db/d12/d5b 0 2024-03-16T09:01:15.877 INFO:tasks.workunit.client.1.smithi019.stdout:7/313: dread d9/db/d12/d5b/d33/f43 [0,4194304] 0 2024-03-16T09:01:15.877 INFO:tasks.workunit.client.1.smithi019.stdout:7/314: fdatasync d9/da/d13/f17 0 2024-03-16T09:01:15.877 INFO:tasks.workunit.client.1.smithi019.stdout:7/315: creat d9/da/d13/d18/d30/f5c x:0 0 0 2024-03-16T09:01:15.877 INFO:tasks.workunit.client.1.smithi019.stdout:3/469: link d3/d17/d5a/c5d d3/c8f 0 2024-03-16T09:01:15.878 INFO:tasks.workunit.client.0.smithi007.stdout:6/519: unlink d2/d11/d3b/d93/d8e/l65 0 2024-03-16T09:01:15.878 INFO:tasks.workunit.client.0.smithi007.stdout:7/495: dwrite d0/dc/f15 [0,4194304] 0 2024-03-16T09:01:15.879 INFO:tasks.workunit.client.0.smithi007.stdout:7/496: chown d0/l9b 133653 1 2024-03-16T09:01:15.879 INFO:tasks.workunit.client.1.smithi019.stdout:0/363: mkdir d7/d1d/d63/d72 0 2024-03-16T09:01:15.881 INFO:tasks.workunit.client.0.smithi007.stdout:7/497: link d0/d19/d1f/d42/f62 d0/dc/d5a/fad 0 2024-03-16T09:01:15.881 INFO:tasks.workunit.client.1.smithi019.stdout:9/277: getdents d1/d7/d21/d27 0 2024-03-16T09:01:15.881 INFO:tasks.workunit.client.1.smithi019.stdout:1/358: rename d16/l21 to d16/d31/l64 0 2024-03-16T09:01:15.883 INFO:tasks.workunit.client.1.smithi019.stdout:7/316: mkdir d9/da/d13/d18/d30/d3e/d5d 0 2024-03-16T09:01:15.885 INFO:tasks.workunit.client.1.smithi019.stdout:7/317: rename d9/da to d9/da/d13/d18/d46/d5e 22 2024-03-16T09:01:15.885 INFO:tasks.workunit.client.1.smithi019.stdout:8/441: rename d6/d2a/d45/d57/f62 to d6/d2a/d45/d19/d74/f92 0 2024-03-16T09:01:15.885 INFO:tasks.workunit.client.0.smithi007.stdout:5/591: creat d1/d7/d16/d1b/fd9 x:0 0 0 2024-03-16T09:01:15.886 INFO:tasks.workunit.client.0.smithi007.stdout:9/621: sync 2024-03-16T09:01:15.886 INFO:tasks.workunit.client.0.smithi007.stdout:4/497: sync 2024-03-16T09:01:15.887 INFO:tasks.workunit.client.1.smithi019.stdout:7/318: link d9/f3f d9/da/d13/d18/d1a/f5f 0 2024-03-16T09:01:15.888 INFO:tasks.workunit.client.1.smithi019.stdout:3/470: write d3/d17/d5a/f73 [3033404,72001] 0 2024-03-16T09:01:15.889 INFO:tasks.workunit.client.1.smithi019.stdout:5/409: dwrite d6/dd/f44 [0,4194304] 0 2024-03-16T09:01:15.890 INFO:tasks.workunit.client.1.smithi019.stdout:8/442: truncate d6/de/d24/f35 4806409 0 2024-03-16T09:01:15.890 INFO:tasks.workunit.client.0.smithi007.stdout:9/622: rename fd to d24/d4c/d86/daa/d7f/d80/d87/d89/fb3 0 2024-03-16T09:01:15.895 INFO:tasks.workunit.client.1.smithi019.stdout:3/471: rename d3/d1b/d40/c65 to d3/d17/d6a/c90 0 2024-03-16T09:01:15.896 INFO:tasks.workunit.client.0.smithi007.stdout:1/538: truncate d9/f22 2374540 0 2024-03-16T09:01:15.898 INFO:tasks.workunit.client.0.smithi007.stdout:6/520: write d2/d13/d3f/d50/d72/f69 [21318,16840] 0 2024-03-16T09:01:15.898 INFO:tasks.workunit.client.1.smithi019.stdout:6/524: dwrite de/d26/d32/d97/d66/d72/d7f/f81 [0,4194304] 0 2024-03-16T09:01:15.901 INFO:tasks.workunit.client.1.smithi019.stdout:5/410: mknod d6/d1f/d33/c7e 0 2024-03-16T09:01:15.901 INFO:tasks.workunit.client.1.smithi019.stdout:8/443: unlink d6/d2a/d45/d2c/l3e 0 2024-03-16T09:01:15.905 INFO:tasks.workunit.client.1.smithi019.stdout:8/444: getdents d6/d2a/d45/d57/d72/d79 0 2024-03-16T09:01:15.905 INFO:tasks.workunit.client.0.smithi007.stdout:6/521: symlink d2/d13/d62/lad 0 2024-03-16T09:01:15.905 INFO:tasks.workunit.client.0.smithi007.stdout:2/689: dwrite dc/df/d24/d28/d2e/d6e/f76 [0,4194304] 0 2024-03-16T09:01:15.906 INFO:tasks.workunit.client.1.smithi019.stdout:3/472: mkdir d3/d1b/d88/d91 0 2024-03-16T09:01:15.908 INFO:tasks.workunit.client.1.smithi019.stdout:8/445: creat d6/d2a/d45/d57/d72/d79/d7b/f93 x:0 0 0 2024-03-16T09:01:15.909 INFO:tasks.workunit.client.0.smithi007.stdout:6/522: mknod d2/d13/d3f/d50/d72/cae 0 2024-03-16T09:01:15.909 INFO:tasks.workunit.client.0.smithi007.stdout:6/523: chown d2/d11/d3b/d93/d15/f98 8040 1 2024-03-16T09:01:15.910 INFO:tasks.workunit.client.1.smithi019.stdout:3/473: creat d3/d17/d5a/d75/f92 x:0 0 0 2024-03-16T09:01:15.910 INFO:tasks.workunit.client.1.smithi019.stdout:3/474: write d3/f6c [1695245,47650] 0 2024-03-16T09:01:15.912 INFO:tasks.workunit.client.1.smithi019.stdout:8/446: creat d6/f94 x:0 0 0 2024-03-16T09:01:15.912 INFO:tasks.workunit.client.1.smithi019.stdout:8/447: write d6/d2a/d45/f6b [852798,51940] 0 2024-03-16T09:01:15.912 INFO:tasks.workunit.client.0.smithi007.stdout:1/539: dread d9/dd/d5e/f65 [0,4194304] 0 2024-03-16T09:01:15.914 INFO:tasks.workunit.client.1.smithi019.stdout:3/475: truncate d3/d11/f3b 276953 0 2024-03-16T09:01:15.918 INFO:tasks.workunit.client.0.smithi007.stdout:9/623: read d24/d4a/f78 [1447319,10161] 0 2024-03-16T09:01:15.918 INFO:tasks.workunit.client.0.smithi007.stdout:2/690: creat dc/db7/fde x:0 0 0 2024-03-16T09:01:15.918 INFO:tasks.workunit.client.0.smithi007.stdout:2/691: chown dc/df/d24/d28/lcf 1814586074 1 2024-03-16T09:01:15.918 INFO:tasks.workunit.client.0.smithi007.stdout:9/624: write d24/d4c/d86/daa/d7f/d80/f8c [2100039,86969] 0 2024-03-16T09:01:15.918 INFO:tasks.workunit.client.1.smithi019.stdout:3/476: link d3/f6c d3/d17/d6a/f93 0 2024-03-16T09:01:15.919 INFO:tasks.workunit.client.1.smithi019.stdout:8/448: mkdir d6/de/d24/d30/d4a/d67/d95 0 2024-03-16T09:01:15.920 INFO:tasks.workunit.client.0.smithi007.stdout:2/692: creat dc/df/d24/d28/d2e/d6e/fdf x:0 0 0 2024-03-16T09:01:15.920 INFO:tasks.workunit.client.0.smithi007.stdout:2/693: dread - dc/df/d24/fd9 zero size 2024-03-16T09:01:15.920 INFO:tasks.workunit.client.0.smithi007.stdout:2/694: write dc/df/d24/d28/d4c/d5e/fdc [626177,5275] 0 2024-03-16T09:01:15.921 INFO:tasks.workunit.client.1.smithi019.stdout:8/449: mkdir d6/d2a/d45/d57/d96 0 2024-03-16T09:01:15.922 INFO:tasks.workunit.client.0.smithi007.stdout:2/695: rename dc/df/f18 to dc/df/d6c/fe0 0 2024-03-16T09:01:15.923 INFO:tasks.workunit.client.1.smithi019.stdout:8/450: creat d6/d2a/d45/d19/f97 x:0 0 0 2024-03-16T09:01:15.928 INFO:tasks.workunit.client.0.smithi007.stdout:9/625: write d24/f27 [4000378,25052] 0 2024-03-16T09:01:15.928 INFO:tasks.workunit.client.0.smithi007.stdout:9/626: fsync d24/d4a/d55/d56/f60 0 2024-03-16T09:01:15.932 INFO:tasks.workunit.client.0.smithi007.stdout:9/627: getdents d24/d4a/d55/d56/da6 0 2024-03-16T09:01:15.934 INFO:tasks.workunit.client.0.smithi007.stdout:9/628: unlink c5 0 2024-03-16T09:01:15.935 INFO:tasks.workunit.client.1.smithi019.stdout:0/364: dwrite d7/d8/fc [4194304,4194304] 0 2024-03-16T09:01:15.935 INFO:tasks.workunit.client.1.smithi019.stdout:0/365: readlink d7/d1d/d28/l16 0 2024-03-16T09:01:15.937 INFO:tasks.workunit.client.1.smithi019.stdout:0/366: write d7/d29/f47 [656211,126534] 0 2024-03-16T09:01:15.939 INFO:tasks.workunit.client.1.smithi019.stdout:5/411: write d6/d1f/d33/d66/f48 [2753166,12374] 0 2024-03-16T09:01:15.939 INFO:tasks.workunit.client.1.smithi019.stdout:0/367: rename d7/d1d/d28/l16 to d7/d8/d32/d36/l73 0 2024-03-16T09:01:15.939 INFO:tasks.workunit.client.1.smithi019.stdout:5/412: fsync d6/d1f/d33/f52 0 2024-03-16T09:01:15.939 INFO:tasks.workunit.client.1.smithi019.stdout:5/413: getdents d6/d1f/d78 0 2024-03-16T09:01:15.940 INFO:tasks.workunit.client.0.smithi007.stdout:7/498: dwrite d0/dc/d5a/fad [0,4194304] 0 2024-03-16T09:01:15.941 INFO:tasks.workunit.client.1.smithi019.stdout:0/368: mknod d7/d1d/c74 0 2024-03-16T09:01:15.941 INFO:tasks.workunit.client.0.smithi007.stdout:9/629: dread f1d [0,4194304] 0 2024-03-16T09:01:15.942 INFO:tasks.workunit.client.1.smithi019.stdout:3/477: dread d3/ff [0,4194304] 0 2024-03-16T09:01:15.942 INFO:tasks.workunit.client.1.smithi019.stdout:3/478: fsync d3/d17/f59 0 2024-03-16T09:01:15.944 INFO:tasks.workunit.client.0.smithi007.stdout:7/499: symlink d0/d19/lae 0 2024-03-16T09:01:15.944 INFO:tasks.workunit.client.0.smithi007.stdout:7/500: read - d0/d19/d1f/d5d/d9d/fa4 zero size 2024-03-16T09:01:15.946 INFO:tasks.workunit.client.1.smithi019.stdout:0/369: read d7/d8/d32/d4a/d61/f25 [74322,75128] 0 2024-03-16T09:01:15.946 INFO:tasks.workunit.client.1.smithi019.stdout:0/370: write d7/d29/f3e [2306377,62715] 0 2024-03-16T09:01:15.950 INFO:tasks.workunit.client.1.smithi019.stdout:0/371: rename d7/l60 to d7/d1d/d63/d6e/l75 0 2024-03-16T09:01:15.951 INFO:tasks.workunit.client.0.smithi007.stdout:1/540: dwrite d9/dd/d1b/d58/f7d [0,4194304] 0 2024-03-16T09:01:15.951 INFO:tasks.workunit.client.0.smithi007.stdout:1/541: chown d9/dd/d1b/d2d/d30/f8e 23 1 2024-03-16T09:01:15.953 INFO:tasks.workunit.client.1.smithi019.stdout:0/372: rename d7/d8/d32/d4a/d61/f25 to d7/d1d/f76 0 2024-03-16T09:01:15.953 INFO:tasks.workunit.client.1.smithi019.stdout:0/373: stat d7/d8/d32/d66 0 2024-03-16T09:01:15.953 INFO:tasks.workunit.client.1.smithi019.stdout:0/374: truncate d7/d8/f23 754353 0 2024-03-16T09:01:15.955 INFO:tasks.workunit.client.1.smithi019.stdout:0/375: creat d7/d1d/d28/d6c/f77 x:0 0 0 2024-03-16T09:01:15.961 INFO:tasks.workunit.client.1.smithi019.stdout:0/376: dread d7/d8/f23 [0,4194304] 0 2024-03-16T09:01:15.961 INFO:tasks.workunit.client.1.smithi019.stdout:0/377: chown d7/f19 6 1 2024-03-16T09:01:15.962 INFO:tasks.workunit.client.1.smithi019.stdout:0/378: link d7/d1d/d28/d6c/f77 d7/d1d/d63/f78 0 2024-03-16T09:01:15.963 INFO:tasks.workunit.client.0.smithi007.stdout:5/592: dwrite d1/d7/d1c/d19/d34/d3e/f43 [0,4194304] 0 2024-03-16T09:01:15.963 INFO:tasks.workunit.client.0.smithi007.stdout:4/498: dwrite d4/d7/d6b/f6d [0,4194304] 0 2024-03-16T09:01:15.963 INFO:tasks.workunit.client.0.smithi007.stdout:5/593: dread - d1/faa zero size 2024-03-16T09:01:15.964 INFO:tasks.workunit.client.1.smithi019.stdout:0/379: creat d7/f79 x:0 0 0 2024-03-16T09:01:15.964 INFO:tasks.workunit.client.1.smithi019.stdout:6/525: dwrite de/d26/d32/d97/d66/d72/f82 [0,4194304] 0 2024-03-16T09:01:15.964 INFO:tasks.workunit.client.1.smithi019.stdout:7/319: dwrite d9/da/d13/d18/d1a/f2b [0,4194304] 0 2024-03-16T09:01:15.964 INFO:tasks.workunit.client.0.smithi007.stdout:4/499: symlink d4/d7/d78/d80/d84/d93/la6 0 2024-03-16T09:01:15.965 INFO:tasks.workunit.client.0.smithi007.stdout:4/500: creat d4/d14/d1c/d5b/fa7 x:0 0 0 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.1.smithi019.stdout:6/526: dread de/d1c/f56 [0,4194304] 0 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.1.smithi019.stdout:0/380: mknod d7/d8/d32/d4a/c7a 0 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.1.smithi019.stdout:0/381: readlink d7/d8/d32/d36/l4c 0 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.1.smithi019.stdout:0/382: truncate d7/d29/f47 974988 0 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/594: symlink d1/d7/d1c/d88/da2/d89/d9b/lda 0 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/595: write d1/d7/d1c/d19/d34/d3e/f6a [793524,125054] 0 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/596: creat d1/d7/d1c/d19/d34/fdb x:0 0 0 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/597: dread - d1/d7/d1c/d88/da2/d89/db3/fa0 zero size 2024-03-16T09:01:15.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/598: stat d1/d7/d1c/d19/d34/d4d/d99/fd7 0 2024-03-16T09:01:15.971 INFO:tasks.workunit.client.1.smithi019.stdout:7/320: rename d9/da/d13/d18/d30/f37 to d9/d25/d28/f60 0 2024-03-16T09:01:15.971 INFO:tasks.workunit.client.0.smithi007.stdout:5/599: unlink d1/d7/c76 0 2024-03-16T09:01:15.973 INFO:tasks.workunit.client.1.smithi019.stdout:0/383: creat d7/d1d/d63/f7b x:0 0 0 2024-03-16T09:01:15.974 INFO:tasks.workunit.client.1.smithi019.stdout:0/384: write d7/d1d/f4b [1918340,99603] 0 2024-03-16T09:01:15.975 INFO:tasks.workunit.client.1.smithi019.stdout:8/451: dwrite f4 [4194304,4194304] 0 2024-03-16T09:01:15.975 INFO:tasks.workunit.client.1.smithi019.stdout:8/452: fdatasync d6/f94 0 2024-03-16T09:01:15.978 INFO:tasks.workunit.client.0.smithi007.stdout:5/600: write d1/d7/d1c/d19/d34/d5e/d6f/f6c [1845474,111657] 0 2024-03-16T09:01:15.979 INFO:tasks.workunit.client.0.smithi007.stdout:6/524: dwrite d2/d11/d3b/f76 [0,4194304] 0 2024-03-16T09:01:15.979 INFO:tasks.workunit.client.0.smithi007.stdout:6/525: chown d2/f20 3 1 2024-03-16T09:01:15.979 INFO:tasks.workunit.client.1.smithi019.stdout:0/385: mkdir d7/d8/d32/d66/d6d/d7c 0 2024-03-16T09:01:15.981 INFO:tasks.workunit.client.1.smithi019.stdout:8/453: mkdir d6/de/d24/d98 0 2024-03-16T09:01:15.981 INFO:tasks.workunit.client.1.smithi019.stdout:8/454: chown d6/d2a/f8d 8219925 1 2024-03-16T09:01:15.982 INFO:tasks.workunit.client.0.smithi007.stdout:5/601: symlink d1/d7/d1c/d88/da2/ldc 0 2024-03-16T09:01:15.982 INFO:tasks.workunit.client.0.smithi007.stdout:5/602: truncate d1/d7/d1c/d19/d34/d3e/d41/f71 586225 0 2024-03-16T09:01:15.983 INFO:tasks.workunit.client.1.smithi019.stdout:8/455: read d6/de/d24/d30/d4a/f4e [149528,109453] 0 2024-03-16T09:01:15.984 INFO:tasks.workunit.client.1.smithi019.stdout:8/456: symlink d6/d2a/d45/d57/d96/l99 0 2024-03-16T09:01:15.985 INFO:tasks.workunit.client.1.smithi019.stdout:8/457: mkdir d6/de/d24/d30/d4a/d67/d95/d9a 0 2024-03-16T09:01:15.987 INFO:tasks.workunit.client.1.smithi019.stdout:8/458: creat d6/d2a/d45/d57/d96/f9b x:0 0 0 2024-03-16T09:01:15.987 INFO:tasks.workunit.client.1.smithi019.stdout:8/459: chown d6/d2a/d45/d2c/l89 42 1 2024-03-16T09:01:15.988 INFO:tasks.workunit.client.0.smithi007.stdout:9/630: dwrite d24/d4a/d55/f65 [0,4194304] 0 2024-03-16T09:01:15.988 INFO:tasks.workunit.client.0.smithi007.stdout:9/631: dread - d24/d4c/d86/daa/d95/f6f zero size 2024-03-16T09:01:15.988 INFO:tasks.workunit.client.0.smithi007.stdout:9/632: fdatasync f7 0 2024-03-16T09:01:15.990 INFO:tasks.workunit.client.1.smithi019.stdout:7/321: dwrite d9/da/d13/d18/d1a/f5f [0,4194304] 0 2024-03-16T09:01:15.992 INFO:tasks.workunit.client.0.smithi007.stdout:7/501: dwrite d0/d6c/f64 [0,4194304] 0 2024-03-16T09:01:15.992 INFO:tasks.workunit.client.0.smithi007.stdout:7/502: stat d0/d6c/d25/da1/fab 0 2024-03-16T09:01:15.992 INFO:tasks.workunit.client.0.smithi007.stdout:7/503: chown d0/f55 3 1 2024-03-16T09:01:15.993 INFO:tasks.workunit.client.0.smithi007.stdout:1/542: dwrite d9/dd/d1b/d2d/d30/d45/f76 [0,4194304] 0 2024-03-16T09:01:15.993 INFO:tasks.workunit.client.1.smithi019.stdout:5/414: dwrite d6/dd/f1d [0,4194304] 0 2024-03-16T09:01:15.993 INFO:tasks.workunit.client.1.smithi019.stdout:8/460: getdents d6/de/d24/d30/d4a/d67 0 2024-03-16T09:01:15.993 INFO:tasks.workunit.client.0.smithi007.stdout:6/526: read d2/d11/d3b/d93/d8e/d4e/f5b [70531,23884] 0 2024-03-16T09:01:15.996 INFO:tasks.workunit.client.1.smithi019.stdout:8/461: unlink d6/d2a/d45/d5d/f6d 0 2024-03-16T09:01:15.996 INFO:tasks.workunit.client.1.smithi019.stdout:8/462: creat d6/d2a/d45/d57/d96/f9c x:0 0 0 2024-03-16T09:01:15.996 INFO:tasks.workunit.client.1.smithi019.stdout:8/463: dread - d6/d2a/d45/d57/d96/f9b zero size 2024-03-16T09:01:15.997 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:15 smithi007 ceph-mon[29470]: pgmap v21: 65 pgs: 65 active+clean; 932 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 89 MiB/s rd, 113 MiB/s wr, 373 op/s 2024-03-16T09:01:15.998 INFO:tasks.workunit.client.1.smithi019.stdout:8/464: mkdir d6/d2a/d45/d57/d72/d9d 0 2024-03-16T09:01:15.998 INFO:tasks.workunit.client.1.smithi019.stdout:8/465: creat d6/f9e x:0 0 0 2024-03-16T09:01:16.002 INFO:tasks.workunit.client.0.smithi007.stdout:9/633: dread d24/f46 [0,4194304] 0 2024-03-16T09:01:16.002 INFO:tasks.workunit.client.0.smithi007.stdout:5/603: dwrite d1/faa [0,4194304] 0 2024-03-16T09:01:16.003 INFO:tasks.workunit.client.1.smithi019.stdout:0/386: getdents d7/d1d 0 2024-03-16T09:01:16.004 INFO:tasks.workunit.client.1.smithi019.stdout:5/415: write d6/dd/f1d [2786201,55699] 0 2024-03-16T09:01:16.007 INFO:tasks.workunit.client.1.smithi019.stdout:5/416: creat d6/d19/f7f x:0 0 0 2024-03-16T09:01:16.008 INFO:tasks.workunit.client.1.smithi019.stdout:5/417: readlink d6/dd/d37/l50 0 2024-03-16T09:01:16.008 INFO:tasks.workunit.client.1.smithi019.stdout:5/418: chown d6/f74 1149104 1 2024-03-16T09:01:16.008 INFO:tasks.workunit.client.1.smithi019.stdout:5/419: fsync d6/dd/f16 0 2024-03-16T09:01:16.008 INFO:tasks.workunit.client.1.smithi019.stdout:5/420: write d6/fb [9229624,67805] 0 2024-03-16T09:01:16.009 INFO:tasks.workunit.client.1.smithi019.stdout:5/421: creat d6/dd/d37/d54/f80 x:0 0 0 2024-03-16T09:01:16.009 INFO:tasks.workunit.client.0.smithi007.stdout:9/634: dread d24/d4c/d86/daa/d7f/d80/d87/d89/fb3 [0,4194304] 0 2024-03-16T09:01:16.010 INFO:tasks.workunit.client.1.smithi019.stdout:5/422: symlink d6/d19/d34/l81 0 2024-03-16T09:01:16.015 INFO:tasks.workunit.client.1.smithi019.stdout:8/466: dwrite d6/d2a/d45/d57/d96/f9b [0,4194304] 0 2024-03-16T09:01:16.016 INFO:tasks.workunit.client.1.smithi019.stdout:8/467: mkdir d6/d2a/d45/d57/d72/d9f 0 2024-03-16T09:01:16.016 INFO:tasks.workunit.client.1.smithi019.stdout:8/468: mknod d6/de/ca0 0 2024-03-16T09:01:16.016 INFO:tasks.workunit.client.0.smithi007.stdout:9/635: dread d24/f26 [0,4194304] 0 2024-03-16T09:01:16.037 INFO:tasks.workunit.client.1.smithi019.stdout:5/423: dwrite d6/d1f/d33/d66/f48 [0,4194304] 0 2024-03-16T09:01:16.039 INFO:tasks.workunit.client.1.smithi019.stdout:0/387: dwrite d7/d29/f41 [0,4194304] 0 2024-03-16T09:01:16.040 INFO:tasks.workunit.client.0.smithi007.stdout:9/636: dwrite d24/d4c/d86/daa/d95/f6f [0,4194304] 0 2024-03-16T09:01:16.040 INFO:tasks.workunit.client.1.smithi019.stdout:5/424: link d6/dd/f67 d6/dd/f82 0 2024-03-16T09:01:16.041 INFO:tasks.workunit.client.1.smithi019.stdout:5/425: mknod d6/d19/c83 0 2024-03-16T09:01:16.042 INFO:tasks.workunit.client.1.smithi019.stdout:5/426: write d6/d19/f1b [1637532,55349] 0 2024-03-16T09:01:16.051 INFO:tasks.workunit.client.1.smithi019.stdout:0/388: write d7/fd [2659335,114965] 0 2024-03-16T09:01:16.051 INFO:tasks.workunit.client.1.smithi019.stdout:0/389: fsync d7/d8/d32/d4a/f4d 0 2024-03-16T09:01:16.051 INFO:tasks.workunit.client.1.smithi019.stdout:0/390: chown d7/d8/d32 129112743 1 2024-03-16T09:01:16.052 INFO:tasks.workunit.client.0.smithi007.stdout:2/696: rmdir dc/df/d24/d28 39 2024-03-16T09:01:16.052 INFO:tasks.workunit.client.1.smithi019.stdout:0/391: creat d7/d8/d32/d66/f7d x:0 0 0 2024-03-16T09:01:16.053 INFO:tasks.workunit.client.1.smithi019.stdout:0/392: mknod d7/d8/d32/d66/c7e 0 2024-03-16T09:01:16.056 INFO:tasks.workunit.client.1.smithi019.stdout:5/427: dwrite d6/d1f/f20 [0,4194304] 0 2024-03-16T09:01:16.056 INFO:tasks.workunit.client.1.smithi019.stdout:5/428: stat d6/dd/f44 0 2024-03-16T09:01:16.058 INFO:tasks.workunit.client.1.smithi019.stdout:5/429: rename d6/dd/d37/d54/d7c to d6/d1f/d77/d84 0 2024-03-16T09:01:16.058 INFO:tasks.workunit.client.1.smithi019.stdout:5/430: chown d6/fb 13 1 2024-03-16T09:01:16.058 INFO:tasks.workunit.client.1.smithi019.stdout:5/431: chown d6/d19/f3c 1 1 2024-03-16T09:01:16.058 INFO:tasks.workunit.client.1.smithi019.stdout:5/432: chown d6/d1f/d33/f4f 1303 1 2024-03-16T09:01:16.061 INFO:tasks.workunit.client.0.smithi007.stdout:7/504: mkdir d0/d6c/d25/d3a/daf 0 2024-03-16T09:01:16.061 INFO:tasks.workunit.client.0.smithi007.stdout:7/505: creat d0/d6c/d25/d3a/fb0 x:0 0 0 2024-03-16T09:01:16.062 INFO:tasks.workunit.client.1.smithi019.stdout:0/393: write d7/d1d/d28/f64 [3780324,31371] 0 2024-03-16T09:01:16.064 INFO:tasks.workunit.client.0.smithi007.stdout:1/543: mknod d9/dd/d1b/d58/cae 0 2024-03-16T09:01:16.064 INFO:tasks.workunit.client.1.smithi019.stdout:0/394: creat d7/d1d/d28/d48/f7f x:0 0 0 2024-03-16T09:01:16.064 INFO:tasks.workunit.client.1.smithi019.stdout:0/395: fsync d7/f79 0 2024-03-16T09:01:16.065 INFO:tasks.workunit.client.1.smithi019.stdout:0/396: dread - d7/d8/d32/d66/f7d zero size 2024-03-16T09:01:16.065 INFO:tasks.workunit.client.1.smithi019.stdout:0/397: symlink d7/d29/l80 0 2024-03-16T09:01:16.065 INFO:tasks.workunit.client.1.smithi019.stdout:0/398: creat d7/d29/f81 x:0 0 0 2024-03-16T09:01:16.065 INFO:tasks.workunit.client.1.smithi019.stdout:0/399: write d7/d1d/d63/f7b [43329,109491] 0 2024-03-16T09:01:16.066 INFO:tasks.workunit.client.1.smithi019.stdout:0/400: write d7/d29/f81 [229630,4835] 0 2024-03-16T09:01:16.066 INFO:tasks.workunit.client.0.smithi007.stdout:1/544: dread d9/d6b/d7e/f9a [0,4194304] 0 2024-03-16T09:01:16.068 INFO:tasks.workunit.client.0.smithi007.stdout:5/604: mknod d1/d7/d16/d1b/d3c/cdd 0 2024-03-16T09:01:16.069 INFO:tasks.workunit.client.1.smithi019.stdout:0/401: truncate d7/f2a 1266560 0 2024-03-16T09:01:16.070 INFO:tasks.workunit.client.1.smithi019.stdout:0/402: write d7/d29/f47 [964494,112302] 0 2024-03-16T09:01:16.070 INFO:tasks.workunit.client.1.smithi019.stdout:0/403: readlink l5 0 2024-03-16T09:01:16.071 INFO:tasks.workunit.client.1.smithi019.stdout:0/404: rmdir d7/d8/d32/d4a 39 2024-03-16T09:01:16.072 INFO:tasks.workunit.client.1.smithi019.stdout:0/405: write d7/d1d/f67 [365570,7000] 0 2024-03-16T09:01:16.073 INFO:tasks.workunit.client.0.smithi007.stdout:9/637: creat d24/d4a/fb4 x:0 0 0 2024-03-16T09:01:16.075 INFO:tasks.workunit.client.0.smithi007.stdout:2/697: creat dc/df/d24/d28/d4c/daf/fe1 x:0 0 0 2024-03-16T09:01:16.079 INFO:tasks.workunit.client.1.smithi019.stdout:0/406: rename d7/d29 to d7/d8/d32/d36/d82 0 2024-03-16T09:01:16.080 INFO:tasks.workunit.client.0.smithi007.stdout:4/501: rename d4/d7 to d4/d14/d1c/d5b/d67/d8f/da8 0 2024-03-16T09:01:16.083 INFO:tasks.workunit.client.0.smithi007.stdout:7/506: link d0/dc/c79 d0/d19/d1f/d76/cb1 0 2024-03-16T09:01:16.084 INFO:tasks.workunit.client.0.smithi007.stdout:1/545: symlink d9/dd/laf 0 2024-03-16T09:01:16.084 INFO:tasks.workunit.client.0.smithi007.stdout:1/546: fsync d9/dd/d1b/d2d/f99 0 2024-03-16T09:01:16.084 INFO:tasks.workunit.client.0.smithi007.stdout:1/547: chown d9/dd/d1b/d58/cae 1 1 2024-03-16T09:01:16.085 INFO:tasks.workunit.client.0.smithi007.stdout:5/605: link d1/d7/d1c/d19/d34/d3e/f43 d1/d7/d1c/d88/da2/dae/fde 0 2024-03-16T09:01:16.087 INFO:tasks.workunit.client.0.smithi007.stdout:1/548: chown d9/dd/d1b/d2d/l5a 940 1 2024-03-16T09:01:16.089 INFO:tasks.workunit.client.0.smithi007.stdout:5/606: creat d1/d7/d1c/d19/d34/d3e/d62/fdf x:0 0 0 2024-03-16T09:01:16.089 INFO:tasks.workunit.client.1.smithi019.stdout:5/433: dwrite d6/d1f/f55 [0,4194304] 0 2024-03-16T09:01:16.090 INFO:tasks.workunit.client.0.smithi007.stdout:7/507: write d0/d6c/d25/d73/f91 [528832,81389] 0 2024-03-16T09:01:16.091 INFO:tasks.workunit.client.0.smithi007.stdout:1/549: mkdir d9/d6b/d7e/db0 0 2024-03-16T09:01:16.091 INFO:tasks.workunit.client.1.smithi019.stdout:5/434: creat d6/dd/d37/d76/f85 x:0 0 0 2024-03-16T09:01:16.092 INFO:tasks.workunit.client.1.smithi019.stdout:5/435: creat d6/dd/d47/f86 x:0 0 0 2024-03-16T09:01:16.093 INFO:tasks.workunit.client.0.smithi007.stdout:5/607: symlink d1/le0 0 2024-03-16T09:01:16.095 INFO:tasks.workunit.client.1.smithi019.stdout:5/436: mkdir d6/dd/d47/d56/d87 0 2024-03-16T09:01:16.095 INFO:tasks.workunit.client.1.smithi019.stdout:5/437: creat d6/d19/f88 x:0 0 0 2024-03-16T09:01:16.095 INFO:tasks.workunit.client.1.smithi019.stdout:5/438: fdatasync d6/dd/d47/f64 0 2024-03-16T09:01:16.095 INFO:tasks.workunit.client.1.smithi019.stdout:5/439: write d6/dd/d37/d54/f80 [48462,6462] 0 2024-03-16T09:01:16.095 INFO:tasks.workunit.client.1.smithi019.stdout:5/440: fdatasync d6/dd/f82 0 2024-03-16T09:01:16.096 INFO:tasks.workunit.client.1.smithi019.stdout:0/407: dread d7/d8/f12 [0,4194304] 0 2024-03-16T09:01:16.099 INFO:tasks.workunit.client.1.smithi019.stdout:0/408: symlink d7/d8/d32/d4a/d61/d68/l83 0 2024-03-16T09:01:16.099 INFO:tasks.workunit.client.1.smithi019.stdout:0/409: write d7/d8/d32/d36/f35 [3668099,70370] 0 2024-03-16T09:01:16.100 INFO:tasks.workunit.client.0.smithi007.stdout:1/550: dread d9/dd/d1b/d2d/d30/d45/f5c [0,4194304] 0 2024-03-16T09:01:16.102 INFO:tasks.workunit.client.0.smithi007.stdout:1/551: rename d9/dd/c3b to d9/d6b/d7e/db0/cb1 0 2024-03-16T09:01:16.103 INFO:tasks.workunit.client.1.smithi019.stdout:0/410: link d7/d1d/c74 d7/d8/d32/c84 0 2024-03-16T09:01:16.103 INFO:tasks.workunit.client.1.smithi019.stdout:0/411: dread - d7/f79 zero size 2024-03-16T09:01:16.105 INFO:tasks.workunit.client.0.smithi007.stdout:4/502: write d4/f10 [1012472,70889] 0 2024-03-16T09:01:16.105 INFO:tasks.workunit.client.0.smithi007.stdout:6/527: dwrite d2/d11/d3b/f76 [4194304,4194304] 0 2024-03-16T09:01:16.107 INFO:tasks.workunit.client.0.smithi007.stdout:6/528: stat d2/c9 0 2024-03-16T09:01:16.112 INFO:tasks.workunit.client.0.smithi007.stdout:5/608: dread d1/d7/d1c/d19/d34/d5e/d6f/d72/fab [0,4194304] 0 2024-03-16T09:01:16.115 INFO:tasks.workunit.client.1.smithi019.stdout:0/412: link d7/l24 d7/d8/d32/d36/l85 0 2024-03-16T09:01:16.115 INFO:tasks.workunit.client.1.smithi019.stdout:0/413: write d7/d8/d32/d36/d82/f71 [106162,36728] 0 2024-03-16T09:01:16.116 INFO:tasks.workunit.client.1.smithi019.stdout:0/414: creat d7/d1d/f86 x:0 0 0 2024-03-16T09:01:16.116 INFO:tasks.workunit.client.1.smithi019.stdout:0/415: chown l5 1308414 1 2024-03-16T09:01:16.117 INFO:tasks.workunit.client.1.smithi019.stdout:0/416: mkdir d7/d1d/d28/d6c/d87 0 2024-03-16T09:01:16.119 INFO:tasks.workunit.client.1.smithi019.stdout:0/417: rename d7/d8/fc to d7/d8/f88 0 2024-03-16T09:01:16.119 INFO:tasks.workunit.client.1.smithi019.stdout:0/418: chown d7/d1d/d28/f34 166399 1 2024-03-16T09:01:16.119 INFO:tasks.workunit.client.1.smithi019.stdout:0/419: truncate d7/d1d/d28/f34 3939797 0 2024-03-16T09:01:16.121 INFO:tasks.workunit.client.0.smithi007.stdout:2/698: dwrite dc/df/d24/d28/d2e/dc5/fd5 [0,4194304] 0 2024-03-16T09:01:16.122 INFO:tasks.workunit.client.0.smithi007.stdout:1/552: dwrite d9/dd/f67 [0,4194304] 0 2024-03-16T09:01:16.123 INFO:tasks.workunit.client.0.smithi007.stdout:2/699: creat dc/df/daa/fe2 x:0 0 0 2024-03-16T09:01:16.125 INFO:tasks.workunit.client.0.smithi007.stdout:1/553: getdents d9/dd/d1b/d28 0 2024-03-16T09:01:16.129 INFO:tasks.workunit.client.0.smithi007.stdout:6/529: dread d2/f46 [0,4194304] 0 2024-03-16T09:01:16.134 INFO:tasks.workunit.client.1.smithi019.stdout:5/441: dwrite d6/d1f/f5a [0,4194304] 0 2024-03-16T09:01:16.134 INFO:tasks.workunit.client.1.smithi019.stdout:5/442: fsync d6/d1f/f55 0 2024-03-16T09:01:16.135 INFO:tasks.workunit.client.0.smithi007.stdout:3/655: sync 2024-03-16T09:01:16.135 INFO:tasks.workunit.client.0.smithi007.stdout:0/715: sync 2024-03-16T09:01:16.135 INFO:tasks.workunit.client.0.smithi007.stdout:5/609: write d1/d7/d1c/d19/d34/d1d/f5a [2327387,37924] 0 2024-03-16T09:01:16.136 INFO:tasks.workunit.client.0.smithi007.stdout:8/908: sync 2024-03-16T09:01:16.137 INFO:tasks.workunit.client.0.smithi007.stdout:4/503: dwrite d4/f54 [0,4194304] 0 2024-03-16T09:01:16.137 INFO:tasks.workunit.client.0.smithi007.stdout:7/508: dwrite d0/d19/d1f/f2c [0,4194304] 0 2024-03-16T09:01:16.139 INFO:tasks.workunit.client.0.smithi007.stdout:3/656: unlink d15/d18/l3d 0 2024-03-16T09:01:16.143 INFO:tasks.workunit.client.0.smithi007.stdout:0/716: dread d3/d1f/f71 [0,4194304] 0 2024-03-16T09:01:16.146 INFO:tasks.workunit.client.0.smithi007.stdout:5/610: mknod d1/d7/d1c/d19/d34/de/ce1 0 2024-03-16T09:01:16.146 INFO:tasks.workunit.client.0.smithi007.stdout:5/611: truncate d1/d7/d1c/d19/d34/d5e/d6f/d72/fab 8499852 0 2024-03-16T09:01:16.146 INFO:tasks.workunit.client.0.smithi007.stdout:5/612: readlink d1/d7/d1c/d88/da2/d89/db3/d2a/l4b 0 2024-03-16T09:01:16.146 INFO:tasks.workunit.client.0.smithi007.stdout:5/613: write d1/d7/d1c/d19/d34/d1d/da8/fc7 [4498809,63974] 0 2024-03-16T09:01:16.146 INFO:tasks.workunit.client.0.smithi007.stdout:5/614: write d1/d7/f12 [6657419,100019] 0 2024-03-16T09:01:16.147 INFO:tasks.workunit.client.0.smithi007.stdout:8/909: mknod d0/dc/d50/c13f 0 2024-03-16T09:01:16.147 INFO:tasks.workunit.client.0.smithi007.stdout:8/910: creat d0/d26/d2d/d43/d4c/dba/f140 x:0 0 0 2024-03-16T09:01:16.147 INFO:tasks.workunit.client.1.smithi019.stdout:0/420: read d7/d1d/d28/f34 [857261,54972] 0 2024-03-16T09:01:16.148 INFO:tasks.workunit.client.0.smithi007.stdout:7/509: read d0/d19/d1f/f54 [5559440,30546] 0 2024-03-16T09:01:16.148 INFO:tasks.workunit.client.1.smithi019.stdout:0/421: getdents d7/d8/d32/d4a/d61/d68 0 2024-03-16T09:01:16.149 INFO:tasks.workunit.client.1.smithi019.stdout:0/422: mkdir d7/d1d/d63/d89 0 2024-03-16T09:01:16.150 INFO:tasks.workunit.client.1.smithi019.stdout:0/423: mkdir d7/d1d/d63/d89/d8a 0 2024-03-16T09:01:16.151 INFO:tasks.workunit.client.1.smithi019.stdout:0/424: truncate d7/d1d/f4b 1214112 0 2024-03-16T09:01:16.152 INFO:tasks.workunit.client.1.smithi019.stdout:0/425: chown d7/le 15989 1 2024-03-16T09:01:16.152 INFO:tasks.workunit.client.1.smithi019.stdout:0/426: chown d7/d1d/l33 6105 1 2024-03-16T09:01:16.152 INFO:tasks.workunit.client.0.smithi007.stdout:4/504: creat d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/fa9 x:0 0 0 2024-03-16T09:01:16.152 INFO:tasks.workunit.client.0.smithi007.stdout:4/505: creat d4/d14/d1c/d5b/d67/faa x:0 0 0 2024-03-16T09:01:16.152 INFO:tasks.workunit.client.0.smithi007.stdout:4/506: write d4/d14/f95 [4228796,72515] 0 2024-03-16T09:01:16.153 INFO:tasks.workunit.client.0.smithi007.stdout:3/657: creat d15/d18/d1f/d29/d3e/d85/fcb x:0 0 0 2024-03-16T09:01:16.154 INFO:tasks.workunit.client.1.smithi019.stdout:0/427: unlink d7/d1d/c1e 0 2024-03-16T09:01:16.154 INFO:tasks.workunit.client.0.smithi007.stdout:0/717: getdents d3/d6c/dbd/dd7/d7b 0 2024-03-16T09:01:16.155 INFO:tasks.workunit.client.0.smithi007.stdout:5/615: truncate d1/d7/d1c/d88/da2/f74 2041405 0 2024-03-16T09:01:16.155 INFO:tasks.workunit.client.0.smithi007.stdout:5/616: fdatasync d1/d7/d1c/d88/da2/fcd 0 2024-03-16T09:01:16.155 INFO:tasks.workunit.client.1.smithi019.stdout:0/428: symlink d7/d8/d32/l8b 0 2024-03-16T09:01:16.156 INFO:tasks.workunit.client.0.smithi007.stdout:8/911: rmdir d0/dc/d50/d8e/dd2/dea/d18 39 2024-03-16T09:01:16.157 INFO:tasks.workunit.client.1.smithi019.stdout:0/429: mkdir d7/d8/d32/d66/d6d/d8c 0 2024-03-16T09:01:16.159 INFO:tasks.workunit.client.0.smithi007.stdout:7/510: mknod d0/d19/cb2 0 2024-03-16T09:01:16.159 INFO:tasks.workunit.client.0.smithi007.stdout:7/511: fdatasync d0/d6c/d25/d3a/f44 0 2024-03-16T09:01:16.159 INFO:tasks.workunit.client.1.smithi019.stdout:0/430: symlink d7/d8/d32/d4a/l8d 0 2024-03-16T09:01:16.160 INFO:tasks.workunit.client.1.smithi019.stdout:4/390: sync 2024-03-16T09:01:16.161 INFO:tasks.workunit.client.0.smithi007.stdout:3/658: mkdir d15/d18/d1f/d29/dc3/dcc 0 2024-03-16T09:01:16.161 INFO:tasks.workunit.client.0.smithi007.stdout:3/659: chown d15/c30 4306 1 2024-03-16T09:01:16.161 INFO:tasks.workunit.client.0.smithi007.stdout:3/660: getdents d15/d18/d1f/d29/dc3/dcc 0 2024-03-16T09:01:16.162 INFO:tasks.workunit.client.1.smithi019.stdout:0/431: rename d7/d1d/d28/d48/f7f to d7/d1d/d28/d6c/f8e 0 2024-03-16T09:01:16.162 INFO:tasks.workunit.client.1.smithi019.stdout:0/432: creat d7/d8/d32/d36/f8f x:0 0 0 2024-03-16T09:01:16.163 INFO:tasks.workunit.client.1.smithi019.stdout:0/433: read - d7/d1d/d28/d6c/f77 zero size 2024-03-16T09:01:16.163 INFO:tasks.workunit.client.1.smithi019.stdout:0/434: write d7/d8/d32/d36/f69 [1023640,111400] 0 2024-03-16T09:01:16.163 INFO:tasks.workunit.client.0.smithi007.stdout:0/718: link d3/d9/f93 d3/d9/da/d18/d27/d34/fea 0 2024-03-16T09:01:16.164 INFO:tasks.workunit.client.0.smithi007.stdout:1/554: dwrite d9/dd/d1b/d28/f36 [0,4194304] 0 2024-03-16T09:01:16.164 INFO:tasks.workunit.client.0.smithi007.stdout:8/912: creat d0/d26/d2d/d43/d4c/f141 x:0 0 0 2024-03-16T09:01:16.165 INFO:tasks.workunit.client.0.smithi007.stdout:8/913: readlink d0/dc/d50/d8e/dd2/dea/d7e/lbf 0 2024-03-16T09:01:16.165 INFO:tasks.workunit.client.1.smithi019.stdout:0/435: creat d7/d8/d32/d4a/d61/d68/f90 x:0 0 0 2024-03-16T09:01:16.168 INFO:tasks.workunit.client.0.smithi007.stdout:0/719: link d3/d9/da/d18/lae d3/d9/da/d48/d25/d43/d1a/leb 0 2024-03-16T09:01:16.170 INFO:tasks.workunit.client.1.smithi019.stdout:0/436: unlink d7/fd 0 2024-03-16T09:01:16.171 INFO:tasks.workunit.client.0.smithi007.stdout:1/555: rename d9/d6b/l8f to d9/d6b/lb2 0 2024-03-16T09:01:16.171 INFO:tasks.workunit.client.0.smithi007.stdout:2/700: dwrite dc/df/d24/d28/d2e/f85 [0,4194304] 0 2024-03-16T09:01:16.171 INFO:tasks.workunit.client.0.smithi007.stdout:6/530: dwrite d2/d13/d3f/d50/d72/fa4 [0,4194304] 0 2024-03-16T09:01:16.172 INFO:tasks.workunit.client.0.smithi007.stdout:8/914: mkdir d0/d26/d2d/d43/d6f/da5/d142 0 2024-03-16T09:01:16.172 INFO:tasks.workunit.client.0.smithi007.stdout:8/915: write d0/dc/d50/d8e/dd2/dea/d18/da7/dc5/def/f13e [3977,80993] 0 2024-03-16T09:01:16.175 INFO:tasks.workunit.client.0.smithi007.stdout:0/720: mkdir d3/d6c/dbd/dd7/d7c/dec 0 2024-03-16T09:01:16.176 INFO:tasks.workunit.client.0.smithi007.stdout:4/507: dwrite d4/d14/d64/d9f/f61 [0,4194304] 0 2024-03-16T09:01:16.177 INFO:tasks.workunit.client.0.smithi007.stdout:4/508: stat d4/d14/d1c/d5b/d67/d8f/d90 0 2024-03-16T09:01:16.177 INFO:tasks.workunit.client.0.smithi007.stdout:2/701: rename dc/df/fc6 to dc/df/d24/d28/d2e/d35/da2/fe3 0 2024-03-16T09:01:16.181 INFO:tasks.workunit.client.0.smithi007.stdout:6/531: creat d2/d11/d1e/d54/faf x:0 0 0 2024-03-16T09:01:16.181 INFO:tasks.workunit.client.0.smithi007.stdout:6/532: stat d2/d11/d3b/d74/d8f 0 2024-03-16T09:01:16.181 INFO:tasks.workunit.client.0.smithi007.stdout:6/533: stat d2/d10 0 2024-03-16T09:01:16.182 INFO:tasks.workunit.client.0.smithi007.stdout:8/916: link d0/dc/d50/d8e/dd2/dea/l1c d0/d26/d2d/d43/d4c/deb/l143 0 2024-03-16T09:01:16.182 INFO:tasks.workunit.client.0.smithi007.stdout:8/917: write d0/dc/d75/dc3/f107 [1356313,38517] 0 2024-03-16T09:01:16.183 INFO:tasks.workunit.client.0.smithi007.stdout:8/918: write d0/dc/d50/d8e/db8/fd6 [729469,108600] 0 2024-03-16T09:01:16.183 INFO:tasks.workunit.client.0.smithi007.stdout:8/919: write d0/dc/d75/f11b [2355943,11055] 0 2024-03-16T09:01:16.185 INFO:tasks.workunit.client.0.smithi007.stdout:0/721: mknod d3/d9/da/d92/d9f/de8/ced 0 2024-03-16T09:01:16.186 INFO:tasks.workunit.client.0.smithi007.stdout:3/661: dwrite d15/d18/d1f/d3b/d63/f66 [0,4194304] 0 2024-03-16T09:01:16.186 INFO:tasks.workunit.client.0.smithi007.stdout:3/662: write d15/f16 [4934908,5662] 0 2024-03-16T09:01:16.188 INFO:tasks.workunit.client.0.smithi007.stdout:4/509: mknod d4/d14/d1c/d5b/cab 0 2024-03-16T09:01:16.189 INFO:tasks.workunit.client.0.smithi007.stdout:2/702: mknod dc/df/d24/d28/d2e/d6e/ce4 0 2024-03-16T09:01:16.191 INFO:tasks.workunit.client.1.smithi019.stdout:4/391: dwrite d4/d6/f32 [0,4194304] 0 2024-03-16T09:01:16.191 INFO:tasks.workunit.client.1.smithi019.stdout:0/437: dwrite d7/d1d/f76 [0,4194304] 0 2024-03-16T09:01:16.191 INFO:tasks.workunit.client.1.smithi019.stdout:0/438: creat d7/d8/d32/d66/f91 x:0 0 0 2024-03-16T09:01:16.191 INFO:tasks.workunit.client.1.smithi019.stdout:0/439: write d7/d8/d32/d36/d82/f6f [352063,21640] 0 2024-03-16T09:01:16.196 INFO:tasks.workunit.client.0.smithi007.stdout:6/534: rename d2/d11/d3b/d93/l77 to d2/d13/d4b/d94/lb0 0 2024-03-16T09:01:16.198 INFO:tasks.workunit.client.0.smithi007.stdout:7/512: getdents d0/d19 0 2024-03-16T09:01:16.198 INFO:tasks.workunit.client.0.smithi007.stdout:7/513: fsync d0/d6c/f5e 0 2024-03-16T09:01:16.199 INFO:tasks.workunit.client.0.smithi007.stdout:2/703: write dc/df/d24/f63 [4106165,69389] 0 2024-03-16T09:01:16.203 INFO:tasks.workunit.client.0.smithi007.stdout:8/920: truncate d0/d26/f2f 5657953 0 2024-03-16T09:01:16.203 INFO:tasks.workunit.client.0.smithi007.stdout:8/921: truncate d0/dc/d50/d8e/dd2/dea/d18/da7/fed 475875 0 2024-03-16T09:01:16.203 INFO:tasks.workunit.client.0.smithi007.stdout:4/510: mkdir d4/d14/d64/d68/d82/dac 0 2024-03-16T09:01:16.203 INFO:tasks.workunit.client.0.smithi007.stdout:4/511: write d4/d14/d64/f88 [1127992,4159] 0 2024-03-16T09:01:16.204 INFO:tasks.workunit.client.0.smithi007.stdout:7/514: dread d0/d6c/d25/d73/f91 [0,4194304] 0 2024-03-16T09:01:16.209 INFO:tasks.workunit.client.0.smithi007.stdout:2/704: rename dc/f3f to dc/fe5 0 2024-03-16T09:01:16.210 INFO:tasks.workunit.client.0.smithi007.stdout:0/722: dwrite d3/d9/db4/d60/fe5 [0,4194304] 0 2024-03-16T09:01:16.212 INFO:tasks.workunit.client.0.smithi007.stdout:0/723: read d3/d9/da/d48/d25/d43/d1a/f20 [75288,127798] 0 2024-03-16T09:01:16.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:15 smithi019 ceph-mon[36407]: pgmap v21: 65 pgs: 65 active+clean; 932 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 89 MiB/s rd, 113 MiB/s wr, 373 op/s 2024-03-16T09:01:16.214 INFO:tasks.workunit.client.0.smithi007.stdout:4/512: rename d4/d14/d1c/f26 to d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/d89/fad 0 2024-03-16T09:01:16.217 INFO:tasks.workunit.client.0.smithi007.stdout:7/515: mkdir d0/d6c/d25/d3a/db3 0 2024-03-16T09:01:16.217 INFO:tasks.workunit.client.0.smithi007.stdout:7/516: chown d0/d19/d1f/d69 1 1 2024-03-16T09:01:16.219 INFO:tasks.workunit.client.0.smithi007.stdout:2/705: mknod dc/d27/ce6 0 2024-03-16T09:01:16.224 INFO:tasks.workunit.client.0.smithi007.stdout:0/724: unlink d3/d9/da/d48/d9e/fe6 0 2024-03-16T09:01:16.224 INFO:tasks.workunit.client.0.smithi007.stdout:0/725: chown d3/d9/da/d48/d25/d87/dbb/dde 807 1 2024-03-16T09:01:16.227 INFO:tasks.workunit.client.0.smithi007.stdout:2/706: getdents dc/df/d24/d28/d2e/d35/da2 0 2024-03-16T09:01:16.227 INFO:tasks.workunit.client.1.smithi019.stdout:0/440: dwrite d7/d8/f12 [0,4194304] 0 2024-03-16T09:01:16.229 INFO:tasks.workunit.client.0.smithi007.stdout:2/707: mknod dc/ce7 0 2024-03-16T09:01:16.229 INFO:tasks.workunit.client.0.smithi007.stdout:2/708: fdatasync dc/d27/f7f 0 2024-03-16T09:01:16.229 INFO:tasks.workunit.client.0.smithi007.stdout:2/709: chown dc/df/d24/d28 3890 1 2024-03-16T09:01:16.229 INFO:tasks.workunit.client.0.smithi007.stdout:2/710: dread - dc/df/d24/d28/d2e/d6e/f6f zero size 2024-03-16T09:01:16.229 INFO:tasks.workunit.client.0.smithi007.stdout:2/711: write dc/fab [449648,99705] 0 2024-03-16T09:01:16.250 INFO:tasks.workunit.client.0.smithi007.stdout:3/663: dwrite d15/f16 [0,4194304] 0 2024-03-16T09:01:16.250 INFO:tasks.workunit.client.0.smithi007.stdout:6/535: dwrite d2/d11/d1e/d54/d7a/f97 [0,4194304] 0 2024-03-16T09:01:16.250 INFO:tasks.workunit.client.0.smithi007.stdout:6/536: write d2/d11/d3b/d93/d15/f98 [597723,78776] 0 2024-03-16T09:01:16.251 INFO:tasks.workunit.client.1.smithi019.stdout:0/441: truncate d7/d1d/f76 1907532 0 2024-03-16T09:01:16.252 INFO:tasks.workunit.client.1.smithi019.stdout:0/442: unlink d7/d1d/f86 0 2024-03-16T09:01:16.252 INFO:tasks.workunit.client.1.smithi019.stdout:0/443: truncate d7/f46 1284432 0 2024-03-16T09:01:16.253 INFO:tasks.workunit.client.1.smithi019.stdout:0/444: mkdir d7/d1d/d28/d92 0 2024-03-16T09:01:16.253 INFO:tasks.workunit.client.1.smithi019.stdout:0/445: fsync d7/d1d/d28/f5a 0 2024-03-16T09:01:16.254 INFO:tasks.workunit.client.1.smithi019.stdout:0/446: creat d7/d1d/d63/d6e/f93 x:0 0 0 2024-03-16T09:01:16.254 INFO:tasks.workunit.client.1.smithi019.stdout:0/447: creat d7/d8/d32/d4a/d61/d68/f94 x:0 0 0 2024-03-16T09:01:16.259 INFO:tasks.workunit.client.1.smithi019.stdout:4/392: dwrite d4/d6/d15/f67 [0,4194304] 0 2024-03-16T09:01:16.259 INFO:tasks.workunit.client.1.smithi019.stdout:4/393: truncate d4/d6/d15/d1f/d2c/f6b 479575 0 2024-03-16T09:01:16.259 INFO:tasks.workunit.client.0.smithi007.stdout:2/712: dwrite dc/df/daa/fb3 [0,4194304] 0 2024-03-16T09:01:16.260 INFO:tasks.workunit.client.0.smithi007.stdout:2/713: write dc/df/d24/fd9 [724872,657] 0 2024-03-16T09:01:16.260 INFO:tasks.workunit.client.0.smithi007.stdout:2/714: read dc/df/d24/d28/d2e/d35/f5f [225935,114564] 0 2024-03-16T09:01:16.260 INFO:tasks.workunit.client.0.smithi007.stdout:2/715: read - dc/df/d24/d28/d97/da4/fa6 zero size 2024-03-16T09:01:16.261 INFO:tasks.workunit.client.0.smithi007.stdout:0/726: dwrite d3/d9/db4/fcf [0,4194304] 0 2024-03-16T09:01:16.261 INFO:tasks.workunit.client.0.smithi007.stdout:6/537: dread d2/d11/d1e/d54/f63 [0,4194304] 0 2024-03-16T09:01:16.261 INFO:tasks.workunit.client.0.smithi007.stdout:7/517: dwrite d0/d19/d1f/d42/f7b [0,4194304] 0 2024-03-16T09:01:16.263 INFO:tasks.workunit.client.0.smithi007.stdout:8/922: dwrite d0/dc/d50/d8e/dd2/dea/d18/f4e [4194304,4194304] 0 2024-03-16T09:01:16.264 INFO:tasks.workunit.client.1.smithi019.stdout:0/448: dread d7/d8/f50 [0,4194304] 0 2024-03-16T09:01:16.264 INFO:tasks.workunit.client.1.smithi019.stdout:0/449: creat d7/d1d/d28/d6c/f95 x:0 0 0 2024-03-16T09:01:16.264 INFO:tasks.workunit.client.1.smithi019.stdout:0/450: write d7/d8/d32/d36/d82/f71 [415991,119295] 0 2024-03-16T09:01:16.264 INFO:tasks.workunit.client.1.smithi019.stdout:0/451: write d7/d8/d32/d36/d82/f47 [816289,33061] 0 2024-03-16T09:01:16.264 INFO:tasks.workunit.client.1.smithi019.stdout:4/394: unlink d4/d6/d15/d1f/d2c/f71 0 2024-03-16T09:01:16.265 INFO:tasks.workunit.client.0.smithi007.stdout:3/664: read d15/d18/d6a/d78/fa8 [3330185,25811] 0 2024-03-16T09:01:16.266 INFO:tasks.workunit.client.0.smithi007.stdout:2/716: truncate dc/df/d24/d28/d2e/f5c 1717250 0 2024-03-16T09:01:16.266 INFO:tasks.workunit.client.0.smithi007.stdout:2/717: stat dc/df/daa 0 2024-03-16T09:01:16.266 INFO:tasks.workunit.client.0.smithi007.stdout:2/718: dread - dc/df/d24/d28/d97/da4/fa6 zero size 2024-03-16T09:01:16.266 INFO:tasks.workunit.client.0.smithi007.stdout:2/719: chown dc/df/d24/d28/d2e/d35/da2 75854 1 2024-03-16T09:01:16.266 INFO:tasks.workunit.client.0.smithi007.stdout:2/720: fsync dc/df/d6c/fcd 0 2024-03-16T09:01:16.267 INFO:tasks.workunit.client.0.smithi007.stdout:2/721: dread - dc/df/d24/d28/d4c/daf/fe1 zero size 2024-03-16T09:01:16.267 INFO:tasks.workunit.client.0.smithi007.stdout:2/722: creat dc/db7/fe8 x:0 0 0 2024-03-16T09:01:16.267 INFO:tasks.workunit.client.1.smithi019.stdout:0/452: symlink d7/l96 0 2024-03-16T09:01:16.267 INFO:tasks.workunit.client.1.smithi019.stdout:0/453: dread - d7/d1d/d28/d6c/f77 zero size 2024-03-16T09:01:16.268 INFO:tasks.workunit.client.0.smithi007.stdout:2/723: dread dc/df/d6c/fcd [0,4194304] 0 2024-03-16T09:01:16.269 INFO:tasks.workunit.client.1.smithi019.stdout:0/454: chown l6 55659 1 2024-03-16T09:01:16.274 INFO:tasks.workunit.client.1.smithi019.stdout:2/484: sync 2024-03-16T09:01:16.274 INFO:tasks.workunit.client.1.smithi019.stdout:2/485: truncate d5/d6/d2b/d53/f71 1036077 0 2024-03-16T09:01:16.274 INFO:tasks.workunit.client.1.smithi019.stdout:2/486: fdatasync d5/d2e/f75 0 2024-03-16T09:01:16.274 INFO:tasks.workunit.client.1.smithi019.stdout:9/278: sync 2024-03-16T09:01:16.274 INFO:tasks.workunit.client.1.smithi019.stdout:9/279: dread - d1/d7/f50 zero size 2024-03-16T09:01:16.277 INFO:tasks.workunit.client.0.smithi007.stdout:0/727: rename d3/d9/db4/d2c/d5b/cc6 to d3/d6c/cee 0 2024-03-16T09:01:16.277 INFO:tasks.workunit.client.0.smithi007.stdout:0/728: creat d3/d9/da/d18/d23/d36/fef x:0 0 0 2024-03-16T09:01:16.277 INFO:tasks.workunit.client.1.smithi019.stdout:9/280: getdents d1 0 2024-03-16T09:01:16.280 INFO:tasks.workunit.client.0.smithi007.stdout:8/923: creat d0/d26/d2d/d43/d72/dad/f144 x:0 0 0 2024-03-16T09:01:16.284 INFO:tasks.workunit.client.1.smithi019.stdout:4/395: rmdir d4/d6/d15/d1f/d2c 39 2024-03-16T09:01:16.285 INFO:tasks.workunit.client.0.smithi007.stdout:2/724: mknod dc/df/d24/d28/d97/ce9 0 2024-03-16T09:01:16.287 INFO:tasks.workunit.client.0.smithi007.stdout:6/538: dwrite d2/d13/d24/f6a [4194304,4194304] 0 2024-03-16T09:01:16.289 INFO:tasks.workunit.client.0.smithi007.stdout:7/518: rename d0/d6c/f52 to d0/dc/d38/d4e/fb4 0 2024-03-16T09:01:16.289 INFO:tasks.workunit.client.0.smithi007.stdout:7/519: fdatasync d0/d6c/d25/d3a/f4f 0 2024-03-16T09:01:16.289 INFO:tasks.workunit.client.0.smithi007.stdout:7/520: truncate d0/d19/d1f/f82 565826 0 2024-03-16T09:01:16.289 INFO:tasks.workunit.client.1.smithi019.stdout:0/455: dwrite d7/d8/d32/d66/f7d [0,4194304] 0 2024-03-16T09:01:16.290 INFO:tasks.workunit.client.0.smithi007.stdout:0/729: symlink d3/d1f/dca/lf0 0 2024-03-16T09:01:16.291 INFO:tasks.workunit.client.0.smithi007.stdout:0/730: chown d3/d9/da/d18/d23/lb5 722939 1 2024-03-16T09:01:16.293 INFO:tasks.workunit.client.0.smithi007.stdout:8/924: unlink d0/d26/d2d/d43/d72/dad/ffa 0 2024-03-16T09:01:16.293 INFO:tasks.workunit.client.1.smithi019.stdout:0/456: creat d7/f97 x:0 0 0 2024-03-16T09:01:16.294 INFO:tasks.workunit.client.0.smithi007.stdout:3/665: truncate d15/d18/d1f/d29/f1b 981783 0 2024-03-16T09:01:16.294 INFO:tasks.workunit.client.1.smithi019.stdout:4/396: dread d4/d6/f39 [0,4194304] 0 2024-03-16T09:01:16.294 INFO:tasks.workunit.client.1.smithi019.stdout:2/487: dwrite d5/de/f32 [0,4194304] 0 2024-03-16T09:01:16.295 INFO:tasks.workunit.client.0.smithi007.stdout:2/725: mkdir dc/df/d24/d28/dea 0 2024-03-16T09:01:16.295 INFO:tasks.workunit.client.0.smithi007.stdout:2/726: fdatasync dc/df/d24/d28/d2e/d35/f98 0 2024-03-16T09:01:16.296 INFO:tasks.workunit.client.1.smithi019.stdout:0/457: link d7/d1d/d2e/c37 d7/d1d/d28/c98 0 2024-03-16T09:01:16.298 INFO:tasks.workunit.client.1.smithi019.stdout:4/397: rename d4/d1b/d49/l4b to d4/d64/l84 0 2024-03-16T09:01:16.298 INFO:tasks.workunit.client.1.smithi019.stdout:9/281: dwrite d1/d7/d21/d27/f14 [0,4194304] 0 2024-03-16T09:01:16.299 INFO:tasks.workunit.client.0.smithi007.stdout:0/731: symlink d3/d74/lf1 0 2024-03-16T09:01:16.300 INFO:tasks.workunit.client.1.smithi019.stdout:2/488: mkdir d5/de/da2 0 2024-03-16T09:01:16.300 INFO:tasks.workunit.client.1.smithi019.stdout:2/489: creat d5/d4e/d74/fa3 x:0 0 0 2024-03-16T09:01:16.300 INFO:tasks.workunit.client.1.smithi019.stdout:2/490: read - d5/de/f55 zero size 2024-03-16T09:01:16.300 INFO:tasks.workunit.client.1.smithi019.stdout:4/398: mknod d4/d1b/d49/c85 0 2024-03-16T09:01:16.301 INFO:tasks.workunit.client.0.smithi007.stdout:8/925: symlink d0/dc/d50/d8e/dd2/dea/d18/da7/de2/d113/l145 0 2024-03-16T09:01:16.301 INFO:tasks.workunit.client.0.smithi007.stdout:3/666: write d15/d18/d1f/d56/f88 [663343,26157] 0 2024-03-16T09:01:16.303 INFO:tasks.workunit.client.0.smithi007.stdout:2/727: creat dc/df/d24/d28/d97/d99/feb x:0 0 0 2024-03-16T09:01:16.304 INFO:tasks.workunit.client.0.smithi007.stdout:6/539: getdents d2/d13/d4b 0 2024-03-16T09:01:16.305 INFO:tasks.workunit.client.0.smithi007.stdout:7/521: rmdir d0/d19/d1f/d5d/d8a/d93 39 2024-03-16T09:01:16.306 INFO:tasks.workunit.client.0.smithi007.stdout:3/667: dread d15/d18/d1f/d29/d3e/d85/fac [0,4194304] 0 2024-03-16T09:01:16.306 INFO:tasks.workunit.client.0.smithi007.stdout:3/668: creat d15/d18/d1f/d29/d3e/d85/fcd x:0 0 0 2024-03-16T09:01:16.307 INFO:tasks.workunit.client.1.smithi019.stdout:0/458: read d7/d8/f12 [3539608,58955] 0 2024-03-16T09:01:16.312 INFO:tasks.workunit.client.1.smithi019.stdout:9/282: mkdir d1/d7/de/d24/d44/d60 0 2024-03-16T09:01:16.314 INFO:tasks.workunit.client.1.smithi019.stdout:2/491: creat d5/d6/d2b/d2f/fa4 x:0 0 0 2024-03-16T09:01:16.316 INFO:tasks.workunit.client.0.smithi007.stdout:7/522: dread d0/d19/d1f/d42/f7b [0,4194304] 0 2024-03-16T09:01:16.317 INFO:tasks.workunit.client.1.smithi019.stdout:1/359: sync 2024-03-16T09:01:16.317 INFO:tasks.workunit.client.1.smithi019.stdout:1/360: fsync d16/d18/f39 0 2024-03-16T09:01:16.317 INFO:tasks.workunit.client.1.smithi019.stdout:4/399: rename d4/d6/l4c to d4/d1b/d2a/d3d/l86 0 2024-03-16T09:01:16.317 INFO:tasks.workunit.client.1.smithi019.stdout:4/400: write d4/d6/f5f [350979,108591] 0 2024-03-16T09:01:16.318 INFO:tasks.workunit.client.0.smithi007.stdout:0/732: creat d3/d9/db4/d1c/ff2 x:0 0 0 2024-03-16T09:01:16.319 INFO:tasks.workunit.client.1.smithi019.stdout:0/459: unlink d7/d8/d32/d36/d82/f3e 0 2024-03-16T09:01:16.319 INFO:tasks.workunit.client.1.smithi019.stdout:0/460: stat d7/d1d/d28/l1c 0 2024-03-16T09:01:16.319 INFO:tasks.workunit.client.0.smithi007.stdout:6/540: dwrite d2/d11/d3b/f87 [0,4194304] 0 2024-03-16T09:01:16.320 INFO:tasks.workunit.client.0.smithi007.stdout:8/926: unlink d0/d26/d2d/d43/d4c/dba/f140 0 2024-03-16T09:01:16.321 INFO:tasks.workunit.client.0.smithi007.stdout:2/728: write dc/df/d24/d28/d2e/f78 [1027849,2347] 0 2024-03-16T09:01:16.321 INFO:tasks.workunit.client.0.smithi007.stdout:2/729: dread - dc/df/d24/d28/d2e/d6e/fdf zero size 2024-03-16T09:01:16.322 INFO:tasks.workunit.client.1.smithi019.stdout:9/283: mknod d1/d7/c61 0 2024-03-16T09:01:16.322 INFO:tasks.workunit.client.1.smithi019.stdout:9/284: truncate d1/d7/f3e 939531 0 2024-03-16T09:01:16.322 INFO:tasks.workunit.client.1.smithi019.stdout:9/285: dread - d1/d7/d21/d27/f4e zero size 2024-03-16T09:01:16.322 INFO:tasks.workunit.client.1.smithi019.stdout:2/492: mknod d5/d2e/ca5 0 2024-03-16T09:01:16.323 INFO:tasks.workunit.client.1.smithi019.stdout:2/493: truncate d5/d6/d2b/d2f/d35/f39 823044 0 2024-03-16T09:01:16.324 INFO:tasks.workunit.client.1.smithi019.stdout:0/461: rename d7/d8/f12 to d7/d1d/d28/d6c/f99 0 2024-03-16T09:01:16.324 INFO:tasks.workunit.client.1.smithi019.stdout:0/462: fsync d7/f2a 0 2024-03-16T09:01:16.325 INFO:tasks.workunit.client.0.smithi007.stdout:7/523: creat d0/d6c/fb5 x:0 0 0 2024-03-16T09:01:16.325 INFO:tasks.workunit.client.1.smithi019.stdout:1/361: dread d16/f37 [0,4194304] 0 2024-03-16T09:01:16.326 INFO:tasks.workunit.client.1.smithi019.stdout:1/362: write f9 [5205350,91097] 0 2024-03-16T09:01:16.326 INFO:tasks.workunit.client.1.smithi019.stdout:1/363: creat d16/f65 x:0 0 0 2024-03-16T09:01:16.326 INFO:tasks.workunit.client.1.smithi019.stdout:9/286: symlink d1/d7/de/d24/d5b/l62 0 2024-03-16T09:01:16.326 INFO:tasks.workunit.client.0.smithi007.stdout:0/733: creat d3/d6c/dbd/dd7/d7b/ff3 x:0 0 0 2024-03-16T09:01:16.326 INFO:tasks.workunit.client.0.smithi007.stdout:0/734: truncate d3/d9/da/d18/d23/d36/f86 4701655 0 2024-03-16T09:01:16.327 INFO:tasks.workunit.client.0.smithi007.stdout:6/541: read d2/d11/d3b/f87 [2047351,70808] 0 2024-03-16T09:01:16.327 INFO:tasks.workunit.client.0.smithi007.stdout:6/542: truncate d2/d11/d3b/d74/f80 676256 0 2024-03-16T09:01:16.327 INFO:tasks.workunit.client.1.smithi019.stdout:2/494: rename d5/d6/d2b/d89/d7d to d5/d6/d64/d6f/da6 0 2024-03-16T09:01:16.329 INFO:tasks.workunit.client.1.smithi019.stdout:0/463: unlink d7/d8/d32/d66/f7d 0 2024-03-16T09:01:16.329 INFO:tasks.workunit.client.1.smithi019.stdout:2/495: write d5/d6/d2b/d2f/d35/d4b/f5c [524731,115990] 0 2024-03-16T09:01:16.329 INFO:tasks.workunit.client.0.smithi007.stdout:8/927: rename d0/dc/dd/f13a to d0/dc/d50/d8e/dd2/dea/d18/da7/dc5/d130/f146 0 2024-03-16T09:01:16.330 INFO:tasks.workunit.client.0.smithi007.stdout:8/928: chown d0/dc/d50/d8e/dd2/dea/le7 29386195 1 2024-03-16T09:01:16.330 INFO:tasks.workunit.client.0.smithi007.stdout:8/929: write d0/dc/d50/d8e/dd2/dea/f122 [2210112,73545] 0 2024-03-16T09:01:16.330 INFO:tasks.workunit.client.0.smithi007.stdout:8/930: read - d0/dc/d80/f136 zero size 2024-03-16T09:01:16.331 INFO:tasks.workunit.client.0.smithi007.stdout:2/730: unlink dc/d27/f7f 0 2024-03-16T09:01:16.331 INFO:tasks.workunit.client.0.smithi007.stdout:7/524: write d0/dc/d5a/fad [3754826,61153] 0 2024-03-16T09:01:16.331 INFO:tasks.workunit.client.1.smithi019.stdout:1/364: mknod d16/d18/c66 0 2024-03-16T09:01:16.332 INFO:tasks.workunit.client.1.smithi019.stdout:9/287: mknod d1/c63 0 2024-03-16T09:01:16.332 INFO:tasks.workunit.client.0.smithi007.stdout:3/669: mkdir d15/d18/d1f/dce 0 2024-03-16T09:01:16.334 INFO:tasks.workunit.client.1.smithi019.stdout:1/365: rename d16/f49 to d16/d1f/d3c/f67 0 2024-03-16T09:01:16.335 INFO:tasks.workunit.client.0.smithi007.stdout:0/735: mknod d3/d6c/dbd/dd7/cf4 0 2024-03-16T09:01:16.335 INFO:tasks.workunit.client.0.smithi007.stdout:0/736: write d3/d1f/fb7 [1577410,125557] 0 2024-03-16T09:01:16.337 INFO:tasks.workunit.client.1.smithi019.stdout:1/366: getdents d16/d1f 0 2024-03-16T09:01:16.338 INFO:tasks.workunit.client.0.smithi007.stdout:6/543: unlink d2/c75 0 2024-03-16T09:01:16.339 INFO:tasks.workunit.client.1.smithi019.stdout:1/367: mkdir d16/d1f/d4f/d68 0 2024-03-16T09:01:16.339 INFO:tasks.workunit.client.1.smithi019.stdout:2/496: dread d5/d6/d2b/f6d [0,4194304] 0 2024-03-16T09:01:16.342 INFO:tasks.workunit.client.0.smithi007.stdout:8/931: rename d0/dc/c4d to d0/dc/d75/dc3/c147 0 2024-03-16T09:01:16.342 INFO:tasks.workunit.client.0.smithi007.stdout:2/731: rmdir dc/df/d24/d28/d2e/d35 39 2024-03-16T09:01:16.342 INFO:tasks.workunit.client.0.smithi007.stdout:2/732: creat dc/df/d6c/fec x:0 0 0 2024-03-16T09:01:16.342 INFO:tasks.workunit.client.0.smithi007.stdout:2/733: fdatasync dc/f10 0 2024-03-16T09:01:16.344 INFO:tasks.workunit.client.0.smithi007.stdout:3/670: symlink d15/d18/d6a/d78/lcf 0 2024-03-16T09:01:16.348 INFO:tasks.workunit.client.0.smithi007.stdout:6/544: creat d2/d11/d3b/d74/d8f/fb1 x:0 0 0 2024-03-16T09:01:16.351 INFO:tasks.workunit.client.1.smithi019.stdout:1/368: dread d16/d18/f1a [0,4194304] 0 2024-03-16T09:01:16.352 INFO:tasks.workunit.client.0.smithi007.stdout:6/545: rename d2/d11/d3b/la8 to d2/d13/d3f/d50/d78/d81/lb2 0 2024-03-16T09:01:16.352 INFO:tasks.workunit.client.0.smithi007.stdout:6/546: read d2/f46 [410982,20012] 0 2024-03-16T09:01:16.353 INFO:tasks.workunit.client.1.smithi019.stdout:1/369: unlink d16/d1f/f30 0 2024-03-16T09:01:16.354 INFO:tasks.workunit.client.0.smithi007.stdout:6/547: symlink d2/d13/d3f/d50/d78/d81/lb3 0 2024-03-16T09:01:16.354 INFO:tasks.workunit.client.0.smithi007.stdout:6/548: write d2/d11/d3b/d93/d8e/d49/d84/f9e [1405889,120955] 0 2024-03-16T09:01:16.354 INFO:tasks.workunit.client.0.smithi007.stdout:8/932: write d0/dc/d50/d8e/dd2/dea/d18/f3f [2722773,124120] 0 2024-03-16T09:01:16.354 INFO:tasks.workunit.client.0.smithi007.stdout:8/933: stat d0/dc/f89 0 2024-03-16T09:01:16.355 INFO:tasks.workunit.client.0.smithi007.stdout:8/934: chown d0/d26/d2d/d43/d4c/deb/f10c 51 1 2024-03-16T09:01:16.355 INFO:tasks.workunit.client.0.smithi007.stdout:8/935: fdatasync d0/dc/d50/d8e/dd2/dea/d18/f8d 0 2024-03-16T09:01:16.355 INFO:tasks.workunit.client.0.smithi007.stdout:8/936: chown d0/dbd 913805624 1 2024-03-16T09:01:16.356 INFO:tasks.workunit.client.1.smithi019.stdout:2/497: read d5/de/f5d [418437,18345] 0 2024-03-16T09:01:16.356 INFO:tasks.workunit.client.1.smithi019.stdout:2/498: chown d5/ff 217930681 1 2024-03-16T09:01:16.356 INFO:tasks.workunit.client.1.smithi019.stdout:2/499: fsync d5/d6/d2b/d2f/d35/f56 0 2024-03-16T09:01:16.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/464: dwrite d7/d8/d32/d36/f8f [0,4194304] 0 2024-03-16T09:01:16.356 INFO:tasks.workunit.client.1.smithi019.stdout:1/370: creat d16/d31/f69 x:0 0 0 2024-03-16T09:01:16.356 INFO:tasks.workunit.client.1.smithi019.stdout:2/500: mknod d5/d6/d2b/d2f/d35/d4b/ca7 0 2024-03-16T09:01:16.356 INFO:tasks.workunit.client.1.smithi019.stdout:2/501: fsync d5/d6/d2b/d2f/d35/d4b/f93 0 2024-03-16T09:01:16.356 INFO:tasks.workunit.client.1.smithi019.stdout:2/502: dread - d5/d2e/f9a zero size 2024-03-16T09:01:16.357 INFO:tasks.workunit.client.0.smithi007.stdout:8/937: symlink d0/dc/dd/d8b/d9e/l148 0 2024-03-16T09:01:16.357 INFO:tasks.workunit.client.0.smithi007.stdout:8/938: write d0/dc/d75/fb1 [792128,79245] 0 2024-03-16T09:01:16.357 INFO:tasks.workunit.client.0.smithi007.stdout:8/939: mkdir d0/d26/d95/d149 0 2024-03-16T09:01:16.358 INFO:tasks.workunit.client.1.smithi019.stdout:0/465: symlink d7/d8/d32/d36/l9a 0 2024-03-16T09:01:16.359 INFO:tasks.workunit.client.1.smithi019.stdout:1/371: link d16/d18/c36 d16/d1f/d4f/d61/c6a 0 2024-03-16T09:01:16.359 INFO:tasks.workunit.client.1.smithi019.stdout:1/372: chown d16/d1f/d4f/f62 0 1 2024-03-16T09:01:16.361 INFO:tasks.workunit.client.1.smithi019.stdout:2/503: rename d5/d6/d64/d6f/d87/l9f to d5/d6/d2b/la8 0 2024-03-16T09:01:16.361 INFO:tasks.workunit.client.1.smithi019.stdout:0/466: mknod d7/d8/d32/d4a/d61/c9b 0 2024-03-16T09:01:16.361 INFO:tasks.workunit.client.1.smithi019.stdout:0/467: getdents d7/d1d/d63/d89/d8a 0 2024-03-16T09:01:16.362 INFO:tasks.workunit.client.0.smithi007.stdout:7/525: dwrite d0/f3c [0,4194304] 0 2024-03-16T09:01:16.362 INFO:tasks.workunit.client.0.smithi007.stdout:7/526: fsync d0/d6c/d25/da1/fab 0 2024-03-16T09:01:16.362 INFO:tasks.workunit.client.0.smithi007.stdout:7/527: chown d0/d19/d1f/d5d/d8a 20772 1 2024-03-16T09:01:16.363 INFO:tasks.workunit.client.0.smithi007.stdout:0/737: dwrite d3/d9/da/f4c [4194304,4194304] 0 2024-03-16T09:01:16.363 INFO:tasks.workunit.client.1.smithi019.stdout:9/288: dwrite d1/d7/de/d24/d44/f53 [0,4194304] 0 2024-03-16T09:01:16.363 INFO:tasks.workunit.client.1.smithi019.stdout:9/289: read - d1/d7/d21/d27/f4e zero size 2024-03-16T09:01:16.363 INFO:tasks.workunit.client.1.smithi019.stdout:9/290: fdatasync d1/d7/d21/d27/f22 0 2024-03-16T09:01:16.363 INFO:tasks.workunit.client.1.smithi019.stdout:1/373: creat d16/d1f/d4f/d61/f6b x:0 0 0 2024-03-16T09:01:16.364 INFO:tasks.workunit.client.0.smithi007.stdout:0/738: write d3/d9/da/d48/fa9 [513937,4136] 0 2024-03-16T09:01:16.364 INFO:tasks.workunit.client.0.smithi007.stdout:0/739: chown d3/d9/db4/fcf 3483 1 2024-03-16T09:01:16.369 INFO:tasks.workunit.client.0.smithi007.stdout:0/740: rmdir d3/d9/da/d48 39 2024-03-16T09:01:16.369 INFO:tasks.workunit.client.0.smithi007.stdout:0/741: write d3/d9/db4/d2c/fc1 [224308,92282] 0 2024-03-16T09:01:16.370 INFO:tasks.workunit.client.0.smithi007.stdout:6/549: dread d2/d11/d3b/d93/d8e/d4e/f5b [0,4194304] 0 2024-03-16T09:01:16.370 INFO:tasks.workunit.client.0.smithi007.stdout:6/550: creat d2/d13/d3f/d50/d72/fb4 x:0 0 0 2024-03-16T09:01:16.370 INFO:tasks.workunit.client.0.smithi007.stdout:6/551: chown d2/d13/d24/f6a 6397 1 2024-03-16T09:01:16.372 INFO:tasks.workunit.client.1.smithi019.stdout:0/468: dread d7/d1d/d28/f64 [0,4194304] 0 2024-03-16T09:01:16.372 INFO:tasks.workunit.client.1.smithi019.stdout:1/374: creat d16/d18/f6c x:0 0 0 2024-03-16T09:01:16.373 INFO:tasks.workunit.client.1.smithi019.stdout:0/469: symlink d7/d8/d32/d66/d6d/d8c/l9c 0 2024-03-16T09:01:16.373 INFO:tasks.workunit.client.1.smithi019.stdout:2/504: write d5/de/f45 [3157590,5775] 0 2024-03-16T09:01:16.374 INFO:tasks.workunit.client.0.smithi007.stdout:7/528: dread d0/d6c/d25/d3a/f77 [0,4194304] 0 2024-03-16T09:01:16.375 INFO:tasks.workunit.client.1.smithi019.stdout:1/375: getdents d16/d1f/d43/d55 0 2024-03-16T09:01:16.375 INFO:tasks.workunit.client.1.smithi019.stdout:1/376: chown d16/f22 13765 1 2024-03-16T09:01:16.375 INFO:tasks.workunit.client.1.smithi019.stdout:9/291: dread d1/d7/d21/f38 [0,4194304] 0 2024-03-16T09:01:16.376 INFO:tasks.workunit.client.0.smithi007.stdout:0/742: dread d3/d9/da/d48/d25/f68 [0,4194304] 0 2024-03-16T09:01:16.376 INFO:tasks.workunit.client.0.smithi007.stdout:0/743: fdatasync d3/d9/da/d18/d23/f32 0 2024-03-16T09:01:16.376 INFO:tasks.workunit.client.0.smithi007.stdout:7/529: mknod d0/d19/d1f/d42/cb6 0 2024-03-16T09:01:16.381 INFO:tasks.workunit.client.0.smithi007.stdout:7/530: unlink d0/dc/f15 0 2024-03-16T09:01:16.381 INFO:tasks.workunit.client.0.smithi007.stdout:6/552: dread d2/d11/f25 [4194304,4194304] 0 2024-03-16T09:01:16.383 INFO:tasks.workunit.client.0.smithi007.stdout:7/531: mkdir d0/db7 0 2024-03-16T09:01:16.383 INFO:tasks.workunit.client.0.smithi007.stdout:7/532: fsync d0/dc/d5a/fad 0 2024-03-16T09:01:16.384 INFO:tasks.workunit.client.0.smithi007.stdout:6/553: mknod d2/d11/d3b/d93/d8e/d4e/cb5 0 2024-03-16T09:01:16.385 INFO:tasks.workunit.client.0.smithi007.stdout:7/533: symlink d0/dc/d38/d4e/lb8 0 2024-03-16T09:01:16.387 INFO:tasks.workunit.client.0.smithi007.stdout:6/554: rename d2/d11/d3b/f76 to d2/d11/d3b/d93/d8e/fb6 0 2024-03-16T09:01:16.388 INFO:tasks.workunit.client.1.smithi019.stdout:1/377: dread d16/f2f [0,4194304] 0 2024-03-16T09:01:16.389 INFO:tasks.workunit.client.1.smithi019.stdout:9/292: dread d1/d7/d21/d27/f3c [0,4194304] 0 2024-03-16T09:01:16.389 INFO:tasks.workunit.client.0.smithi007.stdout:7/534: creat d0/d19/d1f/d5d/d8a/fb9 x:0 0 0 2024-03-16T09:01:16.390 INFO:tasks.workunit.client.0.smithi007.stdout:7/535: mkdir d0/dc/d5a/dba 0 2024-03-16T09:01:16.391 INFO:tasks.workunit.client.1.smithi019.stdout:1/378: rename d16/d1f/f51 to d16/d1f/d43/f6d 0 2024-03-16T09:01:16.391 INFO:tasks.workunit.client.1.smithi019.stdout:1/379: dread - d16/d1f/d4f/f60 zero size 2024-03-16T09:01:16.392 INFO:tasks.workunit.client.0.smithi007.stdout:7/536: unlink d0/d19/d1f/d5d/d8a/d93/f9e 0 2024-03-16T09:01:16.393 INFO:tasks.workunit.client.1.smithi019.stdout:9/293: creat d1/d7/de/d24/d44/d60/f64 x:0 0 0 2024-03-16T09:01:16.393 INFO:tasks.workunit.client.1.smithi019.stdout:9/294: readlink d1/d7/l3d 0 2024-03-16T09:01:16.394 INFO:tasks.workunit.client.1.smithi019.stdout:9/295: rmdir d1/d7/d33 39 2024-03-16T09:01:16.394 INFO:tasks.workunit.client.0.smithi007.stdout:7/537: symlink d0/d19/d1f/d5d/lbb 0 2024-03-16T09:01:16.395 INFO:tasks.workunit.client.1.smithi019.stdout:9/296: symlink d1/d7/d21/l65 0 2024-03-16T09:01:16.397 INFO:tasks.workunit.client.1.smithi019.stdout:9/297: symlink d1/d7/l66 0 2024-03-16T09:01:16.397 INFO:tasks.workunit.client.1.smithi019.stdout:9/298: chown d1/d7/de/d24/f40 15 1 2024-03-16T09:01:16.399 INFO:tasks.workunit.client.0.smithi007.stdout:8/940: dwrite d0/d26/d2d/d43/d6f/f7d [0,4194304] 0 2024-03-16T09:01:16.403 INFO:tasks.workunit.client.1.smithi019.stdout:2/505: dwrite d5/de/fd [0,4194304] 0 2024-03-16T09:01:16.403 INFO:tasks.workunit.client.1.smithi019.stdout:2/506: fdatasync d5/d2e/f3b 0 2024-03-16T09:01:16.405 INFO:tasks.workunit.client.1.smithi019.stdout:1/380: dread d16/f2b [0,4194304] 0 2024-03-16T09:01:16.405 INFO:tasks.workunit.client.1.smithi019.stdout:0/470: dwrite d7/f2a [0,4194304] 0 2024-03-16T09:01:16.405 INFO:tasks.workunit.client.1.smithi019.stdout:1/381: write d16/d1f/d43/f4c [983255,42924] 0 2024-03-16T09:01:16.408 INFO:tasks.workunit.client.1.smithi019.stdout:0/471: mknod d7/d1d/d28/d92/c9d 0 2024-03-16T09:01:16.408 INFO:tasks.workunit.client.1.smithi019.stdout:0/472: write d7/d8/d32/d66/f91 [773289,88778] 0 2024-03-16T09:01:16.408 INFO:tasks.workunit.client.1.smithi019.stdout:0/473: write d7/d1d/d63/f7b [1144614,87394] 0 2024-03-16T09:01:16.409 INFO:tasks.workunit.client.0.smithi007.stdout:7/538: dread d0/d19/f2b [0,4194304] 0 2024-03-16T09:01:16.411 INFO:tasks.workunit.client.1.smithi019.stdout:1/382: rename d16/c38 to d16/c6e 0 2024-03-16T09:01:16.413 INFO:tasks.workunit.client.1.smithi019.stdout:9/299: dwrite d1/d7/d21/d27/f57 [0,4194304] 0 2024-03-16T09:01:16.413 INFO:tasks.workunit.client.0.smithi007.stdout:6/555: dwrite d2/d11/f47 [0,4194304] 0 2024-03-16T09:01:16.415 INFO:tasks.workunit.client.1.smithi019.stdout:0/474: mkdir d7/d1d/d63/d89/d9e 0 2024-03-16T09:01:16.415 INFO:tasks.workunit.client.1.smithi019.stdout:0/475: chown d7/d1d 35974362 1 2024-03-16T09:01:16.416 INFO:tasks.workunit.client.1.smithi019.stdout:1/383: stat d16/d31/l3a 0 2024-03-16T09:01:16.416 INFO:tasks.workunit.client.0.smithi007.stdout:7/539: symlink d0/d6c/d25/d7a/d9a/lbc 0 2024-03-16T09:01:16.418 INFO:tasks.workunit.client.1.smithi019.stdout:9/300: mkdir d1/d7/d67 0 2024-03-16T09:01:16.419 INFO:tasks.workunit.client.1.smithi019.stdout:0/476: creat d7/d1d/d28/d6c/d87/f9f x:0 0 0 2024-03-16T09:01:16.419 INFO:tasks.workunit.client.0.smithi007.stdout:7/540: mknod d0/d19/d1f/d69/cbd 0 2024-03-16T09:01:16.420 INFO:tasks.workunit.client.1.smithi019.stdout:9/301: link d1/d7/d21/d27/f13 d1/d7/f68 0 2024-03-16T09:01:16.420 INFO:tasks.workunit.client.1.smithi019.stdout:9/302: fsync d1/d7/de/d24/f47 0 2024-03-16T09:01:16.421 INFO:tasks.workunit.client.0.smithi007.stdout:6/556: getdents d2/d11/d3b/d93/d8e 0 2024-03-16T09:01:16.421 INFO:tasks.workunit.client.1.smithi019.stdout:0/477: mknod d7/d8/d32/d66/d6d/d7c/ca0 0 2024-03-16T09:01:16.422 INFO:tasks.workunit.client.0.smithi007.stdout:7/541: creat d0/d19/d1f/d5d/d9d/fbe x:0 0 0 2024-03-16T09:01:16.423 INFO:tasks.workunit.client.0.smithi007.stdout:7/542: chown d0/d6c/d25/d3a/db3 561 1 2024-03-16T09:01:16.423 INFO:tasks.workunit.client.1.smithi019.stdout:9/303: symlink d1/d7/de/l69 0 2024-03-16T09:01:16.423 INFO:tasks.workunit.client.1.smithi019.stdout:9/304: write d1/d7/de/d24/f2c [1347844,94345] 0 2024-03-16T09:01:16.423 INFO:tasks.workunit.client.1.smithi019.stdout:9/305: write d1/f3 [996047,116636] 0 2024-03-16T09:01:16.423 INFO:tasks.workunit.client.1.smithi019.stdout:9/306: readlink d1/d7/l46 0 2024-03-16T09:01:16.423 INFO:tasks.workunit.client.1.smithi019.stdout:9/307: fsync d1/d7/de/d24/f39 0 2024-03-16T09:01:16.424 INFO:tasks.workunit.client.1.smithi019.stdout:0/478: mkdir d7/d1d/d28/da1 0 2024-03-16T09:01:16.424 INFO:tasks.workunit.client.1.smithi019.stdout:0/479: stat d7/d1d/d28/f64 0 2024-03-16T09:01:16.424 INFO:tasks.workunit.client.1.smithi019.stdout:1/384: write d16/d18/f39 [6693872,116507] 0 2024-03-16T09:01:16.424 INFO:tasks.workunit.client.1.smithi019.stdout:1/385: readlink d16/d1f/d4f/l63 0 2024-03-16T09:01:16.425 INFO:tasks.workunit.client.1.smithi019.stdout:0/480: rename d7/d1d/d28/l3d to d7/d1d/d63/la2 0 2024-03-16T09:01:16.425 INFO:tasks.workunit.client.0.smithi007.stdout:7/543: mknod d0/d19/d1f/d69/cbf 0 2024-03-16T09:01:16.426 INFO:tasks.workunit.client.1.smithi019.stdout:1/386: creat d16/d1f/d43/f6f x:0 0 0 2024-03-16T09:01:16.426 INFO:tasks.workunit.client.1.smithi019.stdout:1/387: truncate d16/f65 887193 0 2024-03-16T09:01:16.426 INFO:tasks.workunit.client.1.smithi019.stdout:1/388: write d16/d1f/d4f/f60 [304514,61399] 0 2024-03-16T09:01:16.426 INFO:tasks.workunit.client.1.smithi019.stdout:1/389: creat d16/d1f/d43/f70 x:0 0 0 2024-03-16T09:01:16.427 INFO:tasks.workunit.client.1.smithi019.stdout:0/481: mkdir d7/d1d/d28/d6c/d87/da3 0 2024-03-16T09:01:16.427 INFO:tasks.workunit.client.1.smithi019.stdout:0/482: write d7/d8/d32/d36/f35 [625102,32557] 0 2024-03-16T09:01:16.427 INFO:tasks.workunit.client.1.smithi019.stdout:0/483: write d7/d1d/d28/d6c/f8e [904916,105893] 0 2024-03-16T09:01:16.428 INFO:tasks.workunit.client.1.smithi019.stdout:1/390: mkdir d16/d31/d71 0 2024-03-16T09:01:16.428 INFO:tasks.workunit.client.1.smithi019.stdout:1/391: stat d16/d1f/d43/c4d 0 2024-03-16T09:01:16.428 INFO:tasks.workunit.client.0.smithi007.stdout:6/557: rmdir d2/d13/d3f/d50 39 2024-03-16T09:01:16.429 INFO:tasks.workunit.client.1.smithi019.stdout:0/484: mknod d7/d1d/d28/d6c/d87/da3/ca4 0 2024-03-16T09:01:16.429 INFO:tasks.workunit.client.1.smithi019.stdout:0/485: chown d7/d8/d32/d4a/d61/f27 1 1 2024-03-16T09:01:16.429 INFO:tasks.workunit.client.1.smithi019.stdout:0/486: dread - d7/d8/d32/d4a/d61/d68/f94 zero size 2024-03-16T09:01:16.429 INFO:tasks.workunit.client.1.smithi019.stdout:2/507: dwrite d5/d2e/f3b [0,4194304] 0 2024-03-16T09:01:16.430 INFO:tasks.workunit.client.0.smithi007.stdout:8/941: dwrite d0/dc/d50/d8e/dd2/dea/f104 [0,4194304] 0 2024-03-16T09:01:16.430 INFO:tasks.workunit.client.0.smithi007.stdout:8/942: fsync d0/dc/d50/d8e/dd2/dea/d18/f4e 0 2024-03-16T09:01:16.430 INFO:tasks.workunit.client.0.smithi007.stdout:8/943: read - d0/d26/d2d/d43/d4c/f141 zero size 2024-03-16T09:01:16.430 INFO:tasks.workunit.client.0.smithi007.stdout:8/944: read d0/dc/d75/fb1 [835952,62562] 0 2024-03-16T09:01:16.431 INFO:tasks.workunit.client.0.smithi007.stdout:6/558: symlink d2/d11/d3b/d93/d8e/lb7 0 2024-03-16T09:01:16.431 INFO:tasks.workunit.client.0.smithi007.stdout:6/559: write d2/d11/d3b/d74/d8f/fb1 [963714,15513] 0 2024-03-16T09:01:16.431 INFO:tasks.workunit.client.1.smithi019.stdout:1/392: link d16/f42 d16/d31/f72 0 2024-03-16T09:01:16.431 INFO:tasks.workunit.client.1.smithi019.stdout:0/487: creat d7/d8/d32/fa5 x:0 0 0 2024-03-16T09:01:16.435 INFO:tasks.workunit.client.1.smithi019.stdout:2/508: mknod d5/d4e/d78/ca9 0 2024-03-16T09:01:16.435 INFO:tasks.workunit.client.1.smithi019.stdout:9/308: dread d1/d7/d21/d27/f14 [0,4194304] 0 2024-03-16T09:01:16.436 INFO:tasks.workunit.client.0.smithi007.stdout:8/945: getdents d0/d26/dfe 0 2024-03-16T09:01:16.436 INFO:tasks.workunit.client.1.smithi019.stdout:0/488: mkdir d7/d1d/d28/d92/da6 0 2024-03-16T09:01:16.437 INFO:tasks.workunit.client.1.smithi019.stdout:0/489: truncate d7/d8/d32/d36/f8f 3987309 0 2024-03-16T09:01:16.438 INFO:tasks.workunit.client.1.smithi019.stdout:0/490: truncate d7/d8/d32/d4a/d61/d68/f90 633250 0 2024-03-16T09:01:16.439 INFO:tasks.workunit.client.1.smithi019.stdout:0/491: getdents d7/d8/d32/d36/d82 0 2024-03-16T09:01:16.439 INFO:tasks.workunit.client.0.smithi007.stdout:6/560: rmdir d2/d13/d4b 39 2024-03-16T09:01:16.440 INFO:tasks.workunit.client.1.smithi019.stdout:9/309: dread d1/d7/f68 [0,4194304] 0 2024-03-16T09:01:16.442 INFO:tasks.workunit.client.0.smithi007.stdout:6/561: mknod d2/d11/d1e/cb8 0 2024-03-16T09:01:16.442 INFO:tasks.workunit.client.0.smithi007.stdout:6/562: write d2/f20 [5022260,8609] 0 2024-03-16T09:01:16.442 INFO:tasks.workunit.client.1.smithi019.stdout:9/310: unlink d1/d7/d21/d27/f13 0 2024-03-16T09:01:16.443 INFO:tasks.workunit.client.0.smithi007.stdout:6/563: truncate d2/d11/d3b/d93/f60 12873 0 2024-03-16T09:01:16.443 INFO:tasks.workunit.client.0.smithi007.stdout:6/564: dread - d2/d82/fa0 zero size 2024-03-16T09:01:16.443 INFO:tasks.workunit.client.0.smithi007.stdout:6/565: fsync d2/d10/d3d/f6c 0 2024-03-16T09:01:16.444 INFO:tasks.workunit.client.1.smithi019.stdout:9/311: creat d1/f6a x:0 0 0 2024-03-16T09:01:16.445 INFO:tasks.workunit.client.1.smithi019.stdout:9/312: mkdir d1/d7/de/d24/d44/d6b 0 2024-03-16T09:01:16.445 INFO:tasks.workunit.client.0.smithi007.stdout:6/566: getdents d2/d11/d1e/d54/d7a/d9a 0 2024-03-16T09:01:16.445 INFO:tasks.workunit.client.1.smithi019.stdout:2/509: dread d5/d6/d2b/f6d [0,4194304] 0 2024-03-16T09:01:16.446 INFO:tasks.workunit.client.1.smithi019.stdout:9/313: creat d1/d7/de/d24/d5b/f6c x:0 0 0 2024-03-16T09:01:16.447 INFO:tasks.workunit.client.1.smithi019.stdout:2/510: creat d5/de/da2/faa x:0 0 0 2024-03-16T09:01:16.447 INFO:tasks.workunit.client.1.smithi019.stdout:2/511: write d5/d6/d2b/d2f/f3a [1104578,109333] 0 2024-03-16T09:01:16.451 INFO:tasks.workunit.client.1.smithi019.stdout:9/314: dread d1/d7/de/f10 [0,4194304] 0 2024-03-16T09:01:16.451 INFO:tasks.workunit.client.1.smithi019.stdout:9/315: creat d1/d7/de/d24/f6d x:0 0 0 2024-03-16T09:01:16.452 INFO:tasks.workunit.client.1.smithi019.stdout:9/316: creat d1/d7/de/d24/d5b/f6e x:0 0 0 2024-03-16T09:01:16.452 INFO:tasks.workunit.client.1.smithi019.stdout:9/317: write d1/d7/de/f3a [1391237,104310] 0 2024-03-16T09:01:16.452 INFO:tasks.workunit.client.0.smithi007.stdout:8/946: dread d0/d26/d2d/f32 [0,4194304] 0 2024-03-16T09:01:16.453 INFO:tasks.workunit.client.1.smithi019.stdout:5/443: sync 2024-03-16T09:01:16.453 INFO:tasks.workunit.client.1.smithi019.stdout:3/479: sync 2024-03-16T09:01:16.453 INFO:tasks.workunit.client.1.smithi019.stdout:8/469: sync 2024-03-16T09:01:16.453 INFO:tasks.workunit.client.1.smithi019.stdout:6/527: sync 2024-03-16T09:01:16.453 INFO:tasks.workunit.client.1.smithi019.stdout:3/480: dread - d3/d17/d5a/f8c zero size 2024-03-16T09:01:16.453 INFO:tasks.workunit.client.1.smithi019.stdout:5/444: write d6/d19/f3c [75833,24127] 0 2024-03-16T09:01:16.453 INFO:tasks.workunit.client.1.smithi019.stdout:3/481: write d3/d17/d5a/f62 [705451,26320] 0 2024-03-16T09:01:16.453 INFO:tasks.workunit.client.1.smithi019.stdout:9/318: mknod d1/d7/d67/c6f 0 2024-03-16T09:01:16.454 INFO:tasks.workunit.client.1.smithi019.stdout:7/322: sync 2024-03-16T09:01:16.454 INFO:tasks.workunit.client.1.smithi019.stdout:8/470: symlink d6/de/d24/d30/d4a/la1 0 2024-03-16T09:01:16.455 INFO:tasks.workunit.client.1.smithi019.stdout:6/528: creat de/d26/d32/d4e/d57/d9f/fac x:0 0 0 2024-03-16T09:01:16.455 INFO:tasks.workunit.client.1.smithi019.stdout:6/529: read - de/d26/d32/d97/d51/d70/f91 zero size 2024-03-16T09:01:16.456 INFO:tasks.workunit.client.1.smithi019.stdout:3/482: symlink d3/d1b/d40/d4a/l94 0 2024-03-16T09:01:16.456 INFO:tasks.workunit.client.1.smithi019.stdout:3/483: fdatasync d3/d1b/d40/d4a/f41 0 2024-03-16T09:01:16.457 INFO:tasks.workunit.client.1.smithi019.stdout:9/319: mknod d1/d7/de/d24/d44/d60/c70 0 2024-03-16T09:01:16.457 INFO:tasks.workunit.client.1.smithi019.stdout:9/320: write d1/d7/de/d24/d5b/f6c [46504,13020] 0 2024-03-16T09:01:16.458 INFO:tasks.workunit.client.1.smithi019.stdout:1/393: dwrite d16/d1f/d43/f6d [0,4194304] 0 2024-03-16T09:01:16.458 INFO:tasks.workunit.client.1.smithi019.stdout:1/394: fsync d16/d1f/d4f/f57 0 2024-03-16T09:01:16.460 INFO:tasks.workunit.client.1.smithi019.stdout:6/530: link de/l31 de/d26/d32/d97/d66/lad 0 2024-03-16T09:01:16.460 INFO:tasks.workunit.client.1.smithi019.stdout:3/484: mknod d3/d17/d3f/c95 0 2024-03-16T09:01:16.460 INFO:tasks.workunit.client.1.smithi019.stdout:3/485: write d3/d17/d5a/d75/d7a/f87 [1043222,25694] 0 2024-03-16T09:01:16.460 INFO:tasks.workunit.client.1.smithi019.stdout:3/486: truncate d3/d17/d5a/f8c 939389 0 2024-03-16T09:01:16.461 INFO:tasks.workunit.client.1.smithi019.stdout:7/323: getdents d9/da/d13/d18 0 2024-03-16T09:01:16.462 INFO:tasks.workunit.client.0.smithi007.stdout:7/544: dwrite d0/d6c/fb5 [0,4194304] 0 2024-03-16T09:01:16.466 INFO:tasks.workunit.client.1.smithi019.stdout:8/471: rmdir d6/d2a/d45/d57/d72/d79/d7b 39 2024-03-16T09:01:16.466 INFO:tasks.workunit.client.1.smithi019.stdout:8/472: fdatasync d6/d2a/d45/d19/f97 0 2024-03-16T09:01:16.468 INFO:tasks.workunit.client.1.smithi019.stdout:9/321: dread d1/d7/d21/d27/f22 [4194304,4194304] 0 2024-03-16T09:01:16.468 INFO:tasks.workunit.client.1.smithi019.stdout:9/322: chown d1/d7/de/d24/d44/d60 382223 1 2024-03-16T09:01:16.469 INFO:tasks.workunit.client.1.smithi019.stdout:2/512: write d5/de/fd [2881343,25964] 0 2024-03-16T09:01:16.471 INFO:tasks.workunit.client.1.smithi019.stdout:6/531: mkdir de/d26/d32/d97/dae 0 2024-03-16T09:01:16.471 INFO:tasks.workunit.client.1.smithi019.stdout:3/487: symlink d3/d17/l96 0 2024-03-16T09:01:16.471 INFO:tasks.workunit.client.1.smithi019.stdout:3/488: readlink d3/d1b/d40/d4a/l60 0 2024-03-16T09:01:16.472 INFO:tasks.workunit.client.0.smithi007.stdout:6/567: truncate d2/d11/d3b/d93/f60 101238 0 2024-03-16T09:01:16.473 INFO:tasks.workunit.client.0.smithi007.stdout:7/545: getdents d0/d19/d1f/d5d/d8a/d93 0 2024-03-16T09:01:16.473 INFO:tasks.workunit.client.0.smithi007.stdout:7/546: chown d0/d6c/d25/d73/f5c 426450 1 2024-03-16T09:01:16.473 INFO:tasks.workunit.client.1.smithi019.stdout:7/324: link d9/da/l52 d9/da/d13/d18/d1a/l61 0 2024-03-16T09:01:16.475 INFO:tasks.workunit.client.0.smithi007.stdout:6/568: getdents d2/d11/d3b/d93/d15 0 2024-03-16T09:01:16.475 INFO:tasks.workunit.client.0.smithi007.stdout:6/569: readlink d2/d13/d4b/d94/lb0 0 2024-03-16T09:01:16.475 INFO:tasks.workunit.client.1.smithi019.stdout:8/473: mknod d6/d2a/d45/d57/d96/ca2 0 2024-03-16T09:01:16.476 INFO:tasks.workunit.client.1.smithi019.stdout:1/395: write f10 [3777989,67087] 0 2024-03-16T09:01:16.476 INFO:tasks.workunit.client.1.smithi019.stdout:1/396: chown d16/d1f/d4f/d61/f6b 9319639 1 2024-03-16T09:01:16.476 INFO:tasks.workunit.client.1.smithi019.stdout:1/397: chown c7 2897 1 2024-03-16T09:01:16.476 INFO:tasks.workunit.client.0.smithi007.stdout:7/547: symlink d0/d19/d1f/d5d/d8a/lc0 0 2024-03-16T09:01:16.477 INFO:tasks.workunit.client.1.smithi019.stdout:9/323: mknod d1/d7/de/d24/d5b/c71 0 2024-03-16T09:01:16.477 INFO:tasks.workunit.client.1.smithi019.stdout:9/324: creat d1/d7/de/d24/d44/d60/f72 x:0 0 0 2024-03-16T09:01:16.477 INFO:tasks.workunit.client.1.smithi019.stdout:9/325: truncate d1/d7/f45 469035 0 2024-03-16T09:01:16.477 INFO:tasks.workunit.client.1.smithi019.stdout:9/326: truncate d1/d7/de/f1a 1661712 0 2024-03-16T09:01:16.477 INFO:tasks.workunit.client.1.smithi019.stdout:9/327: fdatasync d1/d7/f34 0 2024-03-16T09:01:16.478 INFO:tasks.workunit.client.0.smithi007.stdout:6/570: link d2/d11/d3b/d93/l23 d2/d13/d3f/d50/d78/d81/lb9 0 2024-03-16T09:01:16.479 INFO:tasks.workunit.client.1.smithi019.stdout:6/532: truncate de/f15 3230328 0 2024-03-16T09:01:16.479 INFO:tasks.workunit.client.1.smithi019.stdout:6/533: fdatasync de/d26/d32/d4e/d57/d9f/fac 0 2024-03-16T09:01:16.480 INFO:tasks.workunit.client.1.smithi019.stdout:1/398: write d16/f2f [1515245,67787] 0 2024-03-16T09:01:16.480 INFO:tasks.workunit.client.1.smithi019.stdout:1/399: dread - d16/d1f/f34 zero size 2024-03-16T09:01:16.485 INFO:tasks.workunit.client.1.smithi019.stdout:3/489: rename d3/d17/d3f/c95 to d3/d8b/c97 0 2024-03-16T09:01:16.487 INFO:tasks.workunit.client.1.smithi019.stdout:3/490: read d3/d17/d6a/f93 [1658252,83532] 0 2024-03-16T09:01:16.487 INFO:tasks.workunit.client.0.smithi007.stdout:6/571: dread d2/d11/d3b/f87 [0,4194304] 0 2024-03-16T09:01:16.487 INFO:tasks.workunit.client.0.smithi007.stdout:6/572: write d2/d13/d3f/d50/d78/d81/f8c [795659,60003] 0 2024-03-16T09:01:16.488 INFO:tasks.workunit.client.1.smithi019.stdout:9/328: creat d1/d7/d33/d59/f73 x:0 0 0 2024-03-16T09:01:16.488 INFO:tasks.workunit.client.1.smithi019.stdout:9/329: chown d1/d7/f34 6774550 1 2024-03-16T09:01:16.488 INFO:tasks.workunit.client.1.smithi019.stdout:9/330: chown d1/d7/l46 430795103 1 2024-03-16T09:01:16.490 INFO:tasks.workunit.client.1.smithi019.stdout:7/325: rename d9/db/f3d to d9/da/d13/d18/d30/d3e/f62 0 2024-03-16T09:01:16.493 INFO:tasks.workunit.client.1.smithi019.stdout:3/491: rename d3/d17/d5a/f62 to d3/d1b/d40/d4a/f98 0 2024-03-16T09:01:16.493 INFO:tasks.workunit.client.1.smithi019.stdout:3/492: write d3/d17/d3f/f77 [493378,68947] 0 2024-03-16T09:01:16.494 INFO:tasks.workunit.client.1.smithi019.stdout:7/326: unlink d9/da/d13/l14 0 2024-03-16T09:01:16.494 INFO:tasks.workunit.client.1.smithi019.stdout:8/474: dread d6/de/d24/f35 [0,4194304] 0 2024-03-16T09:01:16.495 INFO:tasks.workunit.client.1.smithi019.stdout:8/475: fsync f2 0 2024-03-16T09:01:16.495 INFO:tasks.workunit.client.1.smithi019.stdout:3/493: symlink d3/d17/d6a/l99 0 2024-03-16T09:01:16.495 INFO:tasks.workunit.client.1.smithi019.stdout:3/494: creat d3/f9a x:0 0 0 2024-03-16T09:01:16.496 INFO:tasks.workunit.client.1.smithi019.stdout:7/327: mknod d9/da/d13/d18/c63 0 2024-03-16T09:01:16.497 INFO:tasks.workunit.client.1.smithi019.stdout:9/331: write d1/d7/de/d24/f47 [2376282,36163] 0 2024-03-16T09:01:16.502 INFO:tasks.workunit.client.1.smithi019.stdout:3/495: write d3/d1b/d40/d4a/f41 [2302033,27314] 0 2024-03-16T09:01:16.504 INFO:tasks.workunit.client.1.smithi019.stdout:8/476: dread f4 [0,4194304] 0 2024-03-16T09:01:16.504 INFO:tasks.workunit.client.1.smithi019.stdout:3/496: mkdir d3/d9b 0 2024-03-16T09:01:16.505 INFO:tasks.workunit.client.1.smithi019.stdout:8/477: symlink d6/de/d24/d30/d4a/d67/la3 0 2024-03-16T09:01:16.505 INFO:tasks.workunit.client.0.smithi007.stdout:6/573: dread d2/d13/d3f/d50/d72/fa4 [0,4194304] 0 2024-03-16T09:01:16.509 INFO:tasks.workunit.client.1.smithi019.stdout:7/328: write d9/da/d13/f17 [1943644,20335] 0 2024-03-16T09:01:16.512 INFO:tasks.workunit.client.1.smithi019.stdout:7/329: mknod d9/da/c64 0 2024-03-16T09:01:16.515 INFO:tasks.workunit.client.1.smithi019.stdout:7/330: dread d9/da/d13/d18/d30/f4a [0,4194304] 0 2024-03-16T09:01:16.515 INFO:tasks.workunit.client.1.smithi019.stdout:7/331: readlink d9/d25/l4d 0 2024-03-16T09:01:16.516 INFO:tasks.workunit.client.1.smithi019.stdout:0/492: dwrite d7/d1d/d28/d6c/f8e [0,4194304] 0 2024-03-16T09:01:16.516 INFO:tasks.workunit.client.1.smithi019.stdout:5/445: dwrite d6/f74 [0,4194304] 0 2024-03-16T09:01:16.516 INFO:tasks.workunit.client.1.smithi019.stdout:5/446: dread - d6/dd/d47/f86 zero size 2024-03-16T09:01:16.516 INFO:tasks.workunit.client.1.smithi019.stdout:7/332: creat d9/db/d12/d5b/f65 x:0 0 0 2024-03-16T09:01:16.517 INFO:tasks.workunit.client.1.smithi019.stdout:0/493: read d7/d8/d32/d4a/d61/d68/f90 [52479,34976] 0 2024-03-16T09:01:16.518 INFO:tasks.workunit.client.1.smithi019.stdout:3/497: dread d3/d17/f72 [0,4194304] 0 2024-03-16T09:01:16.518 INFO:tasks.workunit.client.1.smithi019.stdout:3/498: chown d3/d11/f3b 49239452 1 2024-03-16T09:01:16.524 INFO:tasks.workunit.client.1.smithi019.stdout:5/447: mkdir d6/d1f/d33/d89 0 2024-03-16T09:01:16.524 INFO:tasks.workunit.client.0.smithi007.stdout:7/548: dwrite d0/d6c/f64 [0,4194304] 0 2024-03-16T09:01:16.525 INFO:tasks.workunit.client.0.smithi007.stdout:8/947: dwrite d0/dc/dd/f2c [0,4194304] 0 2024-03-16T09:01:16.525 INFO:tasks.workunit.client.1.smithi019.stdout:2/513: dwrite d5/d6/d2b/f3c [4194304,4194304] 0 2024-03-16T09:01:16.526 INFO:tasks.workunit.client.0.smithi007.stdout:7/549: read d0/d6c/d25/d7a/f71 [278468,117610] 0 2024-03-16T09:01:16.527 INFO:tasks.workunit.client.1.smithi019.stdout:7/333: rename d9/da/d13/d18/d1a/l21 to d9/da/d13/d18/l66 0 2024-03-16T09:01:16.527 INFO:tasks.workunit.client.1.smithi019.stdout:7/334: truncate d9/db/d12/d5b/d33/f58 334002 0 2024-03-16T09:01:16.528 INFO:tasks.workunit.client.0.smithi007.stdout:8/948: mkdir d0/dc/dd/d8b/d9e/d14a 0 2024-03-16T09:01:16.528 INFO:tasks.workunit.client.1.smithi019.stdout:9/332: dwrite d1/d7/de/d24/d44/d60/f64 [0,4194304] 0 2024-03-16T09:01:16.528 INFO:tasks.workunit.client.1.smithi019.stdout:9/333: write d1/d7/de/d2d/f5f [625400,62488] 0 2024-03-16T09:01:16.529 INFO:tasks.workunit.client.1.smithi019.stdout:0/494: mkdir d7/d1d/d28/da1/da7 0 2024-03-16T09:01:16.530 INFO:tasks.workunit.client.0.smithi007.stdout:7/550: creat d0/d19/fc1 x:0 0 0 2024-03-16T09:01:16.531 INFO:tasks.workunit.client.0.smithi007.stdout:8/949: mkdir d0/dc/d50/d8e/dd2/dea/d14b 0 2024-03-16T09:01:16.531 INFO:tasks.workunit.client.0.smithi007.stdout:8/950: chown d0/d26/d2d/le8 33356 1 2024-03-16T09:01:16.532 INFO:tasks.workunit.client.0.smithi007.stdout:8/951: chown d0/d26/d2d/d64 252818638 1 2024-03-16T09:01:16.533 INFO:tasks.workunit.client.0.smithi007.stdout:8/952: symlink d0/dc/d50/d8e/dd2/dea/d10a/l14c 0 2024-03-16T09:01:16.535 INFO:tasks.workunit.client.0.smithi007.stdout:8/953: symlink d0/d26/d95/l14d 0 2024-03-16T09:01:16.535 INFO:tasks.workunit.client.1.smithi019.stdout:5/448: symlink d6/dd/d47/d56/l8a 0 2024-03-16T09:01:16.540 INFO:tasks.workunit.client.1.smithi019.stdout:8/478: dwrite d6/de/f6a [0,4194304] 0 2024-03-16T09:01:16.540 INFO:tasks.workunit.client.0.smithi007.stdout:8/954: dread d0/d26/d2d/d43/ff6 [0,4194304] 0 2024-03-16T09:01:16.546 INFO:tasks.workunit.client.1.smithi019.stdout:2/514: mkdir d5/de/dab 0 2024-03-16T09:01:16.546 INFO:tasks.workunit.client.1.smithi019.stdout:9/334: write d1/d7/f45 [1431198,111445] 0 2024-03-16T09:01:16.547 INFO:tasks.workunit.client.1.smithi019.stdout:3/499: dwrite d3/d89/f38 [0,4194304] 0 2024-03-16T09:01:16.547 INFO:tasks.workunit.client.1.smithi019.stdout:3/500: truncate d3/d17/d5a/d75/f92 88192 0 2024-03-16T09:01:16.547 INFO:tasks.workunit.client.0.smithi007.stdout:6/574: dwrite d2/d11/f25 [0,4194304] 0 2024-03-16T09:01:16.547 INFO:tasks.workunit.client.0.smithi007.stdout:6/575: write d2/d11/d3b/d93/d8e/d4e/f64 [523263,123099] 0 2024-03-16T09:01:16.549 INFO:tasks.workunit.client.0.smithi007.stdout:7/551: dwrite d0/d19/d1f/d69/f6b [0,4194304] 0 2024-03-16T09:01:16.549 INFO:tasks.workunit.client.0.smithi007.stdout:7/552: readlink d0/d19/l60 0 2024-03-16T09:01:16.552 INFO:tasks.workunit.client.0.smithi007.stdout:6/576: rename d2/d11/d3b/d93/c16 to d2/d11/cba 0 2024-03-16T09:01:16.556 INFO:tasks.workunit.client.0.smithi007.stdout:7/553: mkdir d0/d6c/d25/d3a/daf/dc2 0 2024-03-16T09:01:16.557 INFO:tasks.workunit.client.0.smithi007.stdout:6/577: creat d2/d13/fbb x:0 0 0 2024-03-16T09:01:16.558 INFO:tasks.workunit.client.1.smithi019.stdout:0/495: creat d7/d1d/d63/d89/d8a/fa8 x:0 0 0 2024-03-16T09:01:16.558 INFO:tasks.workunit.client.1.smithi019.stdout:0/496: dread - d7/f79 zero size 2024-03-16T09:01:16.559 INFO:tasks.workunit.client.0.smithi007.stdout:9/638: sync 2024-03-16T09:01:16.561 INFO:tasks.workunit.client.0.smithi007.stdout:6/578: rename d2/d11/d3b/d93/d15/f28 to d2/d13/d4b/d94/fbc 0 2024-03-16T09:01:16.562 INFO:tasks.workunit.client.1.smithi019.stdout:8/479: mknod d6/de/d24/d30/d4a/d87/ca4 0 2024-03-16T09:01:16.562 INFO:tasks.workunit.client.1.smithi019.stdout:8/480: readlink d6/de/d24/d30/d4a/l53 0 2024-03-16T09:01:16.563 INFO:tasks.workunit.client.1.smithi019.stdout:7/335: symlink d9/l67 0 2024-03-16T09:01:16.564 INFO:tasks.workunit.client.1.smithi019.stdout:9/335: chown d1/d7/d21/l4b 2689833 1 2024-03-16T09:01:16.565 INFO:tasks.workunit.client.0.smithi007.stdout:6/579: rename d2/d11/d3b/d93/c41 to d2/d13/d3f/d86/cbd 0 2024-03-16T09:01:16.566 INFO:tasks.workunit.client.0.smithi007.stdout:9/639: truncate d24/d4c/d86/daa/d7f/d80/d87/d89/f8a 877296 0 2024-03-16T09:01:16.566 INFO:tasks.workunit.client.1.smithi019.stdout:0/497: truncate d7/d8/f88 3584077 0 2024-03-16T09:01:16.567 INFO:tasks.workunit.client.1.smithi019.stdout:8/481: rmdir d6/d2a/d45/d57/d72/d79 39 2024-03-16T09:01:16.569 INFO:tasks.workunit.client.0.smithi007.stdout:9/640: rename d24/d4a/d55/d56/da6 to d24/d4c/d86/daa/d7f/d8f/db5 0 2024-03-16T09:01:16.570 INFO:tasks.workunit.client.1.smithi019.stdout:7/336: rename d9/d25/l4d to d9/da/d13/d18/d30/d3e/l68 0 2024-03-16T09:01:16.572 INFO:tasks.workunit.client.1.smithi019.stdout:9/336: mkdir d1/d7/de/d24/d44/d60/d74 0 2024-03-16T09:01:16.572 INFO:tasks.workunit.client.1.smithi019.stdout:9/337: write d1/d7/de/f2a [3542797,57894] 0 2024-03-16T09:01:16.573 INFO:tasks.workunit.client.0.smithi007.stdout:9/641: unlink c19 0 2024-03-16T09:01:16.574 INFO:tasks.workunit.client.1.smithi019.stdout:8/482: mkdir d6/d2a/d45/d57/d72/d79/da5 0 2024-03-16T09:01:16.574 INFO:tasks.workunit.client.1.smithi019.stdout:8/483: readlink d6/de/d24/d30/d4a/la1 0 2024-03-16T09:01:16.574 INFO:tasks.workunit.client.0.smithi007.stdout:9/642: rmdir d24/d4c/d86 39 2024-03-16T09:01:16.576 INFO:tasks.workunit.client.1.smithi019.stdout:9/338: symlink d1/d7/de/d24/l75 0 2024-03-16T09:01:16.577 INFO:tasks.workunit.client.0.smithi007.stdout:9/643: mknod d24/cb6 0 2024-03-16T09:01:16.578 INFO:tasks.workunit.client.1.smithi019.stdout:9/339: fdatasync d1/d7/f3e 0 2024-03-16T09:01:16.579 INFO:tasks.workunit.client.1.smithi019.stdout:9/340: fdatasync d1/d7/f34 0 2024-03-16T09:01:16.583 INFO:tasks.workunit.client.1.smithi019.stdout:7/337: write d9/d25/f31 [2914402,95604] 0 2024-03-16T09:01:16.583 INFO:tasks.workunit.client.1.smithi019.stdout:5/449: dwrite d6/d19/f59 [0,4194304] 0 2024-03-16T09:01:16.584 INFO:tasks.workunit.client.1.smithi019.stdout:5/450: fsync d6/f9 0 2024-03-16T09:01:16.585 INFO:tasks.workunit.client.1.smithi019.stdout:9/341: rename d1/d7/de/d2d to d1/d7/de/d24/d44/d76 0 2024-03-16T09:01:16.586 INFO:tasks.workunit.client.0.smithi007.stdout:9/644: write f16 [157043,121729] 0 2024-03-16T09:01:16.586 INFO:tasks.workunit.client.0.smithi007.stdout:9/645: chown d24/d4c/d86/daa/d7f/d80/l83 1 1 2024-03-16T09:01:16.586 INFO:tasks.workunit.client.0.smithi007.stdout:9/646: mkdir d24/d4a/da2/db7 0 2024-03-16T09:01:16.587 INFO:tasks.workunit.client.0.smithi007.stdout:9/647: write d24/f31 [376085,77329] 0 2024-03-16T09:01:16.590 INFO:tasks.workunit.client.0.smithi007.stdout:9/648: write d24/f26 [85860,50734] 0 2024-03-16T09:01:16.590 INFO:tasks.workunit.client.0.smithi007.stdout:9/649: chown d24/d4c/d86/daa 431 1 2024-03-16T09:01:16.592 INFO:tasks.workunit.client.0.smithi007.stdout:9/650: creat d24/d4c/d86/daa/d7f/d8f/fb8 x:0 0 0 2024-03-16T09:01:16.592 INFO:tasks.workunit.client.1.smithi019.stdout:5/451: creat d6/d1f/d33/d7d/f8b x:0 0 0 2024-03-16T09:01:16.592 INFO:tasks.workunit.client.1.smithi019.stdout:5/452: dread - d6/d19/f7f zero size 2024-03-16T09:01:16.593 INFO:tasks.workunit.client.1.smithi019.stdout:7/338: rename d9/da/d13/d18/d30/f5c to d9/d25/d28/f69 0 2024-03-16T09:01:16.594 INFO:tasks.workunit.client.1.smithi019.stdout:7/339: read d9/d25/d28/f34 [259467,55439] 0 2024-03-16T09:01:16.595 INFO:tasks.workunit.client.1.smithi019.stdout:5/453: mknod d6/d19/d34/c8c 0 2024-03-16T09:01:16.598 INFO:tasks.workunit.client.1.smithi019.stdout:5/454: dread d6/dd/d37/d54/f5f [0,4194304] 0 2024-03-16T09:01:16.599 INFO:tasks.workunit.client.1.smithi019.stdout:3/501: dwrite d3/f51 [0,4194304] 0 2024-03-16T09:01:16.599 INFO:tasks.workunit.client.1.smithi019.stdout:3/502: chown d3/d89/l68 1 1 2024-03-16T09:01:16.599 INFO:tasks.workunit.client.1.smithi019.stdout:5/455: rmdir d6/dd/d47/d70 39 2024-03-16T09:01:16.600 INFO:tasks.workunit.client.1.smithi019.stdout:2/515: dwrite d5/d2e/d4f/f5e [0,4194304] 0 2024-03-16T09:01:16.601 INFO:tasks.workunit.client.1.smithi019.stdout:2/516: chown d5/d6/d2b/d53/f73 4739994 1 2024-03-16T09:01:16.601 INFO:tasks.workunit.client.1.smithi019.stdout:7/340: dread d9/da/d13/d18/d1a/f1f [0,4194304] 0 2024-03-16T09:01:16.601 INFO:tasks.workunit.client.1.smithi019.stdout:7/341: fdatasync d9/db/d12/f55 0 2024-03-16T09:01:16.601 INFO:tasks.workunit.client.1.smithi019.stdout:7/342: write d9/db/d12/f26 [332548,96108] 0 2024-03-16T09:01:16.604 INFO:tasks.workunit.client.0.smithi007.stdout:7/554: rmdir d0/d6c 39 2024-03-16T09:01:16.604 INFO:tasks.workunit.client.0.smithi007.stdout:7/555: chown d0/d19/l60 99562 1 2024-03-16T09:01:16.606 INFO:tasks.workunit.client.1.smithi019.stdout:4/401: sync 2024-03-16T09:01:16.608 INFO:tasks.workunit.client.0.smithi007.stdout:7/556: dread d0/d6c/d25/d73/f58 [0,4194304] 0 2024-03-16T09:01:16.608 INFO:tasks.workunit.client.1.smithi019.stdout:7/343: creat d9/da/d13/f6a x:0 0 0 2024-03-16T09:01:16.609 INFO:tasks.workunit.client.0.smithi007.stdout:6/580: dwrite d2/d11/d3b/d93/d15/f98 [0,4194304] 0 2024-03-16T09:01:16.609 INFO:tasks.workunit.client.1.smithi019.stdout:5/456: dread f4 [0,4194304] 0 2024-03-16T09:01:16.610 INFO:tasks.workunit.client.1.smithi019.stdout:8/484: dwrite d6/de/d24/d30/d4a/f4e [0,4194304] 0 2024-03-16T09:01:16.610 INFO:tasks.workunit.client.1.smithi019.stdout:0/498: dwrite d7/d1d/d28/f5a [0,4194304] 0 2024-03-16T09:01:16.611 INFO:tasks.workunit.client.1.smithi019.stdout:5/457: symlink d6/d1f/d78/l8d 0 2024-03-16T09:01:16.611 INFO:tasks.workunit.client.1.smithi019.stdout:4/402: dread d4/d6/d15/d1f/d2c/f6b [0,4194304] 0 2024-03-16T09:01:16.612 INFO:tasks.workunit.client.0.smithi007.stdout:6/581: mkdir d2/d13/d3f/d50/d78/dbe 0 2024-03-16T09:01:16.613 INFO:tasks.workunit.client.0.smithi007.stdout:6/582: rmdir d2/d13/d3f 39 2024-03-16T09:01:16.616 INFO:tasks.workunit.client.1.smithi019.stdout:9/342: dwrite d1/d7/de/f1a [0,4194304] 0 2024-03-16T09:01:16.621 INFO:tasks.workunit.client.1.smithi019.stdout:0/499: creat d7/d1d/d63/d89/fa9 x:0 0 0 2024-03-16T09:01:16.622 INFO:tasks.workunit.client.1.smithi019.stdout:0/500: write d7/d8/d32/d36/d82/f71 [881215,66561] 0 2024-03-16T09:01:16.622 INFO:tasks.workunit.client.1.smithi019.stdout:4/403: mknod d4/d1b/c87 0 2024-03-16T09:01:16.623 INFO:tasks.workunit.client.1.smithi019.stdout:5/458: dread d6/f17 [0,4194304] 0 2024-03-16T09:01:16.624 INFO:tasks.workunit.client.0.smithi007.stdout:9/651: rename d24/d4a/d55/d56 to d24/d4a/d96/db9 0 2024-03-16T09:01:16.626 INFO:tasks.workunit.client.1.smithi019.stdout:2/517: dwrite d5/de/f32 [0,4194304] 0 2024-03-16T09:01:16.628 INFO:tasks.workunit.client.0.smithi007.stdout:6/583: write d2/d11/d3b/d93/d15/f98 [3512637,66061] 0 2024-03-16T09:01:16.628 INFO:tasks.workunit.client.1.smithi019.stdout:8/485: rmdir d6/d2a/d45/d57/d96 39 2024-03-16T09:01:16.628 INFO:tasks.workunit.client.1.smithi019.stdout:8/486: dread d6/d2a/d45/d19/d74/f8a [0,4194304] 0 2024-03-16T09:01:16.628 INFO:tasks.workunit.client.1.smithi019.stdout:0/501: creat d7/d8/d32/d36/d82/faa x:0 0 0 2024-03-16T09:01:16.630 INFO:tasks.workunit.client.1.smithi019.stdout:7/344: dread d9/d25/f31 [0,4194304] 0 2024-03-16T09:01:16.631 INFO:tasks.workunit.client.1.smithi019.stdout:7/345: creat d9/da/d13/d18/d46/f6b x:0 0 0 2024-03-16T09:01:16.632 INFO:tasks.workunit.client.1.smithi019.stdout:5/459: rename d6/dd/f40 to d6/dd/d37/d76/f8e 0 2024-03-16T09:01:16.632 INFO:tasks.workunit.client.1.smithi019.stdout:5/460: chown d6/d1f/d78/l8d 1733 1 2024-03-16T09:01:16.632 INFO:tasks.workunit.client.1.smithi019.stdout:5/461: stat d6/f9 0 2024-03-16T09:01:16.632 INFO:tasks.workunit.client.1.smithi019.stdout:3/503: dwrite d3/d17/d6a/f7b [0,4194304] 0 2024-03-16T09:01:16.635 INFO:tasks.workunit.client.1.smithi019.stdout:9/343: mkdir d1/d7/d67/d77 0 2024-03-16T09:01:16.637 INFO:tasks.workunit.client.1.smithi019.stdout:5/462: mknod d6/d1f/d77/d84/c8f 0 2024-03-16T09:01:16.637 INFO:tasks.workunit.client.1.smithi019.stdout:5/463: getdents d6/dd/d47/d7a 0 2024-03-16T09:01:16.638 INFO:tasks.workunit.client.0.smithi007.stdout:6/584: mkdir d2/d10/d3d/d6d/dbf 0 2024-03-16T09:01:16.640 INFO:tasks.workunit.client.1.smithi019.stdout:6/534: sync 2024-03-16T09:01:16.640 INFO:tasks.workunit.client.1.smithi019.stdout:1/400: sync 2024-03-16T09:01:16.640 INFO:tasks.workunit.client.1.smithi019.stdout:6/535: dread - de/d26/d32/d97/d66/f74 zero size 2024-03-16T09:01:16.641 INFO:tasks.workunit.client.1.smithi019.stdout:9/344: truncate d1/d7/f50 487183 0 2024-03-16T09:01:16.642 INFO:tasks.workunit.client.0.smithi007.stdout:6/585: rename d2/d11/f47 to d2/d11/d3b/d74/fc0 0 2024-03-16T09:01:16.642 INFO:tasks.workunit.client.0.smithi007.stdout:6/586: write d2/d10/d3d/d6d/fa1 [1207654,21304] 0 2024-03-16T09:01:16.643 INFO:tasks.workunit.client.0.smithi007.stdout:6/587: chown d2/d13/d3f/d50/d78/d81/lb2 592842 1 2024-03-16T09:01:16.643 INFO:tasks.workunit.client.1.smithi019.stdout:5/464: symlink d6/d1f/d33/d89/l90 0 2024-03-16T09:01:16.645 INFO:tasks.workunit.client.0.smithi007.stdout:6/588: mkdir d2/d11/d3b/d93/dc1 0 2024-03-16T09:01:16.645 INFO:tasks.workunit.client.1.smithi019.stdout:9/345: chown d1/d7/l48 5925 1 2024-03-16T09:01:16.646 INFO:tasks.workunit.client.1.smithi019.stdout:5/465: link d6/dd/d47/f69 d6/d1f/d33/d4c/f91 0 2024-03-16T09:01:16.648 INFO:tasks.workunit.client.1.smithi019.stdout:9/346: truncate d1/d7/de/d24/f39 70120 0 2024-03-16T09:01:16.648 INFO:tasks.workunit.client.1.smithi019.stdout:8/487: write d6/de/f6a [2396981,88626] 0 2024-03-16T09:01:16.649 INFO:tasks.workunit.client.1.smithi019.stdout:8/488: chown d6/de/d24/d30/f3b 50263 1 2024-03-16T09:01:16.652 INFO:tasks.workunit.client.1.smithi019.stdout:5/466: creat d6/d1f/d77/d84/f92 x:0 0 0 2024-03-16T09:01:16.653 INFO:tasks.workunit.client.1.smithi019.stdout:8/489: fdatasync d6/de/d24/f40 0 2024-03-16T09:01:16.655 INFO:tasks.workunit.client.0.smithi007.stdout:5/617: sync 2024-03-16T09:01:16.655 INFO:tasks.workunit.client.0.smithi007.stdout:1/556: sync 2024-03-16T09:01:16.658 INFO:tasks.workunit.client.0.smithi007.stdout:1/557: mknod d9/dd/d5e/d89/cb3 0 2024-03-16T09:01:16.658 INFO:tasks.workunit.client.0.smithi007.stdout:1/558: dread - d9/dd/d1b/d2d/d30/f8e zero size 2024-03-16T09:01:16.663 INFO:tasks.workunit.client.0.smithi007.stdout:1/559: mkdir d9/dd/d1b/d28/d29/d7b/db4 0 2024-03-16T09:01:16.663 INFO:tasks.workunit.client.1.smithi019.stdout:8/490: dread d6/d2a/d45/d19/d74/f7e [0,4194304] 0 2024-03-16T09:01:16.663 INFO:tasks.workunit.client.1.smithi019.stdout:8/491: chown d6/de/ff 1312131 1 2024-03-16T09:01:16.665 INFO:tasks.workunit.client.0.smithi007.stdout:1/560: read d9/f12 [4335416,30177] 0 2024-03-16T09:01:16.665 INFO:tasks.workunit.client.0.smithi007.stdout:1/561: fsync f7 0 2024-03-16T09:01:16.666 INFO:tasks.workunit.client.1.smithi019.stdout:8/492: link d6/d2a/d45/f42 d6/d2a/d45/d57/d72/d79/d7b/fa6 0 2024-03-16T09:01:16.666 INFO:tasks.workunit.client.1.smithi019.stdout:8/493: creat d6/d2a/d45/d5d/fa7 x:0 0 0 2024-03-16T09:01:16.667 INFO:tasks.workunit.client.1.smithi019.stdout:8/494: dread d6/de/d24/d30/d4a/d67/f70 [0,4194304] 0 2024-03-16T09:01:16.669 INFO:tasks.workunit.client.1.smithi019.stdout:8/495: creat d6/de/d24/d30/d4a/d67/d95/d9a/fa8 x:0 0 0 2024-03-16T09:01:16.708 INFO:tasks.workunit.client.1.smithi019.stdout:8/496: dread d6/d2a/d45/d2c/f5c [0,4194304] 0 2024-03-16T09:01:16.710 INFO:tasks.workunit.client.1.smithi019.stdout:8/497: creat d6/de/d24/d30/d4a/d67/fa9 x:0 0 0 2024-03-16T09:01:16.710 INFO:tasks.workunit.client.1.smithi019.stdout:8/498: chown d6/d2a/d45/d2c/f5c 9285732 1 2024-03-16T09:01:16.711 INFO:tasks.workunit.client.1.smithi019.stdout:8/499: creat d6/de/d24/faa x:0 0 0 2024-03-16T09:01:16.712 INFO:tasks.workunit.client.1.smithi019.stdout:8/500: dread - d6/de/d24/f90 zero size 2024-03-16T09:01:16.715 INFO:tasks.workunit.client.1.smithi019.stdout:2/518: dwrite d5/d2e/f3b [0,4194304] 0 2024-03-16T09:01:16.715 INFO:tasks.workunit.client.1.smithi019.stdout:4/404: dwrite d4/d6/f39 [4194304,4194304] 0 2024-03-16T09:01:16.715 INFO:tasks.workunit.client.1.smithi019.stdout:0/502: dwrite d7/d1d/d28/d6c/f95 [0,4194304] 0 2024-03-16T09:01:16.715 INFO:tasks.workunit.client.1.smithi019.stdout:4/405: write d4/d6/d15/f2e [1755292,110925] 0 2024-03-16T09:01:16.716 INFO:tasks.workunit.client.1.smithi019.stdout:4/406: fdatasync d4/d6/f32 0 2024-03-16T09:01:16.719 INFO:tasks.workunit.client.1.smithi019.stdout:2/519: creat d5/fac x:0 0 0 2024-03-16T09:01:16.721 INFO:tasks.workunit.client.1.smithi019.stdout:3/504: dwrite d3/d1b/f8e [0,4194304] 0 2024-03-16T09:01:16.722 INFO:tasks.workunit.client.0.smithi007.stdout:9/652: dwrite d24/f39 [0,4194304] 0 2024-03-16T09:01:16.723 INFO:tasks.workunit.client.1.smithi019.stdout:0/503: rename d7/d8/d32/l8b to d7/d8/d32/d4a/d61/d68/lab 0 2024-03-16T09:01:16.723 INFO:tasks.workunit.client.1.smithi019.stdout:0/504: write d7/d1d/d28/d6c/f77 [697630,16443] 0 2024-03-16T09:01:16.723 INFO:tasks.workunit.client.0.smithi007.stdout:1/562: dwrite d9/dd/d1b/f2c [0,4194304] 0 2024-03-16T09:01:16.724 INFO:tasks.workunit.client.1.smithi019.stdout:5/467: dwrite d6/d19/f4e [4194304,4194304] 0 2024-03-16T09:01:16.724 INFO:tasks.workunit.client.1.smithi019.stdout:5/468: creat d6/dd/d37/f93 x:0 0 0 2024-03-16T09:01:16.724 INFO:tasks.workunit.client.1.smithi019.stdout:5/469: readlink d6/d19/l28 0 2024-03-16T09:01:16.724 INFO:tasks.workunit.client.1.smithi019.stdout:5/470: creat d6/dd/d37/d76/f94 x:0 0 0 2024-03-16T09:01:16.724 INFO:tasks.workunit.client.1.smithi019.stdout:9/347: dwrite d1/d7/de/d24/d5b/f6e [0,4194304] 0 2024-03-16T09:01:16.724 INFO:tasks.workunit.client.1.smithi019.stdout:7/346: dwrite d9/db/d12/d5b/d33/f58 [0,4194304] 0 2024-03-16T09:01:16.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/401: dwrite f14 [0,4194304] 0 2024-03-16T09:01:16.724 INFO:tasks.workunit.client.1.smithi019.stdout:1/402: fdatasync d16/d1f/d4f/f62 0 2024-03-16T09:01:16.725 INFO:tasks.workunit.client.1.smithi019.stdout:2/520: unlink d5/d6/d2b/d2f/l54 0 2024-03-16T09:01:16.725 INFO:tasks.workunit.client.1.smithi019.stdout:6/536: dwrite de/d26/d32/d4e/fa5 [0,4194304] 0 2024-03-16T09:01:16.726 INFO:tasks.workunit.client.1.smithi019.stdout:0/505: mknod d7/d1d/d28/d92/da6/cac 0 2024-03-16T09:01:16.726 INFO:tasks.workunit.client.1.smithi019.stdout:0/506: write d7/d8/d32/d36/f5f [141798,60608] 0 2024-03-16T09:01:16.726 INFO:tasks.workunit.client.1.smithi019.stdout:0/507: fdatasync d7/d1d/d28/d48/f65 0 2024-03-16T09:01:16.726 INFO:tasks.workunit.client.1.smithi019.stdout:0/508: write d7/d1d/d28/d6c/f77 [1283152,56381] 0 2024-03-16T09:01:16.727 INFO:tasks.workunit.client.0.smithi007.stdout:1/563: rmdir d9/d6b/d7e 39 2024-03-16T09:01:16.729 INFO:tasks.workunit.client.0.smithi007.stdout:9/653: rmdir d24/d4a 39 2024-03-16T09:01:16.730 INFO:tasks.workunit.client.0.smithi007.stdout:9/654: chown l20 249736301 1 2024-03-16T09:01:16.731 INFO:tasks.workunit.client.0.smithi007.stdout:9/655: link d24/d4a/d55/c76 d24/d4a/d96/db9/d90/cba 0 2024-03-16T09:01:16.731 INFO:tasks.workunit.client.0.smithi007.stdout:9/656: read f16 [519123,9514] 0 2024-03-16T09:01:16.731 INFO:tasks.workunit.client.0.smithi007.stdout:9/657: write d24/d4a/d96/db9/f77 [520313,74804] 0 2024-03-16T09:01:16.732 INFO:tasks.workunit.client.1.smithi019.stdout:5/471: creat d6/dd/d47/d7a/f95 x:0 0 0 2024-03-16T09:01:16.732 INFO:tasks.workunit.client.1.smithi019.stdout:5/472: chown d6/d19/c83 358636 1 2024-03-16T09:01:16.732 INFO:tasks.workunit.client.1.smithi019.stdout:5/473: chown d6/dd/d37 5 1 2024-03-16T09:01:16.733 INFO:tasks.workunit.client.0.smithi007.stdout:6/589: dread d2/d11/d3b/d93/d15/f98 [0,4194304] 0 2024-03-16T09:01:16.733 INFO:tasks.workunit.client.0.smithi007.stdout:9/658: mkdir d24/d4a/d96/db9/db1/dbb 0 2024-03-16T09:01:16.736 INFO:tasks.workunit.client.0.smithi007.stdout:9/659: rename d24/f2a to d24/d4c/d86/daa/d7f/d80/d87/d89/fbc 0 2024-03-16T09:01:16.737 INFO:tasks.workunit.client.1.smithi019.stdout:1/403: dread d16/d18/f1a [0,4194304] 0 2024-03-16T09:01:16.738 INFO:tasks.workunit.client.0.smithi007.stdout:9/660: link l10 d24/d4a/d96/db9/db1/lbd 0 2024-03-16T09:01:16.741 INFO:tasks.workunit.client.1.smithi019.stdout:6/537: read de/d26/d32/d97/f30 [4062461,113310] 0 2024-03-16T09:01:16.741 INFO:tasks.workunit.client.1.smithi019.stdout:9/348: mknod d1/d7/d21/c78 0 2024-03-16T09:01:16.741 INFO:tasks.workunit.client.1.smithi019.stdout:7/347: mkdir d9/d25/d28/d6c 0 2024-03-16T09:01:16.742 INFO:tasks.workunit.client.1.smithi019.stdout:9/349: fdatasync d1/d7/d21/d27/f57 0 2024-03-16T09:01:16.742 INFO:tasks.workunit.client.1.smithi019.stdout:9/350: write d1/d7/d21/d27/f58 [89591,19117] 0 2024-03-16T09:01:16.742 INFO:tasks.workunit.client.1.smithi019.stdout:7/348: creat d9/db/d12/f6d x:0 0 0 2024-03-16T09:01:16.742 INFO:tasks.workunit.client.1.smithi019.stdout:2/521: rmdir d5/d5a/d80 39 2024-03-16T09:01:16.742 INFO:tasks.workunit.client.1.smithi019.stdout:9/351: dread d1/d7/f3e [0,4194304] 0 2024-03-16T09:01:16.742 INFO:tasks.workunit.client.0.smithi007.stdout:1/564: dread d9/dd/d1b/d2d/d30/f4f [0,4194304] 0 2024-03-16T09:01:16.743 INFO:tasks.workunit.client.0.smithi007.stdout:1/565: dread - d9/dd/d1b/d2d/d30/f8e zero size 2024-03-16T09:01:16.743 INFO:tasks.workunit.client.0.smithi007.stdout:1/566: truncate d9/dd/f67 282742 0 2024-03-16T09:01:16.744 INFO:tasks.workunit.client.0.smithi007.stdout:1/567: mknod d9/dd/d1b/d28/cb5 0 2024-03-16T09:01:16.744 INFO:tasks.workunit.client.0.smithi007.stdout:1/568: chown d9/dd/d1b/d28/d29/f5d 806503 1 2024-03-16T09:01:16.745 INFO:tasks.workunit.client.1.smithi019.stdout:0/509: mknod d7/d1d/d63/d89/d8a/cad 0 2024-03-16T09:01:16.745 INFO:tasks.workunit.client.0.smithi007.stdout:1/569: mknod d9/dd/d1b/d28/d4c/cb6 0 2024-03-16T09:01:16.747 INFO:tasks.workunit.client.1.smithi019.stdout:6/538: chown c6 805083 1 2024-03-16T09:01:16.748 INFO:tasks.workunit.client.0.smithi007.stdout:6/590: dread d2/d11/d1e/f85 [0,4194304] 0 2024-03-16T09:01:16.749 INFO:tasks.workunit.client.0.smithi007.stdout:6/591: mkdir d2/d11/d3b/d93/d8e/d49/dc2 0 2024-03-16T09:01:16.750 INFO:tasks.workunit.client.1.smithi019.stdout:7/349: rmdir d9/db 39 2024-03-16T09:01:16.751 INFO:tasks.workunit.client.0.smithi007.stdout:9/661: dread ff [0,4194304] 0 2024-03-16T09:01:16.751 INFO:tasks.workunit.client.0.smithi007.stdout:9/662: readlink l6 0 2024-03-16T09:01:16.751 INFO:tasks.workunit.client.0.smithi007.stdout:6/592: read d2/d11/d3b/d93/d8e/d4e/f64 [224626,76208] 0 2024-03-16T09:01:16.751 INFO:tasks.workunit.client.0.smithi007.stdout:6/593: dread - d2/f59 zero size 2024-03-16T09:01:16.751 INFO:tasks.workunit.client.0.smithi007.stdout:9/663: mknod d24/d4a/cbe 0 2024-03-16T09:01:16.753 INFO:tasks.workunit.client.0.smithi007.stdout:6/594: symlink d2/d13/d4b/d94/lc3 0 2024-03-16T09:01:16.753 INFO:tasks.workunit.client.0.smithi007.stdout:6/595: chown d2/d82/fa2 86030657 1 2024-03-16T09:01:16.753 INFO:tasks.workunit.client.0.smithi007.stdout:1/570: dread f5 [0,4194304] 0 2024-03-16T09:01:16.753 INFO:tasks.workunit.client.0.smithi007.stdout:9/664: mkdir d24/d4a/d55/dbf 0 2024-03-16T09:01:16.753 INFO:tasks.workunit.client.0.smithi007.stdout:9/665: chown d24/d4c/d91/l93 0 1 2024-03-16T09:01:16.754 INFO:tasks.workunit.client.1.smithi019.stdout:2/522: symlink d5/d6/d2b/d2f/d35/lad 0 2024-03-16T09:01:16.754 INFO:tasks.workunit.client.0.smithi007.stdout:1/571: rmdir d9/dd/d1b/d28/d29/d7b 39 2024-03-16T09:01:16.754 INFO:tasks.workunit.client.0.smithi007.stdout:1/572: truncate d9/dd/d1b/d28/f50 323715 0 2024-03-16T09:01:16.755 INFO:tasks.workunit.client.1.smithi019.stdout:0/510: creat d7/d1d/d28/da1/da7/fae x:0 0 0 2024-03-16T09:01:16.755 INFO:tasks.workunit.client.1.smithi019.stdout:0/511: stat d7/d1d/d63/d6e 0 2024-03-16T09:01:16.755 INFO:tasks.workunit.client.1.smithi019.stdout:9/352: read d1/d7/de/d24/d5b/f6e [852247,88671] 0 2024-03-16T09:01:16.756 INFO:tasks.workunit.client.0.smithi007.stdout:9/666: rename d24/d4a/d55/l82 to d24/d4a/d9e/lc0 0 2024-03-16T09:01:16.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/407: dwrite d4/d1b/d2a/f4e [4194304,4194304] 0 2024-03-16T09:01:16.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/408: dread - d4/d1b/d2a/d4f/f82 zero size 2024-03-16T09:01:16.757 INFO:tasks.workunit.client.1.smithi019.stdout:8/501: dwrite d6/f27 [0,4194304] 0 2024-03-16T09:01:16.757 INFO:tasks.workunit.client.1.smithi019.stdout:5/474: dwrite d6/d1f/d33/d7d/f8b [0,4194304] 0 2024-03-16T09:01:16.758 INFO:tasks.workunit.client.0.smithi007.stdout:1/573: symlink d9/d6b/d7e/dac/lb7 0 2024-03-16T09:01:16.758 INFO:tasks.workunit.client.0.smithi007.stdout:6/596: dread d2/d13/d4b/f7c [0,4194304] 0 2024-03-16T09:01:16.759 INFO:tasks.workunit.client.1.smithi019.stdout:3/505: dwrite d3/d17/d5a/f8c [0,4194304] 0 2024-03-16T09:01:16.762 INFO:tasks.workunit.client.1.smithi019.stdout:7/350: rename d9/d25/d28/d6c to d9/da/d13/d18/d1a/d6e 0 2024-03-16T09:01:16.762 INFO:tasks.workunit.client.1.smithi019.stdout:7/351: write f8 [4841611,117329] 0 2024-03-16T09:01:16.763 INFO:tasks.workunit.client.1.smithi019.stdout:0/512: write d7/d8/d32/d4a/d61/f2c [2547837,63154] 0 2024-03-16T09:01:16.767 INFO:tasks.workunit.client.0.smithi007.stdout:1/574: rename d9/d6b/lb2 to d9/d6b/d7e/lb8 0 2024-03-16T09:01:16.770 INFO:tasks.workunit.client.1.smithi019.stdout:5/475: link d6/f8 d6/d1f/d33/d89/f96 0 2024-03-16T09:01:16.771 INFO:tasks.workunit.client.0.smithi007.stdout:6/597: rename d2/d13/d4b/d94/fac to d2/d11/d1e/d54/d7a/d9a/fc4 0 2024-03-16T09:01:16.771 INFO:tasks.workunit.client.0.smithi007.stdout:6/598: write d2/d13/d4b/d94/fbc [4399276,123311] 0 2024-03-16T09:01:16.773 INFO:tasks.workunit.client.0.smithi007.stdout:1/575: mknod d9/d6b/d7e/dac/cb9 0 2024-03-16T09:01:16.773 INFO:tasks.workunit.client.1.smithi019.stdout:6/539: rmdir de/d26/d32/d97/d2c 39 2024-03-16T09:01:16.779 INFO:tasks.workunit.client.0.smithi007.stdout:6/599: rename d2/d13/d3f/d86/f96 to d2/d11/d1e/d54/d7a/fc5 0 2024-03-16T09:01:16.782 INFO:tasks.workunit.client.1.smithi019.stdout:0/513: symlink d7/d8/d32/laf 0 2024-03-16T09:01:16.784 INFO:tasks.workunit.client.0.smithi007.stdout:9/667: dwrite f1e [8388608,4194304] 0 2024-03-16T09:01:16.785 INFO:tasks.workunit.client.1.smithi019.stdout:4/409: rename d4/d6/d15/d1f/c18 to d4/d6c/d79/c88 0 2024-03-16T09:01:16.786 INFO:tasks.workunit.client.0.smithi007.stdout:4/513: sync 2024-03-16T09:01:16.786 INFO:tasks.workunit.client.0.smithi007.stdout:2/734: sync 2024-03-16T09:01:16.786 INFO:tasks.workunit.client.0.smithi007.stdout:2/735: write dc/df/d24/d28/d2e/d35/da2/fe3 [929284,111744] 0 2024-03-16T09:01:16.786 INFO:tasks.workunit.client.0.smithi007.stdout:3/671: sync 2024-03-16T09:01:16.786 INFO:tasks.workunit.client.0.smithi007.stdout:0/744: sync 2024-03-16T09:01:16.786 INFO:tasks.workunit.client.0.smithi007.stdout:3/672: chown d15/d18/d1f/d29/f1b 144 1 2024-03-16T09:01:16.786 INFO:tasks.workunit.client.0.smithi007.stdout:0/745: dread - d3/d9/da/d18/f7d zero size 2024-03-16T09:01:16.786 INFO:tasks.workunit.client.0.smithi007.stdout:0/746: readlink d3/d9/da/d18/d27/lc9 0 2024-03-16T09:01:16.787 INFO:tasks.workunit.client.0.smithi007.stdout:0/747: truncate d3/d9/f93 802885 0 2024-03-16T09:01:16.787 INFO:tasks.workunit.client.1.smithi019.stdout:0/514: mkdir d7/d1d/d63/d6e/db0 0 2024-03-16T09:01:16.788 INFO:tasks.workunit.client.1.smithi019.stdout:1/404: truncate d16/d18/f23 3733546 0 2024-03-16T09:01:16.789 INFO:tasks.workunit.client.1.smithi019.stdout:1/405: dread d16/d1f/d4f/f57 [0,4194304] 0 2024-03-16T09:01:16.789 INFO:tasks.workunit.client.1.smithi019.stdout:1/406: write d16/f42 [651759,29003] 0 2024-03-16T09:01:16.793 INFO:tasks.workunit.client.0.smithi007.stdout:9/668: truncate d24/d4a/d96/db9/f60 340598 0 2024-03-16T09:01:16.793 INFO:tasks.workunit.client.0.smithi007.stdout:9/669: chown d24/d4c/d86/daa/d95/l72 1 1 2024-03-16T09:01:16.794 INFO:tasks.workunit.client.1.smithi019.stdout:4/410: mkdir d4/d6c/d89 0 2024-03-16T09:01:16.796 INFO:tasks.workunit.client.0.smithi007.stdout:9/670: read d24/d4c/f4e [600280,47817] 0 2024-03-16T09:01:16.797 INFO:tasks.workunit.client.0.smithi007.stdout:4/514: mknod d4/d14/d1c/d5b/d67/d8f/da8/d78/cae 0 2024-03-16T09:01:16.797 INFO:tasks.workunit.client.0.smithi007.stdout:4/515: dread - d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/fa9 zero size 2024-03-16T09:01:16.797 INFO:tasks.workunit.client.1.smithi019.stdout:4/411: truncate d4/d6/d15/d1f/f44 4422450 0 2024-03-16T09:01:16.798 INFO:tasks.workunit.client.0.smithi007.stdout:2/736: mkdir dc/df/d24/d28/d4c/ded 0 2024-03-16T09:01:16.798 INFO:tasks.workunit.client.1.smithi019.stdout:8/502: rename d6/de/d24 to d6/d2a/d3c/dab 0 2024-03-16T09:01:16.800 INFO:tasks.workunit.client.0.smithi007.stdout:0/748: mkdir d3/d9/da/d18/d27/d34/df5 0 2024-03-16T09:01:16.800 INFO:tasks.workunit.client.0.smithi007.stdout:0/749: write d3/d9/da/d18/d23/d36/f86 [2663895,34801] 0 2024-03-16T09:01:16.800 INFO:tasks.workunit.client.1.smithi019.stdout:3/506: rename d3/d1b/d40/d4a/f41 to d3/d17/d5a/d80/f9c 0 2024-03-16T09:01:16.801 INFO:tasks.workunit.client.1.smithi019.stdout:8/503: rmdir d6/de/d51 39 2024-03-16T09:01:16.801 INFO:tasks.workunit.client.0.smithi007.stdout:9/671: symlink d24/d4c/d86/daa/d7f/d8f/lc1 0 2024-03-16T09:01:16.802 INFO:tasks.workunit.client.1.smithi019.stdout:3/507: link d3/d17/f72 d3/d1b/d40/f9d 0 2024-03-16T09:01:16.804 INFO:tasks.workunit.client.1.smithi019.stdout:8/504: write d6/d2a/d45/d19/d74/f8a [279315,53335] 0 2024-03-16T09:01:16.804 INFO:tasks.workunit.client.1.smithi019.stdout:3/508: truncate d3/d1b/f1c 71304 0 2024-03-16T09:01:16.804 INFO:tasks.workunit.client.1.smithi019.stdout:8/505: fdatasync d6/d2a/d45/d19/f31 0 2024-03-16T09:01:16.804 INFO:tasks.workunit.client.0.smithi007.stdout:4/516: symlink d4/d14/d64/d68/d82/laf 0 2024-03-16T09:01:16.804 INFO:tasks.workunit.client.0.smithi007.stdout:4/517: fsync d4/d14/d64/d9f/f61 0 2024-03-16T09:01:16.804 INFO:tasks.workunit.client.0.smithi007.stdout:4/518: readlink d4/d14/l57 0 2024-03-16T09:01:16.805 INFO:tasks.workunit.client.0.smithi007.stdout:2/737: symlink dc/lee 0 2024-03-16T09:01:16.805 INFO:tasks.workunit.client.1.smithi019.stdout:3/509: creat d3/d1b/f9e x:0 0 0 2024-03-16T09:01:16.805 INFO:tasks.workunit.client.1.smithi019.stdout:3/510: fsync d3/d17/d3f/f7c 0 2024-03-16T09:01:16.805 INFO:tasks.workunit.client.1.smithi019.stdout:3/511: readlink d3/d17/d6a/l7f 0 2024-03-16T09:01:16.806 INFO:tasks.workunit.client.1.smithi019.stdout:8/506: mknod d6/d2a/d3c/dab/d98/cac 0 2024-03-16T09:01:16.806 INFO:tasks.workunit.client.1.smithi019.stdout:8/507: write d6/d2a/d45/d19/f31 [750744,95758] 0 2024-03-16T09:01:16.806 INFO:tasks.workunit.client.1.smithi019.stdout:8/508: read - d6/d2a/d3c/dab/d30/d4a/d67/fa9 zero size 2024-03-16T09:01:16.806 INFO:tasks.workunit.client.1.smithi019.stdout:8/509: write d6/d2a/d45/d19/d74/f92 [2615997,9260] 0 2024-03-16T09:01:16.808 INFO:tasks.workunit.client.0.smithi007.stdout:0/750: rename d3/d9/da/d18/lae to d3/d9/da/d48/d25/d43/d1a/d58/lf6 0 2024-03-16T09:01:16.808 INFO:tasks.workunit.client.1.smithi019.stdout:8/510: creat d6/d2a/d45/d57/d96/fad x:0 0 0 2024-03-16T09:01:16.809 INFO:tasks.workunit.client.1.smithi019.stdout:8/511: symlink d6/d2a/d68/lae 0 2024-03-16T09:01:16.809 INFO:tasks.workunit.client.1.smithi019.stdout:8/512: creat d6/d2a/d45/d5d/faf x:0 0 0 2024-03-16T09:01:16.812 INFO:tasks.workunit.client.1.smithi019.stdout:8/513: mknod d6/d2a/d3c/dab/cb0 0 2024-03-16T09:01:16.813 INFO:tasks.workunit.client.0.smithi007.stdout:4/519: symlink d4/d14/d64/d68/d82/dac/lb0 0 2024-03-16T09:01:16.814 INFO:tasks.workunit.client.1.smithi019.stdout:0/515: write d7/d8/d32/d36/f8f [2545980,65173] 0 2024-03-16T09:01:16.819 INFO:tasks.workunit.client.0.smithi007.stdout:1/576: dread d9/dd/d1b/d2d/d30/f52 [0,4194304] 0 2024-03-16T09:01:16.820 INFO:tasks.workunit.client.0.smithi007.stdout:1/577: write d9/dd/d1b/d2d/d30/f6d [603701,105320] 0 2024-03-16T09:01:16.821 INFO:tasks.workunit.client.0.smithi007.stdout:0/751: rename d3/d9/da/d48/f47 to d3/d6c/dbd/dd7/ff7 0 2024-03-16T09:01:16.825 INFO:tasks.workunit.client.0.smithi007.stdout:3/673: write d15/d18/d1f/d29/d43/f81 [2585765,48665] 0 2024-03-16T09:01:16.826 INFO:tasks.workunit.client.0.smithi007.stdout:2/738: creat dc/df/d24/d28/d2e/d4a/fef x:0 0 0 2024-03-16T09:01:16.829 INFO:tasks.workunit.client.0.smithi007.stdout:1/578: unlink d9/c4e 0 2024-03-16T09:01:16.830 INFO:tasks.workunit.client.0.smithi007.stdout:0/752: mknod d3/d9/da/d18/d23/d36/cf8 0 2024-03-16T09:01:16.831 INFO:tasks.workunit.client.0.smithi007.stdout:2/739: rename dc/df/d24/f63 to dc/df/d24/ff0 0 2024-03-16T09:01:16.833 INFO:tasks.workunit.client.0.smithi007.stdout:2/740: mkdir dc/df/d24/d28/d97/da4/db8/df1 0 2024-03-16T09:01:16.833 INFO:tasks.workunit.client.0.smithi007.stdout:3/674: dread d15/d18/d1f/d3b/d63/f93 [0,4194304] 0 2024-03-16T09:01:16.834 INFO:tasks.workunit.client.0.smithi007.stdout:2/741: symlink dc/df/d24/d28/d2e/d4a/lf2 0 2024-03-16T09:01:16.836 INFO:tasks.workunit.client.0.smithi007.stdout:3/675: rmdir d15/d18/d1f/d3b/d87 39 2024-03-16T09:01:16.836 INFO:tasks.workunit.client.0.smithi007.stdout:4/520: dread d4/f17 [0,4194304] 0 2024-03-16T09:01:16.839 INFO:tasks.workunit.client.0.smithi007.stdout:4/521: rmdir d4/d14/d64/d9f 39 2024-03-16T09:01:16.840 INFO:tasks.workunit.client.0.smithi007.stdout:0/753: dread d3/d1f/f71 [0,4194304] 0 2024-03-16T09:01:16.840 INFO:tasks.workunit.client.0.smithi007.stdout:0/754: write d3/d9/da/d48/f13 [4587050,73726] 0 2024-03-16T09:01:16.849 INFO:tasks.workunit.client.1.smithi019.stdout:6/540: dwrite de/d26/d32/d4e/f9b [0,4194304] 0 2024-03-16T09:01:16.851 INFO:tasks.workunit.client.1.smithi019.stdout:6/541: rmdir de/d26/d32/d97/d66/d72 39 2024-03-16T09:01:16.853 INFO:tasks.workunit.client.1.smithi019.stdout:9/353: dwrite d1/d7/f51 [0,4194304] 0 2024-03-16T09:01:16.853 INFO:tasks.workunit.client.1.smithi019.stdout:9/354: fsync d1/d7/de/d24/f42 0 2024-03-16T09:01:16.853 INFO:tasks.workunit.client.1.smithi019.stdout:6/542: rename de/d1c/l4b to de/d26/d32/d4e/d75/laf 0 2024-03-16T09:01:16.854 INFO:tasks.workunit.client.1.smithi019.stdout:9/355: chown d1/d7/f4c 1 1 2024-03-16T09:01:16.854 INFO:tasks.workunit.client.1.smithi019.stdout:9/356: chown d1/d7/de/d24/d44 12551 1 2024-03-16T09:01:16.856 INFO:tasks.workunit.client.1.smithi019.stdout:5/476: dwrite d6/f74 [0,4194304] 0 2024-03-16T09:01:16.856 INFO:tasks.workunit.client.1.smithi019.stdout:5/477: read - d6/d19/f7f zero size 2024-03-16T09:01:16.859 INFO:tasks.workunit.client.1.smithi019.stdout:9/357: rename d1/d7/l48 to d1/d7/d21/d27/l79 0 2024-03-16T09:01:16.861 INFO:tasks.workunit.client.1.smithi019.stdout:5/478: creat d6/dd/d47/d70/f97 x:0 0 0 2024-03-16T09:01:16.861 INFO:tasks.workunit.client.1.smithi019.stdout:5/479: creat d6/d1f/d33/d4c/f98 x:0 0 0 2024-03-16T09:01:16.861 INFO:tasks.workunit.client.1.smithi019.stdout:5/480: write d6/dd/d37/d54/f80 [202457,40991] 0 2024-03-16T09:01:16.864 INFO:tasks.workunit.client.1.smithi019.stdout:5/481: unlink d6/d1f/d33/d66/l75 0 2024-03-16T09:01:16.864 INFO:tasks.workunit.client.1.smithi019.stdout:5/482: readlink d6/d1f/d78/l8d 0 2024-03-16T09:01:16.872 INFO:tasks.workunit.client.1.smithi019.stdout:5/483: dread d6/dd/f1a [0,4194304] 0 2024-03-16T09:01:16.874 INFO:tasks.workunit.client.0.smithi007.stdout:0/755: dread d3/d9/da/d92/d9f/fc7 [0,4194304] 0 2024-03-16T09:01:16.875 INFO:tasks.workunit.client.1.smithi019.stdout:5/484: link d6/d1f/d33/f52 d6/dd/f99 0 2024-03-16T09:01:16.875 INFO:tasks.workunit.client.1.smithi019.stdout:5/485: write d6/dd/f5c [996762,83274] 0 2024-03-16T09:01:16.875 INFO:tasks.workunit.client.1.smithi019.stdout:5/486: chown d6/c36 213737331 1 2024-03-16T09:01:16.876 INFO:tasks.workunit.client.1.smithi019.stdout:5/487: creat d6/d1f/d78/f9a x:0 0 0 2024-03-16T09:01:16.876 INFO:tasks.workunit.client.1.smithi019.stdout:5/488: creat d6/d19/f9b x:0 0 0 2024-03-16T09:01:16.876 INFO:tasks.workunit.client.1.smithi019.stdout:5/489: chown d6/f35 2347 1 2024-03-16T09:01:16.876 INFO:tasks.workunit.client.0.smithi007.stdout:0/756: creat d3/d9/da/d92/d9f/de8/ff9 x:0 0 0 2024-03-16T09:01:16.876 INFO:tasks.workunit.client.0.smithi007.stdout:0/757: getdents d3/d9/da/d48/d25/d87 0 2024-03-16T09:01:16.881 INFO:tasks.workunit.client.0.smithi007.stdout:0/758: dread d3/d9/da/d92/d9f/fc4 [0,4194304] 0 2024-03-16T09:01:16.881 INFO:tasks.workunit.client.0.smithi007.stdout:0/759: write d3/d9/da/d18/d27/f83 [606497,70663] 0 2024-03-16T09:01:16.881 INFO:tasks.workunit.client.0.smithi007.stdout:0/760: chown d3/d9/da/d48/d25/d43/cc3 42 1 2024-03-16T09:01:16.881 INFO:tasks.workunit.client.1.smithi019.stdout:9/358: dread d1/d7/d21/d27/f22 [0,4194304] 0 2024-03-16T09:01:16.882 INFO:tasks.workunit.client.1.smithi019.stdout:9/359: write d1/d7/de/d24/d5b/f6c [740408,62519] 0 2024-03-16T09:01:16.882 INFO:tasks.workunit.client.0.smithi007.stdout:0/761: mkdir d3/d9/db4/d2c/dfa 0 2024-03-16T09:01:16.882 INFO:tasks.workunit.client.0.smithi007.stdout:0/762: stat d3/d9/db4/fcf 0 2024-03-16T09:01:16.883 INFO:tasks.workunit.client.0.smithi007.stdout:6/600: dwrite d2/d11/d3b/d93/d8e/d4e/f5b [0,4194304] 0 2024-03-16T09:01:16.883 INFO:tasks.workunit.client.1.smithi019.stdout:4/412: dwrite d4/d6/d15/d1f/d2c/f31 [0,4194304] 0 2024-03-16T09:01:16.884 INFO:tasks.workunit.client.1.smithi019.stdout:7/352: dwrite d9/db/d12/d5b/d33/f58 [0,4194304] 0 2024-03-16T09:01:16.884 INFO:tasks.workunit.client.1.smithi019.stdout:9/360: rename d1/d7/de/d24/d44/d76/f30 to d1/d7/d21/d27/f7a 0 2024-03-16T09:01:16.884 INFO:tasks.workunit.client.1.smithi019.stdout:7/353: dread - d9/d25/d28/d48/f4b zero size 2024-03-16T09:01:16.884 INFO:tasks.workunit.client.1.smithi019.stdout:7/354: rename d9/da to d9/da/d13/d18/d1a/d6e/d6f 22 2024-03-16T09:01:16.884 INFO:tasks.workunit.client.1.smithi019.stdout:3/512: dwrite d3/d1b/d40/f9d [0,4194304] 0 2024-03-16T09:01:16.885 INFO:tasks.workunit.client.1.smithi019.stdout:0/516: dwrite d7/d1d/d28/d6c/f8e [0,4194304] 0 2024-03-16T09:01:16.885 INFO:tasks.workunit.client.0.smithi007.stdout:3/676: dwrite d15/d18/d6a/db1/fa3 [0,4194304] 0 2024-03-16T09:01:16.886 INFO:tasks.workunit.client.0.smithi007.stdout:4/522: dwrite d4/d14/d64/d68/d82/f8d [8388608,4194304] 0 2024-03-16T09:01:16.886 INFO:tasks.workunit.client.0.smithi007.stdout:4/523: fdatasync d4/d14/d1c/d5b/f62 0 2024-03-16T09:01:16.889 INFO:tasks.workunit.client.0.smithi007.stdout:2/742: dwrite dc/df/d24/d28/d2e/d35/f5f [0,4194304] 0 2024-03-16T09:01:16.890 INFO:tasks.workunit.client.1.smithi019.stdout:4/413: unlink d4/d1b/d49/c6a 0 2024-03-16T09:01:16.891 INFO:tasks.workunit.client.0.smithi007.stdout:3/677: dread d15/d18/d6a/d78/f9e [0,4194304] 0 2024-03-16T09:01:16.892 INFO:tasks.workunit.client.1.smithi019.stdout:6/543: dwrite de/d26/d32/d97/d51/d70/f91 [0,4194304] 0 2024-03-16T09:01:16.893 INFO:tasks.workunit.client.1.smithi019.stdout:6/544: write de/d1c/f8d [556483,104096] 0 2024-03-16T09:01:16.894 INFO:tasks.workunit.client.1.smithi019.stdout:7/355: mkdir d9/da/d13/d70 0 2024-03-16T09:01:16.894 INFO:tasks.workunit.client.1.smithi019.stdout:7/356: readlink d9/db/d12/d5b/l4e 0 2024-03-16T09:01:16.894 INFO:tasks.workunit.client.1.smithi019.stdout:7/357: write d9/da/d13/f6a [1021752,61019] 0 2024-03-16T09:01:16.894 INFO:tasks.workunit.client.0.smithi007.stdout:2/743: creat dc/df/d24/d28/d97/ff3 x:0 0 0 2024-03-16T09:01:16.896 INFO:tasks.workunit.client.0.smithi007.stdout:4/524: truncate d4/f2f 4373144 0 2024-03-16T09:01:16.896 INFO:tasks.workunit.client.0.smithi007.stdout:9/672: dwrite d24/f54 [4194304,4194304] 0 2024-03-16T09:01:16.896 INFO:tasks.workunit.client.0.smithi007.stdout:6/601: dread d2/d10/f36 [0,4194304] 0 2024-03-16T09:01:16.898 INFO:tasks.workunit.client.0.smithi007.stdout:3/678: symlink d15/d18/d1f/d3b/ld0 0 2024-03-16T09:01:16.901 INFO:tasks.workunit.client.0.smithi007.stdout:2/744: creat dc/df/d24/d28/d2e/d35/da2/ff4 x:0 0 0 2024-03-16T09:01:16.901 INFO:tasks.workunit.client.0.smithi007.stdout:8/955: sync 2024-03-16T09:01:16.901 INFO:tasks.workunit.client.0.smithi007.stdout:8/956: fsync d0/dc/d50/d8e/dd2/dea/f122 0 2024-03-16T09:01:16.903 INFO:tasks.workunit.client.0.smithi007.stdout:4/525: dread d4/f17 [0,4194304] 0 2024-03-16T09:01:16.903 INFO:tasks.workunit.client.0.smithi007.stdout:4/526: fdatasync d4/d14/d1c/d5b/d67/f79 0 2024-03-16T09:01:16.903 INFO:tasks.workunit.client.0.smithi007.stdout:4/527: creat d4/d14/d1c/d5b/d67/fb1 x:0 0 0 2024-03-16T09:01:16.908 INFO:tasks.workunit.client.0.smithi007.stdout:6/602: dread d2/d11/d3b/d93/d8e/d4e/f52 [0,4194304] 0 2024-03-16T09:01:16.911 INFO:tasks.workunit.client.0.smithi007.stdout:0/763: dwrite d3/f6f [0,4194304] 0 2024-03-16T09:01:16.912 INFO:tasks.workunit.client.1.smithi019.stdout:0/517: rename d7/d8/d32/d4a/d61/d68/lab to d7/d8/d32/d4a/d61/lb1 0 2024-03-16T09:01:16.913 INFO:tasks.workunit.client.1.smithi019.stdout:7/358: rename d9/da/d13/d18/d1a to d9/da/d13/d18/d1a/d6e/d71 22 2024-03-16T09:01:16.913 INFO:tasks.workunit.client.1.smithi019.stdout:6/545: dwrite de/d26/d32/d97/d66/d72/f82 [0,4194304] 0 2024-03-16T09:01:16.914 INFO:tasks.workunit.client.0.smithi007.stdout:9/673: creat d24/d4c/d86/daa/d7f/d80/d87/d89/fc2 x:0 0 0 2024-03-16T09:01:16.914 INFO:tasks.workunit.client.0.smithi007.stdout:6/603: read d2/d10/f36 [2597486,70186] 0 2024-03-16T09:01:16.914 INFO:tasks.workunit.client.0.smithi007.stdout:6/604: chown d2/d13/d4b/fa6 231799 1 2024-03-16T09:01:16.914 INFO:tasks.workunit.client.0.smithi007.stdout:6/605: write d2/d29/fab [565705,5367] 0 2024-03-16T09:01:16.915 INFO:tasks.workunit.client.0.smithi007.stdout:3/679: mknod d15/d18/d1f/d29/dc3/cd1 0 2024-03-16T09:01:16.918 INFO:tasks.workunit.client.0.smithi007.stdout:2/745: creat dc/df/d24/d28/d97/da4/db8/df1/ff5 x:0 0 0 2024-03-16T09:01:16.919 INFO:tasks.workunit.client.0.smithi007.stdout:8/957: mkdir d0/dc/d50/d8e/dd2/dea/d7e/d14e 0 2024-03-16T09:01:16.921 INFO:tasks.workunit.client.1.smithi019.stdout:4/414: mkdir d4/d73/d8a 0 2024-03-16T09:01:16.921 INFO:tasks.workunit.client.1.smithi019.stdout:4/415: creat d4/d1b/d2a/d4f/f8b x:0 0 0 2024-03-16T09:01:16.921 INFO:tasks.workunit.client.1.smithi019.stdout:4/416: truncate d4/d6/d15/d1f/d2c/f7b 428947 0 2024-03-16T09:01:16.924 INFO:tasks.workunit.client.1.smithi019.stdout:9/361: rmdir d1/d7/d33 39 2024-03-16T09:01:16.924 INFO:tasks.workunit.client.1.smithi019.stdout:9/362: creat d1/d7/d21/f7b x:0 0 0 2024-03-16T09:01:16.925 INFO:tasks.workunit.client.0.smithi007.stdout:4/528: rename d4/d14/d1c/d5b/d67/f79 to d4/d14/fb2 0 2024-03-16T09:01:16.925 INFO:tasks.workunit.client.0.smithi007.stdout:4/529: chown d4/d14/d1c/f3e 14421579 1 2024-03-16T09:01:16.925 INFO:tasks.workunit.client.0.smithi007.stdout:4/530: chown d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/fa9 18 1 2024-03-16T09:01:16.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/764: write d3/d9/da/d18/f59 [1195012,76477] 0 2024-03-16T09:01:16.927 INFO:tasks.workunit.client.0.smithi007.stdout:8/958: mkdir d0/dc/d50/d8e/dd2/dea/d18/da7/de2/d14f 0 2024-03-16T09:01:16.928 INFO:tasks.workunit.client.1.smithi019.stdout:0/518: mknod d7/d8/d32/d66/d6d/d8c/cb2 0 2024-03-16T09:01:16.929 INFO:tasks.workunit.client.1.smithi019.stdout:7/359: symlink d9/db/l72 0 2024-03-16T09:01:16.931 INFO:tasks.workunit.client.0.smithi007.stdout:9/674: rename d24/d4a/fb4 to d24/d4a/da2/db7/fc3 0 2024-03-16T09:01:16.931 INFO:tasks.workunit.client.0.smithi007.stdout:9/675: write d24/d4c/d91/fa5 [809243,122457] 0 2024-03-16T09:01:16.932 INFO:tasks.workunit.client.1.smithi019.stdout:6/546: rmdir de/d26/d32/d97/d2c 39 2024-03-16T09:01:16.933 INFO:tasks.workunit.client.0.smithi007.stdout:0/765: symlink d3/d6c/dbd/dd7/d7c/lfb 0 2024-03-16T09:01:16.933 INFO:tasks.workunit.client.0.smithi007.stdout:0/766: stat d3/d6c/dbd/dd7/d7c/c9c 0 2024-03-16T09:01:16.934 INFO:tasks.workunit.client.0.smithi007.stdout:8/959: truncate d0/d26/d2d/ff2 366976 0 2024-03-16T09:01:16.934 INFO:tasks.workunit.client.0.smithi007.stdout:4/531: write d4/d14/d1c/d5b/f74 [321766,89552] 0 2024-03-16T09:01:16.935 INFO:tasks.workunit.client.1.smithi019.stdout:7/360: write d9/db/d12/f2f [584413,47927] 0 2024-03-16T09:01:16.936 INFO:tasks.workunit.client.1.smithi019.stdout:7/361: creat d9/db/d12/f73 x:0 0 0 2024-03-16T09:01:16.937 INFO:tasks.workunit.client.1.smithi019.stdout:4/417: mknod d4/d1b/d49/d58/d80/c8c 0 2024-03-16T09:01:16.937 INFO:tasks.workunit.client.1.smithi019.stdout:4/418: readlink d4/d1b/d2a/d3d/l78 0 2024-03-16T09:01:16.937 INFO:tasks.workunit.client.1.smithi019.stdout:4/419: write d4/d6/d15/d1f/d2c/f3b [495645,45136] 0 2024-03-16T09:01:16.937 INFO:tasks.workunit.client.1.smithi019.stdout:4/420: fsync d4/d6/d15/f2e 0 2024-03-16T09:01:16.938 INFO:tasks.workunit.client.0.smithi007.stdout:6/606: dwrite d2/d11/d1e/f2f [4194304,4194304] 0 2024-03-16T09:01:16.938 INFO:tasks.workunit.client.0.smithi007.stdout:6/607: write d2/d11/d3b/d93/d8e/d4e/f64 [1167143,92898] 0 2024-03-16T09:01:16.938 INFO:tasks.workunit.client.0.smithi007.stdout:6/608: stat d2/d13/d3f/d50/d72/fa4 0 2024-03-16T09:01:16.939 INFO:tasks.workunit.client.0.smithi007.stdout:8/960: rmdir d0/dbd 39 2024-03-16T09:01:16.939 INFO:tasks.workunit.client.1.smithi019.stdout:9/363: mknod d1/d7/de/d24/c7c 0 2024-03-16T09:01:16.940 INFO:tasks.workunit.client.1.smithi019.stdout:9/364: creat d1/d7/de/d24/d44/f7d x:0 0 0 2024-03-16T09:01:16.940 INFO:tasks.workunit.client.0.smithi007.stdout:9/676: link f8 d24/fc4 0 2024-03-16T09:01:16.941 INFO:tasks.workunit.client.1.smithi019.stdout:6/547: truncate de/f45 358311 0 2024-03-16T09:01:16.944 INFO:tasks.workunit.client.1.smithi019.stdout:4/421: dread d4/d6/d15/d1f/d2c/f40 [0,4194304] 0 2024-03-16T09:01:16.944 INFO:tasks.workunit.client.1.smithi019.stdout:7/362: rmdir d9/da/d13/d18 39 2024-03-16T09:01:16.944 INFO:tasks.workunit.client.0.smithi007.stdout:6/609: creat d2/d11/d1e/d54/d9c/fc6 x:0 0 0 2024-03-16T09:01:16.948 INFO:tasks.workunit.client.0.smithi007.stdout:9/677: link d24/f26 d24/d4c/d86/daa/d7f/d80/d87/fc5 0 2024-03-16T09:01:16.948 INFO:tasks.workunit.client.1.smithi019.stdout:4/422: dread d4/d6/d15/f2e [0,4194304] 0 2024-03-16T09:01:16.950 INFO:tasks.workunit.client.0.smithi007.stdout:4/532: creat d4/d99/fb3 x:0 0 0 2024-03-16T09:01:16.950 INFO:tasks.workunit.client.0.smithi007.stdout:4/533: chown d4/d14/d1c/d5b/d67/d8f/da8/d6b/l7b 1496677 1 2024-03-16T09:01:16.950 INFO:tasks.workunit.client.0.smithi007.stdout:4/534: write d4/d14/d1c/d5b/d67/d8f/da8/f8a [952095,42989] 0 2024-03-16T09:01:16.950 INFO:tasks.workunit.client.0.smithi007.stdout:4/535: fsync d4/d14/f19 0 2024-03-16T09:01:16.952 INFO:tasks.workunit.client.1.smithi019.stdout:6/548: truncate de/d26/d32/d97/f5a 2694744 0 2024-03-16T09:01:16.952 INFO:tasks.workunit.client.1.smithi019.stdout:6/549: dread - de/d26/d32/f9c zero size 2024-03-16T09:01:16.952 INFO:tasks.workunit.client.1.smithi019.stdout:6/550: read - de/d26/d32/d97/da1/fa6 zero size 2024-03-16T09:01:16.953 INFO:tasks.workunit.client.0.smithi007.stdout:8/961: creat d0/d26/d2d/d43/d70/d10f/f150 x:0 0 0 2024-03-16T09:01:16.955 INFO:tasks.workunit.client.1.smithi019.stdout:7/363: unlink d9/db/d12/c19 0 2024-03-16T09:01:16.955 INFO:tasks.workunit.client.1.smithi019.stdout:7/364: dread - d9/d25/d28/f69 zero size 2024-03-16T09:01:16.955 INFO:tasks.workunit.client.1.smithi019.stdout:7/365: fsync d9/db/d12/d5b/d33/f43 0 2024-03-16T09:01:16.956 INFO:tasks.workunit.client.0.smithi007.stdout:0/767: dwrite d3/d9/db4/fcf [4194304,4194304] 0 2024-03-16T09:01:16.956 INFO:tasks.workunit.client.0.smithi007.stdout:9/678: symlink d24/d4c/d86/daa/d7f/d80/lc6 0 2024-03-16T09:01:16.956 INFO:tasks.workunit.client.0.smithi007.stdout:9/679: chown d24/d4c/d86/daa/d95/f68 2906 1 2024-03-16T09:01:16.957 INFO:tasks.workunit.client.1.smithi019.stdout:4/423: mknod d4/d1b/d49/d6e/c8d 0 2024-03-16T09:01:16.957 INFO:tasks.workunit.client.1.smithi019.stdout:9/365: mknod d1/d7/c7e 0 2024-03-16T09:01:16.958 INFO:tasks.workunit.client.0.smithi007.stdout:4/536: creat d4/d14/d1c/d5b/d67/d8f/da8/d9d/fb4 x:0 0 0 2024-03-16T09:01:16.958 INFO:tasks.workunit.client.0.smithi007.stdout:4/537: chown d4/d14/d64/d9f 3460370 1 2024-03-16T09:01:16.959 INFO:tasks.workunit.client.1.smithi019.stdout:7/366: creat d9/f74 x:0 0 0 2024-03-16T09:01:16.959 INFO:tasks.workunit.client.1.smithi019.stdout:7/367: fsync d9/d25/d28/f60 0 2024-03-16T09:01:16.961 INFO:tasks.workunit.client.0.smithi007.stdout:8/962: creat d0/d26/d2d/d64/f151 x:0 0 0 2024-03-16T09:01:16.962 INFO:tasks.workunit.client.1.smithi019.stdout:4/424: mkdir d4/d1b/d49/d58/d80/d8e 0 2024-03-16T09:01:16.962 INFO:tasks.workunit.client.1.smithi019.stdout:4/425: chown d4/d6/f24 711 1 2024-03-16T09:01:16.962 INFO:tasks.workunit.client.1.smithi019.stdout:6/551: read de/d26/d32/d97/d2c/f3a [4770513,11680] 0 2024-03-16T09:01:16.965 INFO:tasks.workunit.client.0.smithi007.stdout:0/768: symlink d3/d9/da/d92/d9f/de8/lfc 0 2024-03-16T09:01:16.966 INFO:tasks.workunit.client.1.smithi019.stdout:7/368: unlink d9/da/d13/d18/d30/f4a 0 2024-03-16T09:01:16.966 INFO:tasks.workunit.client.1.smithi019.stdout:7/369: dread - d9/d25/d28/f57 zero size 2024-03-16T09:01:16.966 INFO:tasks.workunit.client.1.smithi019.stdout:6/552: symlink de/d26/d32/d97/d66/da4/lb0 0 2024-03-16T09:01:16.968 INFO:tasks.workunit.client.1.smithi019.stdout:9/366: dread d1/d7/de/d24/d44/f53 [0,4194304] 0 2024-03-16T09:01:16.968 INFO:tasks.workunit.client.0.smithi007.stdout:9/680: readlink d24/l35 0 2024-03-16T09:01:16.969 INFO:tasks.workunit.client.0.smithi007.stdout:8/963: rename d0/dc/dd/ce6 to d0/dc/dd/d139/c152 0 2024-03-16T09:01:16.974 INFO:tasks.workunit.client.0.smithi007.stdout:0/769: symlink d3/d9/da/d48/d25/d43/lfd 0 2024-03-16T09:01:16.974 INFO:tasks.workunit.client.0.smithi007.stdout:9/681: rmdir d24/d4c/d86/daa/d7f/d80/d87 39 2024-03-16T09:01:16.977 INFO:tasks.workunit.client.1.smithi019.stdout:7/370: dread f7 [0,4194304] 0 2024-03-16T09:01:16.977 INFO:tasks.workunit.client.1.smithi019.stdout:7/371: dread - d9/f74 zero size 2024-03-16T09:01:16.977 INFO:tasks.workunit.client.1.smithi019.stdout:7/372: read - d9/db/d12/d5b/f65 zero size 2024-03-16T09:01:16.977 INFO:tasks.workunit.client.1.smithi019.stdout:7/373: readlink d9/da/d13/d18/l40 0 2024-03-16T09:01:16.978 INFO:tasks.workunit.client.0.smithi007.stdout:8/964: creat d0/d26/d2d/d43/d6f/f153 x:0 0 0 2024-03-16T09:01:16.978 INFO:tasks.workunit.client.0.smithi007.stdout:8/965: read - d0/dc/d50/d8e/fa8 zero size 2024-03-16T09:01:16.980 INFO:tasks.workunit.client.1.smithi019.stdout:7/374: mkdir d9/da/d13/d18/d30/d75 0 2024-03-16T09:01:16.981 INFO:tasks.workunit.client.0.smithi007.stdout:8/966: creat d0/d26/d2d/d64/d123/f154 x:0 0 0 2024-03-16T09:01:16.986 INFO:tasks.workunit.client.0.smithi007.stdout:8/967: mknod d0/dc/dd/d8b/c155 0 2024-03-16T09:01:16.987 INFO:tasks.workunit.client.0.smithi007.stdout:4/538: dread d4/d14/d1c/f3b [0,4194304] 0 2024-03-16T09:01:16.990 INFO:tasks.workunit.client.0.smithi007.stdout:8/968: link d0/dc/d50/d8e/dd2/dea/d18/da7/dc5/d130/f146 d0/dc/d75/f156 0 2024-03-16T09:01:16.993 INFO:tasks.workunit.client.0.smithi007.stdout:5/618: sync 2024-03-16T09:01:16.993 INFO:tasks.workunit.client.0.smithi007.stdout:7/557: sync 2024-03-16T09:01:16.993 INFO:tasks.workunit.client.0.smithi007.stdout:4/539: creat d4/d14/d64/fb5 x:0 0 0 2024-03-16T09:01:16.993 INFO:tasks.workunit.client.0.smithi007.stdout:1/579: write d9/dd/d1b/d28/d29/f47 [559284,23961] 0 2024-03-16T09:01:16.994 INFO:tasks.workunit.client.0.smithi007.stdout:8/969: mknod d0/d26/d2d/d43/d4c/dba/c157 0 2024-03-16T09:01:16.995 INFO:tasks.workunit.client.0.smithi007.stdout:4/540: link d4/d14/d1c/f6a d4/d14/d64/d68/d82/dac/fb6 0 2024-03-16T09:01:16.998 INFO:tasks.workunit.client.0.smithi007.stdout:7/558: link d0/d19/d1f/d5d/d8a/lc0 d0/d6c/d25/d3a/daf/dc2/lc3 0 2024-03-16T09:01:17.001 INFO:tasks.workunit.client.0.smithi007.stdout:5/619: symlink d1/d7/d1c/d19/d34/de/le2 0 2024-03-16T09:01:17.001 INFO:tasks.workunit.client.1.smithi019.stdout:9/367: dwrite d1/d7/de/d24/d5b/f6c [0,4194304] 0 2024-03-16T09:01:17.002 INFO:tasks.workunit.client.0.smithi007.stdout:8/970: mkdir d0/d26/d2d/d64/d123/d158 0 2024-03-16T09:01:17.004 INFO:tasks.workunit.client.0.smithi007.stdout:4/541: symlink d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/d93/lb7 0 2024-03-16T09:01:17.005 INFO:tasks.workunit.client.1.smithi019.stdout:9/368: mknod d1/d7/de/d24/d5b/c7f 0 2024-03-16T09:01:17.005 INFO:tasks.workunit.client.1.smithi019.stdout:9/369: readlink d1/d7/d21/d27/l79 0 2024-03-16T09:01:17.005 INFO:tasks.workunit.client.1.smithi019.stdout:4/426: dwrite d4/d1b/f57 [0,4194304] 0 2024-03-16T09:01:17.007 INFO:tasks.workunit.client.1.smithi019.stdout:7/375: dwrite d9/da/d13/f17 [0,4194304] 0 2024-03-16T09:01:17.007 INFO:tasks.workunit.client.1.smithi019.stdout:7/376: write f8 [3585067,13227] 0 2024-03-16T09:01:17.007 INFO:tasks.workunit.client.1.smithi019.stdout:6/553: dwrite de/d26/d32/d4e/f79 [0,4194304] 0 2024-03-16T09:01:17.007 INFO:tasks.workunit.client.1.smithi019.stdout:6/554: chown de/d26/d32/d4e/d75 1166 1 2024-03-16T09:01:17.007 INFO:tasks.workunit.client.1.smithi019.stdout:9/370: creat d1/d7/de/d24/d44/d60/d74/f80 x:0 0 0 2024-03-16T09:01:17.007 INFO:tasks.workunit.client.1.smithi019.stdout:9/371: stat d1/d7/d21/d27/f35 0 2024-03-16T09:01:17.008 INFO:tasks.workunit.client.0.smithi007.stdout:7/559: stat d0/dc/c79 0 2024-03-16T09:01:17.009 INFO:tasks.workunit.client.1.smithi019.stdout:7/377: stat d9/c29 0 2024-03-16T09:01:17.009 INFO:tasks.workunit.client.0.smithi007.stdout:5/620: creat d1/d7/d16/fe3 x:0 0 0 2024-03-16T09:01:17.010 INFO:tasks.workunit.client.0.smithi007.stdout:6/610: truncate d2/d29/fab 138537 0 2024-03-16T09:01:17.012 INFO:tasks.workunit.client.0.smithi007.stdout:4/542: write d4/f4d [4166580,124949] 0 2024-03-16T09:01:17.020 INFO:tasks.workunit.client.1.smithi019.stdout:9/372: read d1/d7/d21/f38 [132819,72123] 0 2024-03-16T09:01:17.020 INFO:tasks.workunit.client.1.smithi019.stdout:9/373: write d1/d7/de/d24/d5b/f6e [2635479,18849] 0 2024-03-16T09:01:17.023 INFO:tasks.workunit.client.1.smithi019.stdout:6/555: read de/d26/d32/d97/d66/f94 [2189813,34421] 0 2024-03-16T09:01:17.023 INFO:tasks.workunit.client.1.smithi019.stdout:6/556: chown de/d26/d32/d4e/d75/cab 2 1 2024-03-16T09:01:17.023 INFO:tasks.workunit.client.1.smithi019.stdout:6/557: truncate de/d26/d73/fa9 264173 0 2024-03-16T09:01:17.023 INFO:tasks.workunit.client.1.smithi019.stdout:6/558: write de/d26/d73/fa9 [314374,41181] 0 2024-03-16T09:01:17.023 INFO:tasks.workunit.client.1.smithi019.stdout:9/374: truncate d1/d7/d21/f38 1818597 0 2024-03-16T09:01:17.023 INFO:tasks.workunit.client.1.smithi019.stdout:9/375: chown d1/d7/de/d24 247 1 2024-03-16T09:01:17.024 INFO:tasks.workunit.client.1.smithi019.stdout:6/559: mkdir de/d26/d32/d97/dae/db1 0 2024-03-16T09:01:17.025 INFO:tasks.workunit.client.0.smithi007.stdout:5/621: write d1/d7/d1c/f80 [420378,45869] 0 2024-03-16T09:01:17.025 INFO:tasks.workunit.client.0.smithi007.stdout:1/580: dwrite d9/f22 [0,4194304] 0 2024-03-16T09:01:17.025 INFO:tasks.workunit.client.1.smithi019.stdout:9/376: creat d1/d7/f81 x:0 0 0 2024-03-16T09:01:17.026 INFO:tasks.workunit.client.1.smithi019.stdout:6/560: rename de/d26/d32/d97/l3f to de/d26/d32/d4e/d75/lb2 0 2024-03-16T09:01:17.027 INFO:tasks.workunit.client.1.smithi019.stdout:4/427: dread d4/d1b/d49/f75 [0,4194304] 0 2024-03-16T09:01:17.027 INFO:tasks.workunit.client.1.smithi019.stdout:4/428: read d4/d6/f24 [123822,64754] 0 2024-03-16T09:01:17.028 INFO:tasks.workunit.client.0.smithi007.stdout:7/560: rename d0/d6c/d25/f90 to d0/d19/d1f/d5d/fc4 0 2024-03-16T09:01:17.028 INFO:tasks.workunit.client.1.smithi019.stdout:9/377: link d1/d7/d21/f7b d1/d7/d67/f82 0 2024-03-16T09:01:17.030 INFO:tasks.workunit.client.1.smithi019.stdout:4/429: rename d4/d6/f24 to d4/d1b/f8f 0 2024-03-16T09:01:17.034 INFO:tasks.workunit.client.0.smithi007.stdout:4/543: truncate d4/d14/d1c/d5b/f62 3144889 0 2024-03-16T09:01:17.034 INFO:tasks.workunit.client.0.smithi007.stdout:4/544: creat d4/d14/d1c/d5b/d67/d8f/d90/fb8 x:0 0 0 2024-03-16T09:01:17.034 INFO:tasks.workunit.client.0.smithi007.stdout:4/545: write d4/d14/d1c/d5b/f8e [503584,62974] 0 2024-03-16T09:01:17.035 INFO:tasks.workunit.client.1.smithi019.stdout:9/378: symlink d1/d7/d33/d59/l83 0 2024-03-16T09:01:17.035 INFO:tasks.workunit.client.1.smithi019.stdout:6/561: dread de/f14 [0,4194304] 0 2024-03-16T09:01:17.036 INFO:tasks.workunit.client.1.smithi019.stdout:4/430: rename d4/d6/f32 to d4/d1b/d49/d6e/f90 0 2024-03-16T09:01:17.036 INFO:tasks.workunit.client.1.smithi019.stdout:6/562: mknod de/d26/cb3 0 2024-03-16T09:01:17.038 INFO:tasks.workunit.client.1.smithi019.stdout:6/563: mknod de/d26/d32/d4e/d75/cb4 0 2024-03-16T09:01:17.039 INFO:tasks.workunit.client.0.smithi007.stdout:5/622: rename d1/d7/d1c/d88/da2/d89/db3/d2a/d79/c83 to d1/d7/d1c/d19/d34/d5e/ce4 0 2024-03-16T09:01:17.041 INFO:tasks.workunit.client.0.smithi007.stdout:7/561: symlink d0/d19/d1f/lc5 0 2024-03-16T09:01:17.044 INFO:tasks.workunit.client.0.smithi007.stdout:4/546: symlink d4/d14/d64/d68/d82/dac/lb9 0 2024-03-16T09:01:17.044 INFO:tasks.workunit.client.0.smithi007.stdout:4/547: chown d4/d14/f96 36 1 2024-03-16T09:01:17.047 INFO:tasks.workunit.client.0.smithi007.stdout:5/623: truncate d1/d7/d1c/d19/d34/d5e/d6f/f54 173258 0 2024-03-16T09:01:17.047 INFO:tasks.workunit.client.0.smithi007.stdout:5/624: fdatasync d1/d7/d1c/d19/d34/d5e/d6f/d72/f85 0 2024-03-16T09:01:17.048 INFO:tasks.workunit.client.0.smithi007.stdout:5/625: readlink d1/d7/d1c/d19/d34/l5d 0 2024-03-16T09:01:17.048 INFO:tasks.workunit.client.1.smithi019.stdout:4/431: write d4/d1b/d2a/f4e [6412489,90503] 0 2024-03-16T09:01:17.050 INFO:tasks.workunit.client.0.smithi007.stdout:7/562: creat d0/d19/d1f/d69/fc6 x:0 0 0 2024-03-16T09:01:17.050 INFO:tasks.workunit.client.0.smithi007.stdout:7/563: chown d0/d6c/fb5 386 1 2024-03-16T09:01:17.051 INFO:tasks.workunit.client.0.smithi007.stdout:4/548: symlink d4/d14/d64/d68/lba 0 2024-03-16T09:01:17.052 INFO:tasks.workunit.client.0.smithi007.stdout:5/626: creat d1/d7/d1c/d19/d34/d3e/fe5 x:0 0 0 2024-03-16T09:01:17.052 INFO:tasks.workunit.client.0.smithi007.stdout:5/627: fdatasync d1/d7/d1c/d19/fad 0 2024-03-16T09:01:17.052 INFO:tasks.workunit.client.0.smithi007.stdout:5/628: readlink d1/d7/d1c/d19/d34/l5d 0 2024-03-16T09:01:17.054 INFO:tasks.workunit.client.0.smithi007.stdout:5/629: getdents d1/d7/d1c/d88 0 2024-03-16T09:01:17.057 INFO:tasks.workunit.client.0.smithi007.stdout:5/630: symlink d1/d7/d1c/d88/da2/d89/d9b/le6 0 2024-03-16T09:01:17.059 INFO:tasks.workunit.client.0.smithi007.stdout:5/631: mkdir d1/d7/d1c/d19/d34/d4d/de7 0 2024-03-16T09:01:17.059 INFO:tasks.workunit.client.0.smithi007.stdout:5/632: chown d1/d7/d1c/d88/da2/d89/db3/fa0 481210 1 2024-03-16T09:01:17.061 INFO:tasks.workunit.client.0.smithi007.stdout:5/633: unlink d1/d7/d1c/d19/d34/d1d/f4f 0 2024-03-16T09:01:17.063 INFO:tasks.workunit.client.1.smithi019.stdout:6/564: dwrite de/d26/d32/d97/f5a [0,4194304] 0 2024-03-16T09:01:17.063 INFO:tasks.workunit.client.1.smithi019.stdout:6/565: getdents de/d26/d32/d97/dae/db1 0 2024-03-16T09:01:17.063 INFO:tasks.workunit.client.1.smithi019.stdout:7/378: dwrite d9/db/d12/d5b/d33/f43 [0,4194304] 0 2024-03-16T09:01:17.065 INFO:tasks.workunit.client.0.smithi007.stdout:4/549: getdents d4/d14/d64 0 2024-03-16T09:01:17.066 INFO:tasks.workunit.client.1.smithi019.stdout:6/566: getdents de/d26/d32/d97/d51 0 2024-03-16T09:01:17.067 INFO:tasks.workunit.client.0.smithi007.stdout:6/611: dwrite d2/d13/d62/f68 [0,4194304] 0 2024-03-16T09:01:17.068 INFO:tasks.workunit.client.0.smithi007.stdout:4/550: mkdir d4/d14/d1c/d5b/d67/d8f/da8/d9d/dbb 0 2024-03-16T09:01:17.068 INFO:tasks.workunit.client.0.smithi007.stdout:4/551: creat d4/d99/fbc x:0 0 0 2024-03-16T09:01:17.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/379: creat d9/d25/d28/d48/f76 x:0 0 0 2024-03-16T09:01:17.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/380: write d9/db/d12/d5b/f38 [1314780,58355] 0 2024-03-16T09:01:17.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/381: dread - d9/db/f2c zero size 2024-03-16T09:01:17.073 INFO:tasks.workunit.client.1.smithi019.stdout:7/382: rename d9/da/d13/f6a to d9/da/d13/d18/d1a/d6e/f77 0 2024-03-16T09:01:17.073 INFO:tasks.workunit.client.1.smithi019.stdout:7/383: creat d9/db/d12/d5b/f78 x:0 0 0 2024-03-16T09:01:17.075 INFO:tasks.workunit.client.1.smithi019.stdout:9/379: dread d1/d7/de/d24/d5b/f6e [0,4194304] 0 2024-03-16T09:01:17.075 INFO:tasks.workunit.client.1.smithi019.stdout:9/380: creat d1/d7/d21/f84 x:0 0 0 2024-03-16T09:01:17.075 INFO:tasks.workunit.client.1.smithi019.stdout:9/381: write d1/d7/d33/d59/f73 [288851,43690] 0 2024-03-16T09:01:17.078 INFO:tasks.workunit.client.1.smithi019.stdout:9/382: rename d1/d7/d21/f84 to d1/d7/d33/d59/f85 0 2024-03-16T09:01:17.078 INFO:tasks.workunit.client.1.smithi019.stdout:6/567: write de/d26/d32/d97/d51/d70/f91 [587936,77328] 0 2024-03-16T09:01:17.081 INFO:tasks.workunit.client.0.smithi007.stdout:4/552: dread d4/d14/d64/d9f/f61 [0,4194304] 0 2024-03-16T09:01:17.082 INFO:tasks.workunit.client.1.smithi019.stdout:9/383: mkdir d1/d7/d86 0 2024-03-16T09:01:17.082 INFO:tasks.workunit.client.1.smithi019.stdout:9/384: chown d1/d7/de/d24/d44/d76/l49 79161 1 2024-03-16T09:01:17.085 INFO:tasks.workunit.client.1.smithi019.stdout:4/432: dwrite d4/d6/d15/d1f/f44 [0,4194304] 0 2024-03-16T09:01:17.086 INFO:tasks.workunit.client.0.smithi007.stdout:7/564: dwrite d0/f49 [0,4194304] 0 2024-03-16T09:01:17.086 INFO:tasks.workunit.client.0.smithi007.stdout:4/553: creat d4/d14/d64/d9f/fbd x:0 0 0 2024-03-16T09:01:17.086 INFO:tasks.workunit.client.0.smithi007.stdout:4/554: chown d4/d14/d1c/d5b/d67/d8f/d90/f91 7426 1 2024-03-16T09:01:17.088 INFO:tasks.workunit.client.0.smithi007.stdout:5/634: dwrite d1/d7/d1c/d19/d34/d3e/d41/f4c [0,4194304] 0 2024-03-16T09:01:17.091 INFO:tasks.workunit.client.0.smithi007.stdout:4/555: write d4/ff [1087104,93708] 0 2024-03-16T09:01:17.092 INFO:tasks.workunit.client.1.smithi019.stdout:6/568: dread de/d26/d32/d97/f36 [0,4194304] 0 2024-03-16T09:01:17.103 INFO:tasks.workunit.client.1.smithi019.stdout:2/523: sync 2024-03-16T09:01:17.103 INFO:tasks.workunit.client.1.smithi019.stdout:3/513: sync 2024-03-16T09:01:17.103 INFO:tasks.workunit.client.1.smithi019.stdout:3/514: stat d3/c8 0 2024-03-16T09:01:17.103 INFO:tasks.workunit.client.1.smithi019.stdout:5/490: sync 2024-03-16T09:01:17.103 INFO:tasks.workunit.client.1.smithi019.stdout:0/519: sync 2024-03-16T09:01:17.103 INFO:tasks.workunit.client.1.smithi019.stdout:8/514: sync 2024-03-16T09:01:17.103 INFO:tasks.workunit.client.1.smithi019.stdout:5/491: write d6/dd/f82 [5025266,85031] 0 2024-03-16T09:01:17.103 INFO:tasks.workunit.client.1.smithi019.stdout:1/407: sync 2024-03-16T09:01:17.106 INFO:tasks.workunit.client.1.smithi019.stdout:2/524: getdents d5/d4e/d74 0 2024-03-16T09:01:17.108 INFO:tasks.workunit.client.0.smithi007.stdout:5/635: dread d1/d7/d1c/d19/d34/d5e/d6f/d72/fab [4194304,4194304] 0 2024-03-16T09:01:17.108 INFO:tasks.workunit.client.0.smithi007.stdout:5/636: write d1/d7/d1c/d19/d34/d5e/d6f/d72/fd3 [705551,15703] 0 2024-03-16T09:01:17.108 INFO:tasks.workunit.client.0.smithi007.stdout:5/637: stat d1/d7/f12 0 2024-03-16T09:01:17.108 INFO:tasks.workunit.client.0.smithi007.stdout:5/638: write d1/d7/d16/d1b/f8d [2451757,12710] 0 2024-03-16T09:01:17.110 INFO:tasks.workunit.client.1.smithi019.stdout:8/515: dread d6/de/f6a [0,4194304] 0 2024-03-16T09:01:17.110 INFO:tasks.workunit.client.0.smithi007.stdout:5/639: rmdir d1/d7/d1c/d88/da2/d89/d9b 39 2024-03-16T09:01:17.112 INFO:tasks.workunit.client.0.smithi007.stdout:5/640: mkdir d1/d7/d1c/d19/d34/d3e/d41/da6/de8 0 2024-03-16T09:01:17.112 INFO:tasks.workunit.client.0.smithi007.stdout:5/641: readlink d1/d7/l42 0 2024-03-16T09:01:17.112 INFO:tasks.workunit.client.1.smithi019.stdout:2/525: rename d5/de/f45 to d5/d4e/d74/fae 0 2024-03-16T09:01:17.113 INFO:tasks.workunit.client.0.smithi007.stdout:5/642: truncate d1/d7/d1c/d19/d34/d4d/d99/fd7 538270 0 2024-03-16T09:01:17.114 INFO:tasks.workunit.client.1.smithi019.stdout:8/516: unlink d6/d2a/d3c/dab/d30/d4a/l53 0 2024-03-16T09:01:17.116 INFO:tasks.workunit.client.0.smithi007.stdout:5/643: mkdir d1/d7/d1c/d19/d34/d3e/d41/da6/de8/de9 0 2024-03-16T09:01:17.116 INFO:tasks.workunit.client.0.smithi007.stdout:5/644: dread - d1/d7/d1c/d19/d34/d3e/fe5 zero size 2024-03-16T09:01:17.116 INFO:tasks.workunit.client.0.smithi007.stdout:4/556: write d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/d89/fad [1665740,32076] 0 2024-03-16T09:01:17.118 INFO:tasks.workunit.client.1.smithi019.stdout:2/526: creat d5/de/faf x:0 0 0 2024-03-16T09:01:17.118 INFO:tasks.workunit.client.1.smithi019.stdout:2/527: write d5/d2e/f9a [382442,22501] 0 2024-03-16T09:01:17.118 INFO:tasks.workunit.client.1.smithi019.stdout:2/528: chown d5/f23 3 1 2024-03-16T09:01:17.119 INFO:tasks.workunit.client.0.smithi007.stdout:5/645: truncate d1/d7/d1c/d19/d34/f8c 658996 0 2024-03-16T09:01:17.120 INFO:tasks.workunit.client.1.smithi019.stdout:3/515: dread d3/d17/d5a/f8c [0,4194304] 0 2024-03-16T09:01:17.120 INFO:tasks.workunit.client.1.smithi019.stdout:8/517: symlink d6/lb1 0 2024-03-16T09:01:17.120 INFO:tasks.workunit.client.1.smithi019.stdout:8/518: write d6/d2a/d45/d5d/fa7 [263326,52357] 0 2024-03-16T09:01:17.122 INFO:tasks.workunit.client.0.smithi007.stdout:4/557: mkdir d4/d14/d1c/d5b/d67/d8f/da8/dbe 0 2024-03-16T09:01:17.122 INFO:tasks.workunit.client.1.smithi019.stdout:3/516: creat d3/d9b/f9f x:0 0 0 2024-03-16T09:01:17.123 INFO:tasks.workunit.client.0.smithi007.stdout:5/646: creat d1/d7/d1c/d88/da2/d89/db3/fea x:0 0 0 2024-03-16T09:01:17.124 INFO:tasks.workunit.client.0.smithi007.stdout:5/647: getdents d1/d7/d1c/d88/da2/d89/db3/d7d 0 2024-03-16T09:01:17.124 INFO:tasks.workunit.client.0.smithi007.stdout:4/558: mkdir d4/d14/d64/d68/dbf 0 2024-03-16T09:01:17.124 INFO:tasks.workunit.client.1.smithi019.stdout:8/519: creat d6/de/fb2 x:0 0 0 2024-03-16T09:01:17.125 INFO:tasks.workunit.client.1.smithi019.stdout:5/492: write d6/d19/f4e [6319437,117976] 0 2024-03-16T09:01:17.126 INFO:tasks.workunit.client.1.smithi019.stdout:5/493: stat d6/dd/c25 0 2024-03-16T09:01:17.126 INFO:tasks.workunit.client.1.smithi019.stdout:7/384: dwrite d9/da/d13/d18/d1a/f2b [4194304,4194304] 0 2024-03-16T09:01:17.126 INFO:tasks.workunit.client.1.smithi019.stdout:6/569: dwrite de/d26/d32/d4e/f59 [0,4194304] 0 2024-03-16T09:01:17.126 INFO:tasks.workunit.client.1.smithi019.stdout:8/520: rename d6/d2a/f85 to d6/d2a/d45/d5d/fb3 0 2024-03-16T09:01:17.127 INFO:tasks.workunit.client.1.smithi019.stdout:4/433: dwrite d4/d1b/d49/f72 [0,4194304] 0 2024-03-16T09:01:17.127 INFO:tasks.workunit.client.0.smithi007.stdout:5/648: unlink d1/d7/d1c/d19/d34/d5e/ce4 0 2024-03-16T09:01:17.128 INFO:tasks.workunit.client.1.smithi019.stdout:9/385: dwrite d1/d7/d21/d27/f3c [0,4194304] 0 2024-03-16T09:01:17.128 INFO:tasks.workunit.client.1.smithi019.stdout:3/517: write d3/d17/f59 [1095918,54106] 0 2024-03-16T09:01:17.128 INFO:tasks.workunit.client.1.smithi019.stdout:5/494: creat d6/d1f/d77/f9c x:0 0 0 2024-03-16T09:01:17.128 INFO:tasks.workunit.client.1.smithi019.stdout:9/386: creat d1/d7/d21/f87 x:0 0 0 2024-03-16T09:01:17.131 INFO:tasks.workunit.client.1.smithi019.stdout:7/385: mknod d9/da/d13/d18/c79 0 2024-03-16T09:01:17.131 INFO:tasks.workunit.client.1.smithi019.stdout:7/386: chown d9/da/d13/l3c 0 1 2024-03-16T09:01:17.131 INFO:tasks.workunit.client.1.smithi019.stdout:7/387: chown d9/da/f59 638256 1 2024-03-16T09:01:17.132 INFO:tasks.workunit.client.0.smithi007.stdout:5/649: rmdir d1/d7/d16/d1b/d3c 39 2024-03-16T09:01:17.134 INFO:tasks.workunit.client.1.smithi019.stdout:8/521: symlink d6/d2a/d45/lb4 0 2024-03-16T09:01:17.134 INFO:tasks.workunit.client.1.smithi019.stdout:8/522: chown d6/d2a/d45/c25 9 1 2024-03-16T09:01:17.135 INFO:tasks.workunit.client.0.smithi007.stdout:4/559: dread d4/d14/d1c/d5b/f5e [0,4194304] 0 2024-03-16T09:01:17.135 INFO:tasks.workunit.client.0.smithi007.stdout:4/560: fdatasync d4/d14/d64/d68/d82/dac/fb6 0 2024-03-16T09:01:17.135 INFO:tasks.workunit.client.0.smithi007.stdout:4/561: write d4/ff [3513144,23367] 0 2024-03-16T09:01:17.135 INFO:tasks.workunit.client.0.smithi007.stdout:5/650: creat d1/d7/d1c/d19/feb x:0 0 0 2024-03-16T09:01:17.138 INFO:tasks.workunit.client.0.smithi007.stdout:4/562: rename d4/d14/l47 to d4/d14/d1c/d5b/lc0 0 2024-03-16T09:01:17.141 INFO:tasks.workunit.client.1.smithi019.stdout:5/495: rename d6/d19/f88 to d6/dd/d37/f9d 0 2024-03-16T09:01:17.142 INFO:tasks.workunit.client.0.smithi007.stdout:5/651: link d1/d7/d1c/d19/d34/de/c11 d1/d7/d1c/d19/d34/d1d/cec 0 2024-03-16T09:01:17.145 INFO:tasks.workunit.client.0.smithi007.stdout:4/563: rename d4/d14/d1c/d5b/d67/d8f/da8/l7f to d4/d99/lc1 0 2024-03-16T09:01:17.147 INFO:tasks.workunit.client.1.smithi019.stdout:4/434: dread d4/d1b/d2a/f4e [4194304,4194304] 0 2024-03-16T09:01:17.147 INFO:tasks.workunit.client.1.smithi019.stdout:5/496: rename d6/d1f/d33/d7d to d6/d1f/d33/d89/d9e 0 2024-03-16T09:01:17.150 INFO:tasks.workunit.client.0.smithi007.stdout:7/565: dwrite d0/d6c/d25/d3a/f77 [0,4194304] 0 2024-03-16T09:01:17.150 INFO:tasks.workunit.client.0.smithi007.stdout:7/566: fdatasync d0/dc/d5a/f89 0 2024-03-16T09:01:17.150 INFO:tasks.workunit.client.0.smithi007.stdout:7/567: chown d0/dc/d5a/f74 933 1 2024-03-16T09:01:17.153 INFO:tasks.workunit.client.0.smithi007.stdout:4/564: creat d4/d14/d1c/d5b/d67/d8f/da8/dbe/fc2 x:0 0 0 2024-03-16T09:01:17.155 INFO:tasks.workunit.client.0.smithi007.stdout:7/568: mkdir d0/d19/d1f/d69/dc7 0 2024-03-16T09:01:17.156 INFO:tasks.workunit.client.1.smithi019.stdout:6/570: read de/d26/d32/d4e/f79 [3013487,114013] 0 2024-03-16T09:01:17.157 INFO:tasks.workunit.client.0.smithi007.stdout:7/569: rename d0/dc/d38/d4e/c7e to d0/d19/cc8 0 2024-03-16T09:01:17.157 INFO:tasks.workunit.client.0.smithi007.stdout:7/570: write d0/d19/d1f/d76/fa3 [4290420,45116] 0 2024-03-16T09:01:17.158 INFO:tasks.workunit.client.1.smithi019.stdout:0/520: dwrite d7/d1d/d63/f7b [0,4194304] 0 2024-03-16T09:01:17.158 INFO:tasks.workunit.client.1.smithi019.stdout:0/521: chown d7/d8/d32/c84 3 1 2024-03-16T09:01:17.158 INFO:tasks.workunit.client.1.smithi019.stdout:0/522: write d7/d8/d32/d36/d82/f71 [1679896,6023] 0 2024-03-16T09:01:17.159 INFO:tasks.workunit.client.1.smithi019.stdout:0/523: truncate d7/d1d/f67 847943 0 2024-03-16T09:01:17.159 INFO:tasks.workunit.client.1.smithi019.stdout:6/571: creat de/d1c/d9a/d9e/fb5 x:0 0 0 2024-03-16T09:01:17.159 INFO:tasks.workunit.client.1.smithi019.stdout:0/524: symlink d7/d1d/d63/d6e/lb3 0 2024-03-16T09:01:17.160 INFO:tasks.workunit.client.1.smithi019.stdout:6/572: symlink de/d26/d32/d97/da1/lb6 0 2024-03-16T09:01:17.160 INFO:tasks.workunit.client.1.smithi019.stdout:6/573: fdatasync de/d26/d32/d4e/fa5 0 2024-03-16T09:01:17.163 INFO:tasks.workunit.client.1.smithi019.stdout:0/525: link d7/d8/l49 d7/d1d/d28/d48/lb4 0 2024-03-16T09:01:17.163 INFO:tasks.workunit.client.1.smithi019.stdout:0/526: truncate d7/d1d/d63/d89/d8a/fa8 446126 0 2024-03-16T09:01:17.163 INFO:tasks.workunit.client.1.smithi019.stdout:6/574: mkdir de/d26/d32/d97/d66/d72/db7 0 2024-03-16T09:01:17.165 INFO:tasks.workunit.client.1.smithi019.stdout:6/575: creat de/d26/d32/d97/fb8 x:0 0 0 2024-03-16T09:01:17.165 INFO:tasks.workunit.client.1.smithi019.stdout:6/576: chown de/d26/f5c 37273471 1 2024-03-16T09:01:17.165 INFO:tasks.workunit.client.1.smithi019.stdout:6/577: truncate de/d26/d32/d97/d66/f90 1846991 0 2024-03-16T09:01:17.165 INFO:tasks.workunit.client.1.smithi019.stdout:6/578: truncate de/d26/d32/d4e/f79 4397035 0 2024-03-16T09:01:17.165 INFO:tasks.workunit.client.1.smithi019.stdout:6/579: read de/d26/d32/d97/d51/d70/f91 [1616995,127757] 0 2024-03-16T09:01:17.165 INFO:tasks.workunit.client.1.smithi019.stdout:6/580: stat de/d1c/d9a/d9e/fb5 0 2024-03-16T09:01:17.166 INFO:tasks.workunit.client.0.smithi007.stdout:7/571: dread d0/d19/d1f/d42/f62 [0,4194304] 0 2024-03-16T09:01:17.168 INFO:tasks.workunit.client.0.smithi007.stdout:7/572: dread d0/d6c/d25/d3a/f96 [0,4194304] 0 2024-03-16T09:01:17.169 INFO:tasks.workunit.client.1.smithi019.stdout:1/408: dwrite d16/d1f/d4f/d61/f6b [0,4194304] 0 2024-03-16T09:01:17.169 INFO:tasks.workunit.client.1.smithi019.stdout:1/409: write d16/f5d [994269,115892] 0 2024-03-16T09:01:17.169 INFO:tasks.workunit.client.1.smithi019.stdout:1/410: truncate d16/d1f/d43/d55/f5e 1123611 0 2024-03-16T09:01:17.169 INFO:tasks.workunit.client.1.smithi019.stdout:6/581: link de/d26/c63 de/d26/d32/cb9 0 2024-03-16T09:01:17.169 INFO:tasks.workunit.client.1.smithi019.stdout:6/582: dread - de/d26/d32/f62 zero size 2024-03-16T09:01:17.171 INFO:tasks.workunit.client.1.smithi019.stdout:1/411: mkdir d16/d1f/d73 0 2024-03-16T09:01:17.171 INFO:tasks.workunit.client.1.smithi019.stdout:1/412: stat d16/d1f/d4f/f62 0 2024-03-16T09:01:17.172 INFO:tasks.workunit.client.1.smithi019.stdout:6/583: creat de/d26/d32/d97/d51/fba x:0 0 0 2024-03-16T09:01:17.173 INFO:tasks.workunit.client.1.smithi019.stdout:6/584: rmdir de/d26/d32/d97/dae 39 2024-03-16T09:01:17.173 INFO:tasks.workunit.client.0.smithi007.stdout:7/573: rename d0/d19/d1f/d42 to d0/d6c/d25/d7a/dc9 0 2024-03-16T09:01:17.174 INFO:tasks.workunit.client.1.smithi019.stdout:6/585: creat de/d26/d32/d97/d66/d72/db7/fbb x:0 0 0 2024-03-16T09:01:17.174 INFO:tasks.workunit.client.1.smithi019.stdout:6/586: truncate de/d26/d32/d97/d66/f90 2275066 0 2024-03-16T09:01:17.175 INFO:tasks.workunit.client.1.smithi019.stdout:6/587: fdatasync de/f1a 0 2024-03-16T09:01:17.175 INFO:tasks.workunit.client.1.smithi019.stdout:6/588: write de/d26/d32/f9c [115579,83539] 0 2024-03-16T09:01:17.175 INFO:tasks.workunit.client.1.smithi019.stdout:6/589: fdatasync de/f15 0 2024-03-16T09:01:17.188 INFO:tasks.workunit.client.0.smithi007.stdout:5/652: dread d1/d7/d1c/d88/da2/d89/db3/f31 [0,4194304] 0 2024-03-16T09:01:17.189 INFO:tasks.workunit.client.0.smithi007.stdout:5/653: rename d1/d7/d1c/d19/d34/d3e/d62/c64 to d1/d7/d1c/d19/ced 0 2024-03-16T09:01:17.191 INFO:tasks.workunit.client.0.smithi007.stdout:5/654: mkdir d1/d7/d1c/d19/d34/d1d/dee 0 2024-03-16T09:01:17.191 INFO:tasks.workunit.client.0.smithi007.stdout:5/655: write d1/d7/d1c/d19/d34/d3e/d41/da6/fc9 [761183,125045] 0 2024-03-16T09:01:17.201 INFO:tasks.workunit.client.0.smithi007.stdout:7/574: write d0/d19/d1f/d5d/f67 [404275,63073] 0 2024-03-16T09:01:17.201 INFO:tasks.workunit.client.0.smithi007.stdout:7/575: write d0/d6c/d25/d3a/f4f [1309811,31334] 0 2024-03-16T09:01:17.202 INFO:tasks.workunit.client.1.smithi019.stdout:2/529: dwrite d5/d2e/d4f/f86 [0,4194304] 0 2024-03-16T09:01:17.202 INFO:tasks.workunit.client.1.smithi019.stdout:2/530: write d5/d6/d2b/d53/f71 [1786678,88556] 0 2024-03-16T09:01:17.202 INFO:tasks.workunit.client.1.smithi019.stdout:2/531: write d5/f4d [911425,7026] 0 2024-03-16T09:01:17.202 INFO:tasks.workunit.client.1.smithi019.stdout:8/523: dwrite d6/d2a/d3c/dab/d30/f4f [0,4194304] 0 2024-03-16T09:01:17.202 INFO:tasks.workunit.client.1.smithi019.stdout:7/388: dwrite d9/db/f2c [0,4194304] 0 2024-03-16T09:01:17.203 INFO:tasks.workunit.client.0.smithi007.stdout:7/576: creat d0/dc/fca x:0 0 0 2024-03-16T09:01:17.203 INFO:tasks.workunit.client.0.smithi007.stdout:7/577: mknod d0/dc/d5a/dba/ccb 0 2024-03-16T09:01:17.203 INFO:tasks.workunit.client.0.smithi007.stdout:5/656: dread d1/d7/d1c/d88/da2/d89/fb8 [0,4194304] 0 2024-03-16T09:01:17.205 INFO:tasks.workunit.client.0.smithi007.stdout:7/578: rmdir d0/db7 0 2024-03-16T09:01:17.206 INFO:tasks.workunit.client.1.smithi019.stdout:2/532: unlink d5/d6/d2b/c24 0 2024-03-16T09:01:17.206 INFO:tasks.workunit.client.0.smithi007.stdout:7/579: creat d0/d6c/d25/d3a/db3/fcc x:0 0 0 2024-03-16T09:01:17.206 INFO:tasks.workunit.client.0.smithi007.stdout:7/580: fdatasync d0/dc/d5a/f74 0 2024-03-16T09:01:17.208 INFO:tasks.workunit.client.0.smithi007.stdout:7/581: creat d0/d19/d1f/d5d/d8a/d93/fcd x:0 0 0 2024-03-16T09:01:17.208 INFO:tasks.workunit.client.0.smithi007.stdout:7/582: write d0/d6c/d25/d7a/f6e [275484,91618] 0 2024-03-16T09:01:17.208 INFO:tasks.workunit.client.0.smithi007.stdout:7/583: chown d0/d6c/d25/d3a 0 1 2024-03-16T09:01:17.208 INFO:tasks.workunit.client.0.smithi007.stdout:7/584: creat d0/d6c/d25/d3a/db3/fce x:0 0 0 2024-03-16T09:01:17.208 INFO:tasks.workunit.client.0.smithi007.stdout:4/565: dwrite d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/fa9 [0,4194304] 0 2024-03-16T09:01:17.209 INFO:tasks.workunit.client.1.smithi019.stdout:3/518: dwrite d3/d17/d5a/f73 [0,4194304] 0 2024-03-16T09:01:17.209 INFO:tasks.workunit.client.1.smithi019.stdout:4/435: dwrite d4/d1b/d49/d6e/f90 [0,4194304] 0 2024-03-16T09:01:17.209 INFO:tasks.workunit.client.1.smithi019.stdout:0/527: dwrite d7/d1d/d28/d6c/f95 [0,4194304] 0 2024-03-16T09:01:17.209 INFO:tasks.workunit.client.1.smithi019.stdout:0/528: chown d7/d1d/d2e/c31 25773 1 2024-03-16T09:01:17.210 INFO:tasks.workunit.client.1.smithi019.stdout:8/524: mknod d6/d2a/d45/d2c/cb5 0 2024-03-16T09:01:17.210 INFO:tasks.workunit.client.1.smithi019.stdout:8/525: chown d6/lb1 2069883481 1 2024-03-16T09:01:17.211 INFO:tasks.workunit.client.1.smithi019.stdout:9/387: dwrite d1/d7/d21/f7b [0,4194304] 0 2024-03-16T09:01:17.213 INFO:tasks.workunit.client.1.smithi019.stdout:7/389: rename d9/da/d13/d18/d1a/f1f to d9/db/d12/d5b/d33/f7a 0 2024-03-16T09:01:17.213 INFO:tasks.workunit.client.0.smithi007.stdout:7/585: creat d0/d6c/d25/fcf x:0 0 0 2024-03-16T09:01:17.213 INFO:tasks.workunit.client.0.smithi007.stdout:7/586: chown d0/d19/f2b 1662870716 1 2024-03-16T09:01:17.213 INFO:tasks.workunit.client.0.smithi007.stdout:7/587: write d0/d19/d1f/d69/fc6 [815504,10342] 0 2024-03-16T09:01:17.216 INFO:tasks.workunit.client.1.smithi019.stdout:6/590: dwrite de/d1c/d9a/d9e/fb5 [0,4194304] 0 2024-03-16T09:01:17.216 INFO:tasks.workunit.client.1.smithi019.stdout:1/413: dwrite f10 [0,4194304] 0 2024-03-16T09:01:17.221 INFO:tasks.workunit.client.1.smithi019.stdout:4/436: mkdir d4/d1b/d49/d58/d80/d91 0 2024-03-16T09:01:17.221 INFO:tasks.workunit.client.1.smithi019.stdout:4/437: dread - d4/d1b/d49/f53 zero size 2024-03-16T09:01:17.223 INFO:tasks.workunit.client.0.smithi007.stdout:7/588: dread d0/d19/f48 [0,4194304] 0 2024-03-16T09:01:17.224 INFO:tasks.workunit.client.1.smithi019.stdout:8/526: mkdir d6/d2a/d45/d57/db6 0 2024-03-16T09:01:17.228 INFO:tasks.workunit.client.0.smithi007.stdout:4/566: chown d4/c58 74624 1 2024-03-16T09:01:17.229 INFO:tasks.workunit.client.1.smithi019.stdout:9/388: mkdir d1/d88 0 2024-03-16T09:01:17.232 INFO:tasks.workunit.client.1.smithi019.stdout:6/591: link de/f49 de/d1c/fbc 0 2024-03-16T09:01:17.233 INFO:tasks.workunit.client.0.smithi007.stdout:4/567: getdents d4/d14/d1c 0 2024-03-16T09:01:17.234 INFO:tasks.workunit.client.1.smithi019.stdout:1/414: creat d16/d1f/f74 x:0 0 0 2024-03-16T09:01:17.234 INFO:tasks.workunit.client.1.smithi019.stdout:1/415: stat d16/f65 0 2024-03-16T09:01:17.235 INFO:tasks.workunit.client.1.smithi019.stdout:4/438: symlink d4/d1b/l92 0 2024-03-16T09:01:17.235 INFO:tasks.workunit.client.1.smithi019.stdout:4/439: truncate d4/d6/d15/f2e 2421445 0 2024-03-16T09:01:17.235 INFO:tasks.workunit.client.1.smithi019.stdout:4/440: write d4/d6/d15/f2e [3104401,51543] 0 2024-03-16T09:01:17.236 INFO:tasks.workunit.client.1.smithi019.stdout:8/527: mkdir d6/d2a/d45/d57/d72/d79/db7 0 2024-03-16T09:01:17.237 INFO:tasks.workunit.client.1.smithi019.stdout:8/528: write d6/d2a/d45/f6b [586441,84432] 0 2024-03-16T09:01:17.237 INFO:tasks.workunit.client.1.smithi019.stdout:8/529: truncate d6/d2a/d3c/dab/d30/d4a/f81 990029 0 2024-03-16T09:01:17.242 INFO:tasks.workunit.client.1.smithi019.stdout:5/497: rmdir d6/d1f/d33 39 2024-03-16T09:01:17.242 INFO:tasks.workunit.client.1.smithi019.stdout:7/390: getdents d9/da/d13 0 2024-03-16T09:01:17.242 INFO:tasks.workunit.client.1.smithi019.stdout:6/592: creat de/d26/d32/d97/d66/d72/d7f/fbd x:0 0 0 2024-03-16T09:01:17.242 INFO:tasks.workunit.client.1.smithi019.stdout:1/416: truncate d16/f2b 75510 0 2024-03-16T09:01:17.243 INFO:tasks.workunit.client.1.smithi019.stdout:1/417: readlink d16/d1f/d4f/l63 0 2024-03-16T09:01:17.244 INFO:tasks.workunit.client.1.smithi019.stdout:4/441: rename d4/d6c/f6d to d4/d6c/d89/f93 0 2024-03-16T09:01:17.247 INFO:tasks.workunit.client.1.smithi019.stdout:9/389: getdents d1/d7/de/d24/d44 0 2024-03-16T09:01:17.249 INFO:tasks.workunit.client.1.smithi019.stdout:6/593: symlink de/d26/d32/d97/d51/lbe 0 2024-03-16T09:01:17.249 INFO:tasks.workunit.client.1.smithi019.stdout:6/594: creat de/d26/d73/fbf x:0 0 0 2024-03-16T09:01:17.249 INFO:tasks.workunit.client.1.smithi019.stdout:6/595: write de/d26/d32/d97/d66/d72/d7f/fbd [370331,36011] 0 2024-03-16T09:01:17.249 INFO:tasks.workunit.client.0.smithi007.stdout:7/589: write d0/f11 [2314643,102789] 0 2024-03-16T09:01:17.250 INFO:tasks.workunit.client.1.smithi019.stdout:1/418: creat d16/d31/d71/f75 x:0 0 0 2024-03-16T09:01:17.251 INFO:tasks.workunit.client.0.smithi007.stdout:7/590: rename d0/dc/c31 to d0/d19/d1f/d69/cd0 0 2024-03-16T09:01:17.251 INFO:tasks.workunit.client.0.smithi007.stdout:7/591: truncate d0/d6c/d25/d73/f91 4482694 0 2024-03-16T09:01:17.253 INFO:tasks.workunit.client.1.smithi019.stdout:7/391: rename d9/da/d13/d18/d46/f50 to d9/d25/d28/d48/f7b 0 2024-03-16T09:01:17.254 INFO:tasks.workunit.client.1.smithi019.stdout:8/530: rmdir d6/d2a/d45/d57/d96 39 2024-03-16T09:01:17.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/592: write d0/dc/fe [4168029,30675] 0 2024-03-16T09:01:17.255 INFO:tasks.workunit.client.0.smithi007.stdout:7/593: creat d0/d6c/d25/d3a/db3/fd1 x:0 0 0 2024-03-16T09:01:17.255 INFO:tasks.workunit.client.1.smithi019.stdout:7/392: write d9/da/d13/d18/d1a/d6e/f77 [716594,27489] 0 2024-03-16T09:01:17.255 INFO:tasks.workunit.client.1.smithi019.stdout:7/393: dread - d9/d25/f54 zero size 2024-03-16T09:01:17.257 INFO:tasks.workunit.client.0.smithi007.stdout:5/657: dwrite d1/d7/d1c/d88/da2/d89/db3/d2a/f57 [4194304,4194304] 0 2024-03-16T09:01:17.257 INFO:tasks.workunit.client.0.smithi007.stdout:5/658: write d1/d7/d1c/d19/d34/d5e/d6f/f51 [1117451,52595] 0 2024-03-16T09:01:17.257 INFO:tasks.workunit.client.0.smithi007.stdout:5/659: dread - d1/d7/d1c/d88/da2/d89/db3/d2a/d79/fbe zero size 2024-03-16T09:01:17.258 INFO:tasks.workunit.client.1.smithi019.stdout:9/390: unlink d1/d7/de/f3a 0 2024-03-16T09:01:17.258 INFO:tasks.workunit.client.1.smithi019.stdout:9/391: truncate d1/d7/d33/d59/f85 253178 0 2024-03-16T09:01:17.258 INFO:tasks.workunit.client.1.smithi019.stdout:9/392: creat d1/d7/d33/d59/f89 x:0 0 0 2024-03-16T09:01:17.258 INFO:tasks.workunit.client.1.smithi019.stdout:0/529: dwrite d7/d8/d32/fa5 [0,4194304] 0 2024-03-16T09:01:17.258 INFO:tasks.workunit.client.1.smithi019.stdout:6/596: rmdir de/d26/d32/d97/d66 39 2024-03-16T09:01:17.260 INFO:tasks.workunit.client.1.smithi019.stdout:4/442: rename d4/d6/d15/d1f/l59 to d4/d6/d15/d1f/l94 0 2024-03-16T09:01:17.261 INFO:tasks.workunit.client.1.smithi019.stdout:7/394: getdents d9/da/d13/d18 0 2024-03-16T09:01:17.261 INFO:tasks.workunit.client.1.smithi019.stdout:7/395: stat d9/db/d12/d5b/d33/f58 0 2024-03-16T09:01:17.261 INFO:tasks.workunit.client.1.smithi019.stdout:7/396: fsync d9/db/d12/d5b/d33/f58 0 2024-03-16T09:01:17.263 INFO:tasks.workunit.client.1.smithi019.stdout:9/393: dread d1/d7/f45 [0,4194304] 0 2024-03-16T09:01:17.263 INFO:tasks.workunit.client.1.smithi019.stdout:2/533: dwrite d5/d4e/d74/f95 [0,4194304] 0 2024-03-16T09:01:17.265 INFO:tasks.workunit.client.0.smithi007.stdout:5/660: dread d1/d7/d1c/d19/d34/d5e/d6f/d72/f85 [0,4194304] 0 2024-03-16T09:01:17.267 INFO:tasks.workunit.client.1.smithi019.stdout:6/597: truncate f7 3223883 0 2024-03-16T09:01:17.267 INFO:tasks.workunit.client.1.smithi019.stdout:6/598: creat de/d26/d32/d97/d66/d72/db7/fc0 x:0 0 0 2024-03-16T09:01:17.267 INFO:tasks.workunit.client.1.smithi019.stdout:6/599: write de/d26/f85 [2440048,123191] 0 2024-03-16T09:01:17.267 INFO:tasks.workunit.client.1.smithi019.stdout:5/498: dwrite d6/d1f/d33/f52 [0,4194304] 0 2024-03-16T09:01:17.267 INFO:tasks.workunit.client.1.smithi019.stdout:5/499: write d6/dd/f1a [4952782,4009] 0 2024-03-16T09:01:17.267 INFO:tasks.workunit.client.1.smithi019.stdout:5/500: fdatasync d6/dd/d47/f64 0 2024-03-16T09:01:17.268 INFO:tasks.workunit.client.1.smithi019.stdout:4/443: mknod d4/d6/d15/c95 0 2024-03-16T09:01:17.272 INFO:tasks.workunit.client.0.smithi007.stdout:4/568: dwrite d4/d14/f96 [0,4194304] 0 2024-03-16T09:01:17.272 INFO:tasks.workunit.client.1.smithi019.stdout:3/519: dwrite d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:17.272 INFO:tasks.workunit.client.1.smithi019.stdout:7/397: unlink d9/db/d12/d5b/f78 0 2024-03-16T09:01:17.275 INFO:tasks.workunit.client.1.smithi019.stdout:3/520: dread d3/d11/f6f [0,4194304] 0 2024-03-16T09:01:17.276 INFO:tasks.workunit.client.1.smithi019.stdout:3/521: read d3/ff [3529825,120065] 0 2024-03-16T09:01:17.276 INFO:tasks.workunit.client.1.smithi019.stdout:3/522: creat d3/d17/d5a/d75/d7a/fa0 x:0 0 0 2024-03-16T09:01:17.276 INFO:tasks.workunit.client.1.smithi019.stdout:3/523: stat d3/d11/f3b 0 2024-03-16T09:01:17.276 INFO:tasks.workunit.client.1.smithi019.stdout:2/534: truncate d5/de/f32 2402680 0 2024-03-16T09:01:17.276 INFO:tasks.workunit.client.1.smithi019.stdout:2/535: chown d5/d2e/d4f 817434 1 2024-03-16T09:01:17.276 INFO:tasks.workunit.client.1.smithi019.stdout:0/530: getdents d7/d1d/d28/d6c 0 2024-03-16T09:01:17.285 INFO:tasks.workunit.client.0.smithi007.stdout:5/661: mkdir d1/d7/d1c/d19/d34/d3e/d62/def 0 2024-03-16T09:01:17.287 INFO:tasks.workunit.client.1.smithi019.stdout:1/419: rename d16/d1f/d4f/d61 to d16/d31/d76 0 2024-03-16T09:01:17.287 INFO:tasks.workunit.client.1.smithi019.stdout:1/420: chown d16/f22 1079718 1 2024-03-16T09:01:17.287 INFO:tasks.workunit.client.1.smithi019.stdout:1/421: write d16/d1f/d4f/f57 [1502986,61853] 0 2024-03-16T09:01:17.288 INFO:tasks.workunit.client.1.smithi019.stdout:7/398: stat d9/da/l52 0 2024-03-16T09:01:17.288 INFO:tasks.workunit.client.1.smithi019.stdout:3/524: write d3/d17/f72 [1906322,123047] 0 2024-03-16T09:01:17.288 INFO:tasks.workunit.client.1.smithi019.stdout:3/525: chown d3/d1b/d40/d4a/f98 0 1 2024-03-16T09:01:17.289 INFO:tasks.workunit.client.1.smithi019.stdout:8/531: dwrite d6/d2a/f8d [0,4194304] 0 2024-03-16T09:01:17.290 INFO:tasks.workunit.client.1.smithi019.stdout:0/531: mkdir d7/d8/d32/d66/d6d/d8c/db5 0 2024-03-16T09:01:17.293 INFO:tasks.workunit.client.0.smithi007.stdout:5/662: unlink d1/d7/d1c/d19/d34/d3e/d41/d48/f75 0 2024-03-16T09:01:17.294 INFO:tasks.workunit.client.0.smithi007.stdout:5/663: readlink d1/d7/d1c/d19/d34/d5e/l5c 0 2024-03-16T09:01:17.295 INFO:tasks.workunit.client.1.smithi019.stdout:1/422: dread d16/d31/d76/f6b [0,4194304] 0 2024-03-16T09:01:17.295 INFO:tasks.workunit.client.1.smithi019.stdout:4/444: symlink d4/d6/d15/d1f/l96 0 2024-03-16T09:01:17.297 INFO:tasks.workunit.client.1.smithi019.stdout:7/399: creat d9/da/d13/d18/d30/d3e/d5d/f7c x:0 0 0 2024-03-16T09:01:17.297 INFO:tasks.workunit.client.1.smithi019.stdout:3/526: symlink d3/d17/d5a/d75/la1 0 2024-03-16T09:01:17.300 INFO:tasks.workunit.client.0.smithi007.stdout:5/664: write d1/d7/d1c/d19/d34/f8c [286445,101533] 0 2024-03-16T09:01:17.303 INFO:tasks.workunit.client.1.smithi019.stdout:8/532: unlink d6/c2d 0 2024-03-16T09:01:17.303 INFO:tasks.workunit.client.1.smithi019.stdout:8/533: fdatasync d6/d2a/d45/d19/f1c 0 2024-03-16T09:01:17.303 INFO:tasks.workunit.client.1.smithi019.stdout:8/534: truncate d6/d2a/d45/d57/f60 1383197 0 2024-03-16T09:01:17.303 INFO:tasks.workunit.client.1.smithi019.stdout:8/535: creat d6/d2a/d45/d19/fb8 x:0 0 0 2024-03-16T09:01:17.303 INFO:tasks.workunit.client.0.smithi007.stdout:5/665: mkdir d1/d7/d1c/df0 0 2024-03-16T09:01:17.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/666: chown d1/d7/d1c/d19/d34/d3e/d62/def 158375 1 2024-03-16T09:01:17.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/667: truncate d1/d7/d1c/d88/da2/d89/db3/f6e 1926727 0 2024-03-16T09:01:17.305 INFO:tasks.workunit.client.1.smithi019.stdout:0/532: symlink d7/d8/d32/d4a/d61/lb6 0 2024-03-16T09:01:17.305 INFO:tasks.workunit.client.1.smithi019.stdout:4/445: dread d4/d6/d15/f66 [0,4194304] 0 2024-03-16T09:01:17.305 INFO:tasks.workunit.client.1.smithi019.stdout:4/446: dread - d4/d1b/d2a/d4f/f82 zero size 2024-03-16T09:01:17.306 INFO:tasks.workunit.client.0.smithi007.stdout:5/668: unlink d1/d7/d1c/d88/da2/d89/fd5 0 2024-03-16T09:01:17.308 INFO:tasks.workunit.client.0.smithi007.stdout:5/669: mkdir d1/d7/d16/d1b/df1 0 2024-03-16T09:01:17.308 INFO:tasks.workunit.client.0.smithi007.stdout:5/670: chown d1/d7/d1c/d19/d34/cb 3331 1 2024-03-16T09:01:17.308 INFO:tasks.workunit.client.0.smithi007.stdout:5/671: readlink d1/d7/d1c/d19/d34/de/l35 0 2024-03-16T09:01:17.308 INFO:tasks.workunit.client.0.smithi007.stdout:5/672: truncate d1/d7/d1c/d19/d34/d3e/d41/d48/fc3 778431 0 2024-03-16T09:01:17.308 INFO:tasks.workunit.client.0.smithi007.stdout:5/673: readlink d1/le0 0 2024-03-16T09:01:17.310 INFO:tasks.workunit.client.1.smithi019.stdout:3/527: symlink d3/d1b/d40/la2 0 2024-03-16T09:01:17.311 INFO:tasks.workunit.client.1.smithi019.stdout:3/528: fdatasync d3/d17/d3f/f7c 0 2024-03-16T09:01:17.315 INFO:tasks.workunit.client.1.smithi019.stdout:0/533: mknod d7/d1d/d63/d89/d8a/cb7 0 2024-03-16T09:01:17.316 INFO:tasks.workunit.client.1.smithi019.stdout:4/447: dread d4/d1b/d49/f72 [0,4194304] 0 2024-03-16T09:01:17.316 INFO:tasks.workunit.client.1.smithi019.stdout:4/448: dread - d4/d1b/d49/f53 zero size 2024-03-16T09:01:17.316 INFO:tasks.workunit.client.1.smithi019.stdout:4/449: write d4/f26 [5824747,9845] 0 2024-03-16T09:01:17.316 INFO:tasks.workunit.client.1.smithi019.stdout:4/450: fsync d4/d1b/f5d 0 2024-03-16T09:01:17.316 INFO:tasks.workunit.client.1.smithi019.stdout:4/451: fdatasync d4/d1b/f28 0 2024-03-16T09:01:17.318 INFO:tasks.workunit.client.1.smithi019.stdout:0/534: rename d7/d8/d32/d36 to d7/d8/d32/d66/d6d/d8c/db8 0 2024-03-16T09:01:17.330 INFO:tasks.workunit.client.1.smithi019.stdout:2/536: dwrite d5/d6/d64/d6f/da6/f8b [0,4194304] 0 2024-03-16T09:01:17.330 INFO:tasks.workunit.client.0.smithi007.stdout:7/594: dwrite d0/d19/d1f/d76/fa3 [0,4194304] 0 2024-03-16T09:01:17.332 INFO:tasks.workunit.client.1.smithi019.stdout:2/537: creat d5/d6/fb0 x:0 0 0 2024-03-16T09:01:17.333 INFO:tasks.workunit.client.1.smithi019.stdout:2/538: mknod d5/d6/d64/d6f/d87/cb1 0 2024-03-16T09:01:17.333 INFO:tasks.workunit.client.1.smithi019.stdout:2/539: write d5/d2e/d4f/f86 [4338496,78075] 0 2024-03-16T09:01:17.334 INFO:tasks.workunit.client.0.smithi007.stdout:7/595: rename d0/f49 to d0/d19/d1f/fd2 0 2024-03-16T09:01:17.335 INFO:tasks.workunit.client.0.smithi007.stdout:7/596: creat d0/d6c/d25/d3a/fd3 x:0 0 0 2024-03-16T09:01:17.335 INFO:tasks.workunit.client.0.smithi007.stdout:7/597: fsync d0/dc/d5a/f74 0 2024-03-16T09:01:17.336 INFO:tasks.workunit.client.0.smithi007.stdout:7/598: creat d0/d6c/d25/d7a/fd4 x:0 0 0 2024-03-16T09:01:17.338 INFO:tasks.workunit.client.1.smithi019.stdout:6/600: dwrite de/f3c [0,4194304] 0 2024-03-16T09:01:17.339 INFO:tasks.workunit.client.0.smithi007.stdout:7/599: dread d0/d6c/d25/d3a/f4f [0,4194304] 0 2024-03-16T09:01:17.339 INFO:tasks.workunit.client.0.smithi007.stdout:7/600: write d0/d19/d1f/d69/f87 [192532,129717] 0 2024-03-16T09:01:17.344 INFO:tasks.workunit.client.0.smithi007.stdout:4/569: dwrite d4/d14/d1c/d5b/fa7 [0,4194304] 0 2024-03-16T09:01:17.346 INFO:tasks.workunit.client.1.smithi019.stdout:1/423: dwrite d16/d1f/d4f/f50 [0,4194304] 0 2024-03-16T09:01:17.347 INFO:tasks.workunit.client.0.smithi007.stdout:7/601: read d0/dc/fe [5154780,55765] 0 2024-03-16T09:01:17.347 INFO:tasks.workunit.client.1.smithi019.stdout:8/536: dwrite d6/de/f6c [0,4194304] 0 2024-03-16T09:01:17.348 INFO:tasks.workunit.client.1.smithi019.stdout:0/535: dwrite d7/d1d/d63/f78 [0,4194304] 0 2024-03-16T09:01:17.348 INFO:tasks.workunit.client.1.smithi019.stdout:1/424: symlink d16/d1f/d43/d55/l77 0 2024-03-16T09:01:17.350 INFO:tasks.workunit.client.0.smithi007.stdout:7/602: creat d0/dc/d38/d4e/fd5 x:0 0 0 2024-03-16T09:01:17.350 INFO:tasks.workunit.client.0.smithi007.stdout:4/570: getdents d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84 0 2024-03-16T09:01:17.350 INFO:tasks.workunit.client.0.smithi007.stdout:4/571: readlink d4/d14/d1c/d5b/d67/d8f/da8/d6b/l92 0 2024-03-16T09:01:17.352 INFO:tasks.workunit.client.1.smithi019.stdout:8/537: creat d6/d2a/d3c/dab/d30/d4a/d67/d95/fb9 x:0 0 0 2024-03-16T09:01:17.352 INFO:tasks.workunit.client.1.smithi019.stdout:8/538: chown d6/d2a/d45/c23 95655 1 2024-03-16T09:01:17.353 INFO:tasks.workunit.client.1.smithi019.stdout:0/536: creat d7/d8/d32/d4a/fb9 x:0 0 0 2024-03-16T09:01:17.354 INFO:tasks.workunit.client.0.smithi007.stdout:7/603: mknod d0/d19/d1f/d69/cd6 0 2024-03-16T09:01:17.356 INFO:tasks.workunit.client.0.smithi007.stdout:7/604: symlink d0/dc/d5a/ld7 0 2024-03-16T09:01:17.357 INFO:tasks.workunit.client.1.smithi019.stdout:1/425: mkdir d16/d1f/d4f/d68/d78 0 2024-03-16T09:01:17.357 INFO:tasks.workunit.client.1.smithi019.stdout:1/426: chown d16/d18/f39 133232901 1 2024-03-16T09:01:17.358 INFO:tasks.workunit.client.0.smithi007.stdout:7/605: creat d0/d19/d1f/d69/dc7/fd8 x:0 0 0 2024-03-16T09:01:17.360 INFO:tasks.workunit.client.1.smithi019.stdout:0/537: getdents d7/d1d 0 2024-03-16T09:01:17.360 INFO:tasks.workunit.client.1.smithi019.stdout:0/538: chown d7/d1d/d28 10901604 1 2024-03-16T09:01:17.361 INFO:tasks.workunit.client.1.smithi019.stdout:0/539: dread d7/f46 [0,4194304] 0 2024-03-16T09:01:17.361 INFO:tasks.workunit.client.1.smithi019.stdout:0/540: chown d7/d8/d32/fa5 176338851 1 2024-03-16T09:01:17.361 INFO:tasks.workunit.client.1.smithi019.stdout:0/541: write d7/d8/d32/d66/d6d/d8c/db8/d82/f6f [1043577,156] 0 2024-03-16T09:01:17.361 INFO:tasks.workunit.client.1.smithi019.stdout:0/542: write d7/d8/d32/d66/d6d/d8c/db8/f35 [1974892,113258] 0 2024-03-16T09:01:17.365 INFO:tasks.workunit.client.1.smithi019.stdout:0/543: rename d7/d8/d32/d4a/d61/f2c to d7/d8/d32/d66/d6d/d8c/db8/d82/fba 0 2024-03-16T09:01:17.366 INFO:tasks.workunit.client.1.smithi019.stdout:1/427: dread f9 [0,4194304] 0 2024-03-16T09:01:17.368 INFO:tasks.workunit.client.1.smithi019.stdout:1/428: mkdir d16/d1f/d3c/d79 0 2024-03-16T09:01:17.369 INFO:tasks.workunit.client.0.smithi007.stdout:4/572: dread f2 [0,4194304] 0 2024-03-16T09:01:17.370 INFO:tasks.workunit.client.0.smithi007.stdout:4/573: readlink d4/l2a 0 2024-03-16T09:01:17.370 INFO:tasks.workunit.client.0.smithi007.stdout:4/574: fsync d4/d14/d64/d9f/f3f 0 2024-03-16T09:01:17.371 INFO:tasks.workunit.client.1.smithi019.stdout:7/400: truncate d9/da/d13/d18/d1a/d6e/f77 728326 0 2024-03-16T09:01:17.371 INFO:tasks.workunit.client.0.smithi007.stdout:4/575: mknod d4/d14/d1c/d5b/d67/d8f/da8/d9d/dbb/cc3 0 2024-03-16T09:01:17.371 INFO:tasks.workunit.client.0.smithi007.stdout:4/576: creat d4/d14/d64/d68/fc4 x:0 0 0 2024-03-16T09:01:17.371 INFO:tasks.workunit.client.0.smithi007.stdout:4/577: read - d4/d99/fb3 zero size 2024-03-16T09:01:17.372 INFO:tasks.workunit.client.0.smithi007.stdout:4/578: rename d4/d14 to d4/d14/d64/d9f/d9a/dc5 22 2024-03-16T09:01:17.372 INFO:tasks.workunit.client.0.smithi007.stdout:4/579: chown d4/d14/f19 2117258 1 2024-03-16T09:01:17.374 INFO:tasks.workunit.client.0.smithi007.stdout:4/580: getdents d4/d14/d1c/d5b 0 2024-03-16T09:01:17.375 INFO:tasks.workunit.client.0.smithi007.stdout:4/581: creat d4/d14/d1c/d5b/d67/d8f/da8/fc6 x:0 0 0 2024-03-16T09:01:17.375 INFO:tasks.workunit.client.0.smithi007.stdout:4/582: chown d4/d14/d1c/l7c 77568705 1 2024-03-16T09:01:17.377 INFO:tasks.workunit.client.0.smithi007.stdout:4/583: unlink d4/d14/d1c/d5b/d67/d8f/d90/f91 0 2024-03-16T09:01:17.377 INFO:tasks.workunit.client.0.smithi007.stdout:4/584: fdatasync d4/d14/d64/d68/f8b 0 2024-03-16T09:01:17.378 INFO:tasks.workunit.client.0.smithi007.stdout:4/585: mknod d4/d14/d1c/d5b/d67/d8f/da8/d9d/cc7 0 2024-03-16T09:01:17.386 INFO:tasks.workunit.client.0.smithi007.stdout:4/586: write d4/d14/d1c/f2c [2188961,101235] 0 2024-03-16T09:01:17.388 INFO:tasks.workunit.client.0.smithi007.stdout:4/587: mknod d4/d14/d1c/d5b/d67/d8f/da8/d9d/cc8 0 2024-03-16T09:01:17.389 INFO:tasks.workunit.client.0.smithi007.stdout:4/588: chown d4/d14/d64/d68/d82/f8d 1 1 2024-03-16T09:01:17.389 INFO:tasks.workunit.client.0.smithi007.stdout:4/589: creat d4/d14/d1c/d5b/d67/d8f/fc9 x:0 0 0 2024-03-16T09:01:17.390 INFO:tasks.workunit.client.0.smithi007.stdout:4/590: rmdir d4/d14/d1c/d5b/d67/d8f/da8/d6b 39 2024-03-16T09:01:17.392 INFO:tasks.workunit.client.0.smithi007.stdout:4/591: mknod d4/d14/d64/d9f/cca 0 2024-03-16T09:01:17.394 INFO:tasks.workunit.client.0.smithi007.stdout:4/592: mkdir d4/dcb 0 2024-03-16T09:01:17.394 INFO:tasks.workunit.client.0.smithi007.stdout:4/593: mknod d4/d99/ccc 0 2024-03-16T09:01:17.394 INFO:tasks.workunit.client.0.smithi007.stdout:4/594: readlink d4/d14/d1c/d5b/d67/d8f/da8/d78/l97 0 2024-03-16T09:01:17.395 INFO:tasks.workunit.client.0.smithi007.stdout:4/595: mkdir d4/dcb/dcd 0 2024-03-16T09:01:17.400 INFO:tasks.workunit.client.0.smithi007.stdout:4/596: rmdir d4/d14/d1c/d5b/d67 39 2024-03-16T09:01:17.400 INFO:tasks.workunit.client.0.smithi007.stdout:4/597: fdatasync d4/f10 0 2024-03-16T09:01:17.407 INFO:tasks.workunit.client.0.smithi007.stdout:4/598: read d4/d14/d1c/f30 [2184384,95671] 0 2024-03-16T09:01:17.414 INFO:tasks.workunit.client.0.smithi007.stdout:4/599: read d4/d14/d1c/d5b/fa7 [1439319,21149] 0 2024-03-16T09:01:17.416 INFO:tasks.workunit.client.0.smithi007.stdout:4/600: dread d4/d14/d1c/d5b/d67/d8f/da8/f8a [0,4194304] 0 2024-03-16T09:01:17.418 INFO:tasks.workunit.client.0.smithi007.stdout:4/601: dread d4/d14/d64/f7d [0,4194304] 0 2024-03-16T09:01:17.432 INFO:tasks.workunit.client.0.smithi007.stdout:3/680: sync 2024-03-16T09:01:17.432 INFO:tasks.workunit.client.0.smithi007.stdout:2/746: sync 2024-03-16T09:01:17.432 INFO:tasks.workunit.client.0.smithi007.stdout:0/770: sync 2024-03-16T09:01:17.434 INFO:tasks.workunit.client.0.smithi007.stdout:3/681: rename d15/lc2 to d15/d18/d1f/d3b/d63/ld2 0 2024-03-16T09:01:17.434 INFO:tasks.workunit.client.1.smithi019.stdout:4/452: dwrite d4/d6/d15/d1f/d2c/f6b [0,4194304] 0 2024-03-16T09:01:17.436 INFO:tasks.workunit.client.0.smithi007.stdout:2/747: creat dc/dba/ff6 x:0 0 0 2024-03-16T09:01:17.436 INFO:tasks.workunit.client.0.smithi007.stdout:2/748: write dc/df/d24/d28/d2e/d4a/fc1 [878921,86376] 0 2024-03-16T09:01:17.438 INFO:tasks.workunit.client.0.smithi007.stdout:3/682: mknod d15/d18/d1f/d29/d3e/d85/cd3 0 2024-03-16T09:01:17.447 INFO:tasks.workunit.client.1.smithi019.stdout:4/453: rename d4/d6/d15/d1f/f30 to d4/d6/d15/d1f/f97 0 2024-03-16T09:01:17.447 INFO:tasks.workunit.client.0.smithi007.stdout:0/771: dread d3/d9/da/d48/fa5 [0,4194304] 0 2024-03-16T09:01:17.447 INFO:tasks.workunit.client.0.smithi007.stdout:0/772: write d3/d9/db4/d1c/d72/f82 [1410748,107531] 0 2024-03-16T09:01:17.447 INFO:tasks.workunit.client.0.smithi007.stdout:4/602: dread d4/d14/f35 [0,4194304] 0 2024-03-16T09:01:17.447 INFO:tasks.workunit.client.0.smithi007.stdout:2/749: dread dc/df/d24/d28/d2e/d35/fb4 [0,4194304] 0 2024-03-16T09:01:17.448 INFO:tasks.workunit.client.1.smithi019.stdout:4/454: mknod d4/d6/d15/d1f/d2c/c98 0 2024-03-16T09:01:17.448 INFO:tasks.workunit.client.0.smithi007.stdout:0/773: rename d3/d9/da/d48/f12 to d3/d9/da/d48/d4b/ffe 0 2024-03-16T09:01:17.449 INFO:tasks.workunit.client.0.smithi007.stdout:2/750: stat dc/df/l33 0 2024-03-16T09:01:17.450 INFO:tasks.workunit.client.1.smithi019.stdout:6/601: dwrite de/d1c/f20 [0,4194304] 0 2024-03-16T09:01:17.450 INFO:tasks.workunit.client.1.smithi019.stdout:7/401: read d9/da/d13/d18/d1a/f2b [4185297,89423] 0 2024-03-16T09:01:17.451 INFO:tasks.workunit.client.1.smithi019.stdout:7/402: dread - d9/d25/f42 zero size 2024-03-16T09:01:17.451 INFO:tasks.workunit.client.1.smithi019.stdout:8/539: dwrite d6/d2a/d3c/f44 [0,4194304] 0 2024-03-16T09:01:17.452 INFO:tasks.workunit.client.1.smithi019.stdout:1/429: dwrite d16/d1f/d4f/f5c [0,4194304] 0 2024-03-16T09:01:17.452 INFO:tasks.workunit.client.1.smithi019.stdout:8/540: creat d6/d2a/d68/fba x:0 0 0 2024-03-16T09:01:17.452 INFO:tasks.workunit.client.1.smithi019.stdout:8/541: fdatasync d6/d2a/d45/d5d/fb3 0 2024-03-16T09:01:17.452 INFO:tasks.workunit.client.1.smithi019.stdout:8/542: creat d6/d2a/d45/d57/fbb x:0 0 0 2024-03-16T09:01:17.453 INFO:tasks.workunit.client.0.smithi007.stdout:0/774: getdents d3/d9/da/d92/d9f/de8 0 2024-03-16T09:01:17.455 INFO:tasks.workunit.client.0.smithi007.stdout:2/751: getdents dc/d27 0 2024-03-16T09:01:17.455 INFO:tasks.workunit.client.0.smithi007.stdout:7/606: dwrite d0/d6c/d25/d73/f91 [0,4194304] 0 2024-03-16T09:01:17.457 INFO:tasks.workunit.client.0.smithi007.stdout:0/775: creat d3/d9/da/d92/d9f/de8/fff x:0 0 0 2024-03-16T09:01:17.457 INFO:tasks.workunit.client.0.smithi007.stdout:0/776: creat d3/d9/da/d18/d23/d36/f100 x:0 0 0 2024-03-16T09:01:17.457 INFO:tasks.workunit.client.0.smithi007.stdout:0/777: readlink d3/d6c/dbd/dd7/l8c 0 2024-03-16T09:01:17.457 INFO:tasks.workunit.client.0.smithi007.stdout:0/778: readlink d3/d6c/dbd/dd7/d7c/lfb 0 2024-03-16T09:01:17.459 INFO:tasks.workunit.client.0.smithi007.stdout:2/752: mknod dc/df/d24/d28/d97/cf7 0 2024-03-16T09:01:17.463 INFO:tasks.workunit.client.1.smithi019.stdout:8/543: dread d6/d2a/d45/d19/d74/f7e [0,4194304] 0 2024-03-16T09:01:17.464 INFO:tasks.workunit.client.0.smithi007.stdout:0/779: dread d3/d9/da/d48/d25/f68 [0,4194304] 0 2024-03-16T09:01:17.464 INFO:tasks.workunit.client.1.smithi019.stdout:1/430: truncate f14 405131 0 2024-03-16T09:01:17.466 INFO:tasks.workunit.client.0.smithi007.stdout:2/753: mknod dc/df/d24/cf8 0 2024-03-16T09:01:17.467 INFO:tasks.workunit.client.1.smithi019.stdout:1/431: truncate d16/d18/f1a 3786579 0 2024-03-16T09:01:17.467 INFO:tasks.workunit.client.1.smithi019.stdout:1/432: dread - d16/d1f/d4f/f62 zero size 2024-03-16T09:01:17.467 INFO:tasks.workunit.client.1.smithi019.stdout:1/433: dread - d16/d31/f69 zero size 2024-03-16T09:01:17.468 INFO:tasks.workunit.client.1.smithi019.stdout:8/544: mknod d6/d2a/d3c/dab/cbc 0 2024-03-16T09:01:17.468 INFO:tasks.workunit.client.1.smithi019.stdout:8/545: chown d6/d2a/d45/d57 120842 1 2024-03-16T09:01:17.469 INFO:tasks.workunit.client.1.smithi019.stdout:1/434: symlink d16/d1f/d4f/d68/l7a 0 2024-03-16T09:01:17.470 INFO:tasks.workunit.client.1.smithi019.stdout:1/435: mkdir d16/d31/d7b 0 2024-03-16T09:01:17.472 INFO:tasks.workunit.client.0.smithi007.stdout:7/607: dread d0/d6c/fb5 [0,4194304] 0 2024-03-16T09:01:17.472 INFO:tasks.workunit.client.0.smithi007.stdout:7/608: dread - d0/d6c/d25/d7a/fd4 zero size 2024-03-16T09:01:17.473 INFO:tasks.workunit.client.1.smithi019.stdout:1/436: mkdir d16/d1f/d4f/d68/d78/d7c 0 2024-03-16T09:01:17.473 INFO:tasks.workunit.client.1.smithi019.stdout:1/437: dread - d16/d31/f69 zero size 2024-03-16T09:01:17.478 INFO:tasks.workunit.client.0.smithi007.stdout:7/609: rename d0/d19/d1f/d5d/d8a to d0/dc/d5a/dba/dd9 0 2024-03-16T09:01:17.478 INFO:tasks.workunit.client.0.smithi007.stdout:7/610: stat d0/d6c/d25/d7a/dc9/f7b 0 2024-03-16T09:01:17.478 INFO:tasks.workunit.client.0.smithi007.stdout:7/611: chown d0/d19/d1f/d69/f87 19 1 2024-03-16T09:01:17.480 INFO:tasks.workunit.client.0.smithi007.stdout:7/612: rename d0/d19/d1f/d5d/fa7 to d0/d6c/d25/d3a/daf/fda 0 2024-03-16T09:01:17.480 INFO:tasks.workunit.client.0.smithi007.stdout:7/613: write d0/d19/d1f/d69/fc6 [851134,12709] 0 2024-03-16T09:01:17.481 INFO:tasks.workunit.client.0.smithi007.stdout:7/614: truncate d0/d6c/d25/d3a/daf/fda 243676 0 2024-03-16T09:01:17.481 INFO:tasks.workunit.client.0.smithi007.stdout:0/780: dread d3/d9/da/d18/f6a [0,4194304] 0 2024-03-16T09:01:17.482 INFO:tasks.workunit.client.0.smithi007.stdout:7/615: creat d0/d6c/d25/d3a/daf/fdb x:0 0 0 2024-03-16T09:01:17.482 INFO:tasks.workunit.client.0.smithi007.stdout:7/616: creat d0/dc/d5a/fdc x:0 0 0 2024-03-16T09:01:17.482 INFO:tasks.workunit.client.0.smithi007.stdout:7/617: fsync d0/d19/d1f/f82 0 2024-03-16T09:01:17.483 INFO:tasks.workunit.client.0.smithi007.stdout:7/618: symlink d0/d6c/d25/da1/ldd 0 2024-03-16T09:01:17.484 INFO:tasks.workunit.client.0.smithi007.stdout:7/619: mknod d0/d6c/d25/da1/cde 0 2024-03-16T09:01:17.485 INFO:tasks.workunit.client.0.smithi007.stdout:7/620: symlink d0/d6c/ldf 0 2024-03-16T09:01:17.485 INFO:tasks.workunit.client.0.smithi007.stdout:7/621: write d0/fa5 [450842,112414] 0 2024-03-16T09:01:17.486 INFO:tasks.workunit.client.0.smithi007.stdout:7/622: symlink d0/d6c/d25/d7a/le0 0 2024-03-16T09:01:17.486 INFO:tasks.workunit.client.0.smithi007.stdout:7/623: fsync d0/d19/d1f/d5d/f67 0 2024-03-16T09:01:17.487 INFO:tasks.workunit.client.0.smithi007.stdout:7/624: mkdir d0/d19/d1f/d69/de1 0 2024-03-16T09:01:17.487 INFO:tasks.workunit.client.0.smithi007.stdout:7/625: chown d0/d6c/d25/d3a/daf/fda 21807960 1 2024-03-16T09:01:17.488 INFO:tasks.workunit.client.0.smithi007.stdout:7/626: creat d0/d6c/d25/d7a/d9a/d9c/fe2 x:0 0 0 2024-03-16T09:01:17.489 INFO:tasks.workunit.client.0.smithi007.stdout:0/781: write d3/d9/da/d48/d4b/f51 [250370,10264] 0 2024-03-16T09:01:17.489 INFO:tasks.workunit.client.0.smithi007.stdout:0/782: readlink d3/d9/da/d48/d25/d43/l2e 0 2024-03-16T09:01:17.489 INFO:tasks.workunit.client.0.smithi007.stdout:0/783: read d3/d9/da/d18/d23/f45 [3316456,45064] 0 2024-03-16T09:01:17.489 INFO:tasks.workunit.client.0.smithi007.stdout:0/784: chown d3/d9/da/d48/d25/d43/d1a/leb 297 1 2024-03-16T09:01:17.490 INFO:tasks.workunit.client.0.smithi007.stdout:3/683: dwrite d15/d18/d6a/db1/fa3 [0,4194304] 0 2024-03-16T09:01:17.490 INFO:tasks.workunit.client.0.smithi007.stdout:3/684: chown d15/d18/d1f/d29/d3e/dc0 1870304 1 2024-03-16T09:01:17.492 INFO:tasks.workunit.client.0.smithi007.stdout:3/685: creat d15/d18/fd4 x:0 0 0 2024-03-16T09:01:17.493 INFO:tasks.workunit.client.0.smithi007.stdout:3/686: mknod d15/d18/d1f/d29/d43/d72/cd5 0 2024-03-16T09:01:17.494 INFO:tasks.workunit.client.0.smithi007.stdout:3/687: chown d15/d18/d6a/db1/fa3 0 1 2024-03-16T09:01:17.495 INFO:tasks.workunit.client.0.smithi007.stdout:3/688: dread d15/d18/d1f/d29/d43/f69 [0,4194304] 0 2024-03-16T09:01:17.495 INFO:tasks.workunit.client.1.smithi019.stdout:7/403: dwrite d9/d25/d28/f69 [0,4194304] 0 2024-03-16T09:01:17.497 INFO:tasks.workunit.client.1.smithi019.stdout:7/404: mknod d9/da/d13/d18/d30/d75/c7d 0 2024-03-16T09:01:17.498 INFO:tasks.workunit.client.1.smithi019.stdout:7/405: fsync d9/da/d13/d18/d30/d3e/d5d/f7c 0 2024-03-16T09:01:17.498 INFO:tasks.workunit.client.1.smithi019.stdout:6/602: dwrite de/f15 [0,4194304] 0 2024-03-16T09:01:17.498 INFO:tasks.workunit.client.0.smithi007.stdout:2/754: dwrite dc/df/d24/d28/d2e/d35/f98 [0,4194304] 0 2024-03-16T09:01:17.499 INFO:tasks.workunit.client.1.smithi019.stdout:1/438: dwrite d16/d1f/d4f/f58 [0,4194304] 0 2024-03-16T09:01:17.499 INFO:tasks.workunit.client.1.smithi019.stdout:1/439: chown d16/d1f/c3e 1048390 1 2024-03-16T09:01:17.499 INFO:tasks.workunit.client.1.smithi019.stdout:1/440: read - d16/d18/f6c zero size 2024-03-16T09:01:17.499 INFO:tasks.workunit.client.1.smithi019.stdout:7/406: symlink d9/db/d12/l7e 0 2024-03-16T09:01:17.499 INFO:tasks.workunit.client.0.smithi007.stdout:3/689: unlink d15/d18/d1f/c6f 0 2024-03-16T09:01:17.500 INFO:tasks.workunit.client.1.smithi019.stdout:6/603: mkdir de/d26/d32/dc1 0 2024-03-16T09:01:17.503 INFO:tasks.workunit.client.1.smithi019.stdout:7/407: unlink d9/d25/d28/f39 0 2024-03-16T09:01:17.503 INFO:tasks.workunit.client.1.smithi019.stdout:7/408: stat d9/da/c64 0 2024-03-16T09:01:17.504 INFO:tasks.workunit.client.1.smithi019.stdout:4/455: rename d4/d6/d15/d1f/d2c to d4/d64/d99 0 2024-03-16T09:01:17.505 INFO:tasks.workunit.client.0.smithi007.stdout:3/690: rename d15/d18/d1f/d29/dc3 to d15/d18/d1f/dce/dd6 0 2024-03-16T09:01:17.506 INFO:tasks.workunit.client.0.smithi007.stdout:2/755: dread dc/f11 [0,4194304] 0 2024-03-16T09:01:17.506 INFO:tasks.workunit.client.0.smithi007.stdout:2/756: write dc/db7/fe8 [534172,34831] 0 2024-03-16T09:01:17.507 INFO:tasks.workunit.client.1.smithi019.stdout:8/546: rename d6/d2a/d45/d57/l64 to d6/d2a/d45/d57/d72/lbd 0 2024-03-16T09:01:17.508 INFO:tasks.workunit.client.0.smithi007.stdout:3/691: symlink d15/d18/d1f/d29/d3e/d85/daf/ld7 0 2024-03-16T09:01:17.508 INFO:tasks.workunit.client.0.smithi007.stdout:3/692: truncate d15/f2c 5148362 0 2024-03-16T09:01:17.509 INFO:tasks.workunit.client.0.smithi007.stdout:2/757: mknod dc/df/d24/d28/d97/da4/db8/df1/cf9 0 2024-03-16T09:01:17.509 INFO:tasks.workunit.client.0.smithi007.stdout:2/758: write dc/df/d6c/fec [74707,86138] 0 2024-03-16T09:01:17.510 INFO:tasks.workunit.client.1.smithi019.stdout:1/441: dread d16/d1f/d4f/f5c [0,4194304] 0 2024-03-16T09:01:17.511 INFO:tasks.workunit.client.0.smithi007.stdout:3/693: mknod d15/d18/d6a/db1/cd8 0 2024-03-16T09:01:17.512 INFO:tasks.workunit.client.0.smithi007.stdout:2/759: symlink dc/db7/lfa 0 2024-03-16T09:01:17.514 INFO:tasks.workunit.client.1.smithi019.stdout:6/604: dread de/d26/d32/d97/f30 [0,4194304] 0 2024-03-16T09:01:17.514 INFO:tasks.workunit.client.0.smithi007.stdout:7/627: dwrite d0/dc/d5a/dba/dd9/d93/fcd [0,4194304] 0 2024-03-16T09:01:17.514 INFO:tasks.workunit.client.0.smithi007.stdout:7/628: readlink d0/d19/d1f/l78 0 2024-03-16T09:01:17.515 INFO:tasks.workunit.client.0.smithi007.stdout:7/629: write d0/fa5 [1297984,35713] 0 2024-03-16T09:01:17.515 INFO:tasks.workunit.client.0.smithi007.stdout:3/694: mknod d15/d18/d1f/d3b/cd9 0 2024-03-16T09:01:17.515 INFO:tasks.workunit.client.0.smithi007.stdout:0/785: dwrite d3/d9/da/d18/f7d [0,4194304] 0 2024-03-16T09:01:17.517 INFO:tasks.workunit.client.0.smithi007.stdout:2/760: write f7 [349081,62435] 0 2024-03-16T09:01:17.517 INFO:tasks.workunit.client.0.smithi007.stdout:2/761: write dc/df/d6c/fec [99872,61463] 0 2024-03-16T09:01:17.517 INFO:tasks.workunit.client.0.smithi007.stdout:2/762: chown dc/df/d24/l7b 178260 1 2024-03-16T09:01:17.517 INFO:tasks.workunit.client.1.smithi019.stdout:6/605: rename de/d26/l99 to de/d26/d32/d97/d51/d70/lc2 0 2024-03-16T09:01:17.520 INFO:tasks.workunit.client.1.smithi019.stdout:4/456: read d4/d64/d99/f3f [2830907,31124] 0 2024-03-16T09:01:17.521 INFO:tasks.workunit.client.0.smithi007.stdout:7/630: link d0/d19/d1f/d69/cd0 d0/ce3 0 2024-03-16T09:01:17.522 INFO:tasks.workunit.client.0.smithi007.stdout:3/695: mkdir d15/d18/d1f/d29/d3e/dda 0 2024-03-16T09:01:17.522 INFO:tasks.workunit.client.1.smithi019.stdout:4/457: truncate d4/d6/d15/f33 2508877 0 2024-03-16T09:01:17.524 INFO:tasks.workunit.client.0.smithi007.stdout:0/786: dread d3/d1f/fb7 [0,4194304] 0 2024-03-16T09:01:17.524 INFO:tasks.workunit.client.0.smithi007.stdout:0/787: chown d3/d9/da/d48/d25/f68 9166 1 2024-03-16T09:01:17.525 INFO:tasks.workunit.client.1.smithi019.stdout:4/458: mkdir d4/d6/d15/d9a 0 2024-03-16T09:01:17.527 INFO:tasks.workunit.client.0.smithi007.stdout:7/631: mknod d0/d6c/d25/ce4 0 2024-03-16T09:01:17.528 INFO:tasks.workunit.client.1.smithi019.stdout:4/459: mkdir d4/d64/d9b 0 2024-03-16T09:01:17.529 INFO:tasks.workunit.client.0.smithi007.stdout:0/788: mknod d3/d1f/c101 0 2024-03-16T09:01:17.532 INFO:tasks.workunit.client.0.smithi007.stdout:0/789: mknod d3/d9/da/d92/d9f/c102 0 2024-03-16T09:01:17.533 INFO:tasks.workunit.client.1.smithi019.stdout:7/409: dwrite d9/d25/d28/d48/f7b [0,4194304] 0 2024-03-16T09:01:17.533 INFO:tasks.workunit.client.1.smithi019.stdout:4/460: link d4/f35 d4/d1b/d49/f9c 0 2024-03-16T09:01:17.533 INFO:tasks.workunit.client.1.smithi019.stdout:7/410: truncate d9/da/d13/d18/d30/d3e/f62 994678 0 2024-03-16T09:01:17.533 INFO:tasks.workunit.client.0.smithi007.stdout:7/632: rmdir d0/d6c/d25/d3a/db3 39 2024-03-16T09:01:17.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/790: rmdir d3/d9/da/d48/d25/d87/dd0 0 2024-03-16T09:01:17.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/791: truncate d3/d9/da/d48/d25/d43/fdc 269401 0 2024-03-16T09:01:17.536 INFO:tasks.workunit.client.1.smithi019.stdout:4/461: mknod d4/d1b/d49/d58/d80/d8e/c9d 0 2024-03-16T09:01:17.536 INFO:tasks.workunit.client.0.smithi007.stdout:0/792: read d3/d9/da/d18/d27/f9a [1000743,85653] 0 2024-03-16T09:01:17.537 INFO:tasks.workunit.client.1.smithi019.stdout:7/411: symlink d9/d25/d28/l7f 0 2024-03-16T09:01:17.537 INFO:tasks.workunit.client.1.smithi019.stdout:7/412: dread - d9/db/d12/f6d zero size 2024-03-16T09:01:17.538 INFO:tasks.workunit.client.1.smithi019.stdout:7/413: dread d9/d25/f31 [4194304,4194304] 0 2024-03-16T09:01:17.539 INFO:tasks.workunit.client.0.smithi007.stdout:0/793: rename d3/d9/da/d48/d25/d87/f98 to d3/d9/da/d48/d25/d43/f103 0 2024-03-16T09:01:17.539 INFO:tasks.workunit.client.0.smithi007.stdout:0/794: fdatasync d3/d9/da/d18/d27/d34/fea 0 2024-03-16T09:01:17.539 INFO:tasks.workunit.client.0.smithi007.stdout:0/795: chown d3/d9/db4/d2c/c53 2 1 2024-03-16T09:01:17.539 INFO:tasks.workunit.client.0.smithi007.stdout:0/796: chown d3/d1f/dca 0 1 2024-03-16T09:01:17.540 INFO:tasks.workunit.client.0.smithi007.stdout:0/797: creat d3/d74/f104 x:0 0 0 2024-03-16T09:01:17.554 INFO:tasks.workunit.client.1.smithi019.stdout:7/414: dread d9/db/d12/d5b/d33/f43 [0,4194304] 0 2024-03-16T09:01:17.554 INFO:tasks.workunit.client.1.smithi019.stdout:7/415: fdatasync d9/db/d12/d5b/d33/f7a 0 2024-03-16T09:01:17.555 INFO:tasks.workunit.client.1.smithi019.stdout:7/416: mknod d9/da/d13/d18/d1a/c80 0 2024-03-16T09:01:17.555 INFO:tasks.workunit.client.0.smithi007.stdout:0/798: dread d3/d9/da/d48/d25/d43/d1a/d58/d61/f7a [0,4194304] 0 2024-03-16T09:01:17.556 INFO:tasks.workunit.client.1.smithi019.stdout:7/417: symlink d9/d25/d28/d48/l81 0 2024-03-16T09:01:17.557 INFO:tasks.workunit.client.1.smithi019.stdout:7/418: symlink d9/da/d13/d18/d30/d75/l82 0 2024-03-16T09:01:17.561 INFO:tasks.workunit.client.1.smithi019.stdout:6/606: dwrite f7 [0,4194304] 0 2024-03-16T09:01:17.561 INFO:tasks.workunit.client.1.smithi019.stdout:6/607: readlink de/d26/d32/d97/d66/la7 0 2024-03-16T09:01:17.561 INFO:tasks.workunit.client.1.smithi019.stdout:6/608: write de/d26/d32/d4e/f79 [5309243,80639] 0 2024-03-16T09:01:17.562 INFO:tasks.workunit.client.0.smithi007.stdout:3/696: dwrite d15/d18/d1f/d29/d7c/fb3 [0,4194304] 0 2024-03-16T09:01:17.563 INFO:tasks.workunit.client.1.smithi019.stdout:1/442: dwrite d16/d31/d76/f6b [0,4194304] 0 2024-03-16T09:01:17.564 INFO:tasks.workunit.client.1.smithi019.stdout:1/443: read d16/d18/f1a [1240197,93607] 0 2024-03-16T09:01:17.564 INFO:tasks.workunit.client.0.smithi007.stdout:2/763: dwrite dc/df/d24/d28/d2e/d6e/f9a [0,4194304] 0 2024-03-16T09:01:17.564 INFO:tasks.workunit.client.0.smithi007.stdout:2/764: write dc/df/d6c/fe0 [1341455,29503] 0 2024-03-16T09:01:17.564 INFO:tasks.workunit.client.0.smithi007.stdout:7/633: dwrite d0/d6c/d25/d3a/daf/fdb [0,4194304] 0 2024-03-16T09:01:17.564 INFO:tasks.workunit.client.0.smithi007.stdout:7/634: read d0/d19/f26 [5631536,35097] 0 2024-03-16T09:01:17.566 INFO:tasks.workunit.client.0.smithi007.stdout:3/697: write d15/d18/d1f/d29/d7c/f8c [183189,114256] 0 2024-03-16T09:01:17.566 INFO:tasks.workunit.client.1.smithi019.stdout:4/462: dwrite d4/d1b/d2a/f4e [0,4194304] 0 2024-03-16T09:01:17.571 INFO:tasks.workunit.client.1.smithi019.stdout:6/609: dread de/d26/d32/d97/d51/d70/f91 [0,4194304] 0 2024-03-16T09:01:17.571 INFO:tasks.workunit.client.1.smithi019.stdout:6/610: write de/d26/d32/d97/fb8 [714474,127151] 0 2024-03-16T09:01:17.571 INFO:tasks.workunit.client.1.smithi019.stdout:6/611: chown f7 0 1 2024-03-16T09:01:17.574 INFO:tasks.workunit.client.1.smithi019.stdout:1/444: mknod d16/d1f/d73/c7d 0 2024-03-16T09:01:17.576 INFO:tasks.workunit.client.0.smithi007.stdout:3/698: write d15/d18/d1f/d56/f88 [23863,10921] 0 2024-03-16T09:01:17.577 INFO:tasks.workunit.client.1.smithi019.stdout:4/463: mkdir d4/d9e 0 2024-03-16T09:01:17.579 INFO:tasks.workunit.client.1.smithi019.stdout:4/464: mknod d4/d6/c9f 0 2024-03-16T09:01:17.580 INFO:tasks.workunit.client.0.smithi007.stdout:7/635: rename d0/d6c/d25/d3a/l47 to d0/d6c/d25/d7a/dc9/le5 0 2024-03-16T09:01:17.581 INFO:tasks.workunit.client.1.smithi019.stdout:4/465: unlink d4/d1b/d49/f56 0 2024-03-16T09:01:17.582 INFO:tasks.workunit.client.0.smithi007.stdout:3/699: dread d15/d18/d1f/d56/f8a [0,4194304] 0 2024-03-16T09:01:17.582 INFO:tasks.workunit.client.0.smithi007.stdout:3/700: fdatasync fa 0 2024-03-16T09:01:17.582 INFO:tasks.workunit.client.0.smithi007.stdout:3/701: write d15/d18/d1f/d3b/d63/fb7 [203674,12956] 0 2024-03-16T09:01:17.582 INFO:tasks.workunit.client.1.smithi019.stdout:4/466: truncate d4/d1b/f28 496078 0 2024-03-16T09:01:17.588 INFO:tasks.workunit.client.1.smithi019.stdout:6/612: dread de/d26/d32/d97/f30 [4194304,4194304] 0 2024-03-16T09:01:17.588 INFO:tasks.workunit.client.1.smithi019.stdout:6/613: chown de/d26/d32/d97/d2c/c5e 15 1 2024-03-16T09:01:17.588 INFO:tasks.workunit.client.0.smithi007.stdout:7/636: rmdir d0/dc/d38/d4e 39 2024-03-16T09:01:17.588 INFO:tasks.workunit.client.0.smithi007.stdout:7/637: fdatasync d0/d6c/d25/d7a/dc9/f98 0 2024-03-16T09:01:17.591 INFO:tasks.workunit.client.0.smithi007.stdout:7/638: creat d0/d6c/d25/d3a/daf/dc2/fe6 x:0 0 0 2024-03-16T09:01:17.592 INFO:tasks.workunit.client.0.smithi007.stdout:0/799: dwrite d3/d9/da/d18/d23/f63 [0,4194304] 0 2024-03-16T09:01:17.593 INFO:tasks.workunit.client.0.smithi007.stdout:0/800: chown d3/d9/da/d48/d4b/f51 203 1 2024-03-16T09:01:17.593 INFO:tasks.workunit.client.0.smithi007.stdout:7/639: truncate d0/f11 1879429 0 2024-03-16T09:01:17.593 INFO:tasks.workunit.client.0.smithi007.stdout:7/640: write d0/d6c/d25/d73/f5c [495509,129481] 0 2024-03-16T09:01:17.593 INFO:tasks.workunit.client.1.smithi019.stdout:7/419: dwrite d9/f23 [0,4194304] 0 2024-03-16T09:01:17.596 INFO:tasks.workunit.client.0.smithi007.stdout:2/765: dwrite dc/df/d24/d28/d2e/f5c [0,4194304] 0 2024-03-16T09:01:17.596 INFO:tasks.workunit.client.0.smithi007.stdout:2/766: fdatasync dc/df/d24/d28/d4c/f59 0 2024-03-16T09:01:17.597 INFO:tasks.workunit.client.0.smithi007.stdout:0/801: mknod d3/d6c/dbd/dd7/d7c/c105 0 2024-03-16T09:01:17.597 INFO:tasks.workunit.client.0.smithi007.stdout:0/802: stat d3/d9/db4/d1c/f2a 0 2024-03-16T09:01:17.598 INFO:tasks.workunit.client.1.smithi019.stdout:1/445: dwrite d16/d18/f23 [0,4194304] 0 2024-03-16T09:01:17.600 INFO:tasks.workunit.client.1.smithi019.stdout:9/394: sync 2024-03-16T09:01:17.600 INFO:tasks.workunit.client.1.smithi019.stdout:2/540: sync 2024-03-16T09:01:17.600 INFO:tasks.workunit.client.1.smithi019.stdout:5/501: sync 2024-03-16T09:01:17.600 INFO:tasks.workunit.client.1.smithi019.stdout:9/395: readlink d1/d7/d21/l4b 0 2024-03-16T09:01:17.600 INFO:tasks.workunit.client.1.smithi019.stdout:0/544: sync 2024-03-16T09:01:17.601 INFO:tasks.workunit.client.1.smithi019.stdout:3/529: sync 2024-03-16T09:01:17.602 INFO:tasks.workunit.client.1.smithi019.stdout:5/502: dread d6/d1f/f5e [4194304,4194304] 0 2024-03-16T09:01:17.603 INFO:tasks.workunit.client.1.smithi019.stdout:1/446: symlink d16/d1f/d3c/d79/l7e 0 2024-03-16T09:01:17.604 INFO:tasks.workunit.client.0.smithi007.stdout:2/767: mkdir dc/df/d24/d28/d2e/d35/dfb 0 2024-03-16T09:01:17.605 INFO:tasks.workunit.client.1.smithi019.stdout:3/530: rename d3/f9a to d3/d8b/fa3 0 2024-03-16T09:01:17.606 INFO:tasks.workunit.client.1.smithi019.stdout:6/614: dread de/d26/f2a [0,4194304] 0 2024-03-16T09:01:17.606 INFO:tasks.workunit.client.1.smithi019.stdout:5/503: link d6/d1f/f55 d6/d1f/d33/d66/f9f 0 2024-03-16T09:01:17.608 INFO:tasks.workunit.client.1.smithi019.stdout:6/615: symlink de/d1c/d9a/d9e/lc3 0 2024-03-16T09:01:17.609 INFO:tasks.workunit.client.1.smithi019.stdout:5/504: truncate d6/f9 7446833 0 2024-03-16T09:01:17.609 INFO:tasks.workunit.client.1.smithi019.stdout:5/505: fdatasync d6/dd/f67 0 2024-03-16T09:01:17.609 INFO:tasks.workunit.client.1.smithi019.stdout:5/506: chown d6/d1f/d33/d89 63498 1 2024-03-16T09:01:17.609 INFO:tasks.workunit.client.0.smithi007.stdout:0/803: dread d3/d9/da/d18/d23/f45 [0,4194304] 0 2024-03-16T09:01:17.610 INFO:tasks.workunit.client.1.smithi019.stdout:6/616: write de/f14 [3854417,65861] 0 2024-03-16T09:01:17.610 INFO:tasks.workunit.client.1.smithi019.stdout:7/420: write d9/d25/d28/d48/f7b [3302467,9631] 0 2024-03-16T09:01:17.610 INFO:tasks.workunit.client.1.smithi019.stdout:6/617: stat de/d26/d32/d97/d66/d72/db7 0 2024-03-16T09:01:17.610 INFO:tasks.workunit.client.1.smithi019.stdout:5/507: rename d6/dd/c1e to d6/dd/d47/d56/ca0 0 2024-03-16T09:01:17.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/421: rename d9/d25/d28/f34 to d9/da/d13/d18/f83 0 2024-03-16T09:01:17.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/422: chown d9/da/d13/d18/d1a/c80 12258911 1 2024-03-16T09:01:17.620 INFO:tasks.workunit.client.1.smithi019.stdout:6/618: rename de/l31 to de/d26/d32/d97/d66/d72/db7/lc4 0 2024-03-16T09:01:17.621 INFO:tasks.workunit.client.1.smithi019.stdout:6/619: stat de/d26/d32/f8a 0 2024-03-16T09:01:17.621 INFO:tasks.workunit.client.1.smithi019.stdout:6/620: creat de/d26/d32/d97/fc5 x:0 0 0 2024-03-16T09:01:17.621 INFO:tasks.workunit.client.1.smithi019.stdout:6/621: dread - de/d26/d32/f8a zero size 2024-03-16T09:01:17.623 INFO:tasks.workunit.client.1.smithi019.stdout:7/423: unlink d9/da/d13/l3c 0 2024-03-16T09:01:17.623 INFO:tasks.workunit.client.1.smithi019.stdout:7/424: read - d9/db/d12/f73 zero size 2024-03-16T09:01:17.624 INFO:tasks.workunit.client.1.smithi019.stdout:6/622: creat de/d26/fc6 x:0 0 0 2024-03-16T09:01:17.626 INFO:tasks.workunit.client.1.smithi019.stdout:6/623: truncate de/d26/d32/d97/f46 2208274 0 2024-03-16T09:01:17.628 INFO:tasks.workunit.client.1.smithi019.stdout:6/624: link de/d26/d32/d97/d66/lad de/d26/d32/d97/d66/d72/db7/lc7 0 2024-03-16T09:01:17.631 INFO:tasks.workunit.client.1.smithi019.stdout:6/625: rename de/d26/d32/d97/d2c/c33 to de/d1c/d9a/d9e/cc8 0 2024-03-16T09:01:17.631 INFO:tasks.workunit.client.1.smithi019.stdout:6/626: fsync de/d1c/f28 0 2024-03-16T09:01:17.631 INFO:tasks.workunit.client.1.smithi019.stdout:3/531: dread d3/d1b/f8e [0,4194304] 0 2024-03-16T09:01:17.631 INFO:tasks.workunit.client.1.smithi019.stdout:4/467: dwrite d4/d1b/f7e [0,4194304] 0 2024-03-16T09:01:17.632 INFO:tasks.workunit.client.1.smithi019.stdout:7/425: read d9/db/d12/d5b/d33/f58 [1895647,99895] 0 2024-03-16T09:01:17.633 INFO:tasks.workunit.client.1.smithi019.stdout:3/532: stat d3/d8b/c97 0 2024-03-16T09:01:17.635 INFO:tasks.workunit.client.1.smithi019.stdout:4/468: mkdir d4/d1b/d2a/d4f/da0 0 2024-03-16T09:01:17.637 INFO:tasks.workunit.client.1.smithi019.stdout:7/426: creat d9/da/d13/f84 x:0 0 0 2024-03-16T09:01:17.637 INFO:tasks.workunit.client.1.smithi019.stdout:7/427: stat f7 0 2024-03-16T09:01:17.638 INFO:tasks.workunit.client.1.smithi019.stdout:3/533: write d3/d1b/f6e [756991,93495] 0 2024-03-16T09:01:17.639 INFO:tasks.workunit.client.1.smithi019.stdout:4/469: unlink d4/d1b/d49/f75 0 2024-03-16T09:01:17.641 INFO:tasks.workunit.client.1.smithi019.stdout:3/534: link d3/d1b/d40/f9d d3/d17/d3f/fa4 0 2024-03-16T09:01:17.641 INFO:tasks.workunit.client.1.smithi019.stdout:3/535: fdatasync d3/d17/d6a/f7e 0 2024-03-16T09:01:17.641 INFO:tasks.workunit.client.1.smithi019.stdout:4/470: rmdir d4/d1b/d49/d58/d80/d8e 39 2024-03-16T09:01:17.643 INFO:tasks.workunit.client.1.smithi019.stdout:3/536: creat d3/d1b/d88/fa5 x:0 0 0 2024-03-16T09:01:17.643 INFO:tasks.workunit.client.1.smithi019.stdout:3/537: fdatasync d3/d1b/f9e 0 2024-03-16T09:01:17.643 INFO:tasks.workunit.client.1.smithi019.stdout:3/538: readlink d3/d17/l8a 0 2024-03-16T09:01:17.644 INFO:tasks.workunit.client.1.smithi019.stdout:3/539: creat d3/d8b/fa6 x:0 0 0 2024-03-16T09:01:17.684 INFO:tasks.workunit.client.1.smithi019.stdout:3/540: dread d3/d1b/d40/f74 [0,4194304] 0 2024-03-16T09:01:17.684 INFO:tasks.workunit.client.1.smithi019.stdout:3/541: rename d3/d17/d5a to d3/d17/d5a/da7 22 2024-03-16T09:01:17.685 INFO:tasks.workunit.client.1.smithi019.stdout:3/542: mkdir d3/d8b/da8 0 2024-03-16T09:01:17.685 INFO:tasks.workunit.client.1.smithi019.stdout:3/543: creat d3/d9b/fa9 x:0 0 0 2024-03-16T09:01:17.686 INFO:tasks.workunit.client.1.smithi019.stdout:5/508: dwrite d6/dd/f82 [0,4194304] 0 2024-03-16T09:01:17.686 INFO:tasks.workunit.client.1.smithi019.stdout:4/471: dwrite d4/d6/d15/f67 [4194304,4194304] 0 2024-03-16T09:01:17.687 INFO:tasks.workunit.client.0.smithi007.stdout:3/702: dwrite d15/d18/d1f/f8d [0,4194304] 0 2024-03-16T09:01:17.687 INFO:tasks.workunit.client.0.smithi007.stdout:3/703: rename d15 to d15/d18/d1f/d3b/d63/ddb 22 2024-03-16T09:01:17.687 INFO:tasks.workunit.client.0.smithi007.stdout:3/704: rename d15 to d15/d18/d1f/d3b/ddc 22 2024-03-16T09:01:17.687 INFO:tasks.workunit.client.0.smithi007.stdout:3/705: chown d15/d18/d1f/d56/f8a 1746406 1 2024-03-16T09:01:17.687 INFO:tasks.workunit.client.1.smithi019.stdout:2/541: dwrite d5/de/f4a [0,4194304] 0 2024-03-16T09:01:17.688 INFO:tasks.workunit.client.1.smithi019.stdout:3/544: rename d3/d17/d6a/f93 to d3/d17/d6a/faa 0 2024-03-16T09:01:17.689 INFO:tasks.workunit.client.1.smithi019.stdout:5/509: rename d6/d1f/f20 to d6/dd/d37/d76/fa1 0 2024-03-16T09:01:17.689 INFO:tasks.workunit.client.1.smithi019.stdout:5/510: creat d6/dd/d37/d54/fa2 x:0 0 0 2024-03-16T09:01:17.689 INFO:tasks.workunit.client.0.smithi007.stdout:2/768: dwrite dc/df/d24/d28/d97/da4/fa6 [0,4194304] 0 2024-03-16T09:01:17.690 INFO:tasks.workunit.client.0.smithi007.stdout:3/706: creat d15/d18/d1f/d3b/d87/fdd x:0 0 0 2024-03-16T09:01:17.690 INFO:tasks.workunit.client.0.smithi007.stdout:3/707: chown d15/d18/f23 390801859 1 2024-03-16T09:01:17.690 INFO:tasks.workunit.client.0.smithi007.stdout:7/641: dwrite d0/d19/d1f/d5d/d9d/fbe [0,4194304] 0 2024-03-16T09:01:17.690 INFO:tasks.workunit.client.0.smithi007.stdout:7/642: chown d0/d6c/d25/d73 59096 1 2024-03-16T09:01:17.691 INFO:tasks.workunit.client.0.smithi007.stdout:0/804: dwrite d3/d9/da/d18/d23/d36/f100 [0,4194304] 0 2024-03-16T09:01:17.691 INFO:tasks.workunit.client.0.smithi007.stdout:0/805: write d3/d9/da/d18/d23/d36/fef [554973,107760] 0 2024-03-16T09:01:17.692 INFO:tasks.workunit.client.1.smithi019.stdout:7/428: dwrite f4 [0,4194304] 0 2024-03-16T09:01:17.696 INFO:tasks.workunit.client.1.smithi019.stdout:5/511: symlink d6/dd/d37/la3 0 2024-03-16T09:01:17.696 INFO:tasks.workunit.client.1.smithi019.stdout:7/429: symlink d9/da/d13/d18/d1a/d6e/l85 0 2024-03-16T09:01:17.696 INFO:tasks.workunit.client.1.smithi019.stdout:7/430: write d9/d25/f31 [93592,12226] 0 2024-03-16T09:01:17.696 INFO:tasks.workunit.client.1.smithi019.stdout:7/431: readlink d9/da/d13/d18/l40 0 2024-03-16T09:01:17.699 INFO:tasks.workunit.client.1.smithi019.stdout:6/627: dwrite de/d26/d32/d97/d2c/f3a [4194304,4194304] 0 2024-03-16T09:01:17.700 INFO:tasks.workunit.client.0.smithi007.stdout:7/643: creat d0/dc/d5a/dba/dd9/d93/fe7 x:0 0 0 2024-03-16T09:01:17.700 INFO:tasks.workunit.client.0.smithi007.stdout:7/644: dread - d0/d6c/d25/d7a/fd4 zero size 2024-03-16T09:01:17.713 INFO:tasks.workunit.client.0.smithi007.stdout:0/806: symlink d3/d1f/l106 0 2024-03-16T09:01:17.716 INFO:tasks.workunit.client.1.smithi019.stdout:4/472: dwrite d4/d1b/d49/d6e/f90 [4194304,4194304] 0 2024-03-16T09:01:17.717 INFO:tasks.workunit.client.0.smithi007.stdout:7/645: rename d0/fa to d0/d6c/d25/d7a/fe8 0 2024-03-16T09:01:17.718 INFO:tasks.workunit.client.1.smithi019.stdout:8/547: sync 2024-03-16T09:01:17.720 INFO:tasks.workunit.client.0.smithi007.stdout:0/807: creat d3/d9/da/d48/d25/d43/d1a/d58/f107 x:0 0 0 2024-03-16T09:01:17.720 INFO:tasks.workunit.client.0.smithi007.stdout:0/808: mknod d3/c108 0 2024-03-16T09:01:17.721 INFO:tasks.workunit.client.1.smithi019.stdout:4/473: creat d4/d64/d9b/fa1 x:0 0 0 2024-03-16T09:01:17.721 INFO:tasks.workunit.client.1.smithi019.stdout:8/548: mknod d6/d2a/d68/cbe 0 2024-03-16T09:01:17.721 INFO:tasks.workunit.client.1.smithi019.stdout:4/474: chown d4/d1b/d2a/d3d 286682 1 2024-03-16T09:01:17.721 INFO:tasks.workunit.client.1.smithi019.stdout:8/549: read - d6/f9e zero size 2024-03-16T09:01:17.724 INFO:tasks.workunit.client.1.smithi019.stdout:2/542: dwrite d5/d2e/f50 [4194304,4194304] 0 2024-03-16T09:01:17.724 INFO:tasks.workunit.client.1.smithi019.stdout:2/543: creat d5/fb2 x:0 0 0 2024-03-16T09:01:17.724 INFO:tasks.workunit.client.1.smithi019.stdout:2/544: chown d5/f4d 30 1 2024-03-16T09:01:17.724 INFO:tasks.workunit.client.0.smithi007.stdout:0/809: link d3/d9/da/d18/d27/d34/f57 d3/d9/db4/d2c/f109 0 2024-03-16T09:01:17.725 INFO:tasks.workunit.client.0.smithi007.stdout:0/810: fdatasync d3/d9/da/d48/d25/d87/dbb/fc5 0 2024-03-16T09:01:17.725 INFO:tasks.workunit.client.0.smithi007.stdout:0/811: readlink d3/d9/da/d48/d25/d87/dbb/ld3 0 2024-03-16T09:01:17.725 INFO:tasks.workunit.client.0.smithi007.stdout:0/812: chown d3 536 1 2024-03-16T09:01:17.725 INFO:tasks.workunit.client.0.smithi007.stdout:0/813: write d3/d9/da/d48/d25/d43/d1a/d58/d61/fe0 [734816,111132] 0 2024-03-16T09:01:17.726 INFO:tasks.workunit.client.0.smithi007.stdout:0/814: mknod d3/d9/db4/d2c/c10a 0 2024-03-16T09:01:17.728 INFO:tasks.workunit.client.0.smithi007.stdout:0/815: mknod d3/d9/db4/c10b 0 2024-03-16T09:01:17.728 INFO:tasks.workunit.client.0.smithi007.stdout:0/816: chown d3/d9/da/d48/d25/d43/l2e 510 1 2024-03-16T09:01:17.730 INFO:tasks.workunit.client.0.smithi007.stdout:0/817: creat d3/d1f/f10c x:0 0 0 2024-03-16T09:01:17.730 INFO:tasks.workunit.client.0.smithi007.stdout:0/818: write d3/d9/da/d48/fa9 [1688826,61247] 0 2024-03-16T09:01:17.730 INFO:tasks.workunit.client.0.smithi007.stdout:0/819: dread - d3/d9/da/d92/d9f/de8/fff zero size 2024-03-16T09:01:17.730 INFO:tasks.workunit.client.0.smithi007.stdout:0/820: stat d3/d1f 0 2024-03-16T09:01:17.731 INFO:tasks.workunit.client.0.smithi007.stdout:0/821: truncate d3/d9/da/d18/d23/f4d 184281 0 2024-03-16T09:01:17.732 INFO:tasks.workunit.client.0.smithi007.stdout:0/822: truncate d3/d9/da/d48/d25/d43/d91/fba 290449 0 2024-03-16T09:01:17.733 INFO:tasks.workunit.client.0.smithi007.stdout:0/823: symlink d3/d9/da/d48/d25/d87/dbb/dde/l10d 0 2024-03-16T09:01:17.735 INFO:tasks.workunit.client.0.smithi007.stdout:0/824: creat d3/d9/da/d18/d23/f10e x:0 0 0 2024-03-16T09:01:17.740 INFO:tasks.workunit.client.1.smithi019.stdout:2/545: dread d5/d6/d64/d6f/da6/d83/f8a [0,4194304] 0 2024-03-16T09:01:17.741 INFO:tasks.workunit.client.1.smithi019.stdout:2/546: creat d5/d5a/d80/fb3 x:0 0 0 2024-03-16T09:01:17.742 INFO:tasks.workunit.client.1.smithi019.stdout:2/547: unlink d5/d6/d64/d6f/d87/l9b 0 2024-03-16T09:01:17.742 INFO:tasks.workunit.client.1.smithi019.stdout:4/475: dread d4/d6c/d89/f93 [0,4194304] 0 2024-03-16T09:01:17.743 INFO:tasks.workunit.client.1.smithi019.stdout:4/476: chown d4/d6/d15/f66 44 1 2024-03-16T09:01:17.743 INFO:tasks.workunit.client.1.smithi019.stdout:2/548: link d5/d6/fb0 d5/d6/d2b/d2f/d35/d65/fb4 0 2024-03-16T09:01:17.744 INFO:tasks.workunit.client.1.smithi019.stdout:4/477: mkdir d4/d6/d15/da2 0 2024-03-16T09:01:17.744 INFO:tasks.workunit.client.1.smithi019.stdout:4/478: chown d4/d6/d15/d1f/f44 48051063 1 2024-03-16T09:01:17.756 INFO:tasks.workunit.client.0.smithi007.stdout:3/708: dwrite d15/d18/d1f/d3b/d63/f66 [0,4194304] 0 2024-03-16T09:01:17.757 INFO:tasks.workunit.client.1.smithi019.stdout:2/549: unlink d5/d2e/d4f/l69 0 2024-03-16T09:01:17.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/479: mkdir d4/da3 0 2024-03-16T09:01:17.758 INFO:tasks.workunit.client.0.smithi007.stdout:3/709: creat d15/d18/d1f/d56/fde x:0 0 0 2024-03-16T09:01:17.758 INFO:tasks.workunit.client.0.smithi007.stdout:3/710: write d15/d18/d1f/d56/f88 [1369022,93653] 0 2024-03-16T09:01:17.758 INFO:tasks.workunit.client.0.smithi007.stdout:3/711: chown d15/d18/d1f/d29/d3e/d85/fcd 1 1 2024-03-16T09:01:17.764 INFO:tasks.workunit.client.0.smithi007.stdout:3/712: write d15/d18/d6a/f7d [1024145,29243] 0 2024-03-16T09:01:17.765 INFO:tasks.workunit.client.0.smithi007.stdout:2/769: dwrite dc/df/d24/d28/d4c/daf/fe1 [0,4194304] 0 2024-03-16T09:01:17.765 INFO:tasks.workunit.client.0.smithi007.stdout:2/770: write dc/df/d24/d28/d4c/d5e/fdc [654796,36712] 0 2024-03-16T09:01:17.765 INFO:tasks.workunit.client.0.smithi007.stdout:2/771: chown dc/df/d24/d28/d2e/d4a/ca1 3683734 1 2024-03-16T09:01:17.766 INFO:tasks.workunit.client.1.smithi019.stdout:5/512: dwrite d6/d1f/f5a [0,4194304] 0 2024-03-16T09:01:17.766 INFO:tasks.workunit.client.1.smithi019.stdout:5/513: stat d6/dd/d47/d56 0 2024-03-16T09:01:17.766 INFO:tasks.workunit.client.1.smithi019.stdout:6/628: dwrite de/f86 [0,4194304] 0 2024-03-16T09:01:17.767 INFO:tasks.workunit.client.0.smithi007.stdout:3/713: mkdir d15/d18/d1f/dce/dd6/ddf 0 2024-03-16T09:01:17.768 INFO:tasks.workunit.client.1.smithi019.stdout:3/545: dwrite d3/d11/f3b [0,4194304] 0 2024-03-16T09:01:17.771 INFO:tasks.workunit.client.0.smithi007.stdout:2/772: rmdir dc/df/d24/d28 39 2024-03-16T09:01:17.771 INFO:tasks.workunit.client.0.smithi007.stdout:2/773: stat dc/df/d6c/fcd 0 2024-03-16T09:01:17.771 INFO:tasks.workunit.client.0.smithi007.stdout:2/774: rename dc/df/d24/d28 to dc/df/d24/d28/d97/da4/db8/df1/dfc 22 2024-03-16T09:01:17.771 INFO:tasks.workunit.client.0.smithi007.stdout:7/646: dwrite d0/d6c/f64 [4194304,4194304] 0 2024-03-16T09:01:17.771 INFO:tasks.workunit.client.0.smithi007.stdout:2/775: write dc/df/d24/d28/d2e/d4a/f7a [4575064,40552] 0 2024-03-16T09:01:17.772 INFO:tasks.workunit.client.0.smithi007.stdout:7/647: fsync d0/d6c/d25/d3a/daf/dc2/fe6 0 2024-03-16T09:01:17.772 INFO:tasks.workunit.client.0.smithi007.stdout:8/971: sync 2024-03-16T09:01:17.772 INFO:tasks.workunit.client.0.smithi007.stdout:9/682: sync 2024-03-16T09:01:17.772 INFO:tasks.workunit.client.0.smithi007.stdout:1/581: sync 2024-03-16T09:01:17.772 INFO:tasks.workunit.client.0.smithi007.stdout:6/612: sync 2024-03-16T09:01:17.772 INFO:tasks.workunit.client.0.smithi007.stdout:6/613: read d2/f46 [2633140,117840] 0 2024-03-16T09:01:17.773 INFO:tasks.workunit.client.1.smithi019.stdout:4/480: dread d4/d6c/f7c [0,4194304] 0 2024-03-16T09:01:17.773 INFO:tasks.workunit.client.1.smithi019.stdout:4/481: stat d4/d1b/d49/d6e/c8d 0 2024-03-16T09:01:17.773 INFO:tasks.workunit.client.1.smithi019.stdout:4/482: stat d4/d1b/f57 0 2024-03-16T09:01:17.775 INFO:tasks.workunit.client.1.smithi019.stdout:8/550: dwrite d6/d2a/d45/d5d/faf [0,4194304] 0 2024-03-16T09:01:17.775 INFO:tasks.workunit.client.1.smithi019.stdout:8/551: chown d6/d2a/d45 153 1 2024-03-16T09:01:17.775 INFO:tasks.workunit.client.1.smithi019.stdout:8/552: dread - d6/de/fb2 zero size 2024-03-16T09:01:17.777 INFO:tasks.workunit.client.1.smithi019.stdout:4/483: creat d4/d6/d15/d9a/fa4 x:0 0 0 2024-03-16T09:01:17.778 INFO:tasks.workunit.client.0.smithi007.stdout:7/648: mkdir d0/d19/d1f/d69/de1/de9 0 2024-03-16T09:01:17.778 INFO:tasks.workunit.client.0.smithi007.stdout:5/674: sync 2024-03-16T09:01:17.779 INFO:tasks.workunit.client.0.smithi007.stdout:5/675: fsync d1/d7/d1c/d88/da2/d89/fb8 0 2024-03-16T09:01:17.779 INFO:tasks.workunit.client.0.smithi007.stdout:5/676: readlink d1/d7/d1c/d88/da2/dae/lbb 0 2024-03-16T09:01:17.779 INFO:tasks.workunit.client.0.smithi007.stdout:8/972: truncate d0/d26/d2d/d64/f7f 3406642 0 2024-03-16T09:01:17.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/973: fdatasync d0/d26/d2d/d43/d72/dad/f144 0 2024-03-16T09:01:17.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/974: chown d0/dc/d54 0 1 2024-03-16T09:01:17.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/975: read - d0/d26/d2d/d64/f151 zero size 2024-03-16T09:01:17.780 INFO:tasks.workunit.client.0.smithi007.stdout:8/976: write d0/d26/d2d/f6c [1038660,129096] 0 2024-03-16T09:01:17.782 INFO:tasks.workunit.client.1.smithi019.stdout:8/553: rmdir d6/d2a/d3c/dab/d30/d4a/d67/d95 39 2024-03-16T09:01:17.783 INFO:tasks.workunit.client.0.smithi007.stdout:9/683: creat d24/d4c/d86/daa/d7f/d8f/db5/fc7 x:0 0 0 2024-03-16T09:01:17.783 INFO:tasks.workunit.client.0.smithi007.stdout:9/684: chown l10 0 1 2024-03-16T09:01:17.784 INFO:tasks.workunit.client.0.smithi007.stdout:4/603: sync 2024-03-16T09:01:17.784 INFO:tasks.workunit.client.1.smithi019.stdout:8/554: mknod d6/d2a/d45/d19/d74/cbf 0 2024-03-16T09:01:17.785 INFO:tasks.workunit.client.1.smithi019.stdout:8/555: fsync d6/d2a/d45/d19/f97 0 2024-03-16T09:01:17.785 INFO:tasks.workunit.client.0.smithi007.stdout:1/582: mkdir d9/dd/d1b/d28/d29/d7b/dba 0 2024-03-16T09:01:17.785 INFO:tasks.workunit.client.0.smithi007.stdout:1/583: chown d9/dd/d1b/d28/d29/f47 0 1 2024-03-16T09:01:17.787 INFO:tasks.workunit.client.0.smithi007.stdout:7/649: creat d0/d6c/d25/d7a/d9a/fea x:0 0 0 2024-03-16T09:01:17.788 INFO:tasks.workunit.client.1.smithi019.stdout:2/550: dwrite d5/f13 [0,4194304] 0 2024-03-16T09:01:17.788 INFO:tasks.workunit.client.1.smithi019.stdout:2/551: write d5/d6/d2b/d53/f71 [2794991,106462] 0 2024-03-16T09:01:17.788 INFO:tasks.workunit.client.0.smithi007.stdout:2/776: write dc/d27/f8e [125257,116254] 0 2024-03-16T09:01:17.789 INFO:tasks.workunit.client.0.smithi007.stdout:4/604: symlink d4/d14/d64/lce 0 2024-03-16T09:01:17.789 INFO:tasks.workunit.client.1.smithi019.stdout:2/552: chown d5/d4e/d74/fa3 0 1 2024-03-16T09:01:17.789 INFO:tasks.workunit.client.1.smithi019.stdout:2/553: truncate d5/d6/d2b/d2f/d35/d4b/f5c 1540327 0 2024-03-16T09:01:17.790 INFO:tasks.workunit.client.0.smithi007.stdout:1/584: mkdir d9/d6b/d7e/dac/dbb 0 2024-03-16T09:01:17.791 INFO:tasks.workunit.client.0.smithi007.stdout:1/585: write d9/dd/d1b/d2d/f3f [1245672,88190] 0 2024-03-16T09:01:17.791 INFO:tasks.workunit.client.0.smithi007.stdout:6/614: dread d2/d11/d3b/d93/d8e/fb6 [4194304,4194304] 0 2024-03-16T09:01:17.792 INFO:tasks.workunit.client.0.smithi007.stdout:7/650: mkdir d0/d19/d1f/d69/deb 0 2024-03-16T09:01:17.792 INFO:tasks.workunit.client.0.smithi007.stdout:1/586: dread d9/dd/f67 [0,4194304] 0 2024-03-16T09:01:17.792 INFO:tasks.workunit.client.0.smithi007.stdout:1/587: write d9/dd/d1b/d2d/d30/f61 [1469551,125218] 0 2024-03-16T09:01:17.792 INFO:tasks.workunit.client.0.smithi007.stdout:1/588: stat d9/dd/d1b/d2d 0 2024-03-16T09:01:17.795 INFO:tasks.workunit.client.0.smithi007.stdout:8/977: mknod d0/d26/c159 0 2024-03-16T09:01:17.795 INFO:tasks.workunit.client.0.smithi007.stdout:8/978: chown d0/dc/d50/d8e/dd2/dea/d10a 62 1 2024-03-16T09:01:17.795 INFO:tasks.workunit.client.0.smithi007.stdout:8/979: truncate d0/dc/dd/fda 4651369 0 2024-03-16T09:01:17.795 INFO:tasks.workunit.client.0.smithi007.stdout:8/980: fsync d0/dc/dd/d8b/d9e/fb2 0 2024-03-16T09:01:17.797 INFO:tasks.workunit.client.0.smithi007.stdout:2/777: truncate dc/df/d24/d28/f66 917670 0 2024-03-16T09:01:17.798 INFO:tasks.workunit.client.0.smithi007.stdout:4/605: mknod d4/d14/d1c/d5b/d67/d8f/da8/d78/ccf 0 2024-03-16T09:01:17.800 INFO:tasks.workunit.client.0.smithi007.stdout:6/615: symlink d2/d11/d3b/lc7 0 2024-03-16T09:01:17.803 INFO:tasks.workunit.client.0.smithi007.stdout:1/589: rename d9/dd/d5e/f84 to d9/fbc 0 2024-03-16T09:01:17.803 INFO:tasks.workunit.client.0.smithi007.stdout:1/590: chown d9/dd/d1b/d28/f36 9509097 1 2024-03-16T09:01:17.804 INFO:tasks.workunit.client.1.smithi019.stdout:8/556: dread d6/d2a/d45/d57/f65 [0,4194304] 0 2024-03-16T09:01:17.804 INFO:tasks.workunit.client.1.smithi019.stdout:8/557: readlink d6/d2a/d3c/dab/d30/d4a/d67/la3 0 2024-03-16T09:01:17.804 INFO:tasks.workunit.client.0.smithi007.stdout:4/606: mknod d4/d14/d64/cd0 0 2024-03-16T09:01:17.805 INFO:tasks.workunit.client.0.smithi007.stdout:6/616: symlink d2/d11/d3b/d93/lc8 0 2024-03-16T09:01:17.806 INFO:tasks.workunit.client.0.smithi007.stdout:2/778: rename dc/d27/da3/fdb to dc/dba/ffd 0 2024-03-16T09:01:17.808 INFO:tasks.workunit.client.0.smithi007.stdout:1/591: unlink d9/dd/d1b/d2d/d30/d45/c70 0 2024-03-16T09:01:17.810 INFO:tasks.workunit.client.0.smithi007.stdout:6/617: mkdir d2/d11/d1e/d54/d7a/d9a/dc9 0 2024-03-16T09:01:17.810 INFO:tasks.workunit.client.0.smithi007.stdout:6/618: creat d2/d82/fca x:0 0 0 2024-03-16T09:01:17.810 INFO:tasks.workunit.client.0.smithi007.stdout:2/779: mkdir dc/df/d24/d28/d4c/dbc/dfe 0 2024-03-16T09:01:17.813 INFO:tasks.workunit.client.0.smithi007.stdout:1/592: rename d9/dd/d1b/f83 to d9/dd/d1b/d2d/d30/d45/fbd 0 2024-03-16T09:01:17.813 INFO:tasks.workunit.client.0.smithi007.stdout:1/593: truncate d9/dd/d5e/f79 632518 0 2024-03-16T09:01:17.813 INFO:tasks.workunit.client.0.smithi007.stdout:1/594: write d9/dd/d1b/d2d/f99 [446410,79284] 0 2024-03-16T09:01:17.814 INFO:tasks.workunit.client.1.smithi019.stdout:2/554: dwrite d5/d6/d2b/f18 [0,4194304] 0 2024-03-16T09:01:17.816 INFO:tasks.workunit.client.1.smithi019.stdout:2/555: symlink d5/d6/d2b/d2f/d35/lb5 0 2024-03-16T09:01:17.819 INFO:tasks.workunit.client.0.smithi007.stdout:1/595: creat d9/d6b/d7e/dac/dbb/fbe x:0 0 0 2024-03-16T09:01:17.820 INFO:tasks.workunit.client.0.smithi007.stdout:2/780: rmdir dc/df/d24/d28/d2e/dad 39 2024-03-16T09:01:17.828 INFO:tasks.workunit.client.0.smithi007.stdout:2/781: mknod dc/df/d24/d28/cff 0 2024-03-16T09:01:17.831 INFO:tasks.workunit.client.0.smithi007.stdout:2/782: rmdir dc/df/daa 39 2024-03-16T09:01:17.831 INFO:tasks.workunit.client.0.smithi007.stdout:2/783: stat dc/df/d24/d28/d2e/d6e/lae 0 2024-03-16T09:01:17.833 INFO:tasks.workunit.client.0.smithi007.stdout:1/596: write d9/dd/d1b/d2d/f6a [4135258,7465] 0 2024-03-16T09:01:17.841 INFO:tasks.workunit.client.0.smithi007.stdout:6/619: rename d2/d11 to d2/d13/d3f/d50/d78/dbe/dcb 0 2024-03-16T09:01:17.841 INFO:tasks.workunit.client.0.smithi007.stdout:6/620: creat d2/d13/fcc x:0 0 0 2024-03-16T09:01:17.843 INFO:tasks.workunit.client.0.smithi007.stdout:2/784: rename dc/df/d24/d28/d2e/d35/f5f to dc/df/d24/d28/d2e/dc5/f100 0 2024-03-16T09:01:17.845 INFO:tasks.workunit.client.0.smithi007.stdout:6/621: link d2/d10/d3d/d6d/fa1 d2/d13/d3f/d50/d78/fcd 0 2024-03-16T09:01:17.847 INFO:tasks.workunit.client.0.smithi007.stdout:2/785: mknod dc/df/d24/c101 0 2024-03-16T09:01:17.847 INFO:tasks.workunit.client.0.smithi007.stdout:2/786: fdatasync dc/dba/ffd 0 2024-03-16T09:01:17.847 INFO:tasks.workunit.client.0.smithi007.stdout:2/787: write dc/df/d24/fb1 [418527,116088] 0 2024-03-16T09:01:17.847 INFO:tasks.workunit.client.0.smithi007.stdout:2/788: fdatasync dc/dba/ff6 0 2024-03-16T09:01:17.847 INFO:tasks.workunit.client.0.smithi007.stdout:2/789: write dc/df/d24/d28/d97/da4/db8/fd7 [320322,91473] 0 2024-03-16T09:01:17.847 INFO:tasks.workunit.client.0.smithi007.stdout:2/790: truncate dc/df/d24/d28/d4c/d5e/f8b 4900918 0 2024-03-16T09:01:17.848 INFO:tasks.workunit.client.0.smithi007.stdout:6/622: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/fce x:0 0 0 2024-03-16T09:01:17.848 INFO:tasks.workunit.client.0.smithi007.stdout:2/791: mkdir dc/df/d24/d28/dea/d102 0 2024-03-16T09:01:17.849 INFO:tasks.workunit.client.0.smithi007.stdout:6/623: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/fcf x:0 0 0 2024-03-16T09:01:17.849 INFO:tasks.workunit.client.0.smithi007.stdout:2/792: stat dc/df/d24/d28/d2e/c43 0 2024-03-16T09:01:17.849 INFO:tasks.workunit.client.0.smithi007.stdout:2/793: stat dc/df/d24/d28/d2e/d4a/fef 0 2024-03-16T09:01:17.850 INFO:tasks.workunit.client.0.smithi007.stdout:2/794: unlink dc/df/d24/l54 0 2024-03-16T09:01:17.850 INFO:tasks.workunit.client.0.smithi007.stdout:2/795: chown dc/df/d24/d28/d97/da4/fa6 3677 1 2024-03-16T09:01:17.854 INFO:tasks.workunit.client.0.smithi007.stdout:2/796: symlink dc/df/d24/d28/dea/d102/l103 0 2024-03-16T09:01:17.855 INFO:tasks.workunit.client.0.smithi007.stdout:2/797: rmdir dc/df/d24/d28/d4c/d5e 39 2024-03-16T09:01:17.859 INFO:tasks.workunit.client.1.smithi019.stdout:4/484: dread d4/d6/f39 [0,4194304] 0 2024-03-16T09:01:17.861 INFO:tasks.workunit.client.0.smithi007.stdout:3/714: dwrite d15/d18/d1f/d29/d3e/d85/fcd [0,4194304] 0 2024-03-16T09:01:17.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/514: dwrite d6/d1f/d33/d66/f9f [0,4194304] 0 2024-03-16T09:01:17.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/515: read d6/dd/d37/f3d [5125814,53819] 0 2024-03-16T09:01:17.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/516: dread - d6/dd/d47/f69 zero size 2024-03-16T09:01:17.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/517: creat d6/d1f/fa4 x:0 0 0 2024-03-16T09:01:17.864 INFO:tasks.workunit.client.1.smithi019.stdout:4/485: rename d4/d6/d15/d1f/f97 to d4/fa5 0 2024-03-16T09:01:17.868 INFO:tasks.workunit.client.1.smithi019.stdout:5/518: mknod d6/d19/d34/ca5 0 2024-03-16T09:01:17.868 INFO:tasks.workunit.client.0.smithi007.stdout:3/715: creat d15/d18/d1f/d77/fe0 x:0 0 0 2024-03-16T09:01:17.869 INFO:tasks.workunit.client.1.smithi019.stdout:4/486: getdents d4/d73 0 2024-03-16T09:01:17.870 INFO:tasks.workunit.client.0.smithi007.stdout:3/716: mkdir d15/d18/de1 0 2024-03-16T09:01:17.873 INFO:tasks.workunit.client.1.smithi019.stdout:5/519: rmdir d6/dd/d37/d76 39 2024-03-16T09:01:17.873 INFO:tasks.workunit.client.1.smithi019.stdout:5/520: readlink d6/d1f/d33/d89/l90 0 2024-03-16T09:01:17.873 INFO:tasks.workunit.client.1.smithi019.stdout:5/521: creat d6/dd/d37/fa6 x:0 0 0 2024-03-16T09:01:17.873 INFO:tasks.workunit.client.1.smithi019.stdout:5/522: truncate d6/dd/d37/f58 170024 0 2024-03-16T09:01:17.873 INFO:tasks.workunit.client.1.smithi019.stdout:2/556: dread d5/ff [0,4194304] 0 2024-03-16T09:01:17.873 INFO:tasks.workunit.client.1.smithi019.stdout:2/557: fdatasync d5/d2e/f3b 0 2024-03-16T09:01:17.874 INFO:tasks.workunit.client.0.smithi007.stdout:3/717: link d15/d18/d1f/d3b/d87/f92 d15/d18/d1f/d29/d3e/d85/fe2 0 2024-03-16T09:01:17.874 INFO:tasks.workunit.client.0.smithi007.stdout:3/718: fsync d15/d18/d1f/d3b/f47 0 2024-03-16T09:01:17.876 INFO:tasks.workunit.client.0.smithi007.stdout:7/651: dwrite d0/d6c/d25/d3a/f96 [0,4194304] 0 2024-03-16T09:01:17.876 INFO:tasks.workunit.client.1.smithi019.stdout:6/629: dwrite de/d26/f2a [0,4194304] 0 2024-03-16T09:01:17.876 INFO:tasks.workunit.client.1.smithi019.stdout:6/630: chown de/c12 144679 1 2024-03-16T09:01:17.876 INFO:tasks.workunit.client.1.smithi019.stdout:6/631: creat de/d26/d32/d97/d66/d72/d7f/fc9 x:0 0 0 2024-03-16T09:01:17.877 INFO:tasks.workunit.client.0.smithi007.stdout:3/719: rmdir d15/d18/d1f/d29/d3e/d85/daf 39 2024-03-16T09:01:17.877 INFO:tasks.workunit.client.0.smithi007.stdout:3/720: write d15/d18/d1f/d3b/fca [364070,121377] 0 2024-03-16T09:01:17.878 INFO:tasks.workunit.client.0.smithi007.stdout:6/624: dread d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/f35 [0,4194304] 0 2024-03-16T09:01:17.878 INFO:tasks.workunit.client.0.smithi007.stdout:6/625: write d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d7a/d9a/fc4 [373107,78945] 0 2024-03-16T09:01:17.879 INFO:tasks.workunit.client.1.smithi019.stdout:4/487: dread d4/d6c/d89/f93 [0,4194304] 0 2024-03-16T09:01:17.879 INFO:tasks.workunit.client.1.smithi019.stdout:2/558: rmdir d5/d6/d2b 39 2024-03-16T09:01:17.881 INFO:tasks.workunit.client.0.smithi007.stdout:1/597: dread d9/dd/d1b/d2d/d30/f61 [0,4194304] 0 2024-03-16T09:01:17.881 INFO:tasks.workunit.client.0.smithi007.stdout:1/598: write d9/dd/d5e/f79 [759511,119757] 0 2024-03-16T09:01:17.881 INFO:tasks.workunit.client.0.smithi007.stdout:1/599: chown d9/dd/d1b/d28/d29/d7b/db4 404 1 2024-03-16T09:01:17.881 INFO:tasks.workunit.client.0.smithi007.stdout:1/600: fsync d9/dd/d1b/d2d/d30/d45/f51 0 2024-03-16T09:01:17.882 INFO:tasks.workunit.client.1.smithi019.stdout:6/632: dread de/d26/d32/d97/d66/d72/d7f/fbd [0,4194304] 0 2024-03-16T09:01:17.882 INFO:tasks.workunit.client.1.smithi019.stdout:4/488: symlink d4/la6 0 2024-03-16T09:01:17.884 INFO:tasks.workunit.client.1.smithi019.stdout:3/546: dwrite d3/d17/f72 [4194304,4194304] 0 2024-03-16T09:01:17.884 INFO:tasks.workunit.client.1.smithi019.stdout:3/547: read - d3/d17/d5a/d75/d7a/fa0 zero size 2024-03-16T09:01:17.884 INFO:tasks.workunit.client.1.smithi019.stdout:2/559: rename d5/d6/d64/d6f/da6/d83/f8a to d5/d6/d64/d6f/fb6 0 2024-03-16T09:01:17.884 INFO:tasks.workunit.client.1.smithi019.stdout:2/560: chown d5/d6/d2b/d2f/d35/d4b/f93 442667 1 2024-03-16T09:01:17.885 INFO:tasks.workunit.client.1.smithi019.stdout:6/633: creat de/d26/d32/dc1/fca x:0 0 0 2024-03-16T09:01:17.888 INFO:tasks.workunit.client.1.smithi019.stdout:4/489: write d4/f26 [5694999,15376] 0 2024-03-16T09:01:17.889 INFO:tasks.workunit.client.0.smithi007.stdout:7/652: truncate d0/dc/d38/d4e/fb4 1134643 0 2024-03-16T09:01:17.890 INFO:tasks.workunit.client.0.smithi007.stdout:1/601: write d9/f12 [975546,39671] 0 2024-03-16T09:01:17.890 INFO:tasks.workunit.client.0.smithi007.stdout:1/602: write d9/dd/d1b/d58/f7d [4383504,102328] 0 2024-03-16T09:01:17.891 INFO:tasks.workunit.client.1.smithi019.stdout:2/561: unlink d5/de/c1a 0 2024-03-16T09:01:17.891 INFO:tasks.workunit.client.1.smithi019.stdout:2/562: truncate d5/f61 493882 0 2024-03-16T09:01:17.893 INFO:tasks.workunit.client.0.smithi007.stdout:2/798: dwrite dc/d27/fd1 [0,4194304] 0 2024-03-16T09:01:17.895 INFO:tasks.workunit.client.0.smithi007.stdout:6/626: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d3b 39 2024-03-16T09:01:17.897 INFO:tasks.workunit.client.0.smithi007.stdout:7/653: mkdir d0/d6c/d25/d3a/daf/dec 0 2024-03-16T09:01:17.900 INFO:tasks.workunit.client.0.smithi007.stdout:1/603: symlink d9/dd/d1b/d58/lbf 0 2024-03-16T09:01:17.900 INFO:tasks.workunit.client.0.smithi007.stdout:1/604: stat d9/dd/c17 0 2024-03-16T09:01:17.900 INFO:tasks.workunit.client.0.smithi007.stdout:1/605: write d9/dd/f59 [5044142,44904] 0 2024-03-16T09:01:17.901 INFO:tasks.workunit.client.1.smithi019.stdout:2/563: link d5/d6/d2b/l70 d5/lb7 0 2024-03-16T09:01:17.901 INFO:tasks.workunit.client.1.smithi019.stdout:2/564: link d5/c16 d5/d6/cb8 0 2024-03-16T09:01:17.901 INFO:tasks.workunit.client.1.smithi019.stdout:2/565: write d5/d2e/f75 [1514328,105741] 0 2024-03-16T09:01:17.902 INFO:tasks.workunit.client.1.smithi019.stdout:2/566: mknod d5/d2e/d4f/cb9 0 2024-03-16T09:01:17.902 INFO:tasks.workunit.client.0.smithi007.stdout:2/799: mknod dc/df/d24/d28/dea/c104 0 2024-03-16T09:01:17.903 INFO:tasks.workunit.client.1.smithi019.stdout:2/567: mkdir d5/d5a/d96/dba 0 2024-03-16T09:01:17.903 INFO:tasks.workunit.client.0.smithi007.stdout:7/654: mkdir d0/d6c/d25/d73/ded 0 2024-03-16T09:01:17.905 INFO:tasks.workunit.client.0.smithi007.stdout:1/606: unlink d9/dd/d1b/d2d/l5a 0 2024-03-16T09:01:17.906 INFO:tasks.workunit.client.0.smithi007.stdout:9/685: dwrite d24/f39 [0,4194304] 0 2024-03-16T09:01:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:7/655: link d0/d19/d1f/d76/fa3 d0/d6c/d25/d3a/db3/fee 0 2024-03-16T09:01:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:7/656: write d0/d6c/d25/da1/fab [39980,130259] 0 2024-03-16T09:01:17.908 INFO:tasks.workunit.client.0.smithi007.stdout:7/657: creat d0/d19/d1f/d69/dc7/fef x:0 0 0 2024-03-16T09:01:17.909 INFO:tasks.workunit.client.0.smithi007.stdout:7/658: chown d0/d19/d1f/d76/cb1 29509913 1 2024-03-16T09:01:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:2/800: dread dc/df/d24/ff0 [0,4194304] 0 2024-03-16T09:01:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:6/627: dread d2/f46 [0,4194304] 0 2024-03-16T09:01:17.910 INFO:tasks.workunit.client.0.smithi007.stdout:5/677: dwrite d1/d7/d1c/d19/d34/d3e/f43 [4194304,4194304] 0 2024-03-16T09:01:17.911 INFO:tasks.workunit.client.1.smithi019.stdout:4/490: dwrite d4/d6/d15/f66 [0,4194304] 0 2024-03-16T09:01:17.911 INFO:tasks.workunit.client.1.smithi019.stdout:4/491: dread - d4/d1b/d2a/f74 zero size 2024-03-16T09:01:17.911 INFO:tasks.workunit.client.1.smithi019.stdout:4/492: stat d4/d64/d99 0 2024-03-16T09:01:17.911 INFO:tasks.workunit.client.1.smithi019.stdout:4/493: readlink d4/l69 0 2024-03-16T09:01:17.911 INFO:tasks.workunit.client.1.smithi019.stdout:4/494: readlink d4/l69 0 2024-03-16T09:01:17.912 INFO:tasks.workunit.client.1.smithi019.stdout:5/523: dwrite d6/fb [0,4194304] 0 2024-03-16T09:01:17.912 INFO:tasks.workunit.client.1.smithi019.stdout:5/524: write d6/dd/f99 [4961555,127720] 0 2024-03-16T09:01:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:1/607: getdents d9 0 2024-03-16T09:01:17.913 INFO:tasks.workunit.client.0.smithi007.stdout:1/608: truncate d9/f8b 981123 0 2024-03-16T09:01:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:7/659: fsync d0/d6c/d25/da1/fab 0 2024-03-16T09:01:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:1/609: write d9/d6b/d7e/d91/fa3 [560284,108067] 0 2024-03-16T09:01:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:7/660: write d0/d6c/d25/d7a/fd4 [12823,122840] 0 2024-03-16T09:01:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:9/686: write ff [4162211,82241] 0 2024-03-16T09:01:17.915 INFO:tasks.workunit.client.0.smithi007.stdout:7/661: write d0/d6c/d25/d3a/daf/dc2/fe6 [511748,87551] 0 2024-03-16T09:01:17.916 INFO:tasks.workunit.client.1.smithi019.stdout:5/525: creat d6/d1f/d33/d89/fa7 x:0 0 0 2024-03-16T09:01:17.916 INFO:tasks.workunit.client.1.smithi019.stdout:5/526: fdatasync d6/f9 0 2024-03-16T09:01:17.916 INFO:tasks.workunit.client.1.smithi019.stdout:5/527: truncate d6/d1f/fa4 464199 0 2024-03-16T09:01:17.917 INFO:tasks.workunit.client.0.smithi007.stdout:2/801: rename dc/df/d24/d28/d4c/fbe to dc/df/d24/d28/dea/f105 0 2024-03-16T09:01:17.918 INFO:tasks.workunit.client.1.smithi019.stdout:5/528: rename d6/f17 to d6/d1f/d33/d4c/fa8 0 2024-03-16T09:01:17.918 INFO:tasks.workunit.client.1.smithi019.stdout:6/634: rmdir de 39 2024-03-16T09:01:17.918 INFO:tasks.workunit.client.1.smithi019.stdout:6/635: fsync de/d26/d32/d97/d66/f94 0 2024-03-16T09:01:17.918 INFO:tasks.workunit.client.1.smithi019.stdout:9/396: sync 2024-03-16T09:01:17.919 INFO:tasks.workunit.client.1.smithi019.stdout:0/545: sync 2024-03-16T09:01:17.919 INFO:tasks.workunit.client.1.smithi019.stdout:7/432: sync 2024-03-16T09:01:17.919 INFO:tasks.workunit.client.1.smithi019.stdout:7/433: readlink d9/db/d12/d5b/l4e 0 2024-03-16T09:01:17.919 INFO:tasks.workunit.client.1.smithi019.stdout:1/447: sync 2024-03-16T09:01:17.922 INFO:tasks.workunit.client.1.smithi019.stdout:9/397: rename d1/d7/d21/f38 to d1/d7/de/d24/f8a 0 2024-03-16T09:01:17.923 INFO:tasks.workunit.client.1.smithi019.stdout:7/434: rmdir d9/da/d13/d18/d30/d75 39 2024-03-16T09:01:17.924 INFO:tasks.workunit.client.1.smithi019.stdout:4/495: dread d4/f12 [0,4194304] 0 2024-03-16T09:01:17.925 INFO:tasks.workunit.client.1.smithi019.stdout:4/496: dread d4/d64/d99/f7b [0,4194304] 0 2024-03-16T09:01:17.927 INFO:tasks.workunit.client.1.smithi019.stdout:1/448: getdents d16/d1f/d3c 0 2024-03-16T09:01:17.927 INFO:tasks.workunit.client.1.smithi019.stdout:1/449: fsync f10 0 2024-03-16T09:01:17.928 INFO:tasks.workunit.client.0.smithi007.stdout:6/628: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/fd0 x:0 0 0 2024-03-16T09:01:17.931 INFO:tasks.workunit.client.1.smithi019.stdout:9/398: rename d1/d88 to d1/d7/de/d24/d44/d60/d8b 0 2024-03-16T09:01:17.931 INFO:tasks.workunit.client.1.smithi019.stdout:9/399: write d1/d7/d21/d27/f22 [5524846,24946] 0 2024-03-16T09:01:17.931 INFO:tasks.workunit.client.0.smithi007.stdout:5/678: symlink d1/d7/d1c/d19/d34/d1d/dee/lf2 0 2024-03-16T09:01:17.932 INFO:tasks.workunit.client.1.smithi019.stdout:2/568: getdents d5/d5a/d96 0 2024-03-16T09:01:17.932 INFO:tasks.workunit.client.1.smithi019.stdout:2/569: truncate d5/d2e/d4f/f57 983133 0 2024-03-16T09:01:17.932 INFO:tasks.workunit.client.1.smithi019.stdout:2/570: truncate d5/de/f4a 4547953 0 2024-03-16T09:01:17.936 INFO:tasks.workunit.client.0.smithi007.stdout:9/687: readlink d24/d4c/l58 0 2024-03-16T09:01:17.937 INFO:tasks.workunit.client.1.smithi019.stdout:0/546: rename d7/d1d/d28/d6c/f95 to d7/d1d/d28/d92/da6/fbb 0 2024-03-16T09:01:17.939 INFO:tasks.workunit.client.1.smithi019.stdout:2/571: stat d5/d6/d64/c81 0 2024-03-16T09:01:17.939 INFO:tasks.workunit.client.0.smithi007.stdout:7/662: creat d0/d6c/d25/d7a/dc9/ff0 x:0 0 0 2024-03-16T09:01:17.939 INFO:tasks.workunit.client.0.smithi007.stdout:7/663: chown d0/d19/fc1 1756284103 1 2024-03-16T09:01:17.940 INFO:tasks.workunit.client.1.smithi019.stdout:7/435: rmdir d9/d25/d28/d48 39 2024-03-16T09:01:17.940 INFO:tasks.workunit.client.1.smithi019.stdout:7/436: fsync d9/f3f 0 2024-03-16T09:01:17.940 INFO:tasks.workunit.client.1.smithi019.stdout:7/437: read d9/db/d12/f2f [354364,51363] 0 2024-03-16T09:01:17.941 INFO:tasks.workunit.client.0.smithi007.stdout:2/802: rename dc/df/d24/d28/d97/da4/db8/cc3 to dc/df/d24/d28/d2e/dad/c106 0 2024-03-16T09:01:17.942 INFO:tasks.workunit.client.1.smithi019.stdout:0/547: symlink d7/d1d/d63/d89/lbc 0 2024-03-16T09:01:17.943 INFO:tasks.workunit.client.1.smithi019.stdout:4/497: rename d4/d1b/l92 to d4/d1b/d2a/la7 0 2024-03-16T09:01:17.944 INFO:tasks.workunit.client.0.smithi007.stdout:6/629: symlink d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/dc1/ld1 0 2024-03-16T09:01:17.944 INFO:tasks.workunit.client.1.smithi019.stdout:9/400: dread d1/f3 [0,4194304] 0 2024-03-16T09:01:17.944 INFO:tasks.workunit.client.1.smithi019.stdout:0/548: mknod d7/d1d/d28/d48/cbd 0 2024-03-16T09:01:17.945 INFO:tasks.workunit.client.1.smithi019.stdout:0/549: chown d7/d8/d32/c84 1227 1 2024-03-16T09:01:17.945 INFO:tasks.workunit.client.1.smithi019.stdout:0/550: readlink d7/d1d/d63/d6e/lb3 0 2024-03-16T09:01:17.947 INFO:tasks.workunit.client.0.smithi007.stdout:9/688: symlink d24/d4a/d96/db9/db1/lc8 0 2024-03-16T09:01:17.949 INFO:tasks.workunit.client.0.smithi007.stdout:5/679: write d1/d7/d1c/d19/d34/d3e/f6a [3024576,77678] 0 2024-03-16T09:01:17.949 INFO:tasks.workunit.client.0.smithi007.stdout:2/803: creat dc/df/d24/d28/d2e/d6e/f107 x:0 0 0 2024-03-16T09:01:17.950 INFO:tasks.workunit.client.1.smithi019.stdout:5/529: dwrite d6/dd/d47/f64 [0,4194304] 0 2024-03-16T09:01:17.950 INFO:tasks.workunit.client.1.smithi019.stdout:6/636: dwrite de/d26/f39 [0,4194304] 0 2024-03-16T09:01:17.950 INFO:tasks.workunit.client.1.smithi019.stdout:6/637: write de/d26/d32/d97/d66/d72/d7f/fc9 [404661,92758] 0 2024-03-16T09:01:17.950 INFO:tasks.workunit.client.1.smithi019.stdout:6/638: stat de/d26/d32/d97/f30 0 2024-03-16T09:01:17.950 INFO:tasks.workunit.client.1.smithi019.stdout:0/551: rename d7/d8/l49 to d7/d8/d32/d4a/lbe 0 2024-03-16T09:01:17.952 INFO:tasks.workunit.client.1.smithi019.stdout:9/401: unlink d1/d7/l16 0 2024-03-16T09:01:17.953 INFO:tasks.workunit.client.1.smithi019.stdout:2/572: dread d5/de/fd [0,4194304] 0 2024-03-16T09:01:17.955 INFO:tasks.workunit.client.0.smithi007.stdout:1/610: dwrite d9/d6b/d7e/d91/fa3 [0,4194304] 0 2024-03-16T09:01:17.956 INFO:tasks.workunit.client.1.smithi019.stdout:6/639: mkdir de/d26/d32/d97/d66/d72/d7f/dcb 0 2024-03-16T09:01:17.958 INFO:tasks.workunit.client.1.smithi019.stdout:0/552: mkdir d7/d8/d32/d4a/d61/dbf 0 2024-03-16T09:01:17.959 INFO:tasks.workunit.client.0.smithi007.stdout:1/611: mknod d9/dd/d1b/d2d/d30/d45/cc0 0 2024-03-16T09:01:17.961 INFO:tasks.workunit.client.1.smithi019.stdout:9/402: creat d1/d7/d86/f8c x:0 0 0 2024-03-16T09:01:17.961 INFO:tasks.workunit.client.1.smithi019.stdout:9/403: stat d1/d7/de/f10 0 2024-03-16T09:01:17.961 INFO:tasks.workunit.client.1.smithi019.stdout:9/404: chown d1/d7/de/d24 1200 1 2024-03-16T09:01:17.961 INFO:tasks.workunit.client.1.smithi019.stdout:9/405: readlink d1/d7/de/l5c 0 2024-03-16T09:01:17.962 INFO:tasks.workunit.client.1.smithi019.stdout:9/406: write d1/d7/d21/f87 [153135,27292] 0 2024-03-16T09:01:17.962 INFO:tasks.workunit.client.1.smithi019.stdout:7/438: dwrite d9/da/d13/d18/d46/f6b [0,4194304] 0 2024-03-16T09:01:17.963 INFO:tasks.workunit.client.0.smithi007.stdout:6/630: read d2/d13/d3f/d50/d72/f4c [1504120,120957] 0 2024-03-16T09:01:17.963 INFO:tasks.workunit.client.0.smithi007.stdout:1/612: rename d9/d6b/ca5 to d9/d6b/d7e/d91/cc1 0 2024-03-16T09:01:17.963 INFO:tasks.workunit.client.0.smithi007.stdout:1/613: rename d9/dd/d1b to d9/dd/d1b/d2d/d30/dc2 22 2024-03-16T09:01:17.963 INFO:tasks.workunit.client.0.smithi007.stdout:1/614: readlink d9/dd/d1b/d58/lbf 0 2024-03-16T09:01:17.963 INFO:tasks.workunit.client.0.smithi007.stdout:1/615: write d9/d6b/d7e/dac/dbb/fbe [1004980,28258] 0 2024-03-16T09:01:17.964 INFO:tasks.workunit.client.1.smithi019.stdout:2/573: mknod d5/d6/d2b/d2f/d35/d4b/cbb 0 2024-03-16T09:01:17.964 INFO:tasks.workunit.client.1.smithi019.stdout:6/640: mknod de/d26/d32/d97/d66/d72/ccc 0 2024-03-16T09:01:17.965 INFO:tasks.workunit.client.1.smithi019.stdout:0/553: mknod d7/d1d/d63/d6e/db0/cc0 0 2024-03-16T09:01:17.965 INFO:tasks.workunit.client.0.smithi007.stdout:7/664: dread d0/d6c/d25/d7a/fd4 [0,4194304] 0 2024-03-16T09:01:17.967 INFO:tasks.workunit.client.0.smithi007.stdout:1/616: creat d9/dd/d1b/d2d/d30/fc3 x:0 0 0 2024-03-16T09:01:17.967 INFO:tasks.workunit.client.0.smithi007.stdout:1/617: rename d9/dd/d1b/d58 to d9/dd/d1b/d58/dc4 22 2024-03-16T09:01:17.969 INFO:tasks.workunit.client.1.smithi019.stdout:6/641: mkdir de/d26/d32/d4e/d75/dcd 0 2024-03-16T09:01:17.969 INFO:tasks.workunit.client.0.smithi007.stdout:7/665: mknod d0/d19/d1f/d5d/d9d/cf1 0 2024-03-16T09:01:17.970 INFO:tasks.workunit.client.1.smithi019.stdout:1/450: dwrite d16/f3d [0,4194304] 0 2024-03-16T09:01:17.971 INFO:tasks.workunit.client.1.smithi019.stdout:0/554: symlink d7/d8/d32/lc1 0 2024-03-16T09:01:17.972 INFO:tasks.workunit.client.0.smithi007.stdout:1/618: link d9/dd/d1b/d2d/d30/c97 d9/d6b/d7e/dac/cc5 0 2024-03-16T09:01:17.972 INFO:tasks.workunit.client.1.smithi019.stdout:6/642: mkdir de/d26/d32/dce 0 2024-03-16T09:01:17.977 INFO:tasks.workunit.client.0.smithi007.stdout:5/680: dwrite d1/d7/d1c/d19/fad [4194304,4194304] 0 2024-03-16T09:01:17.980 INFO:tasks.workunit.client.1.smithi019.stdout:0/555: getdents d7/d1d/d28/d92 0 2024-03-16T09:01:17.981 INFO:tasks.workunit.client.0.smithi007.stdout:7/666: rename d0/d19/d1f/d69/cbd to d0/d6c/cf2 0 2024-03-16T09:01:17.984 INFO:tasks.workunit.client.1.smithi019.stdout:5/530: dread d6/d19/f4e [4194304,4194304] 0 2024-03-16T09:01:17.984 INFO:tasks.workunit.client.1.smithi019.stdout:9/407: dread d1/d7/d33/d59/f85 [0,4194304] 0 2024-03-16T09:01:17.984 INFO:tasks.workunit.client.1.smithi019.stdout:9/408: fsync d1/d7/f50 0 2024-03-16T09:01:17.984 INFO:tasks.workunit.client.1.smithi019.stdout:9/409: fdatasync d1/d7/d21/d27/f14 0 2024-03-16T09:01:17.984 INFO:tasks.workunit.client.1.smithi019.stdout:4/498: dwrite d4/d6/d15/f2e [0,4194304] 0 2024-03-16T09:01:17.985 INFO:tasks.workunit.client.1.smithi019.stdout:4/499: fsync d4/d1b/d49/f53 0 2024-03-16T09:01:17.986 INFO:tasks.workunit.client.0.smithi007.stdout:5/681: mkdir d1/d7/d1c/d88/da2/dae/df3 0 2024-03-16T09:01:17.986 INFO:tasks.workunit.client.0.smithi007.stdout:5/682: write d1/d7/d1c/f15 [200100,99428] 0 2024-03-16T09:01:17.987 INFO:tasks.workunit.client.0.smithi007.stdout:5/683: readlink d1/d7/l18 0 2024-03-16T09:01:17.991 INFO:tasks.workunit.client.0.smithi007.stdout:7/667: rename d0/d6c/d25/d7a/dc9/l85 to d0/d6c/d25/d3a/lf3 0 2024-03-16T09:01:17.991 INFO:tasks.workunit.client.1.smithi019.stdout:7/439: dread d9/da/d13/d18/d1a/f5f [0,4194304] 0 2024-03-16T09:01:17.991 INFO:tasks.workunit.client.1.smithi019.stdout:7/440: write d9/db/d12/d5b/d33/f43 [4487853,105050] 0 2024-03-16T09:01:17.991 INFO:tasks.workunit.client.1.smithi019.stdout:7/441: stat d9/da/d13/d18/l40 0 2024-03-16T09:01:17.993 INFO:tasks.workunit.client.1.smithi019.stdout:9/410: write d1/d7/f56 [125138,31255] 0 2024-03-16T09:01:17.996 INFO:tasks.workunit.client.0.smithi007.stdout:5/684: mkdir d1/d7/d1c/d88/da2/df4 0 2024-03-16T09:01:17.997 INFO:tasks.workunit.client.0.smithi007.stdout:5/685: creat d1/d7/d1c/d19/d34/de/ff5 x:0 0 0 2024-03-16T09:01:17.998 INFO:tasks.workunit.client.0.smithi007.stdout:7/668: symlink d0/d6c/d25/d73/ded/lf4 0 2024-03-16T09:01:17.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:17 smithi007 ceph-mon[29470]: pgmap v22: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 230 MiB/s rd, 285 MiB/s wr, 603 op/s 2024-03-16T09:01:17.999 INFO:tasks.workunit.client.1.smithi019.stdout:7/442: rename d9/da/d13/d18/l40 to d9/db/d12/d5b/l86 0 2024-03-16T09:01:18.000 INFO:tasks.workunit.client.1.smithi019.stdout:7/443: chown d9/db/d12/d5b/l86 959383756 1 2024-03-16T09:01:18.000 INFO:tasks.workunit.client.1.smithi019.stdout:9/411: symlink d1/d7/de/l8d 0 2024-03-16T09:01:18.001 INFO:tasks.workunit.client.0.smithi007.stdout:7/669: getdents d0/d6c/d25/d7a/d9a/d9c 0 2024-03-16T09:01:18.002 INFO:tasks.workunit.client.1.smithi019.stdout:7/444: creat d9/db/f87 x:0 0 0 2024-03-16T09:01:18.027 INFO:tasks.workunit.client.0.smithi007.stdout:6/631: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/fc0 [0,4194304] 0 2024-03-16T09:01:18.029 INFO:tasks.workunit.client.0.smithi007.stdout:6/632: symlink d2/ld2 0 2024-03-16T09:01:18.029 INFO:tasks.workunit.client.0.smithi007.stdout:6/633: dread - d2/d13/d3f/d50/d72/fb4 zero size 2024-03-16T09:01:18.030 INFO:tasks.workunit.client.1.smithi019.stdout:1/451: dwrite d16/d31/d76/f6b [0,4194304] 0 2024-03-16T09:01:18.031 INFO:tasks.workunit.client.0.smithi007.stdout:6/634: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/fd3 x:0 0 0 2024-03-16T09:01:18.031 INFO:tasks.workunit.client.1.smithi019.stdout:1/452: truncate d16/d1f/d43/f6d 1847440 0 2024-03-16T09:01:18.031 INFO:tasks.workunit.client.1.smithi019.stdout:1/453: fdatasync d16/d1f/d43/f6d 0 2024-03-16T09:01:18.031 INFO:tasks.workunit.client.1.smithi019.stdout:1/454: dread - d16/d18/f6c zero size 2024-03-16T09:01:18.031 INFO:tasks.workunit.client.1.smithi019.stdout:1/455: chown d16/d1f/d3c/d79/l7e 24799 1 2024-03-16T09:01:18.031 INFO:tasks.workunit.client.1.smithi019.stdout:1/456: chown d16/d1f/d4f/f50 9998 1 2024-03-16T09:01:18.037 INFO:tasks.workunit.client.1.smithi019.stdout:1/457: dread d16/f5d [0,4194304] 0 2024-03-16T09:01:18.037 INFO:tasks.workunit.client.1.smithi019.stdout:1/458: write d16/f42 [1223534,46011] 0 2024-03-16T09:01:18.042 INFO:tasks.workunit.client.1.smithi019.stdout:4/500: dwrite d4/d64/d99/f6b [0,4194304] 0 2024-03-16T09:01:18.042 INFO:tasks.workunit.client.1.smithi019.stdout:4/501: chown d4/d6/d15/d9a 1338 1 2024-03-16T09:01:18.059 INFO:tasks.workunit.client.1.smithi019.stdout:6/643: dwrite de/d26/d32/d97/d66/d72/d7f/fbd [0,4194304] 0 2024-03-16T09:01:18.059 INFO:tasks.workunit.client.1.smithi019.stdout:5/531: dwrite d6/dd/d37/d54/fa2 [0,4194304] 0 2024-03-16T09:01:18.059 INFO:tasks.workunit.client.1.smithi019.stdout:6/644: link de/d26/d32/d97/f30 de/d26/d32/d97/dae/db1/fcf 0 2024-03-16T09:01:18.059 INFO:tasks.workunit.client.1.smithi019.stdout:6/645: stat de/d26/d32/d4e/d75/l95 0 2024-03-16T09:01:18.064 INFO:tasks.workunit.client.1.smithi019.stdout:6/646: unlink de/d26/d32/d97/d66/d72/c8e 0 2024-03-16T09:01:18.064 INFO:tasks.workunit.client.1.smithi019.stdout:5/532: rmdir d6/d1f/d77 39 2024-03-16T09:01:18.065 INFO:tasks.workunit.client.1.smithi019.stdout:6/647: rename de/d26/d32/d4e/d57/d9f to de/d26/d32/d97/d66/d72/dd0 0 2024-03-16T09:01:18.067 INFO:tasks.workunit.client.1.smithi019.stdout:6/648: creat de/d26/d32/fd1 x:0 0 0 2024-03-16T09:01:18.067 INFO:tasks.workunit.client.1.smithi019.stdout:5/533: rmdir d6/dd/d47/d70 39 2024-03-16T09:01:18.068 INFO:tasks.workunit.client.1.smithi019.stdout:6/649: symlink de/d26/d32/d97/da1/ld2 0 2024-03-16T09:01:18.069 INFO:tasks.workunit.client.1.smithi019.stdout:6/650: creat de/d26/d32/d97/fd3 x:0 0 0 2024-03-16T09:01:18.069 INFO:tasks.workunit.client.1.smithi019.stdout:6/651: readlink de/d26/d32/d97/d51/l64 0 2024-03-16T09:01:18.069 INFO:tasks.workunit.client.1.smithi019.stdout:6/652: write de/d26/d32/d97/d66/d72/d7f/fc9 [669157,22727] 0 2024-03-16T09:01:18.069 INFO:tasks.workunit.client.0.smithi007.stdout:7/670: dwrite d0/d6c/d25/d7a/f71 [0,4194304] 0 2024-03-16T09:01:18.070 INFO:tasks.workunit.client.0.smithi007.stdout:1/619: dwrite d9/dd/f81 [0,4194304] 0 2024-03-16T09:01:18.070 INFO:tasks.workunit.client.0.smithi007.stdout:5/686: dwrite d1/d7/d16/d1b/d3c/fa7 [0,4194304] 0 2024-03-16T09:01:18.070 INFO:tasks.workunit.client.1.smithi019.stdout:1/459: dwrite d16/f65 [0,4194304] 0 2024-03-16T09:01:18.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/445: dwrite d9/da/d13/d18/d30/d3e/f62 [0,4194304] 0 2024-03-16T09:01:18.071 INFO:tasks.workunit.client.1.smithi019.stdout:7/446: creat d9/d25/d28/f88 x:0 0 0 2024-03-16T09:01:18.071 INFO:tasks.workunit.client.1.smithi019.stdout:7/447: fdatasync d9/db/d12/f55 0 2024-03-16T09:01:18.072 INFO:tasks.workunit.client.1.smithi019.stdout:4/502: dwrite d4/d6/d15/d9a/fa4 [0,4194304] 0 2024-03-16T09:01:18.076 INFO:tasks.workunit.client.1.smithi019.stdout:1/460: rmdir d16/d1f/d3c/d79 39 2024-03-16T09:01:18.076 INFO:tasks.workunit.client.0.smithi007.stdout:7/671: mkdir d0/df5 0 2024-03-16T09:01:18.076 INFO:tasks.workunit.client.0.smithi007.stdout:7/672: dread - d0/dc/d38/d4e/f68 zero size 2024-03-16T09:01:18.076 INFO:tasks.workunit.client.1.smithi019.stdout:5/534: dread d6/d19/f4e [4194304,4194304] 0 2024-03-16T09:01:18.077 INFO:tasks.workunit.client.1.smithi019.stdout:6/653: write de/d1c/f8d [384945,97345] 0 2024-03-16T09:01:18.079 INFO:tasks.workunit.client.1.smithi019.stdout:4/503: mknod d4/d73/d8a/ca8 0 2024-03-16T09:01:18.080 INFO:tasks.workunit.client.0.smithi007.stdout:5/687: creat d1/ff6 x:0 0 0 2024-03-16T09:01:18.080 INFO:tasks.workunit.client.0.smithi007.stdout:7/673: dread d0/d6c/d25/d7a/fe8 [0,4194304] 0 2024-03-16T09:01:18.082 INFO:tasks.workunit.client.1.smithi019.stdout:1/461: link d16/d18/c40 d16/d31/d76/c7f 0 2024-03-16T09:01:18.082 INFO:tasks.workunit.client.1.smithi019.stdout:1/462: chown d16/d1f/d43/f6f 2 1 2024-03-16T09:01:18.083 INFO:tasks.workunit.client.0.smithi007.stdout:7/674: link d0/f3 d0/d6c/d25/d3a/ff6 0 2024-03-16T09:01:18.083 INFO:tasks.workunit.client.1.smithi019.stdout:1/463: dread d16/d18/f1e [0,4194304] 0 2024-03-16T09:01:18.083 INFO:tasks.workunit.client.1.smithi019.stdout:5/535: unlink d6/d1f/d33/c7b 0 2024-03-16T09:01:18.085 INFO:tasks.workunit.client.1.smithi019.stdout:6/654: rename de/d26/d32/d4e/f79 to de/d26/d32/d4e/d57/fd4 0 2024-03-16T09:01:18.087 INFO:tasks.workunit.client.1.smithi019.stdout:4/504: mkdir d4/d64/d9b/da9 0 2024-03-16T09:01:18.087 INFO:tasks.workunit.client.1.smithi019.stdout:4/505: readlink d4/l69 0 2024-03-16T09:01:18.087 INFO:tasks.workunit.client.1.smithi019.stdout:4/506: truncate d4/d1b/d49/f53 180940 0 2024-03-16T09:01:18.087 INFO:tasks.workunit.client.1.smithi019.stdout:4/507: fdatasync d4/d6/d15/f60 0 2024-03-16T09:01:18.087 INFO:tasks.workunit.client.1.smithi019.stdout:4/508: chown d4/d64/l84 25624 1 2024-03-16T09:01:18.088 INFO:tasks.workunit.client.1.smithi019.stdout:1/464: symlink d16/d31/d76/l80 0 2024-03-16T09:01:18.089 INFO:tasks.workunit.client.1.smithi019.stdout:5/536: rmdir d6/dd/d47/d70 39 2024-03-16T09:01:18.089 INFO:tasks.workunit.client.1.smithi019.stdout:5/537: dread - d6/dd/d47/f69 zero size 2024-03-16T09:01:18.090 INFO:tasks.workunit.client.1.smithi019.stdout:6/655: mkdir de/d26/d32/d97/d66/d72/d7f/dd5 0 2024-03-16T09:01:18.092 INFO:tasks.workunit.client.1.smithi019.stdout:7/448: dwrite d9/da/f59 [0,4194304] 0 2024-03-16T09:01:18.092 INFO:tasks.workunit.client.1.smithi019.stdout:7/449: chown d9/d25/d28/d48/f4b 1 1 2024-03-16T09:01:18.092 INFO:tasks.workunit.client.1.smithi019.stdout:4/509: creat d4/d64/faa x:0 0 0 2024-03-16T09:01:18.092 INFO:tasks.workunit.client.1.smithi019.stdout:4/510: write d4/d1b/d2a/f4d [957453,45792] 0 2024-03-16T09:01:18.093 INFO:tasks.workunit.client.1.smithi019.stdout:1/465: symlink d16/d31/d7b/l81 0 2024-03-16T09:01:18.095 INFO:tasks.workunit.client.1.smithi019.stdout:6/656: rmdir de/d1c/d9a/d9e 39 2024-03-16T09:01:18.100 INFO:tasks.workunit.client.0.smithi007.stdout:7/675: rename d0/d6c/d25 to d0/d19/d1f/d5d/d9d/df7 0 2024-03-16T09:01:18.100 INFO:tasks.workunit.client.0.smithi007.stdout:7/676: chown d0/d19/d1f/d5d/d9d/df7/f7f 394922 1 2024-03-16T09:01:18.100 INFO:tasks.workunit.client.0.smithi007.stdout:7/677: dread - d0/d19/d1f/d5d/d9d/df7/d7a/fa8 zero size 2024-03-16T09:01:18.102 INFO:tasks.workunit.client.1.smithi019.stdout:7/450: rename d9/db/d12/d5b/l4e to d9/db/l89 0 2024-03-16T09:01:18.102 INFO:tasks.workunit.client.1.smithi019.stdout:7/451: chown d9/da/d13/d18/d30/d3e/d5d/f7c 0 1 2024-03-16T09:01:18.103 INFO:tasks.workunit.client.0.smithi007.stdout:7/678: mkdir d0/d19/d1f/d5d/d9d/df8 0 2024-03-16T09:01:18.103 INFO:tasks.workunit.client.0.smithi007.stdout:7/679: dread - d0/d19/d1f/d5d/d9d/df7/d3a/db3/fce zero size 2024-03-16T09:01:18.103 INFO:tasks.workunit.client.1.smithi019.stdout:4/511: mknod d4/d9e/cab 0 2024-03-16T09:01:18.104 INFO:tasks.workunit.client.0.smithi007.stdout:7/680: mknod d0/dc/d5a/dba/dd9/d93/cf9 0 2024-03-16T09:01:18.104 INFO:tasks.workunit.client.0.smithi007.stdout:7/681: write d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f7b [4197907,36656] 0 2024-03-16T09:01:18.105 INFO:tasks.workunit.client.1.smithi019.stdout:1/466: unlink d16/d18/c36 0 2024-03-16T09:01:18.106 INFO:tasks.workunit.client.1.smithi019.stdout:7/452: getdents d9/da/d13/d18/d1a 0 2024-03-16T09:01:18.107 INFO:tasks.workunit.client.0.smithi007.stdout:7/682: getdents d0/d19 0 2024-03-16T09:01:18.107 INFO:tasks.workunit.client.0.smithi007.stdout:7/683: truncate d0/d19/d1f/d5d/d9d/df7/d3a/daf/dc2/fe6 933371 0 2024-03-16T09:01:18.108 INFO:tasks.workunit.client.0.smithi007.stdout:7/684: creat d0/d19/d1f/d69/deb/ffa x:0 0 0 2024-03-16T09:01:18.109 INFO:tasks.workunit.client.1.smithi019.stdout:1/467: symlink d16/d31/d76/l82 0 2024-03-16T09:01:18.109 INFO:tasks.workunit.client.1.smithi019.stdout:1/468: fsync f9 0 2024-03-16T09:01:18.109 INFO:tasks.workunit.client.1.smithi019.stdout:1/469: stat d16/d1f 0 2024-03-16T09:01:18.109 INFO:tasks.workunit.client.1.smithi019.stdout:1/470: truncate d16/d31/f45 554291 0 2024-03-16T09:01:18.109 INFO:tasks.workunit.client.1.smithi019.stdout:1/471: fdatasync d16/d1f/d4f/f50 0 2024-03-16T09:01:18.116 INFO:tasks.workunit.client.0.smithi007.stdout:7/685: dread d0/d19/d1f/d5d/d9d/df7/d73/f91 [0,4194304] 0 2024-03-16T09:01:18.117 INFO:tasks.workunit.client.0.smithi007.stdout:5/688: dwrite d1/d7/d1c/d88/da2/dae/fde [4194304,4194304] 0 2024-03-16T09:01:18.118 INFO:tasks.workunit.client.1.smithi019.stdout:5/538: dwrite d6/d1f/d33/f4b [0,4194304] 0 2024-03-16T09:01:18.118 INFO:tasks.workunit.client.1.smithi019.stdout:5/539: fdatasync d6/d19/f9b 0 2024-03-16T09:01:18.118 INFO:tasks.workunit.client.1.smithi019.stdout:4/512: fsync d4/d1b/d2a/f4d 0 2024-03-16T09:01:18.118 INFO:tasks.workunit.client.1.smithi019.stdout:4/513: write d4/d64/d99/f3b [909492,320] 0 2024-03-16T09:01:18.127 INFO:tasks.workunit.client.1.smithi019.stdout:4/514: symlink d4/lac 0 2024-03-16T09:01:18.127 INFO:tasks.workunit.client.1.smithi019.stdout:4/515: fdatasync d4/d6/d15/f2e 0 2024-03-16T09:01:18.127 INFO:tasks.workunit.client.1.smithi019.stdout:4/516: chown d4/d1b/d49/d58/d80/d8e 79069 1 2024-03-16T09:01:18.128 INFO:tasks.workunit.client.1.smithi019.stdout:4/517: dread - d4/d6/f81 zero size 2024-03-16T09:01:18.128 INFO:tasks.workunit.client.1.smithi019.stdout:4/518: rename d4/d1b/d2a/d4f/l5a to d4/d6c/d79/lad 0 2024-03-16T09:01:18.129 INFO:tasks.workunit.client.1.smithi019.stdout:4/519: truncate d4/d6/f81 439380 0 2024-03-16T09:01:18.131 INFO:tasks.workunit.client.1.smithi019.stdout:4/520: getdents d4/d1b 0 2024-03-16T09:01:18.132 INFO:tasks.workunit.client.1.smithi019.stdout:4/521: creat d4/d6/d15/da2/fae x:0 0 0 2024-03-16T09:01:18.143 INFO:tasks.workunit.client.1.smithi019.stdout:6/657: dwrite de/d1c/f20 [4194304,4194304] 0 2024-03-16T09:01:18.144 INFO:tasks.workunit.client.1.smithi019.stdout:6/658: dread - de/d26/d32/d97/d51/fba zero size 2024-03-16T09:01:18.144 INFO:tasks.workunit.client.1.smithi019.stdout:6/659: chown c6 32927 1 2024-03-16T09:01:18.151 INFO:tasks.workunit.client.1.smithi019.stdout:8/558: sync 2024-03-16T09:01:18.164 INFO:tasks.workunit.client.1.smithi019.stdout:7/453: dwrite d9/db/d12/f6d [0,4194304] 0 2024-03-16T09:01:18.164 INFO:tasks.workunit.client.1.smithi019.stdout:1/472: dwrite d16/d31/f45 [0,4194304] 0 2024-03-16T09:01:18.166 INFO:tasks.workunit.client.0.smithi007.stdout:5/689: dwrite d1/d7/d16/d1b/f60 [0,4194304] 0 2024-03-16T09:01:18.166 INFO:tasks.workunit.client.1.smithi019.stdout:5/540: dwrite d6/d19/f9b [0,4194304] 0 2024-03-16T09:01:18.167 INFO:tasks.workunit.client.0.smithi007.stdout:7/686: dwrite d0/dc/d5a/fdc [0,4194304] 0 2024-03-16T09:01:18.177 INFO:tasks.workunit.client.1.smithi019.stdout:1/473: dread d16/f37 [0,4194304] 0 2024-03-16T09:01:18.182 INFO:tasks.workunit.client.1.smithi019.stdout:1/474: dread d16/d18/f1e [0,4194304] 0 2024-03-16T09:01:18.182 INFO:tasks.workunit.client.1.smithi019.stdout:1/475: chown d16/d1f/d4f/f50 630793 1 2024-03-16T09:01:18.190 INFO:tasks.workunit.client.1.smithi019.stdout:8/559: dwrite d6/d2a/d45/d2c/f5c [0,4194304] 0 2024-03-16T09:01:18.193 INFO:tasks.workunit.client.1.smithi019.stdout:1/476: dread d16/d31/f45 [0,4194304] 0 2024-03-16T09:01:18.193 INFO:tasks.workunit.client.1.smithi019.stdout:1/477: write d16/d1f/d43/f6f [655558,22256] 0 2024-03-16T09:01:18.200 INFO:tasks.workunit.client.1.smithi019.stdout:7/454: dwrite d9/db/f2c [0,4194304] 0 2024-03-16T09:01:18.200 INFO:tasks.workunit.client.0.smithi007.stdout:5/690: dwrite d1/d7/d1c/d88/da2/d89/db3/f31 [0,4194304] 0 2024-03-16T09:01:18.200 INFO:tasks.workunit.client.0.smithi007.stdout:7/687: dwrite d0/d6c/f5e [0,4194304] 0 2024-03-16T09:01:18.201 INFO:tasks.workunit.client.0.smithi007.stdout:7/688: dread d0/d19/d1f/d5d/d9d/df7/d7a/fe8 [0,4194304] 0 2024-03-16T09:01:18.202 INFO:tasks.workunit.client.0.smithi007.stdout:7/689: readlink d0/l14 0 2024-03-16T09:01:18.202 INFO:tasks.workunit.client.0.smithi007.stdout:5/691: mkdir d1/d7/d1c/d19/df7 0 2024-03-16T09:01:18.202 INFO:tasks.workunit.client.0.smithi007.stdout:5/692: chown d1/d7/d16/d1b/d3c/fa7 871 1 2024-03-16T09:01:18.203 INFO:tasks.workunit.client.0.smithi007.stdout:7/690: symlink d0/d19/d1f/d5d/d9d/df7/d3a/db3/lfb 0 2024-03-16T09:01:18.204 INFO:tasks.workunit.client.0.smithi007.stdout:5/693: creat d1/d7/d1c/d88/da2/dae/ff8 x:0 0 0 2024-03-16T09:01:18.204 INFO:tasks.workunit.client.1.smithi019.stdout:6/660: mknod de/d26/d32/d4e/cd6 0 2024-03-16T09:01:18.205 INFO:tasks.workunit.client.0.smithi007.stdout:7/691: mknod d0/d19/d1f/d69/dc7/cfc 0 2024-03-16T09:01:18.207 INFO:tasks.workunit.client.0.smithi007.stdout:5/694: symlink d1/d7/d1c/d19/d34/d4d/lf9 0 2024-03-16T09:01:18.207 INFO:tasks.workunit.client.0.smithi007.stdout:5/695: truncate d1/d7/d1c/d19/d34/fdb 781000 0 2024-03-16T09:01:18.207 INFO:tasks.workunit.client.0.smithi007.stdout:5/696: creat d1/d7/d16/ffa x:0 0 0 2024-03-16T09:01:18.207 INFO:tasks.workunit.client.0.smithi007.stdout:5/697: dread - d1/d7/d1c/d88/da2/fb1 zero size 2024-03-16T09:01:18.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:17 smithi019 ceph-mon[36407]: pgmap v22: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 230 MiB/s rd, 285 MiB/s wr, 603 op/s 2024-03-16T09:01:18.217 INFO:tasks.workunit.client.1.smithi019.stdout:5/541: unlink d6/dd/d47/f69 0 2024-03-16T09:01:18.218 INFO:tasks.workunit.client.0.smithi007.stdout:5/698: dread d1/d7/d1c/f80 [0,4194304] 0 2024-03-16T09:01:18.219 INFO:tasks.workunit.client.0.smithi007.stdout:5/699: write d1/d7/d1c/d19/d34/d3e/d41/da6/fc9 [1189987,125762] 0 2024-03-16T09:01:18.219 INFO:tasks.workunit.client.1.smithi019.stdout:8/560: creat d6/d2a/d45/d57/db6/fc0 x:0 0 0 2024-03-16T09:01:18.219 INFO:tasks.workunit.client.0.smithi007.stdout:7/692: dwrite d0/d19/f26 [8388608,4194304] 0 2024-03-16T09:01:18.220 INFO:tasks.workunit.client.0.smithi007.stdout:5/700: mknod d1/d7/d1c/d19/d34/d5e/d6f/d72/cfb 0 2024-03-16T09:01:18.220 INFO:tasks.workunit.client.1.smithi019.stdout:1/478: mknod d16/d1f/d4f/c83 0 2024-03-16T09:01:18.223 INFO:tasks.workunit.client.0.smithi007.stdout:5/701: unlink d1/d7/d1c/d19/d34/d5e/d6f/f54 0 2024-03-16T09:01:18.223 INFO:tasks.workunit.client.0.smithi007.stdout:5/702: chown d1/c68 50 1 2024-03-16T09:01:18.223 INFO:tasks.workunit.client.0.smithi007.stdout:5/703: truncate d1/d7/d1c/d19/d34/de/ff5 562359 0 2024-03-16T09:01:18.224 INFO:tasks.workunit.client.0.smithi007.stdout:5/704: mknod d1/d7/d1c/d19/d34/d5e/d6f/d72/cfc 0 2024-03-16T09:01:18.224 INFO:tasks.workunit.client.1.smithi019.stdout:7/455: creat d9/f8a x:0 0 0 2024-03-16T09:01:18.224 INFO:tasks.workunit.client.1.smithi019.stdout:7/456: creat d9/d25/d28/f8b x:0 0 0 2024-03-16T09:01:18.225 INFO:tasks.workunit.client.0.smithi007.stdout:5/705: symlink d1/d7/d1c/df0/lfd 0 2024-03-16T09:01:18.225 INFO:tasks.workunit.client.1.smithi019.stdout:4/522: rmdir d4/d6/d15/d9a 39 2024-03-16T09:01:18.225 INFO:tasks.workunit.client.1.smithi019.stdout:4/523: fdatasync d4/d1b/d49/d6e/f90 0 2024-03-16T09:01:18.226 INFO:tasks.workunit.client.1.smithi019.stdout:3/548: sync 2024-03-16T09:01:18.227 INFO:tasks.workunit.client.0.smithi007.stdout:5/706: mknod d1/d7/d1c/d19/df7/cfe 0 2024-03-16T09:01:18.227 INFO:tasks.workunit.client.1.smithi019.stdout:6/661: unlink de/d1c/c22 0 2024-03-16T09:01:18.229 INFO:tasks.workunit.client.0.smithi007.stdout:7/693: write d0/d19/d1f/d5d/d9d/df7/d3a/f77 [4093034,16911] 0 2024-03-16T09:01:18.233 INFO:tasks.workunit.client.1.smithi019.stdout:5/542: rename d6/d19/f59 to d6/dd/d61/fa9 0 2024-03-16T09:01:18.236 INFO:tasks.workunit.client.1.smithi019.stdout:8/561: truncate d6/de/f49 3672439 0 2024-03-16T09:01:18.236 INFO:tasks.workunit.client.1.smithi019.stdout:0/556: sync 2024-03-16T09:01:18.236 INFO:tasks.workunit.client.1.smithi019.stdout:0/557: truncate d7/d8/d32/d66/d6d/d8c/db8/f69 1986023 0 2024-03-16T09:01:18.236 INFO:tasks.workunit.client.1.smithi019.stdout:2/574: sync 2024-03-16T09:01:18.238 INFO:tasks.workunit.client.1.smithi019.stdout:1/479: truncate d16/f37 518669 0 2024-03-16T09:01:18.238 INFO:tasks.workunit.client.1.smithi019.stdout:1/480: truncate d16/d18/f1b 1557675 0 2024-03-16T09:01:18.238 INFO:tasks.workunit.client.1.smithi019.stdout:1/481: chown d16/d18/c40 222 1 2024-03-16T09:01:18.238 INFO:tasks.workunit.client.0.smithi007.stdout:7/694: dread d0/dc/d5a/fdc [0,4194304] 0 2024-03-16T09:01:18.240 INFO:tasks.workunit.client.1.smithi019.stdout:7/457: truncate d9/db/d12/f2f 805452 0 2024-03-16T09:01:18.240 INFO:tasks.workunit.client.1.smithi019.stdout:7/458: truncate d9/da/d13/f84 1006806 0 2024-03-16T09:01:18.241 INFO:tasks.workunit.client.1.smithi019.stdout:9/412: sync 2024-03-16T09:01:18.242 INFO:tasks.workunit.client.1.smithi019.stdout:4/524: link d4/d1b/d2a/la7 d4/d64/laf 0 2024-03-16T09:01:18.243 INFO:tasks.workunit.client.1.smithi019.stdout:3/549: symlink d3/d17/lab 0 2024-03-16T09:01:18.245 INFO:tasks.workunit.client.1.smithi019.stdout:6/662: symlink de/d26/d32/dce/ld7 0 2024-03-16T09:01:18.245 INFO:tasks.workunit.client.1.smithi019.stdout:6/663: stat de/d26/d32/d97/d51/d70 0 2024-03-16T09:01:18.245 INFO:tasks.workunit.client.1.smithi019.stdout:6/664: fsync de/d26/d32/d97/d66/d72/d7f/fc9 0 2024-03-16T09:01:18.248 INFO:tasks.workunit.client.1.smithi019.stdout:5/543: dread d6/d1f/f55 [0,4194304] 0 2024-03-16T09:01:18.248 INFO:tasks.workunit.client.0.smithi007.stdout:7/695: write d0/d19/d1f/d5d/d9d/df7/d7a/f84 [1972180,65295] 0 2024-03-16T09:01:18.248 INFO:tasks.workunit.client.0.smithi007.stdout:5/707: dwrite d1/d7/d1c/d19/d34/d5e/f8e [0,4194304] 0 2024-03-16T09:01:18.249 INFO:tasks.workunit.client.0.smithi007.stdout:5/708: mkdir d1/d7/d1c/d88/da2/d89/db3/d2a/d81/dff 0 2024-03-16T09:01:18.249 INFO:tasks.workunit.client.0.smithi007.stdout:5/709: dread - d1/d7/d16/d1b/fd9 zero size 2024-03-16T09:01:18.254 INFO:tasks.workunit.client.1.smithi019.stdout:0/558: creat d7/d8/d32/d66/d6d/d8c/db8/d82/fc2 x:0 0 0 2024-03-16T09:01:18.254 INFO:tasks.workunit.client.1.smithi019.stdout:0/559: chown d7/d1d/c5c 188338969 1 2024-03-16T09:01:18.254 INFO:tasks.workunit.client.1.smithi019.stdout:0/560: creat d7/d1d/d2e/fc3 x:0 0 0 2024-03-16T09:01:18.262 INFO:tasks.workunit.client.1.smithi019.stdout:2/575: mknod d5/d6/d2b/d2f/d35/cbc 0 2024-03-16T09:01:18.263 INFO:tasks.workunit.client.1.smithi019.stdout:7/459: link d9/f3f d9/da/d13/d18/d30/d75/f8c 0 2024-03-16T09:01:18.263 INFO:tasks.workunit.client.0.smithi007.stdout:5/710: dread d1/d7/d1c/d19/d34/d5e/d6f/d72/f92 [0,4194304] 0 2024-03-16T09:01:18.263 INFO:tasks.workunit.client.0.smithi007.stdout:5/711: symlink d1/d7/d1c/d88/da2/d89/db3/d7d/l100 0 2024-03-16T09:01:18.267 INFO:tasks.workunit.client.1.smithi019.stdout:4/525: rename d4/d1b/d49/f9c to d4/d73/fb0 0 2024-03-16T09:01:18.269 INFO:tasks.workunit.client.1.smithi019.stdout:3/550: rmdir d3/d17/d5a/d80 39 2024-03-16T09:01:18.270 INFO:tasks.workunit.client.1.smithi019.stdout:8/562: dwrite d6/d2a/d3c/f46 [0,4194304] 0 2024-03-16T09:01:18.270 INFO:tasks.workunit.client.1.smithi019.stdout:2/576: dread d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:18.270 INFO:tasks.workunit.client.1.smithi019.stdout:8/563: chown d6/d2a/d3c/f44 106147340 1 2024-03-16T09:01:18.270 INFO:tasks.workunit.client.1.smithi019.stdout:2/577: truncate d5/d5a/f9d 241392 0 2024-03-16T09:01:18.271 INFO:tasks.workunit.client.1.smithi019.stdout:6/665: truncate de/f86 4086891 0 2024-03-16T09:01:18.273 INFO:tasks.workunit.client.1.smithi019.stdout:5/544: unlink d6/d19/f2e 0 2024-03-16T09:01:18.273 INFO:tasks.workunit.client.1.smithi019.stdout:5/545: write d6/dd/f5c [1022547,77143] 0 2024-03-16T09:01:18.276 INFO:tasks.workunit.client.0.smithi007.stdout:5/712: dread d1/d7/d1c/d88/da2/d89/fb8 [0,4194304] 0 2024-03-16T09:01:18.276 INFO:tasks.workunit.client.0.smithi007.stdout:5/713: stat d1/d7/d1c/d19/d34/d4d/l8a 0 2024-03-16T09:01:18.276 INFO:tasks.workunit.client.0.smithi007.stdout:5/714: creat d1/d7/d1c/d19/d34/d4d/f101 x:0 0 0 2024-03-16T09:01:18.279 INFO:tasks.workunit.client.1.smithi019.stdout:1/482: dwrite d16/d1f/f2e [0,4194304] 0 2024-03-16T09:01:18.280 INFO:tasks.workunit.client.1.smithi019.stdout:8/564: dread d6/d2a/f88 [0,4194304] 0 2024-03-16T09:01:18.280 INFO:tasks.workunit.client.1.smithi019.stdout:1/483: write fb [4791090,60182] 0 2024-03-16T09:01:18.280 INFO:tasks.workunit.client.1.smithi019.stdout:0/561: dwrite d7/d8/d32/d66/d6d/d8c/db8/f35 [0,4194304] 0 2024-03-16T09:01:18.280 INFO:tasks.workunit.client.0.smithi007.stdout:7/696: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/db3/fce [0,4194304] 0 2024-03-16T09:01:18.281 INFO:tasks.workunit.client.0.smithi007.stdout:5/715: dread f0 [0,4194304] 0 2024-03-16T09:01:18.281 INFO:tasks.workunit.client.0.smithi007.stdout:5/716: stat d1/d7/d1c/d19/d34/d5e/f56 0 2024-03-16T09:01:18.281 INFO:tasks.workunit.client.1.smithi019.stdout:7/460: mkdir d9/d25/d28/d48/d8d 0 2024-03-16T09:01:18.283 INFO:tasks.workunit.client.1.smithi019.stdout:4/526: creat d4/d6/d15/d9a/fb1 x:0 0 0 2024-03-16T09:01:18.283 INFO:tasks.workunit.client.1.smithi019.stdout:3/551: creat d3/d11/fac x:0 0 0 2024-03-16T09:01:18.283 INFO:tasks.workunit.client.1.smithi019.stdout:3/552: creat d3/d17/d3f/fad x:0 0 0 2024-03-16T09:01:18.283 INFO:tasks.workunit.client.1.smithi019.stdout:3/553: write d3/d17/d5a/d75/d7a/f87 [1661415,56629] 0 2024-03-16T09:01:18.284 INFO:tasks.workunit.client.1.smithi019.stdout:4/527: read d4/d1b/d49/f53 [14543,85548] 0 2024-03-16T09:01:18.285 INFO:tasks.workunit.client.0.smithi007.stdout:5/717: mkdir d1/d7/d1c/d19/dcc/d102 0 2024-03-16T09:01:18.288 INFO:tasks.workunit.client.1.smithi019.stdout:2/578: creat d5/d6/d64/d6f/da6/d83/fbd x:0 0 0 2024-03-16T09:01:18.288 INFO:tasks.workunit.client.0.smithi007.stdout:5/718: symlink d1/d7/d1c/d88/da2/l103 0 2024-03-16T09:01:18.288 INFO:tasks.workunit.client.1.smithi019.stdout:4/528: dread d4/d1b/f28 [0,4194304] 0 2024-03-16T09:01:18.290 INFO:tasks.workunit.client.0.smithi007.stdout:5/719: rename d1/d7/cc2 to d1/d7/d1c/d19/d34/d4d/de7/c104 0 2024-03-16T09:01:18.290 INFO:tasks.workunit.client.0.smithi007.stdout:5/720: write d1/d7/d1c/d19/d34/d3e/f6a [3650026,38700] 0 2024-03-16T09:01:18.290 INFO:tasks.workunit.client.0.smithi007.stdout:5/721: chown d1/d7/d16/d1b/df1 834729 1 2024-03-16T09:01:18.291 INFO:tasks.workunit.client.1.smithi019.stdout:6/666: rename de/d1c/c6b to de/d1c/d9a/cd8 0 2024-03-16T09:01:18.291 INFO:tasks.workunit.client.1.smithi019.stdout:6/667: chown de/d1c/f56 28787488 1 2024-03-16T09:01:18.293 INFO:tasks.workunit.client.0.smithi007.stdout:5/722: stat d1/d7/d1c/d19/d34/d3e/c63 0 2024-03-16T09:01:18.300 INFO:tasks.workunit.client.1.smithi019.stdout:7/461: symlink d9/da/d13/l8e 0 2024-03-16T09:01:18.300 INFO:tasks.workunit.client.1.smithi019.stdout:4/529: write d4/d1b/d49/f72 [1574371,38674] 0 2024-03-16T09:01:18.302 INFO:tasks.workunit.client.1.smithi019.stdout:2/579: truncate d5/d6/d64/d6f/fb6 1785633 0 2024-03-16T09:01:18.303 INFO:tasks.workunit.client.1.smithi019.stdout:6/668: getdents de/d26/d32/d4e/d75/dcd 0 2024-03-16T09:01:18.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/723: mkdir d1/d7/d1c/d19/df7/d105 0 2024-03-16T09:01:18.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/724: dread - d1/d7/d1c/d19/d34/d3e/fe5 zero size 2024-03-16T09:01:18.305 INFO:tasks.workunit.client.0.smithi007.stdout:5/725: mknod d1/d7/d1c/d19/d34/d3e/d62/c106 0 2024-03-16T09:01:18.310 INFO:tasks.workunit.client.1.smithi019.stdout:2/580: unlink d5/d4e/d74/fae 0 2024-03-16T09:01:18.310 INFO:tasks.workunit.client.1.smithi019.stdout:0/562: dwrite d7/d1d/d63/d6e/f93 [0,4194304] 0 2024-03-16T09:01:18.310 INFO:tasks.workunit.client.1.smithi019.stdout:0/563: dread - d7/d1d/d2e/fc3 zero size 2024-03-16T09:01:18.310 INFO:tasks.workunit.client.1.smithi019.stdout:5/546: dwrite d6/d1f/d78/f9a [0,4194304] 0 2024-03-16T09:01:18.310 INFO:tasks.workunit.client.1.smithi019.stdout:1/484: dwrite d16/d18/f1a [0,4194304] 0 2024-03-16T09:01:18.312 INFO:tasks.workunit.client.0.smithi007.stdout:7/697: dwrite d0/dc/d38/d4e/fd5 [0,4194304] 0 2024-03-16T09:01:18.312 INFO:tasks.workunit.client.0.smithi007.stdout:7/698: dread - d0/d19/d1f/d5d/d9d/df7/d3a/db3/fcc zero size 2024-03-16T09:01:18.312 INFO:tasks.workunit.client.0.smithi007.stdout:7/699: chown d0/d19/f48 960 1 2024-03-16T09:01:18.312 INFO:tasks.workunit.client.0.smithi007.stdout:7/700: fsync d0/d19/d1f/d5d/d9d/df7/d7a/f6e 0 2024-03-16T09:01:18.315 INFO:tasks.workunit.client.0.smithi007.stdout:7/701: mkdir d0/d6c/dfd 0 2024-03-16T09:01:18.317 INFO:tasks.workunit.client.0.smithi007.stdout:7/702: symlink d0/d6c/dfd/lfe 0 2024-03-16T09:01:18.317 INFO:tasks.workunit.client.0.smithi007.stdout:7/703: readlink d0/d6c/ldf 0 2024-03-16T09:01:18.339 INFO:tasks.workunit.client.0.smithi007.stdout:0/825: sync 2024-03-16T09:01:18.339 INFO:tasks.workunit.client.0.smithi007.stdout:0/826: readlink d3/d9/da/d48/d25/d87/dbb/ld3 0 2024-03-16T09:01:18.340 INFO:tasks.workunit.client.0.smithi007.stdout:8/981: sync 2024-03-16T09:01:18.340 INFO:tasks.workunit.client.0.smithi007.stdout:8/982: fdatasync d0/d26/d2d/ff2 0 2024-03-16T09:01:18.340 INFO:tasks.workunit.client.0.smithi007.stdout:0/827: rmdir d3/d9/da/d48/d25/d43/d1a 39 2024-03-16T09:01:18.340 INFO:tasks.workunit.client.0.smithi007.stdout:0/828: chown d3/f8b 3416667 1 2024-03-16T09:01:18.340 INFO:tasks.workunit.client.0.smithi007.stdout:0/829: fdatasync d3/d9/da/d48/d25/d43/d1a/f30 0 2024-03-16T09:01:18.341 INFO:tasks.workunit.client.1.smithi019.stdout:4/530: getdents d4/d6 0 2024-03-16T09:01:18.342 INFO:tasks.workunit.client.0.smithi007.stdout:0/830: getdents d3/d9/da/d48/d25/d43/d1a 0 2024-03-16T09:01:18.342 INFO:tasks.workunit.client.0.smithi007.stdout:0/831: stat d3/d9/db4/cd8 0 2024-03-16T09:01:18.342 INFO:tasks.workunit.client.0.smithi007.stdout:0/832: dread - d3/faf zero size 2024-03-16T09:01:18.343 INFO:tasks.workunit.client.0.smithi007.stdout:0/833: symlink d3/d9/da/d48/d25/d43/d1a/l10f 0 2024-03-16T09:01:18.343 INFO:tasks.workunit.client.0.smithi007.stdout:0/834: fdatasync d3/d9/db4/d2c/fc1 0 2024-03-16T09:01:18.344 INFO:tasks.workunit.client.1.smithi019.stdout:0/564: mknod d7/d8/d32/d66/d6d/d7c/cc4 0 2024-03-16T09:01:18.345 INFO:tasks.workunit.client.1.smithi019.stdout:0/565: write d7/d8/d32/d66/d6d/d8c/db8/f5f [791037,38826] 0 2024-03-16T09:01:18.348 INFO:tasks.workunit.client.1.smithi019.stdout:5/547: getdents d6/d1f/d77/d84 0 2024-03-16T09:01:18.350 INFO:tasks.workunit.client.1.smithi019.stdout:1/485: mkdir d16/d84 0 2024-03-16T09:01:18.352 INFO:tasks.workunit.client.1.smithi019.stdout:8/565: rename d6/d2a/d45/d57/db6 to d6/d2a/d45/d57/d72/d79/dc1 0 2024-03-16T09:01:18.352 INFO:tasks.workunit.client.1.smithi019.stdout:8/566: fdatasync d6/de/ff 0 2024-03-16T09:01:18.353 INFO:tasks.workunit.client.1.smithi019.stdout:4/531: creat d4/d1b/d49/d6e/fb2 x:0 0 0 2024-03-16T09:01:18.357 INFO:tasks.workunit.client.1.smithi019.stdout:8/567: symlink d6/d2a/d45/d57/d72/d79/d7b/lc2 0 2024-03-16T09:01:18.358 INFO:tasks.workunit.client.1.smithi019.stdout:5/548: dread d6/dd/f44 [0,4194304] 0 2024-03-16T09:01:18.359 INFO:tasks.workunit.client.1.smithi019.stdout:7/462: rename d9/l67 to d9/d25/d28/d48/l8f 0 2024-03-16T09:01:18.361 INFO:tasks.workunit.client.1.smithi019.stdout:4/532: truncate d4/d64/d99/f40 27434 0 2024-03-16T09:01:18.361 INFO:tasks.workunit.client.1.smithi019.stdout:4/533: creat d4/d6/d15/fb3 x:0 0 0 2024-03-16T09:01:18.361 INFO:tasks.workunit.client.1.smithi019.stdout:8/568: mknod d6/d2a/d3c/dab/d30/d4a/d87/cc3 0 2024-03-16T09:01:18.362 INFO:tasks.workunit.client.1.smithi019.stdout:8/569: fsync d6/d2a/d3c/dab/faa 0 2024-03-16T09:01:18.362 INFO:tasks.workunit.client.1.smithi019.stdout:2/581: rename d5/d2e/c48 to d5/d5a/d96/dba/cbe 0 2024-03-16T09:01:18.363 INFO:tasks.workunit.client.1.smithi019.stdout:2/582: truncate d5/d6/d2b/d2f/d35/d4b/f93 187926 0 2024-03-16T09:01:18.363 INFO:tasks.workunit.client.1.smithi019.stdout:7/463: rmdir d9/da/d13/d18/d30/d3e 39 2024-03-16T09:01:18.364 INFO:tasks.workunit.client.1.smithi019.stdout:4/534: truncate d4/d6/f81 279988 0 2024-03-16T09:01:18.364 INFO:tasks.workunit.client.1.smithi019.stdout:4/535: readlink d4/l69 0 2024-03-16T09:01:18.364 INFO:tasks.workunit.client.1.smithi019.stdout:4/536: readlink d4/d6/d15/l2b 0 2024-03-16T09:01:18.365 INFO:tasks.workunit.client.1.smithi019.stdout:1/486: rename d16/d18/l5b to d16/d1f/l85 0 2024-03-16T09:01:18.367 INFO:tasks.workunit.client.1.smithi019.stdout:2/583: creat d5/d6/d64/d6f/fbf x:0 0 0 2024-03-16T09:01:18.378 INFO:tasks.workunit.client.1.smithi019.stdout:7/464: write d9/da/d13/f17 [1787078,7649] 0 2024-03-16T09:01:18.380 INFO:tasks.workunit.client.1.smithi019.stdout:7/465: symlink d9/db/d12/l90 0 2024-03-16T09:01:18.380 INFO:tasks.workunit.client.1.smithi019.stdout:7/466: readlink d9/da/d13/d18/d30/d3e/l68 0 2024-03-16T09:01:18.381 INFO:tasks.workunit.client.1.smithi019.stdout:7/467: creat d9/da/d13/d18/d1a/f91 x:0 0 0 2024-03-16T09:01:18.381 INFO:tasks.workunit.client.1.smithi019.stdout:7/468: stat d9/da/f24 0 2024-03-16T09:01:18.381 INFO:tasks.workunit.client.1.smithi019.stdout:7/469: write d9/da/f24 [933018,3535] 0 2024-03-16T09:01:18.385 INFO:tasks.workunit.client.1.smithi019.stdout:2/584: write d5/d2e/f3b [1652230,22273] 0 2024-03-16T09:01:18.387 INFO:tasks.workunit.client.1.smithi019.stdout:2/585: rename d5/d6/d64/d6f/fbf to d5/d6/d64/d6f/da6/fc0 0 2024-03-16T09:01:18.389 INFO:tasks.workunit.client.1.smithi019.stdout:2/586: rename d5/d6/d2b/d53/f73 to d5/de/da2/fc1 0 2024-03-16T09:01:18.389 INFO:tasks.workunit.client.0.smithi007.stdout:8/983: dwrite d0/d26/d2d/d64/d123/f12d [0,4194304] 0 2024-03-16T09:01:18.390 INFO:tasks.workunit.client.1.smithi019.stdout:2/587: truncate d5/de/f32 395325 0 2024-03-16T09:01:18.390 INFO:tasks.workunit.client.1.smithi019.stdout:2/588: rmdir d5/d6/d2b/d2f/d35/d65 39 2024-03-16T09:01:18.391 INFO:tasks.workunit.client.1.smithi019.stdout:2/589: fdatasync d5/d4e/d74/f90 0 2024-03-16T09:01:18.391 INFO:tasks.workunit.client.0.smithi007.stdout:8/984: mkdir d0/d26/d2d/d43/d70/d10f/d15a 0 2024-03-16T09:01:18.392 INFO:tasks.workunit.client.1.smithi019.stdout:2/590: creat d5/d6/d64/d6f/da6/fc2 x:0 0 0 2024-03-16T09:01:18.392 INFO:tasks.workunit.client.0.smithi007.stdout:8/985: dread d0/dc/dd/ff3 [0,4194304] 0 2024-03-16T09:01:18.393 INFO:tasks.workunit.client.0.smithi007.stdout:8/986: read d0/f3c [1815400,20884] 0 2024-03-16T09:01:18.393 INFO:tasks.workunit.client.1.smithi019.stdout:2/591: mknod d5/d4e/d74/cc3 0 2024-03-16T09:01:18.400 INFO:tasks.workunit.client.1.smithi019.stdout:5/549: dwrite d6/d1f/d33/d89/f96 [0,4194304] 0 2024-03-16T09:01:18.401 INFO:tasks.workunit.client.0.smithi007.stdout:0/835: dwrite d3/d9/da/d48/d25/d43/d1a/d58/d61/f8f [0,4194304] 0 2024-03-16T09:01:18.402 INFO:tasks.workunit.client.1.smithi019.stdout:4/537: dwrite d4/d6/f83 [0,4194304] 0 2024-03-16T09:01:18.402 INFO:tasks.workunit.client.1.smithi019.stdout:4/538: chown d4/lac 542725937 1 2024-03-16T09:01:18.402 INFO:tasks.workunit.client.1.smithi019.stdout:1/487: dwrite d16/f22 [0,4194304] 0 2024-03-16T09:01:18.402 INFO:tasks.workunit.client.1.smithi019.stdout:1/488: fdatasync d16/f22 0 2024-03-16T09:01:18.402 INFO:tasks.workunit.client.1.smithi019.stdout:2/592: read d5/d2e/d4f/f5e [294290,15740] 0 2024-03-16T09:01:18.402 INFO:tasks.workunit.client.1.smithi019.stdout:2/593: truncate d5/d6/d2b/f72 5041649 0 2024-03-16T09:01:18.403 INFO:tasks.workunit.client.1.smithi019.stdout:5/550: creat d6/dd/d47/d56/faa x:0 0 0 2024-03-16T09:01:18.403 INFO:tasks.workunit.client.1.smithi019.stdout:5/551: chown d6/d19/d34/c8c 1356153136 1 2024-03-16T09:01:18.403 INFO:tasks.workunit.client.1.smithi019.stdout:7/470: dwrite d9/da/d13/d18/d46/f6b [0,4194304] 0 2024-03-16T09:01:18.406 INFO:tasks.workunit.client.1.smithi019.stdout:4/539: stat d4/d64/laf 0 2024-03-16T09:01:18.408 INFO:tasks.workunit.client.1.smithi019.stdout:5/552: write d6/d1f/f3f [369017,20235] 0 2024-03-16T09:01:18.411 INFO:tasks.workunit.client.1.smithi019.stdout:2/594: getdents d5/d6/d2b/d2f/d35/d65 0 2024-03-16T09:01:18.411 INFO:tasks.workunit.client.1.smithi019.stdout:2/595: chown d5/d6/d64 335 1 2024-03-16T09:01:18.413 INFO:tasks.workunit.client.1.smithi019.stdout:4/540: unlink d4/d6/d15/f66 0 2024-03-16T09:01:18.414 INFO:tasks.workunit.client.1.smithi019.stdout:5/553: symlink d6/d1f/d77/d84/lab 0 2024-03-16T09:01:18.416 INFO:tasks.workunit.client.1.smithi019.stdout:4/541: creat d4/d6/d15/d1f/fb4 x:0 0 0 2024-03-16T09:01:18.417 INFO:tasks.workunit.client.1.smithi019.stdout:5/554: symlink d6/dd/lac 0 2024-03-16T09:01:18.417 INFO:tasks.workunit.client.1.smithi019.stdout:5/555: fdatasync d6/dd/d37/d76/f8e 0 2024-03-16T09:01:18.419 INFO:tasks.workunit.client.1.smithi019.stdout:4/542: mknod d4/d1b/d49/cb5 0 2024-03-16T09:01:18.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/543: truncate d4/fa5 325220 0 2024-03-16T09:01:18.430 INFO:tasks.workunit.client.1.smithi019.stdout:5/556: write d6/d19/f9b [515100,235] 0 2024-03-16T09:01:18.430 INFO:tasks.workunit.client.1.smithi019.stdout:5/557: write f4 [2500438,2414] 0 2024-03-16T09:01:18.430 INFO:tasks.workunit.client.1.smithi019.stdout:5/558: write d6/dd/d37/f58 [383504,59922] 0 2024-03-16T09:01:18.431 INFO:tasks.workunit.client.0.smithi007.stdout:0/836: dwrite d3/d6c/dbd/dd7/d7b/ff3 [0,4194304] 0 2024-03-16T09:01:18.432 INFO:tasks.workunit.client.1.smithi019.stdout:7/471: dwrite d9/da/f24 [0,4194304] 0 2024-03-16T09:01:18.432 INFO:tasks.workunit.client.1.smithi019.stdout:1/489: dwrite d16/d1f/d43/f6f [0,4194304] 0 2024-03-16T09:01:18.435 INFO:tasks.workunit.client.1.smithi019.stdout:1/490: mknod d16/d1f/d4f/c86 0 2024-03-16T09:01:18.438 INFO:tasks.workunit.client.1.smithi019.stdout:2/596: dread d5/d2e/f3b [0,4194304] 0 2024-03-16T09:01:18.439 INFO:tasks.workunit.client.1.smithi019.stdout:9/413: sync 2024-03-16T09:01:18.439 INFO:tasks.workunit.client.1.smithi019.stdout:9/414: truncate d1/d7/de/f10 3230486 0 2024-03-16T09:01:18.441 INFO:tasks.workunit.client.0.smithi007.stdout:0/837: dread d3/d9/da/d48/fa9 [0,4194304] 0 2024-03-16T09:01:18.441 INFO:tasks.workunit.client.0.smithi007.stdout:0/838: creat d3/d9/da/d48/d25/d43/d1a/d58/d61/f110 x:0 0 0 2024-03-16T09:01:18.441 INFO:tasks.workunit.client.0.smithi007.stdout:0/839: fsync d3/d9/da/d48/d4b/f52 0 2024-03-16T09:01:18.444 INFO:tasks.workunit.client.1.smithi019.stdout:9/415: mknod d1/d7/c8e 0 2024-03-16T09:01:18.444 INFO:tasks.workunit.client.1.smithi019.stdout:9/416: chown d1/d7/d67 96 1 2024-03-16T09:01:18.450 INFO:tasks.workunit.client.1.smithi019.stdout:9/417: symlink d1/d7/d21/l8f 0 2024-03-16T09:01:18.450 INFO:tasks.workunit.client.1.smithi019.stdout:9/418: creat d1/d7/de/d24/f90 x:0 0 0 2024-03-16T09:01:18.450 INFO:tasks.workunit.client.1.smithi019.stdout:9/419: dread - d1/d7/d21/d27/f55 zero size 2024-03-16T09:01:18.450 INFO:tasks.workunit.client.1.smithi019.stdout:9/420: write d1/d7/de/d24/f42 [4597216,97157] 0 2024-03-16T09:01:18.450 INFO:tasks.workunit.client.1.smithi019.stdout:9/421: creat d1/d7/de/d24/d44/d60/d74/f91 x:0 0 0 2024-03-16T09:01:18.453 INFO:tasks.workunit.client.1.smithi019.stdout:7/472: dwrite d9/db/f2c [0,4194304] 0 2024-03-16T09:01:18.457 INFO:tasks.workunit.client.1.smithi019.stdout:9/422: mkdir d1/d92 0 2024-03-16T09:01:18.458 INFO:tasks.workunit.client.1.smithi019.stdout:7/473: mknod d9/da/d13/c92 0 2024-03-16T09:01:18.462 INFO:tasks.workunit.client.1.smithi019.stdout:9/423: mknod d1/d7/de/d24/d5b/c93 0 2024-03-16T09:01:18.466 INFO:tasks.workunit.client.1.smithi019.stdout:7/474: mkdir d9/d25/d93 0 2024-03-16T09:01:18.466 INFO:tasks.workunit.client.1.smithi019.stdout:9/424: symlink d1/d7/d33/l94 0 2024-03-16T09:01:18.469 INFO:tasks.workunit.client.0.smithi007.stdout:0/840: dwrite d3/d9/da/d18/d27/d34/f89 [0,4194304] 0 2024-03-16T09:01:18.469 INFO:tasks.workunit.client.0.smithi007.stdout:0/841: chown d3/d9/db4/d1c/cad 0 1 2024-03-16T09:01:18.473 INFO:tasks.workunit.client.1.smithi019.stdout:2/597: dwrite d5/d6/d64/d6f/da6/fa1 [0,4194304] 0 2024-03-16T09:01:18.473 INFO:tasks.workunit.client.1.smithi019.stdout:2/598: chown d5/d2e/d4f/f57 11687 1 2024-03-16T09:01:18.479 INFO:tasks.workunit.client.1.smithi019.stdout:1/491: dwrite d16/f20 [0,4194304] 0 2024-03-16T09:01:18.482 INFO:tasks.workunit.client.1.smithi019.stdout:1/492: symlink d16/d18/l87 0 2024-03-16T09:01:18.482 INFO:tasks.workunit.client.1.smithi019.stdout:1/493: stat d16/d18/c66 0 2024-03-16T09:01:18.482 INFO:tasks.workunit.client.1.smithi019.stdout:9/425: dwrite d1/d7/f68 [0,4194304] 0 2024-03-16T09:01:18.484 INFO:tasks.workunit.client.1.smithi019.stdout:1/494: link d16/d1f/d3c/c4b d16/d84/c88 0 2024-03-16T09:01:18.488 INFO:tasks.workunit.client.1.smithi019.stdout:1/495: mkdir d16/d31/d71/d89 0 2024-03-16T09:01:18.488 INFO:tasks.workunit.client.0.smithi007.stdout:0/842: dwrite d3/d9/da/d48/d25/d43/d91/fba [0,4194304] 0 2024-03-16T09:01:18.488 INFO:tasks.workunit.client.0.smithi007.stdout:0/843: creat d3/d9/da/d92/d9f/de8/f111 x:0 0 0 2024-03-16T09:01:18.488 INFO:tasks.workunit.client.0.smithi007.stdout:0/844: read d3/f22 [3570448,2392] 0 2024-03-16T09:01:18.490 INFO:tasks.workunit.client.1.smithi019.stdout:1/496: mkdir d16/d1f/d4f/d8a 0 2024-03-16T09:01:18.492 INFO:tasks.workunit.client.0.smithi007.stdout:0/845: creat d3/d9/db4/d60/f112 x:0 0 0 2024-03-16T09:01:18.492 INFO:tasks.workunit.client.0.smithi007.stdout:0/846: fdatasync d3/d6c/dbd/dd7/f67 0 2024-03-16T09:01:18.493 INFO:tasks.workunit.client.1.smithi019.stdout:7/475: truncate d9/db/f2c 2378730 0 2024-03-16T09:01:18.493 INFO:tasks.workunit.client.1.smithi019.stdout:7/476: dread - d9/da/d13/d18/f5a zero size 2024-03-16T09:01:18.493 INFO:tasks.workunit.client.1.smithi019.stdout:7/477: creat d9/da/d13/f94 x:0 0 0 2024-03-16T09:01:18.493 INFO:tasks.workunit.client.1.smithi019.stdout:7/478: chown d9/da/d13/d18/d1a/c80 47733 1 2024-03-16T09:01:18.493 INFO:tasks.workunit.client.0.smithi007.stdout:0/847: symlink d3/l113 0 2024-03-16T09:01:18.495 INFO:tasks.workunit.client.1.smithi019.stdout:7/479: link d9/d25/d28/d48/l81 d9/d25/d28/d48/l95 0 2024-03-16T09:01:18.495 INFO:tasks.workunit.client.1.smithi019.stdout:7/480: truncate d9/d25/f54 791590 0 2024-03-16T09:01:18.495 INFO:tasks.workunit.client.1.smithi019.stdout:7/481: write d9/db/d12/d5b/d33/f7a [1495180,70681] 0 2024-03-16T09:01:18.496 INFO:tasks.workunit.client.1.smithi019.stdout:7/482: mknod d9/da/d13/d18/d30/d3e/d5d/c96 0 2024-03-16T09:01:18.496 INFO:tasks.workunit.client.1.smithi019.stdout:7/483: chown d9/f74 0 1 2024-03-16T09:01:18.500 INFO:tasks.workunit.client.1.smithi019.stdout:7/484: rmdir d9/da/d13/d70 0 2024-03-16T09:01:18.500 INFO:tasks.workunit.client.1.smithi019.stdout:7/485: creat d9/da/d13/d18/d30/f97 x:0 0 0 2024-03-16T09:01:18.500 INFO:tasks.workunit.client.1.smithi019.stdout:7/486: readlink d9/d25/d28/d48/l95 0 2024-03-16T09:01:18.501 INFO:tasks.workunit.client.1.smithi019.stdout:7/487: chown d9/da/d13/d18/d30/d75/c7d 435062836 1 2024-03-16T09:01:18.501 INFO:tasks.workunit.client.1.smithi019.stdout:7/488: readlink d9/d25/d28/d48/l81 0 2024-03-16T09:01:18.502 INFO:tasks.workunit.client.1.smithi019.stdout:7/489: symlink d9/db/d12/l98 0 2024-03-16T09:01:18.503 INFO:tasks.workunit.client.1.smithi019.stdout:1/497: read d16/d1f/d43/f47 [3546849,38577] 0 2024-03-16T09:01:18.503 INFO:tasks.workunit.client.1.smithi019.stdout:2/599: dwrite d5/d6/d64/d6f/da6/d83/fbd [0,4194304] 0 2024-03-16T09:01:18.503 INFO:tasks.workunit.client.1.smithi019.stdout:2/600: write d5/d2e/f9a [1050363,106192] 0 2024-03-16T09:01:18.503 INFO:tasks.workunit.client.1.smithi019.stdout:7/490: symlink d9/da/d13/d18/d30/d3e/d5d/l99 0 2024-03-16T09:01:18.504 INFO:tasks.workunit.client.1.smithi019.stdout:7/491: stat d9/db/d12 0 2024-03-16T09:01:18.505 INFO:tasks.workunit.client.1.smithi019.stdout:9/426: dwrite d1/d7/d21/f7b [0,4194304] 0 2024-03-16T09:01:18.505 INFO:tasks.workunit.client.1.smithi019.stdout:9/427: dread - d1/d7/de/d24/d44/d60/d74/f80 zero size 2024-03-16T09:01:18.505 INFO:tasks.workunit.client.1.smithi019.stdout:9/428: creat d1/d7/d33/d59/f95 x:0 0 0 2024-03-16T09:01:18.506 INFO:tasks.workunit.client.1.smithi019.stdout:2/601: rmdir d5/d4e 39 2024-03-16T09:01:18.506 INFO:tasks.workunit.client.1.smithi019.stdout:2/602: readlink d5/d6/d2b/d2f/d35/l8e 0 2024-03-16T09:01:18.508 INFO:tasks.workunit.client.1.smithi019.stdout:7/492: rename d9/da/d13/d18/d1a/d6e to d9/da/d13/d9a 0 2024-03-16T09:01:18.514 INFO:tasks.workunit.client.1.smithi019.stdout:2/603: dread d5/d6/d2b/f3c [8388608,4194304] 0 2024-03-16T09:01:18.515 INFO:tasks.workunit.client.1.smithi019.stdout:9/429: truncate d1/d7/de/d24/f2c 139605 0 2024-03-16T09:01:18.515 INFO:tasks.workunit.client.1.smithi019.stdout:9/430: write d1/d7/de/d24/d44/d60/d74/f91 [405509,111103] 0 2024-03-16T09:01:18.516 INFO:tasks.workunit.client.1.smithi019.stdout:7/493: rename d9/db/l72 to d9/da/d13/l9b 0 2024-03-16T09:01:18.517 INFO:tasks.workunit.client.1.smithi019.stdout:2/604: symlink d5/d6/d64/d6f/lc4 0 2024-03-16T09:01:18.517 INFO:tasks.workunit.client.1.smithi019.stdout:2/605: creat d5/fc5 x:0 0 0 2024-03-16T09:01:18.517 INFO:tasks.workunit.client.1.smithi019.stdout:2/606: creat d5/de/da2/fc6 x:0 0 0 2024-03-16T09:01:18.517 INFO:tasks.workunit.client.1.smithi019.stdout:2/607: write d5/d6/f28 [1933560,62222] 0 2024-03-16T09:01:18.518 INFO:tasks.workunit.client.1.smithi019.stdout:9/431: symlink d1/d7/de/l96 0 2024-03-16T09:01:18.518 INFO:tasks.workunit.client.1.smithi019.stdout:9/432: chown d1/d7/de/d24/d44/d76/f5f 4493 1 2024-03-16T09:01:18.518 INFO:tasks.workunit.client.1.smithi019.stdout:7/494: link d9/da/d13/d18/d30/d3e/d5d/f7c d9/d25/d28/f9c 0 2024-03-16T09:01:18.520 INFO:tasks.workunit.client.1.smithi019.stdout:2/608: creat d5/de/dab/fc7 x:0 0 0 2024-03-16T09:01:18.520 INFO:tasks.workunit.client.1.smithi019.stdout:2/609: fsync d5/de/f32 0 2024-03-16T09:01:18.521 INFO:tasks.workunit.client.1.smithi019.stdout:2/610: write d5/d2e/d4f/f57 [709510,100088] 0 2024-03-16T09:01:18.521 INFO:tasks.workunit.client.1.smithi019.stdout:2/611: fsync d5/d6/d64/d6f/fb6 0 2024-03-16T09:01:18.522 INFO:tasks.workunit.client.1.smithi019.stdout:7/495: creat d9/da/d13/d18/d30/d75/f9d x:0 0 0 2024-03-16T09:01:18.522 INFO:tasks.workunit.client.1.smithi019.stdout:2/612: symlink d5/d6/lc8 0 2024-03-16T09:01:18.523 INFO:tasks.workunit.client.1.smithi019.stdout:2/613: write d5/d6/d64/d6f/da6/fc2 [325892,104600] 0 2024-03-16T09:01:18.524 INFO:tasks.workunit.client.1.smithi019.stdout:7/496: unlink d9/da/d13/d18/d30/d75/l82 0 2024-03-16T09:01:18.525 INFO:tasks.workunit.client.1.smithi019.stdout:1/498: dwrite d16/d18/f24 [0,4194304] 0 2024-03-16T09:01:18.526 INFO:tasks.workunit.client.1.smithi019.stdout:7/497: mkdir d9/da/d13/d18/d30/d9e 0 2024-03-16T09:01:18.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/848: dwrite d3/d9/da/d48/d25/d43/d1a/d58/d61/f7a [4194304,4194304] 0 2024-03-16T09:01:18.528 INFO:tasks.workunit.client.0.smithi007.stdout:0/849: rmdir d3/d6c 39 2024-03-16T09:01:18.529 INFO:tasks.workunit.client.0.smithi007.stdout:0/850: mkdir d3/d9/da/d48/d25/d43/d1a/d58/d61/d114 0 2024-03-16T09:01:18.530 INFO:tasks.workunit.client.0.smithi007.stdout:4/607: sync 2024-03-16T09:01:18.530 INFO:tasks.workunit.client.0.smithi007.stdout:4/608: readlink d4/d14/l57 0 2024-03-16T09:01:18.530 INFO:tasks.workunit.client.0.smithi007.stdout:4/609: dread - d4/d14/d1c/d5b/d67/d8f/da8/f70 zero size 2024-03-16T09:01:18.539 INFO:tasks.workunit.client.1.smithi019.stdout:9/433: rename d1/d7/de/d24/f2c to d1/d7/de/d24/d44/d60/f97 0 2024-03-16T09:01:18.541 INFO:tasks.workunit.client.1.smithi019.stdout:2/614: rename d5/d6/d2b/d2f/d35/d65/c7c to d5/d6/d2b/d2f/d35/cc9 0 2024-03-16T09:01:18.543 INFO:tasks.workunit.client.1.smithi019.stdout:7/498: rename d9/db/d12/d5b/d33/f43 to d9/db/d12/d5b/f9f 0 2024-03-16T09:01:18.543 INFO:tasks.workunit.client.1.smithi019.stdout:7/499: creat d9/da/d13/d18/d30/d75/fa0 x:0 0 0 2024-03-16T09:01:18.543 INFO:tasks.workunit.client.1.smithi019.stdout:7/500: fdatasync d9/db/d12/d5b/d33/f7a 0 2024-03-16T09:01:18.544 INFO:tasks.workunit.client.1.smithi019.stdout:7/501: dread - d9/d25/d28/f88 zero size 2024-03-16T09:01:18.545 INFO:tasks.workunit.client.1.smithi019.stdout:2/615: mkdir d5/d6/d2b/d2f/d35/d4b/dca 0 2024-03-16T09:01:18.546 INFO:tasks.workunit.client.1.smithi019.stdout:7/502: mknod d9/da/d13/d9a/ca1 0 2024-03-16T09:01:18.556 INFO:tasks.workunit.client.1.smithi019.stdout:1/499: dwrite d16/f20 [0,4194304] 0 2024-03-16T09:01:18.556 INFO:tasks.workunit.client.1.smithi019.stdout:2/616: write d5/d6/d2b/f18 [1647371,118027] 0 2024-03-16T09:01:18.556 INFO:tasks.workunit.client.1.smithi019.stdout:2/617: write d5/d6/d2b/d2f/d35/f56 [388685,17228] 0 2024-03-16T09:01:18.557 INFO:tasks.workunit.client.1.smithi019.stdout:2/618: mknod d5/d6/d64/ccb 0 2024-03-16T09:01:18.560 INFO:tasks.workunit.client.1.smithi019.stdout:2/619: getdents d5/d6/d2b/d2f/d35/d65 0 2024-03-16T09:01:18.560 INFO:tasks.workunit.client.1.smithi019.stdout:2/620: write d5/de/da2/fc1 [651180,78988] 0 2024-03-16T09:01:18.560 INFO:tasks.workunit.client.1.smithi019.stdout:2/621: write d5/d6/d2b/f98 [43860,21507] 0 2024-03-16T09:01:18.560 INFO:tasks.workunit.client.1.smithi019.stdout:2/622: creat d5/d6/fcc x:0 0 0 2024-03-16T09:01:18.561 INFO:tasks.workunit.client.1.smithi019.stdout:2/623: getdents d5/d6/d2b/d2f/d35/d65 0 2024-03-16T09:01:18.561 INFO:tasks.workunit.client.0.smithi007.stdout:4/610: dwrite d4/d14/d64/d68/f6f [0,4194304] 0 2024-03-16T09:01:18.562 INFO:tasks.workunit.client.1.smithi019.stdout:2/624: mknod d5/de/da2/ccd 0 2024-03-16T09:01:18.563 INFO:tasks.workunit.client.1.smithi019.stdout:2/625: mknod d5/cce 0 2024-03-16T09:01:18.564 INFO:tasks.workunit.client.1.smithi019.stdout:2/626: rmdir d5/d6/d64/d6f/da6/d83 39 2024-03-16T09:01:18.565 INFO:tasks.workunit.client.1.smithi019.stdout:3/554: sync 2024-03-16T09:01:18.565 INFO:tasks.workunit.client.0.smithi007.stdout:4/611: unlink d4/d14/d1c/d5b/d67/f81 0 2024-03-16T09:01:18.566 INFO:tasks.workunit.client.1.smithi019.stdout:2/627: rename d5/d4e/d74/f90 to d5/d6/d2b/d89/fcf 0 2024-03-16T09:01:18.566 INFO:tasks.workunit.client.1.smithi019.stdout:2/628: write d5/de/da2/faa [87469,104500] 0 2024-03-16T09:01:18.566 INFO:tasks.workunit.client.1.smithi019.stdout:7/503: dwrite d9/d25/d28/f69 [0,4194304] 0 2024-03-16T09:01:18.567 INFO:tasks.workunit.client.1.smithi019.stdout:2/629: link d5/d6/d2b/l20 d5/d6/d2b/ld0 0 2024-03-16T09:01:18.567 INFO:tasks.workunit.client.1.smithi019.stdout:9/434: dwrite d1/d7/f81 [0,4194304] 0 2024-03-16T09:01:18.569 INFO:tasks.workunit.client.0.smithi007.stdout:4/612: mknod d4/d14/d1c/d5b/d67/d8f/da8/d78/cd1 0 2024-03-16T09:01:18.569 INFO:tasks.workunit.client.1.smithi019.stdout:1/500: truncate d16/d1f/d4f/f5c 2755050 0 2024-03-16T09:01:18.571 INFO:tasks.workunit.client.0.smithi007.stdout:4/613: mkdir d4/d14/d1c/d5b/d67/d8f/da8/d9d/dbb/dd2 0 2024-03-16T09:01:18.573 INFO:tasks.workunit.client.0.smithi007.stdout:4/614: mkdir d4/d14/d1c/d5b/d67/dd3 0 2024-03-16T09:01:18.574 INFO:tasks.workunit.client.1.smithi019.stdout:2/630: creat d5/d6/d2b/d89/fd1 x:0 0 0 2024-03-16T09:01:18.576 INFO:tasks.workunit.client.1.smithi019.stdout:9/435: mkdir d1/d98 0 2024-03-16T09:01:18.576 INFO:tasks.workunit.client.1.smithi019.stdout:9/436: creat d1/d7/de/d24/f99 x:0 0 0 2024-03-16T09:01:18.576 INFO:tasks.workunit.client.0.smithi007.stdout:4/615: mknod d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/cd4 0 2024-03-16T09:01:18.579 INFO:tasks.workunit.client.1.smithi019.stdout:9/437: creat d1/d7/de/d24/d44/d76/f9a x:0 0 0 2024-03-16T09:01:18.584 INFO:tasks.workunit.client.0.smithi007.stdout:3/721: sync 2024-03-16T09:01:18.586 INFO:tasks.workunit.client.1.smithi019.stdout:9/438: mknod d1/d7/d21/c9b 0 2024-03-16T09:01:18.587 INFO:tasks.workunit.client.0.smithi007.stdout:4/616: read d4/d14/f35 [2792970,45447] 0 2024-03-16T09:01:18.588 INFO:tasks.workunit.client.1.smithi019.stdout:6/669: sync 2024-03-16T09:01:18.588 INFO:tasks.workunit.client.1.smithi019.stdout:3/555: dwrite d3/d17/d6a/f7e [0,4194304] 0 2024-03-16T09:01:18.590 INFO:tasks.workunit.client.1.smithi019.stdout:6/670: rename de/d26/d32/d97/fd3 to de/d26/d32/d97/d66/d72/d7f/dcb/fd9 0 2024-03-16T09:01:18.590 INFO:tasks.workunit.client.1.smithi019.stdout:9/439: dread d1/d7/d21/d27/f35 [0,4194304] 0 2024-03-16T09:01:18.591 INFO:tasks.workunit.client.1.smithi019.stdout:2/631: dread d5/d6/d2b/d2f/f46 [0,4194304] 0 2024-03-16T09:01:18.591 INFO:tasks.workunit.client.1.smithi019.stdout:2/632: chown d5/d5a 829564 1 2024-03-16T09:01:18.591 INFO:tasks.workunit.client.1.smithi019.stdout:2/633: chown d5/d5a/d96/dba 0 1 2024-03-16T09:01:18.592 INFO:tasks.workunit.client.1.smithi019.stdout:2/634: read d5/d6/d2b/f72 [4740878,21857] 0 2024-03-16T09:01:18.592 INFO:tasks.workunit.client.1.smithi019.stdout:2/635: getdents d5/d6/d2b/d2f/d35/d65 0 2024-03-16T09:01:18.592 INFO:tasks.workunit.client.1.smithi019.stdout:2/636: write d5/d6/d2b/d89/fd1 [461290,35062] 0 2024-03-16T09:01:18.592 INFO:tasks.workunit.client.1.smithi019.stdout:2/637: fdatasync d5/d6/d64/d6f/da6/fa1 0 2024-03-16T09:01:18.593 INFO:tasks.workunit.client.1.smithi019.stdout:9/440: mkdir d1/d7/de/d24/d44/d76/d9c 0 2024-03-16T09:01:18.594 INFO:tasks.workunit.client.0.smithi007.stdout:3/722: dread d15/d18/d1f/f54 [0,4194304] 0 2024-03-16T09:01:18.595 INFO:tasks.workunit.client.1.smithi019.stdout:9/441: rename d1/d7/de/d24/f40 to d1/d7/d33/d59/f9d 0 2024-03-16T09:01:18.595 INFO:tasks.workunit.client.0.smithi007.stdout:3/723: symlink d15/d18/d1f/d56/d98/le3 0 2024-03-16T09:01:18.598 INFO:tasks.workunit.client.0.smithi007.stdout:3/724: symlink d15/d18/d1f/d56/d98/le4 0 2024-03-16T09:01:18.601 INFO:tasks.workunit.client.1.smithi019.stdout:3/556: dread d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:18.601 INFO:tasks.workunit.client.0.smithi007.stdout:3/725: write d15/d18/d1f/f2e [4266689,93095] 0 2024-03-16T09:01:18.604 INFO:tasks.workunit.client.1.smithi019.stdout:7/504: dwrite d9/f3f [0,4194304] 0 2024-03-16T09:01:18.605 INFO:tasks.workunit.client.1.smithi019.stdout:6/671: dread de/d26/f39 [0,4194304] 0 2024-03-16T09:01:18.609 INFO:tasks.workunit.client.1.smithi019.stdout:7/505: symlink d9/db/d12/d5b/d33/la2 0 2024-03-16T09:01:18.609 INFO:tasks.workunit.client.1.smithi019.stdout:7/506: write d9/da/d13/d9a/f77 [1464175,20480] 0 2024-03-16T09:01:18.610 INFO:tasks.workunit.client.0.smithi007.stdout:3/726: dread fa [0,4194304] 0 2024-03-16T09:01:18.610 INFO:tasks.workunit.client.1.smithi019.stdout:3/557: rename d3/d17 to d3/d8b/dae 0 2024-03-16T09:01:18.612 INFO:tasks.workunit.client.1.smithi019.stdout:3/558: creat d3/d8b/dae/d3f/faf x:0 0 0 2024-03-16T09:01:18.614 INFO:tasks.workunit.client.1.smithi019.stdout:3/559: creat d3/d1b/d88/d91/fb0 x:0 0 0 2024-03-16T09:01:18.616 INFO:tasks.workunit.client.1.smithi019.stdout:9/442: dwrite d1/d7/d21/d27/f57 [0,4194304] 0 2024-03-16T09:01:18.616 INFO:tasks.workunit.client.1.smithi019.stdout:9/443: chown d1/d7/de/l1b 420922311 1 2024-03-16T09:01:18.617 INFO:tasks.workunit.client.1.smithi019.stdout:9/444: creat d1/d7/de/d24/d44/d6b/f9e x:0 0 0 2024-03-16T09:01:18.617 INFO:tasks.workunit.client.1.smithi019.stdout:6/672: dread de/d1c/f20 [0,4194304] 0 2024-03-16T09:01:18.618 INFO:tasks.workunit.client.1.smithi019.stdout:6/673: mknod de/d26/cda 0 2024-03-16T09:01:18.619 INFO:tasks.workunit.client.1.smithi019.stdout:6/674: chown de/d26/d32/d4e/d75/l77 3148 1 2024-03-16T09:01:18.619 INFO:tasks.workunit.client.1.smithi019.stdout:9/445: getdents d1/d7/d21/d27 0 2024-03-16T09:01:18.619 INFO:tasks.workunit.client.1.smithi019.stdout:9/446: creat d1/d7/de/d24/d44/d60/f9f x:0 0 0 2024-03-16T09:01:18.624 INFO:tasks.workunit.client.1.smithi019.stdout:7/507: write f4 [2615398,49970] 0 2024-03-16T09:01:18.627 INFO:tasks.workunit.client.1.smithi019.stdout:3/560: dread d3/d11/f49 [0,4194304] 0 2024-03-16T09:01:18.628 INFO:tasks.workunit.client.1.smithi019.stdout:3/561: write d3/d8b/dae/d5a/d75/d7a/fa0 [643254,11460] 0 2024-03-16T09:01:18.628 INFO:tasks.workunit.client.1.smithi019.stdout:3/562: dread - d3/d8b/fa6 zero size 2024-03-16T09:01:18.629 INFO:tasks.workunit.client.1.smithi019.stdout:1/501: dwrite d16/f2b [0,4194304] 0 2024-03-16T09:01:18.629 INFO:tasks.workunit.client.1.smithi019.stdout:1/502: truncate d16/d1f/d4f/f60 404746 0 2024-03-16T09:01:18.629 INFO:tasks.workunit.client.1.smithi019.stdout:1/503: truncate d16/d18/f1e 502031 0 2024-03-16T09:01:18.629 INFO:tasks.workunit.client.1.smithi019.stdout:1/504: fdatasync d16/f65 0 2024-03-16T09:01:18.629 INFO:tasks.workunit.client.1.smithi019.stdout:3/563: symlink d3/d9b/lb1 0 2024-03-16T09:01:18.631 INFO:tasks.workunit.client.1.smithi019.stdout:1/505: write f9 [4551248,105836] 0 2024-03-16T09:01:18.634 INFO:tasks.workunit.client.1.smithi019.stdout:3/564: mknod d3/d89/cb2 0 2024-03-16T09:01:18.634 INFO:tasks.workunit.client.1.smithi019.stdout:6/675: write de/f15 [3708212,1199] 0 2024-03-16T09:01:18.634 INFO:tasks.workunit.client.1.smithi019.stdout:6/676: fsync de/d26/fc6 0 2024-03-16T09:01:18.635 INFO:tasks.workunit.client.1.smithi019.stdout:6/677: creat de/d26/d32/d97/dae/fdb x:0 0 0 2024-03-16T09:01:18.637 INFO:tasks.workunit.client.1.smithi019.stdout:3/565: dread d3/d11/f6f [0,4194304] 0 2024-03-16T09:01:18.647 INFO:tasks.workunit.client.1.smithi019.stdout:1/506: dread d16/d1f/d43/f6f [0,4194304] 0 2024-03-16T09:01:18.648 INFO:tasks.workunit.client.1.smithi019.stdout:1/507: rmdir d16/d1f/d3c 39 2024-03-16T09:01:18.651 INFO:tasks.workunit.client.1.smithi019.stdout:6/678: write de/d26/d32/d4e/f9b [1420325,58311] 0 2024-03-16T09:01:18.652 INFO:tasks.workunit.client.1.smithi019.stdout:2/638: dwrite d5/d2e/f50 [0,4194304] 0 2024-03-16T09:01:18.652 INFO:tasks.workunit.client.1.smithi019.stdout:6/679: mkdir de/d26/d32/d97/d66/ddc 0 2024-03-16T09:01:18.654 INFO:tasks.workunit.client.1.smithi019.stdout:2/639: unlink d5/d6/d2b/f6d 0 2024-03-16T09:01:18.654 INFO:tasks.workunit.client.1.smithi019.stdout:2/640: fdatasync d5/d5a/f9d 0 2024-03-16T09:01:18.656 INFO:tasks.workunit.client.1.smithi019.stdout:2/641: unlink d5/d6/d2b/d2f/d35/f58 0 2024-03-16T09:01:18.658 INFO:tasks.workunit.client.1.smithi019.stdout:2/642: mknod d5/d5a/d96/cd2 0 2024-03-16T09:01:18.659 INFO:tasks.workunit.client.1.smithi019.stdout:2/643: getdents d5/d6/d64/d6f 0 2024-03-16T09:01:18.659 INFO:tasks.workunit.client.1.smithi019.stdout:2/644: fsync d5/d6/d2b/f3c 0 2024-03-16T09:01:18.663 INFO:tasks.workunit.client.1.smithi019.stdout:1/508: dwrite d16/f37 [0,4194304] 0 2024-03-16T09:01:18.663 INFO:tasks.workunit.client.1.smithi019.stdout:1/509: fdatasync d16/d1f/f74 0 2024-03-16T09:01:18.663 INFO:tasks.workunit.client.1.smithi019.stdout:1/510: dread - d16/d31/d71/f75 zero size 2024-03-16T09:01:18.663 INFO:tasks.workunit.client.1.smithi019.stdout:1/511: stat d16/d1f/d43/f4c 0 2024-03-16T09:01:18.663 INFO:tasks.workunit.client.1.smithi019.stdout:6/680: dread de/d26/d32/d4e/d75/fa2 [0,4194304] 0 2024-03-16T09:01:18.665 INFO:tasks.workunit.client.1.smithi019.stdout:6/681: mknod de/d26/d32/d97/d2c/cdd 0 2024-03-16T09:01:18.665 INFO:tasks.workunit.client.1.smithi019.stdout:6/682: dread - de/d26/d32/f62 zero size 2024-03-16T09:01:18.665 INFO:tasks.workunit.client.1.smithi019.stdout:6/683: truncate de/d26/d32/d97/d51/fba 956561 0 2024-03-16T09:01:18.670 INFO:tasks.workunit.client.0.smithi007.stdout:4/617: dwrite d4/d14/d1c/f6a [0,4194304] 0 2024-03-16T09:01:18.670 INFO:tasks.workunit.client.0.smithi007.stdout:3/727: dwrite d15/d18/d1f/d3b/f57 [4194304,4194304] 0 2024-03-16T09:01:18.670 INFO:tasks.workunit.client.1.smithi019.stdout:7/508: dwrite d9/db/d12/d5b/f9f [0,4194304] 0 2024-03-16T09:01:18.671 INFO:tasks.workunit.client.1.smithi019.stdout:3/566: dwrite d3/d1b/f8e [0,4194304] 0 2024-03-16T09:01:18.671 INFO:tasks.workunit.client.1.smithi019.stdout:3/567: mknod d3/d8b/da8/cb3 0 2024-03-16T09:01:18.671 INFO:tasks.workunit.client.1.smithi019.stdout:3/568: chown d3/d1b 44348 1 2024-03-16T09:01:18.671 INFO:tasks.workunit.client.1.smithi019.stdout:3/569: truncate d3/d1b/f1c 77463 0 2024-03-16T09:01:18.675 INFO:tasks.workunit.client.0.smithi007.stdout:3/728: creat d15/d18/d1f/d29/d3e/dda/fe5 x:0 0 0 2024-03-16T09:01:18.676 INFO:tasks.workunit.client.0.smithi007.stdout:3/729: getdents d15/d18/d6a 0 2024-03-16T09:01:18.676 INFO:tasks.workunit.client.0.smithi007.stdout:3/730: write d15/d18/d1f/d29/d7c/fb0 [430756,33128] 0 2024-03-16T09:01:18.678 INFO:tasks.workunit.client.1.smithi019.stdout:7/509: dread d9/f23 [0,4194304] 0 2024-03-16T09:01:18.679 INFO:tasks.workunit.client.0.smithi007.stdout:3/731: unlink d15/d18/d1f/d3b/d87/fdd 0 2024-03-16T09:01:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:3/732: creat d15/d18/d1f/d29/d43/fe6 x:0 0 0 2024-03-16T09:01:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:9/689: sync 2024-03-16T09:01:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:8/987: sync 2024-03-16T09:01:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:6/635: sync 2024-03-16T09:01:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:6/636: chown d2/d13/d3f/d50/d78/dbe/dcb/c43 0 1 2024-03-16T09:01:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:5/726: sync 2024-03-16T09:01:18.682 INFO:tasks.workunit.client.0.smithi007.stdout:2/804: sync 2024-03-16T09:01:18.683 INFO:tasks.workunit.client.0.smithi007.stdout:1/620: sync 2024-03-16T09:01:18.683 INFO:tasks.workunit.client.0.smithi007.stdout:0/851: sync 2024-03-16T09:01:18.683 INFO:tasks.workunit.client.0.smithi007.stdout:7/704: sync 2024-03-16T09:01:18.683 INFO:tasks.workunit.client.0.smithi007.stdout:1/621: write d9/dd/d1b/f4d [659300,62008] 0 2024-03-16T09:01:18.683 INFO:tasks.workunit.client.0.smithi007.stdout:1/622: write d9/dd/d1b/d28/d29/f2a [7551919,55873] 0 2024-03-16T09:01:18.685 INFO:tasks.workunit.client.0.smithi007.stdout:9/690: truncate ff 3085044 0 2024-03-16T09:01:18.685 INFO:tasks.workunit.client.0.smithi007.stdout:9/691: creat d24/d4c/d86/daa/d7f/d8f/db5/fc9 x:0 0 0 2024-03-16T09:01:18.685 INFO:tasks.workunit.client.0.smithi007.stdout:9/692: dread - d24/d4a/d96/fa8 zero size 2024-03-16T09:01:18.685 INFO:tasks.workunit.client.0.smithi007.stdout:9/693: read d24/d4c/d86/daa/d7f/d80/d87/fc5 [127871,68398] 0 2024-03-16T09:01:18.685 INFO:tasks.workunit.client.0.smithi007.stdout:9/694: write d24/d4a/d96/db9/fa3 [426270,69372] 0 2024-03-16T09:01:18.686 INFO:tasks.workunit.client.0.smithi007.stdout:8/988: mkdir d0/dc/dd/d8b/d15b 0 2024-03-16T09:01:18.686 INFO:tasks.workunit.client.1.smithi019.stdout:7/510: getdents d9/da/d13/d18/d30/d75 0 2024-03-16T09:01:18.686 INFO:tasks.workunit.client.1.smithi019.stdout:7/511: fdatasync d9/f74 0 2024-03-16T09:01:18.687 INFO:tasks.workunit.client.1.smithi019.stdout:7/512: truncate d9/da/d13/d18/f5a 46660 0 2024-03-16T09:01:18.687 INFO:tasks.workunit.client.1.smithi019.stdout:7/513: write d9/d25/d28/f69 [4963863,36292] 0 2024-03-16T09:01:18.687 INFO:tasks.workunit.client.1.smithi019.stdout:0/566: sync 2024-03-16T09:01:18.687 INFO:tasks.workunit.client.1.smithi019.stdout:4/544: sync 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.0.smithi007.stdout:6/637: truncate d2/d13/d24/f6a 5583987 0 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.0.smithi007.stdout:6/638: write d2/d13/d62/f6b [176263,90462] 0 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.0.smithi007.stdout:6/639: stat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/lb7 0 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.1.smithi019.stdout:8/570: sync 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.1.smithi019.stdout:8/571: write d6/d2a/d45/d5d/fb3 [495182,74447] 0 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.1.smithi019.stdout:8/572: write d6/d2a/d3c/dab/faa [400505,24769] 0 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.1.smithi019.stdout:8/573: chown d6/d2a/d45/c22 776377263 1 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.1.smithi019.stdout:5/559: sync 2024-03-16T09:01:18.688 INFO:tasks.workunit.client.1.smithi019.stdout:5/560: truncate f4 3221097 0 2024-03-16T09:01:18.689 INFO:tasks.workunit.client.1.smithi019.stdout:5/561: chown d6/dd/d37/d76 629 1 2024-03-16T09:01:18.689 INFO:tasks.workunit.client.1.smithi019.stdout:7/514: creat d9/da/d13/d18/fa3 x:0 0 0 2024-03-16T09:01:18.689 INFO:tasks.workunit.client.1.smithi019.stdout:8/574: dread d6/f77 [0,4194304] 0 2024-03-16T09:01:18.689 INFO:tasks.workunit.client.1.smithi019.stdout:8/575: write d6/d2a/d45/f42 [783750,128509] 0 2024-03-16T09:01:18.689 INFO:tasks.workunit.client.0.smithi007.stdout:5/727: unlink d1/d7/d1c/d19/d34/d5e/d6f/da9/fd1 0 2024-03-16T09:01:18.691 INFO:tasks.workunit.client.1.smithi019.stdout:0/567: unlink d7/d1d/d28/l1c 0 2024-03-16T09:01:18.691 INFO:tasks.workunit.client.1.smithi019.stdout:4/545: mknod d4/d64/d9b/cb6 0 2024-03-16T09:01:18.691 INFO:tasks.workunit.client.1.smithi019.stdout:4/546: creat d4/d64/d9b/fb7 x:0 0 0 2024-03-16T09:01:18.692 INFO:tasks.workunit.client.1.smithi019.stdout:7/515: mkdir d9/da/d13/d18/d30/d75/da4 0 2024-03-16T09:01:18.693 INFO:tasks.workunit.client.1.smithi019.stdout:1/512: dwrite d16/d1f/d4f/f58 [0,4194304] 0 2024-03-16T09:01:18.693 INFO:tasks.workunit.client.0.smithi007.stdout:2/805: mkdir dc/df/d24/d28/d2e/d4a/d108 0 2024-03-16T09:01:18.694 INFO:tasks.workunit.client.0.smithi007.stdout:0/852: mknod d3/d6c/c115 0 2024-03-16T09:01:18.694 INFO:tasks.workunit.client.0.smithi007.stdout:0/853: truncate d3/d9/da/d18/d27/fbc 981788 0 2024-03-16T09:01:18.696 INFO:tasks.workunit.client.0.smithi007.stdout:7/705: mknod d0/d19/d1f/d5d/d9d/df7/d3a/daf/dc2/cff 0 2024-03-16T09:01:18.696 INFO:tasks.workunit.client.0.smithi007.stdout:7/706: write d0/dc/d38/d4e/f68 [507411,38000] 0 2024-03-16T09:01:18.696 INFO:tasks.workunit.client.1.smithi019.stdout:7/516: mkdir d9/da/d13/d18/d30/d3e/da5 0 2024-03-16T09:01:18.696 INFO:tasks.workunit.client.1.smithi019.stdout:7/517: stat d9/db/d12/l98 0 2024-03-16T09:01:18.697 INFO:tasks.workunit.client.1.smithi019.stdout:7/518: truncate d9/db/d12/d5b/f65 134962 0 2024-03-16T09:01:18.697 INFO:tasks.workunit.client.1.smithi019.stdout:7/519: creat d9/d25/d28/fa6 x:0 0 0 2024-03-16T09:01:18.697 INFO:tasks.workunit.client.1.smithi019.stdout:7/520: truncate d9/da/d13/d18/d30/d75/fa0 92307 0 2024-03-16T09:01:18.697 INFO:tasks.workunit.client.1.smithi019.stdout:7/521: chown d9/da/d13/f17 890 1 2024-03-16T09:01:18.698 INFO:tasks.workunit.client.0.smithi007.stdout:1/623: link d9/dd/d1b/d58/lbf d9/dd/d1b/d2d/d30/d45/lc6 0 2024-03-16T09:01:18.698 INFO:tasks.workunit.client.0.smithi007.stdout:3/733: dread d15/d18/d1f/f58 [0,4194304] 0 2024-03-16T09:01:18.698 INFO:tasks.workunit.client.0.smithi007.stdout:3/734: write d15/d18/d1f/d29/d3e/d4c/f64 [3705271,50026] 0 2024-03-16T09:01:18.700 INFO:tasks.workunit.client.1.smithi019.stdout:1/513: mknod d16/d1f/d4f/c8b 0 2024-03-16T09:01:18.700 INFO:tasks.workunit.client.1.smithi019.stdout:1/514: fdatasync d16/f5d 0 2024-03-16T09:01:18.702 INFO:tasks.workunit.client.1.smithi019.stdout:4/547: rename d4/d6/d15/d1f/fb4 to d4/d6/d15/d1f/fb8 0 2024-03-16T09:01:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:8/989: mknod d0/d26/d2d/d43/d4c/deb/d128/c15c 0 2024-03-16T09:01:18.705 INFO:tasks.workunit.client.1.smithi019.stdout:1/515: symlink d16/d1f/d3c/l8c 0 2024-03-16T09:01:18.705 INFO:tasks.workunit.client.1.smithi019.stdout:1/516: stat d16/d31/f72 0 2024-03-16T09:01:18.705 INFO:tasks.workunit.client.1.smithi019.stdout:1/517: creat d16/d31/d71/f8d x:0 0 0 2024-03-16T09:01:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:5/728: link d1/c68 d1/d7/d1c/d19/d34/de/c107 0 2024-03-16T09:01:18.707 INFO:tasks.workunit.client.1.smithi019.stdout:4/548: link d4/d1b/f8f d4/d64/d99/fb9 0 2024-03-16T09:01:18.707 INFO:tasks.workunit.client.1.smithi019.stdout:4/549: readlink d4/d6c/l77 0 2024-03-16T09:01:18.707 INFO:tasks.workunit.client.1.smithi019.stdout:4/550: stat d4/d1b/d49/d58/d80/d8e 0 2024-03-16T09:01:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:7/707: mknod d0/d19/d1f/d5d/d9d/df7/d73/ded/c100 0 2024-03-16T09:01:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:7/708: chown d0/dc/d38/d4e 809756631 1 2024-03-16T09:01:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:7/709: stat d0/d19/d1f/d5d/d9d/df7/d3a/daf/fdb 0 2024-03-16T09:01:18.709 INFO:tasks.workunit.client.1.smithi019.stdout:1/518: creat d16/d1f/d4f/d68/f8e x:0 0 0 2024-03-16T09:01:18.711 INFO:tasks.workunit.client.0.smithi007.stdout:8/990: truncate d0/dc/f23 6873712 0 2024-03-16T09:01:18.711 INFO:tasks.workunit.client.0.smithi007.stdout:2/806: rmdir dc/df/d24/d28/d2e/dcc 0 2024-03-16T09:01:18.712 INFO:tasks.workunit.client.1.smithi019.stdout:1/519: rename d16/d1f/d4f/d68/d78/d7c to d16/d1f/d4f/d68/d8f 0 2024-03-16T09:01:18.715 INFO:tasks.workunit.client.1.smithi019.stdout:8/576: dread f2 [0,4194304] 0 2024-03-16T09:01:18.715 INFO:tasks.workunit.client.1.smithi019.stdout:8/577: write d6/d2a/d45/d57/fbb [864809,21683] 0 2024-03-16T09:01:18.716 INFO:tasks.workunit.client.0.smithi007.stdout:0/854: getdents d3/d6c/dbd/dd7/d7b 0 2024-03-16T09:01:18.717 INFO:tasks.workunit.client.1.smithi019.stdout:5/562: read d6/dd/f67 [1043805,90694] 0 2024-03-16T09:01:18.717 INFO:tasks.workunit.client.0.smithi007.stdout:2/807: dread dc/df/d6c/fa9 [0,4194304] 0 2024-03-16T09:01:18.720 INFO:tasks.workunit.client.1.smithi019.stdout:5/563: rename d6/c13 to d6/dd/d47/d56/cad 0 2024-03-16T09:01:18.721 INFO:tasks.workunit.client.1.smithi019.stdout:4/551: write d4/d1b/f7e [108350,70972] 0 2024-03-16T09:01:18.723 INFO:tasks.workunit.client.1.smithi019.stdout:0/568: write d7/d1d/d28/d6c/f77 [1579211,75158] 0 2024-03-16T09:01:18.723 INFO:tasks.workunit.client.1.smithi019.stdout:0/569: chown d7/d8/d32/d66/d6d/d8c/db8/d82/c5e 221756 1 2024-03-16T09:01:18.723 INFO:tasks.workunit.client.0.smithi007.stdout:7/710: symlink d0/d19/d1f/d76/l101 0 2024-03-16T09:01:18.724 INFO:tasks.workunit.client.0.smithi007.stdout:3/735: dread d15/d18/d1f/d29/d43/f81 [0,4194304] 0 2024-03-16T09:01:18.725 INFO:tasks.workunit.client.1.smithi019.stdout:4/552: unlink d4/l11 0 2024-03-16T09:01:18.725 INFO:tasks.workunit.client.1.smithi019.stdout:1/520: write d16/d1f/f2e [3522264,131014] 0 2024-03-16T09:01:18.725 INFO:tasks.workunit.client.1.smithi019.stdout:1/521: creat d16/d18/f90 x:0 0 0 2024-03-16T09:01:18.725 INFO:tasks.workunit.client.1.smithi019.stdout:1/522: chown d16/d1f/d73 359827974 1 2024-03-16T09:01:18.726 INFO:tasks.workunit.client.0.smithi007.stdout:0/855: mknod d3/d9/da/d48/d25/d43/d1a/c116 0 2024-03-16T09:01:18.726 INFO:tasks.workunit.client.0.smithi007.stdout:0/856: creat d3/d9/da/d48/d9e/f117 x:0 0 0 2024-03-16T09:01:18.726 INFO:tasks.workunit.client.0.smithi007.stdout:0/857: readlink d3/d9/da/d48/d25/d43/d1a/leb 0 2024-03-16T09:01:18.727 INFO:tasks.workunit.client.0.smithi007.stdout:0/858: truncate d3/d9/da/d48/d4b/f52 1255991 0 2024-03-16T09:01:18.727 INFO:tasks.workunit.client.1.smithi019.stdout:0/570: truncate d7/d1d/f76 799789 0 2024-03-16T09:01:18.727 INFO:tasks.workunit.client.1.smithi019.stdout:0/571: creat d7/d1d/d2e/fc5 x:0 0 0 2024-03-16T09:01:18.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/808: symlink dc/df/d24/d28/d4c/dbc/l109 0 2024-03-16T09:01:18.729 INFO:tasks.workunit.client.0.smithi007.stdout:7/711: creat d0/d19/d1f/d69/de1/de9/f102 x:0 0 0 2024-03-16T09:01:18.729 INFO:tasks.workunit.client.0.smithi007.stdout:2/809: dread dc/df/d24/fd9 [0,4194304] 0 2024-03-16T09:01:18.730 INFO:tasks.workunit.client.1.smithi019.stdout:1/523: chown d16/d31/d76/l80 14083 1 2024-03-16T09:01:18.731 INFO:tasks.workunit.client.0.smithi007.stdout:3/736: dread d15/d18/d1f/f2e [0,4194304] 0 2024-03-16T09:01:18.733 INFO:tasks.workunit.client.1.smithi019.stdout:0/572: mkdir d7/d1d/d28/d92/dc6 0 2024-03-16T09:01:18.734 INFO:tasks.workunit.client.0.smithi007.stdout:7/712: creat d0/d19/d1f/d5d/d9d/df8/f103 x:0 0 0 2024-03-16T09:01:18.734 INFO:tasks.workunit.client.0.smithi007.stdout:7/713: write d0/d19/d1f/d69/dc7/fd8 [37528,53844] 0 2024-03-16T09:01:18.735 INFO:tasks.workunit.client.1.smithi019.stdout:4/553: unlink d4/d6/d15/d1f/c29 0 2024-03-16T09:01:18.735 INFO:tasks.workunit.client.1.smithi019.stdout:4/554: fdatasync d4/d6/d15/d1f/fb8 0 2024-03-16T09:01:18.735 INFO:tasks.workunit.client.1.smithi019.stdout:4/555: fsync d4/f12 0 2024-03-16T09:01:18.735 INFO:tasks.workunit.client.0.smithi007.stdout:2/810: mknod dc/df/d24/d28/d2e/d4a/d108/c10a 0 2024-03-16T09:01:18.736 INFO:tasks.workunit.client.1.smithi019.stdout:0/573: unlink d7/d8/d32/d66/d6d/d8c/db8/d82/c54 0 2024-03-16T09:01:18.737 INFO:tasks.workunit.client.0.smithi007.stdout:2/811: creat dc/df/d24/d28/d4c/dbc/dfe/f10b x:0 0 0 2024-03-16T09:01:18.737 INFO:tasks.workunit.client.0.smithi007.stdout:2/812: creat dc/db7/f10c x:0 0 0 2024-03-16T09:01:18.737 INFO:tasks.workunit.client.0.smithi007.stdout:2/813: chown dc/df/d24/d28/d4c/d5e/c70 579 1 2024-03-16T09:01:18.738 INFO:tasks.workunit.client.1.smithi019.stdout:4/556: creat d4/d1b/d49/d58/d80/d91/fba x:0 0 0 2024-03-16T09:01:18.738 INFO:tasks.workunit.client.1.smithi019.stdout:4/557: truncate d4/d1b/d49/d6e/fb2 672388 0 2024-03-16T09:01:18.739 INFO:tasks.workunit.client.1.smithi019.stdout:0/574: rename d7/d1d/d63/d89/d8a/fa8 to d7/d1d/d63/d89/d9e/fc7 0 2024-03-16T09:01:18.741 INFO:tasks.workunit.client.0.smithi007.stdout:2/814: rename dc/df/d24/d28/d2e/d35/dfb to dc/df/d24/d28/dea/d10d 0 2024-03-16T09:01:18.741 INFO:tasks.workunit.client.0.smithi007.stdout:2/815: fsync dc/fab 0 2024-03-16T09:01:18.741 INFO:tasks.workunit.client.1.smithi019.stdout:0/575: rename d7/d1d/d28/f64 to d7/d8/d32/d66/d6d/fc8 0 2024-03-16T09:01:18.741 INFO:tasks.workunit.client.1.smithi019.stdout:0/576: creat d7/d1d/d63/d6e/fc9 x:0 0 0 2024-03-16T09:01:18.743 INFO:tasks.workunit.client.1.smithi019.stdout:0/577: getdents d7/d1d/d28/da1 0 2024-03-16T09:01:18.745 INFO:tasks.workunit.client.1.smithi019.stdout:4/558: dread d4/d1b/d49/d6e/fb2 [0,4194304] 0 2024-03-16T09:01:18.747 INFO:tasks.workunit.client.1.smithi019.stdout:6/684: dwrite de/d26/d32/f8a [0,4194304] 0 2024-03-16T09:01:18.748 INFO:tasks.workunit.client.1.smithi019.stdout:3/570: dwrite d3/d1b/f8e [4194304,4194304] 0 2024-03-16T09:01:18.748 INFO:tasks.workunit.client.1.smithi019.stdout:4/559: symlink d4/d6c/d89/lbb 0 2024-03-16T09:01:18.748 INFO:tasks.workunit.client.1.smithi019.stdout:4/560: creat d4/d1b/d2a/d4f/fbc x:0 0 0 2024-03-16T09:01:18.749 INFO:tasks.workunit.client.1.smithi019.stdout:4/561: stat d4/d6/d15/f60 0 2024-03-16T09:01:18.749 INFO:tasks.workunit.client.1.smithi019.stdout:4/562: stat d4/d6/d15/d9a 0 2024-03-16T09:01:18.749 INFO:tasks.workunit.client.0.smithi007.stdout:2/816: rename dc/db7/cd4 to dc/df/daa/c10e 0 2024-03-16T09:01:18.749 INFO:tasks.workunit.client.0.smithi007.stdout:2/817: write dc/df/d24/d28/d97/da4/db8/df1/ff5 [540190,124573] 0 2024-03-16T09:01:18.750 INFO:tasks.workunit.client.1.smithi019.stdout:8/578: fsync d6/d2a/d3c/dab/faa 0 2024-03-16T09:01:18.751 INFO:tasks.workunit.client.0.smithi007.stdout:6/640: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/fb6 [8388608,4194304] 0 2024-03-16T09:01:18.752 INFO:tasks.workunit.client.0.smithi007.stdout:2/818: getdents dc/df/d24/d28/d97/da4/db8 0 2024-03-16T09:01:18.755 INFO:tasks.workunit.client.0.smithi007.stdout:1/624: dwrite d9/dd/d1b/d28/d4c/f63 [0,4194304] 0 2024-03-16T09:01:18.755 INFO:tasks.workunit.client.0.smithi007.stdout:1/625: stat d9/dd/d1b/d2d/d30 0 2024-03-16T09:01:18.755 INFO:tasks.workunit.client.1.smithi019.stdout:7/522: dwrite d9/db/f87 [0,4194304] 0 2024-03-16T09:01:18.756 INFO:tasks.workunit.client.1.smithi019.stdout:7/523: truncate d9/d25/d28/f9c 985017 0 2024-03-16T09:01:18.756 INFO:tasks.workunit.client.1.smithi019.stdout:7/524: creat d9/db/fa7 x:0 0 0 2024-03-16T09:01:18.756 INFO:tasks.workunit.client.0.smithi007.stdout:2/819: truncate dc/df/d24/d28/d2e/d35/fb4 2550218 0 2024-03-16T09:01:18.756 INFO:tasks.workunit.client.0.smithi007.stdout:2/820: write dc/df/d24/d28/d2e/d35/f91 [442197,124664] 0 2024-03-16T09:01:18.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/563: creat d4/d6/d15/d1f/fbd x:0 0 0 2024-03-16T09:01:18.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/564: read d4/d6/d15/f60 [5764823,41269] 0 2024-03-16T09:01:18.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/565: creat d4/d6/d15/da2/fbe x:0 0 0 2024-03-16T09:01:18.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/566: stat d4/l5 0 2024-03-16T09:01:18.757 INFO:tasks.workunit.client.1.smithi019.stdout:4/567: truncate d4/d1b/f5d 249377 0 2024-03-16T09:01:18.758 INFO:tasks.workunit.client.1.smithi019.stdout:4/568: write d4/d64/d99/fb9 [401255,130176] 0 2024-03-16T09:01:18.760 INFO:tasks.workunit.client.0.smithi007.stdout:9/695: dread ff [0,4194304] 0 2024-03-16T09:01:18.760 INFO:tasks.workunit.client.0.smithi007.stdout:4/618: dwrite d4/f2f [0,4194304] 0 2024-03-16T09:01:18.761 INFO:tasks.workunit.client.1.smithi019.stdout:8/579: unlink d6/d2a/d45/d19/f31 0 2024-03-16T09:01:18.761 INFO:tasks.workunit.client.1.smithi019.stdout:8/580: write d6/d2a/d68/f86 [407657,103733] 0 2024-03-16T09:01:18.763 INFO:tasks.workunit.client.1.smithi019.stdout:3/571: read d3/d8b/dae/d6a/f7e [4175878,34477] 0 2024-03-16T09:01:18.763 INFO:tasks.workunit.client.1.smithi019.stdout:3/572: write d3/d8b/fa3 [74239,99896] 0 2024-03-16T09:01:18.764 INFO:tasks.workunit.client.0.smithi007.stdout:1/626: mknod d9/d6b/d7e/cc7 0 2024-03-16T09:01:18.764 INFO:tasks.workunit.client.0.smithi007.stdout:1/627: read f0 [647320,95478] 0 2024-03-16T09:01:18.767 INFO:tasks.workunit.client.1.smithi019.stdout:1/524: dwrite d16/d1f/f34 [0,4194304] 0 2024-03-16T09:01:18.767 INFO:tasks.workunit.client.0.smithi007.stdout:2/821: write dc/df/daa/fb3 [2826211,130034] 0 2024-03-16T09:01:18.768 INFO:tasks.workunit.client.0.smithi007.stdout:1/628: write d9/f62 [121251,97623] 0 2024-03-16T09:01:18.771 INFO:tasks.workunit.client.1.smithi019.stdout:6/685: dread de/d1c/f5d [0,4194304] 0 2024-03-16T09:01:18.776 INFO:tasks.workunit.client.1.smithi019.stdout:3/573: dread d3/d8b/dae/d5a/f73 [0,4194304] 0 2024-03-16T09:01:18.780 INFO:tasks.workunit.client.1.smithi019.stdout:7/525: link d9/da/d13/d18/d1a/f91 d9/da/d13/d18/d1a/fa8 0 2024-03-16T09:01:18.780 INFO:tasks.workunit.client.1.smithi019.stdout:7/526: dread - d9/d25/d28/f57 zero size 2024-03-16T09:01:18.780 INFO:tasks.workunit.client.1.smithi019.stdout:7/527: dread - d9/d25/f42 zero size 2024-03-16T09:01:18.780 INFO:tasks.workunit.client.1.smithi019.stdout:7/528: read d9/db/d12/d5b/d33/f7a [1376224,17905] 0 2024-03-16T09:01:18.780 INFO:tasks.workunit.client.1.smithi019.stdout:7/529: creat d9/da/d13/d18/d30/fa9 x:0 0 0 2024-03-16T09:01:18.780 INFO:tasks.workunit.client.1.smithi019.stdout:7/530: fdatasync d9/db/d12/d5b/d33/f7a 0 2024-03-16T09:01:18.780 INFO:tasks.workunit.client.1.smithi019.stdout:7/531: chown d9/da/d13/d9a/ca1 2078621 1 2024-03-16T09:01:18.782 INFO:tasks.workunit.client.0.smithi007.stdout:9/696: mkdir d24/d4a/d96/db9/dca 0 2024-03-16T09:01:18.782 INFO:tasks.workunit.client.0.smithi007.stdout:9/697: readlink d24/d4a/d96/db9/db1/lc8 0 2024-03-16T09:01:18.782 INFO:tasks.workunit.client.0.smithi007.stdout:9/698: truncate d24/d4c/d86/daa/d95/f62 487934 0 2024-03-16T09:01:18.782 INFO:tasks.workunit.client.0.smithi007.stdout:9/699: write f1c [2216441,47856] 0 2024-03-16T09:01:18.782 INFO:tasks.workunit.client.1.smithi019.stdout:8/581: creat d6/d2a/d45/d57/d72/d9f/fc4 x:0 0 0 2024-03-16T09:01:18.786 INFO:tasks.workunit.client.1.smithi019.stdout:1/525: readlink d16/d31/d76/l82 0 2024-03-16T09:01:18.788 INFO:tasks.workunit.client.0.smithi007.stdout:4/619: dread d4/d14/d1c/f6a [0,4194304] 0 2024-03-16T09:01:18.788 INFO:tasks.workunit.client.1.smithi019.stdout:6/686: unlink de/d26/d32/d97/d2c/c5e 0 2024-03-16T09:01:18.790 INFO:tasks.workunit.client.1.smithi019.stdout:3/574: link d3/d1b/d40/la2 d3/d8b/dae/d6a/lb4 0 2024-03-16T09:01:18.791 INFO:tasks.workunit.client.1.smithi019.stdout:8/582: dread f4 [0,4194304] 0 2024-03-16T09:01:18.791 INFO:tasks.workunit.client.1.smithi019.stdout:8/583: stat d6/d2a/d45/d57/d96/fad 0 2024-03-16T09:01:18.791 INFO:tasks.workunit.client.1.smithi019.stdout:8/584: truncate d6/f94 732470 0 2024-03-16T09:01:18.791 INFO:tasks.workunit.client.1.smithi019.stdout:7/532: creat d9/faa x:0 0 0 2024-03-16T09:01:18.791 INFO:tasks.workunit.client.0.smithi007.stdout:1/629: rename d9/dd/f81 to d9/dd/d1b/d2d/d30/d45/fc8 0 2024-03-16T09:01:18.792 INFO:tasks.workunit.client.1.smithi019.stdout:6/687: mknod de/d26/d32/d97/dae/db1/cde 0 2024-03-16T09:01:18.792 INFO:tasks.workunit.client.1.smithi019.stdout:6/688: fsync de/d26/d32/d97/fc5 0 2024-03-16T09:01:18.792 INFO:tasks.workunit.client.1.smithi019.stdout:6/689: fdatasync de/d1c/f8d 0 2024-03-16T09:01:18.793 INFO:tasks.workunit.client.0.smithi007.stdout:9/700: symlink d24/d4a/d55/dbf/lcb 0 2024-03-16T09:01:18.793 INFO:tasks.workunit.client.1.smithi019.stdout:8/585: unlink d6/d2a/d45/d19/c1f 0 2024-03-16T09:01:18.793 INFO:tasks.workunit.client.1.smithi019.stdout:8/586: write d6/d2a/d3f/f7d [257463,130545] 0 2024-03-16T09:01:18.793 INFO:tasks.workunit.client.0.smithi007.stdout:4/620: symlink d4/d14/d1c/d5b/d67/d8f/da8/d6b/ld5 0 2024-03-16T09:01:18.794 INFO:tasks.workunit.client.0.smithi007.stdout:4/621: chown d4/d14/d1c/d5b/d67/d8f/d90/fb8 7 1 2024-03-16T09:01:18.795 INFO:tasks.workunit.client.0.smithi007.stdout:9/701: creat d24/d4a/fcc x:0 0 0 2024-03-16T09:01:18.795 INFO:tasks.workunit.client.1.smithi019.stdout:7/533: rename d9/da/d13/d18/d46 to d9/da/d13/d9a/dab 0 2024-03-16T09:01:18.797 INFO:tasks.workunit.client.0.smithi007.stdout:3/737: dwrite d15/d18/d1f/d29/d3e/d85/fe2 [0,4194304] 0 2024-03-16T09:01:18.797 INFO:tasks.workunit.client.0.smithi007.stdout:3/738: fdatasync d15/d18/f23 0 2024-03-16T09:01:18.798 INFO:tasks.workunit.client.1.smithi019.stdout:4/569: dwrite d4/d6/d15/fb3 [0,4194304] 0 2024-03-16T09:01:18.798 INFO:tasks.workunit.client.1.smithi019.stdout:4/570: dread - d4/d6/d15/d9a/fb1 zero size 2024-03-16T09:01:18.798 INFO:tasks.workunit.client.0.smithi007.stdout:9/702: symlink d24/d4a/d96/lcd 0 2024-03-16T09:01:18.798 INFO:tasks.workunit.client.0.smithi007.stdout:9/703: fdatasync d24/d4c/f4e 0 2024-03-16T09:01:18.798 INFO:tasks.workunit.client.1.smithi019.stdout:1/526: write d16/d1f/f34 [667214,88643] 0 2024-03-16T09:01:18.799 INFO:tasks.workunit.client.0.smithi007.stdout:4/622: mknod d4/dcb/cd6 0 2024-03-16T09:01:18.801 INFO:tasks.workunit.client.0.smithi007.stdout:4/623: dread d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/d89/f9b [0,4194304] 0 2024-03-16T09:01:18.801 INFO:tasks.workunit.client.0.smithi007.stdout:4/624: fsync d4/d14/d64/d68/f6f 0 2024-03-16T09:01:18.801 INFO:tasks.workunit.client.0.smithi007.stdout:0/859: dwrite d3/d9/da/d48/d25/d43/d1a/d58/d61/f7a [0,4194304] 0 2024-03-16T09:01:18.801 INFO:tasks.workunit.client.0.smithi007.stdout:2/822: dwrite dc/df/d24/d28/d97/f9e [0,4194304] 0 2024-03-16T09:01:18.801 INFO:tasks.workunit.client.0.smithi007.stdout:6/641: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/f98 [0,4194304] 0 2024-03-16T09:01:18.802 INFO:tasks.workunit.client.1.smithi019.stdout:8/587: dread d6/d2a/d3f/f7d [0,4194304] 0 2024-03-16T09:01:18.802 INFO:tasks.workunit.client.1.smithi019.stdout:3/575: dread d3/d1b/d40/f9d [0,4194304] 0 2024-03-16T09:01:18.802 INFO:tasks.workunit.client.1.smithi019.stdout:0/578: dwrite d7/f5d [0,4194304] 0 2024-03-16T09:01:18.802 INFO:tasks.workunit.client.1.smithi019.stdout:3/576: stat d3/d8b/dae/d5a/d75/la1 0 2024-03-16T09:01:18.802 INFO:tasks.workunit.client.1.smithi019.stdout:3/577: write d3/d11/f69 [1332412,116194] 0 2024-03-16T09:01:18.804 INFO:tasks.workunit.client.0.smithi007.stdout:3/739: rename d15/d18/d1f/l84 to d15/d18/d1f/dce/dd6/ddf/le7 0 2024-03-16T09:01:18.804 INFO:tasks.workunit.client.0.smithi007.stdout:3/740: write d15/d18/d1f/d29/d43/f69 [88760,19781] 0 2024-03-16T09:01:18.807 INFO:tasks.workunit.client.1.smithi019.stdout:0/579: dread d7/d8/d32/d66/d6d/d8c/db8/d82/f71 [0,4194304] 0 2024-03-16T09:01:18.807 INFO:tasks.workunit.client.1.smithi019.stdout:0/580: chown d7/d8/d32/d4a/fb9 246 1 2024-03-16T09:01:18.808 INFO:tasks.workunit.client.0.smithi007.stdout:9/704: write d24/f48 [2716303,53404] 0 2024-03-16T09:01:18.808 INFO:tasks.workunit.client.0.smithi007.stdout:9/705: stat d24/d4a/f74 0 2024-03-16T09:01:18.808 INFO:tasks.workunit.client.0.smithi007.stdout:9/706: read d24/d4c/f7a [498907,52453] 0 2024-03-16T09:01:18.814 INFO:tasks.workunit.client.0.smithi007.stdout:0/860: creat d3/d1f/dca/f118 x:0 0 0 2024-03-16T09:01:18.815 INFO:tasks.workunit.client.1.smithi019.stdout:8/588: dread d6/d2a/d45/d57/d96/f9b [0,4194304] 0 2024-03-16T09:01:18.815 INFO:tasks.workunit.client.1.smithi019.stdout:8/589: creat d6/d2a/d45/d19/fc5 x:0 0 0 2024-03-16T09:01:18.816 INFO:tasks.workunit.client.0.smithi007.stdout:2/823: getdents dc/df/d24/d28/d2e/dc5 0 2024-03-16T09:01:18.820 INFO:tasks.workunit.client.0.smithi007.stdout:6/642: link d2/d13/d3f/d50/d78/dbe/dcb/d3b/f87 d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/fd4 0 2024-03-16T09:01:18.820 INFO:tasks.workunit.client.0.smithi007.stdout:6/643: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/fd3 50617 0 2024-03-16T09:01:18.823 INFO:tasks.workunit.client.0.smithi007.stdout:0/861: dread d3/d9/db4/d1c/d72/f82 [0,4194304] 0 2024-03-16T09:01:18.830 INFO:tasks.workunit.client.0.smithi007.stdout:2/824: write dc/d27/fd1 [199927,62207] 0 2024-03-16T09:01:18.830 INFO:tasks.workunit.client.0.smithi007.stdout:2/825: creat dc/db7/f10f x:0 0 0 2024-03-16T09:01:18.831 INFO:tasks.workunit.client.0.smithi007.stdout:9/707: getdents d24/d4c/d86/daa/d7f/d80/d87 0 2024-03-16T09:01:18.832 INFO:tasks.workunit.client.0.smithi007.stdout:9/708: chown d24/d4c/d86/daa/d95/f6f 336217 1 2024-03-16T09:01:18.832 INFO:tasks.workunit.client.0.smithi007.stdout:9/709: fsync d24/d4a/d96/db9/fa3 0 2024-03-16T09:01:18.834 INFO:tasks.workunit.client.0.smithi007.stdout:0/862: link d3/d9/da/d48/d9e/lb3 d3/d9/da/d92/d9f/de8/l119 0 2024-03-16T09:01:18.838 INFO:tasks.workunit.client.0.smithi007.stdout:2/826: unlink dc/d27/f65 0 2024-03-16T09:01:18.839 INFO:tasks.workunit.client.0.smithi007.stdout:9/710: creat d24/d4a/d96/db9/db1/fce x:0 0 0 2024-03-16T09:01:18.840 INFO:tasks.workunit.client.0.smithi007.stdout:3/741: dwrite d15/d18/d1f/d3b/d63/f93 [4194304,4194304] 0 2024-03-16T09:01:18.840 INFO:tasks.workunit.client.0.smithi007.stdout:3/742: write d15/d18/d1f/d56/f65 [1044119,46967] 0 2024-03-16T09:01:18.840 INFO:tasks.workunit.client.0.smithi007.stdout:2/827: symlink dc/df/d24/d28/dea/d10d/l110 0 2024-03-16T09:01:18.841 INFO:tasks.workunit.client.0.smithi007.stdout:9/711: dread d24/d4c/f7a [0,4194304] 0 2024-03-16T09:01:18.844 INFO:tasks.workunit.client.1.smithi019.stdout:8/590: dwrite d6/d2a/d3c/dab/d30/d4a/d67/d95/d9a/fa8 [0,4194304] 0 2024-03-16T09:01:18.844 INFO:tasks.workunit.client.1.smithi019.stdout:7/534: link d9/da/f59 d9/da/d13/d18/fac 0 2024-03-16T09:01:18.844 INFO:tasks.workunit.client.1.smithi019.stdout:7/535: fsync d9/d25/d28/d48/f76 0 2024-03-16T09:01:18.845 INFO:tasks.workunit.client.1.smithi019.stdout:7/536: chown d9/db/d12/f55 3604 1 2024-03-16T09:01:18.846 INFO:tasks.workunit.client.0.smithi007.stdout:2/828: symlink dc/df/d24/d28/d4c/d5e/l111 0 2024-03-16T09:01:18.846 INFO:tasks.workunit.client.0.smithi007.stdout:2/829: chown dc/df/d24/d28/d97/da4/db8/df1 347954 1 2024-03-16T09:01:18.846 INFO:tasks.workunit.client.0.smithi007.stdout:2/830: write dc/d27/f8e [4506597,67337] 0 2024-03-16T09:01:18.847 INFO:tasks.workunit.client.1.smithi019.stdout:4/571: mknod d4/d1b/d49/d58/cbf 0 2024-03-16T09:01:18.847 INFO:tasks.workunit.client.0.smithi007.stdout:4/625: dwrite d4/f40 [0,4194304] 0 2024-03-16T09:01:18.847 INFO:tasks.workunit.client.0.smithi007.stdout:4/626: chown d4/d14/d1c/d5b/d67/d8f/da8/d6b/c76 149213304 1 2024-03-16T09:01:18.848 INFO:tasks.workunit.client.0.smithi007.stdout:4/627: chown d4/d14/d64/cd0 129212686 1 2024-03-16T09:01:18.848 INFO:tasks.workunit.client.0.smithi007.stdout:9/712: creat d24/d4c/d86/daa/d7f/d80/d87/fcf x:0 0 0 2024-03-16T09:01:18.848 INFO:tasks.workunit.client.1.smithi019.stdout:1/527: creat d16/d1f/d3c/d79/f91 x:0 0 0 2024-03-16T09:01:18.849 INFO:tasks.workunit.client.0.smithi007.stdout:3/743: getdents d15/d18/d1f/d29/d3e/dda 0 2024-03-16T09:01:18.850 INFO:tasks.workunit.client.0.smithi007.stdout:6/644: dwrite d2/d10/f3a [4194304,4194304] 0 2024-03-16T09:01:18.850 INFO:tasks.workunit.client.0.smithi007.stdout:6/645: fsync d2/d13/d4b/f55 0 2024-03-16T09:01:18.851 INFO:tasks.workunit.client.1.smithi019.stdout:3/578: getdents d3/d1b/d40 0 2024-03-16T09:01:18.852 INFO:tasks.workunit.client.0.smithi007.stdout:2/831: symlink dc/df/d24/d28/d4c/daf/l112 0 2024-03-16T09:01:18.853 INFO:tasks.workunit.client.1.smithi019.stdout:0/581: mknod d7/d1d/d28/d92/da6/cca 0 2024-03-16T09:01:18.857 INFO:tasks.workunit.client.0.smithi007.stdout:9/713: dread d24/f48 [0,4194304] 0 2024-03-16T09:01:18.857 INFO:tasks.workunit.client.0.smithi007.stdout:9/714: truncate d24/d4a/da2/db7/fc3 676988 0 2024-03-16T09:01:18.857 INFO:tasks.workunit.client.0.smithi007.stdout:9/715: readlink d24/d4a/d9e/lc0 0 2024-03-16T09:01:18.857 INFO:tasks.workunit.client.0.smithi007.stdout:9/716: read d24/f31 [3961257,59381] 0 2024-03-16T09:01:18.860 INFO:tasks.workunit.client.0.smithi007.stdout:0/863: dwrite d3/d1f/f7f [0,4194304] 0 2024-03-16T09:01:18.863 INFO:tasks.workunit.client.0.smithi007.stdout:6/646: link d2/d13/d24/f6a d2/d10/d3d/d6d/dbf/fd5 0 2024-03-16T09:01:18.863 INFO:tasks.workunit.client.0.smithi007.stdout:6/647: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/f60 [643170,85767] 0 2024-03-16T09:01:18.865 INFO:tasks.workunit.client.1.smithi019.stdout:0/582: read d7/f2a [1701125,86937] 0 2024-03-16T09:01:18.865 INFO:tasks.workunit.client.1.smithi019.stdout:0/583: write d7/d1d/d2e/f3c [1226010,62240] 0 2024-03-16T09:01:18.865 INFO:tasks.workunit.client.1.smithi019.stdout:0/584: dread - d7/d8/d32/d4a/d61/d68/f94 zero size 2024-03-16T09:01:18.866 INFO:tasks.workunit.client.1.smithi019.stdout:8/591: symlink d6/d2a/d3c/dab/d30/d4a/d67/d95/d9a/lc6 0 2024-03-16T09:01:18.868 INFO:tasks.workunit.client.1.smithi019.stdout:4/572: symlink d4/lc0 0 2024-03-16T09:01:18.868 INFO:tasks.workunit.client.1.smithi019.stdout:4/573: readlink d4/lac 0 2024-03-16T09:01:18.868 INFO:tasks.workunit.client.1.smithi019.stdout:4/574: write d4/d6/f5f [1105566,62832] 0 2024-03-16T09:01:18.870 INFO:tasks.workunit.client.0.smithi007.stdout:2/832: creat dc/df/d24/d28/dea/d102/f113 x:0 0 0 2024-03-16T09:01:18.870 INFO:tasks.workunit.client.1.smithi019.stdout:1/528: unlink d16/d1f/d4f/f5c 0 2024-03-16T09:01:18.872 INFO:tasks.workunit.client.1.smithi019.stdout:0/585: rename d7/d1d/d63/d89/fa9 to d7/d8/d32/fcb 0 2024-03-16T09:01:18.873 INFO:tasks.workunit.client.1.smithi019.stdout:0/586: stat d7/d8/d32/d4a/d61/d68/l83 0 2024-03-16T09:01:18.873 INFO:tasks.workunit.client.1.smithi019.stdout:0/587: readlink d7/d8/d32/d4a/d61/lb1 0 2024-03-16T09:01:18.878 INFO:tasks.workunit.client.0.smithi007.stdout:3/744: rmdir d15/d18/d1f/d3b/d87 39 2024-03-16T09:01:18.878 INFO:tasks.workunit.client.1.smithi019.stdout:0/588: mkdir d7/d1d/d28/d92/dcc 0 2024-03-16T09:01:18.879 INFO:tasks.workunit.client.1.smithi019.stdout:0/589: chown d7/d8/d32/d66/d6d/d8c/db8/c58 8682 1 2024-03-16T09:01:18.879 INFO:tasks.workunit.client.1.smithi019.stdout:0/590: dread - d7/d8/d32/d66/d6d/d8c/db8/d82/faa zero size 2024-03-16T09:01:18.880 INFO:tasks.workunit.client.1.smithi019.stdout:1/529: mkdir d16/d31/d76/d92 0 2024-03-16T09:01:18.881 INFO:tasks.workunit.client.0.smithi007.stdout:0/864: rename d3/d9/da/d18/d27/d34/d3a/l5a to d3/d9/da/d48/d25/d43/l11a 0 2024-03-16T09:01:18.881 INFO:tasks.workunit.client.0.smithi007.stdout:9/717: read d24/f44 [3065811,117772] 0 2024-03-16T09:01:18.882 INFO:tasks.workunit.client.1.smithi019.stdout:0/591: dread d7/d8/d32/d66/d6d/d8c/db8/f5f [0,4194304] 0 2024-03-16T09:01:18.883 INFO:tasks.workunit.client.1.smithi019.stdout:1/530: symlink d16/d1f/d4f/d68/l93 0 2024-03-16T09:01:18.884 INFO:tasks.workunit.client.1.smithi019.stdout:0/592: creat d7/d8/d32/d66/d6d/d7c/fcd x:0 0 0 2024-03-16T09:01:18.884 INFO:tasks.workunit.client.0.smithi007.stdout:6/648: link d2/d13/d3f/d50/d78/dbe/dcb/d1e/cb8 d2/d13/d62/d91/cd6 0 2024-03-16T09:01:18.885 INFO:tasks.workunit.client.1.smithi019.stdout:1/531: truncate d16/f22 2245433 0 2024-03-16T09:01:18.886 INFO:tasks.workunit.client.0.smithi007.stdout:2/833: mkdir dc/df/d24/d28/d2e/d6e/d114 0 2024-03-16T09:01:18.886 INFO:tasks.workunit.client.0.smithi007.stdout:2/834: chown dc/df/d24/d28/dea/d102/l103 749875415 1 2024-03-16T09:01:18.887 INFO:tasks.workunit.client.0.smithi007.stdout:0/865: symlink d3/d9/da/d48/d4b/l11b 0 2024-03-16T09:01:18.887 INFO:tasks.workunit.client.1.smithi019.stdout:1/532: rename d16/d18/c66 to d16/d1f/c94 0 2024-03-16T09:01:18.887 INFO:tasks.workunit.client.1.smithi019.stdout:1/533: getdents d16/d31/d76/d92 0 2024-03-16T09:01:18.888 INFO:tasks.workunit.client.1.smithi019.stdout:1/534: truncate f12 4920260 0 2024-03-16T09:01:18.888 INFO:tasks.workunit.client.1.smithi019.stdout:1/535: chown d16/f2f 1190 1 2024-03-16T09:01:18.888 INFO:tasks.workunit.client.0.smithi007.stdout:9/718: fsync d24/d8e/fa7 0 2024-03-16T09:01:18.889 INFO:tasks.workunit.client.0.smithi007.stdout:2/835: mknod dc/df/d24/d28/d2e/d4a/d108/c115 0 2024-03-16T09:01:18.890 INFO:tasks.workunit.client.0.smithi007.stdout:9/719: mkdir d24/d4c/d86/daa/d7f/d8f/dd0 0 2024-03-16T09:01:18.891 INFO:tasks.workunit.client.0.smithi007.stdout:6/649: dread d2/d13/d24/f6a [4194304,4194304] 0 2024-03-16T09:01:18.892 INFO:tasks.workunit.client.1.smithi019.stdout:2/645: sync 2024-03-16T09:01:18.893 INFO:tasks.workunit.client.1.smithi019.stdout:9/447: sync 2024-03-16T09:01:18.893 INFO:tasks.workunit.client.1.smithi019.stdout:9/448: getdents d1/d7/de/d24/d44/d76/d9c 0 2024-03-16T09:01:18.893 INFO:tasks.workunit.client.1.smithi019.stdout:9/449: dread - d1/d7/de/d24/f99 zero size 2024-03-16T09:01:18.893 INFO:tasks.workunit.client.0.smithi007.stdout:2/836: creat dc/df/daa/f116 x:0 0 0 2024-03-16T09:01:18.893 INFO:tasks.workunit.client.0.smithi007.stdout:2/837: dread - dc/df/d24/d28/d4c/d5e/fd8 zero size 2024-03-16T09:01:18.894 INFO:tasks.workunit.client.1.smithi019.stdout:0/593: dread d7/d8/f50 [0,4194304] 0 2024-03-16T09:01:18.895 INFO:tasks.workunit.client.0.smithi007.stdout:9/720: symlink d24/d4c/d86/daa/d7f/ld1 0 2024-03-16T09:01:18.895 INFO:tasks.workunit.client.1.smithi019.stdout:7/537: truncate d9/d25/d28/f9c 244626 0 2024-03-16T09:01:18.895 INFO:tasks.workunit.client.1.smithi019.stdout:7/538: chown f8 89 1 2024-03-16T09:01:18.897 INFO:tasks.workunit.client.1.smithi019.stdout:2/646: mknod d5/d5a/d80/cd3 0 2024-03-16T09:01:18.897 INFO:tasks.workunit.client.0.smithi007.stdout:4/628: dwrite d4/f4d [0,4194304] 0 2024-03-16T09:01:18.898 INFO:tasks.workunit.client.1.smithi019.stdout:4/575: dwrite d4/d6/f81 [0,4194304] 0 2024-03-16T09:01:18.900 INFO:tasks.workunit.client.0.smithi007.stdout:2/838: unlink dc/df/d24/d28/d4c/d5e/c5d 0 2024-03-16T09:01:18.902 INFO:tasks.workunit.client.0.smithi007.stdout:0/866: write d3/d9/da/d18/d23/d36/f100 [3201184,34435] 0 2024-03-16T09:01:18.902 INFO:tasks.workunit.client.1.smithi019.stdout:0/594: getdents d7/d1d/d28 0 2024-03-16T09:01:18.906 INFO:tasks.workunit.client.0.smithi007.stdout:2/839: rename dc/c25 to dc/df/d24/d28/d97/c117 0 2024-03-16T09:01:18.906 INFO:tasks.workunit.client.0.smithi007.stdout:2/840: fsync dc/d27/f8f 0 2024-03-16T09:01:18.906 INFO:tasks.workunit.client.0.smithi007.stdout:2/841: write dc/df/d24/d28/d2e/d6e/fdf [163201,1586] 0 2024-03-16T09:01:18.907 INFO:tasks.workunit.client.1.smithi019.stdout:4/576: mkdir d4/d1b/dc1 0 2024-03-16T09:01:18.907 INFO:tasks.workunit.client.0.smithi007.stdout:4/629: dread d4/d14/d1c/d5b/f62 [0,4194304] 0 2024-03-16T09:01:18.908 INFO:tasks.workunit.client.1.smithi019.stdout:6/690: rmdir de/d26/d32/d97/dae 39 2024-03-16T09:01:18.908 INFO:tasks.workunit.client.1.smithi019.stdout:6/691: read - de/d26/d32/d97/d66/d72/db7/fbb zero size 2024-03-16T09:01:18.910 INFO:tasks.workunit.client.1.smithi019.stdout:0/595: mknod d7/d1d/d63/d89/d8a/cce 0 2024-03-16T09:01:18.910 INFO:tasks.workunit.client.1.smithi019.stdout:0/596: chown d7/d1d/d2e/f39 3470470 1 2024-03-16T09:01:18.911 INFO:tasks.workunit.client.1.smithi019.stdout:0/597: dread - d7/f6a zero size 2024-03-16T09:01:18.914 INFO:tasks.workunit.client.1.smithi019.stdout:4/577: link d4/d6/d15/d1f/l94 d4/d1b/d49/d58/d80/d8e/lc2 0 2024-03-16T09:01:18.914 INFO:tasks.workunit.client.1.smithi019.stdout:2/647: write d5/d6/d64/d6f/da6/fa1 [1531600,8246] 0 2024-03-16T09:01:18.914 INFO:tasks.workunit.client.1.smithi019.stdout:8/592: dwrite d6/d2a/d45/d19/d74/f7e [0,4194304] 0 2024-03-16T09:01:18.915 INFO:tasks.workunit.client.1.smithi019.stdout:4/578: symlink d4/d64/d9b/da9/lc3 0 2024-03-16T09:01:18.915 INFO:tasks.workunit.client.1.smithi019.stdout:4/579: read d4/f26 [5579669,70119] 0 2024-03-16T09:01:18.915 INFO:tasks.workunit.client.1.smithi019.stdout:4/580: readlink d4/l5 0 2024-03-16T09:01:18.917 INFO:tasks.workunit.client.0.smithi007.stdout:0/867: rename d3/d9/da/d18/f59 to d3/d9/db4/de3/f11c 0 2024-03-16T09:01:18.918 INFO:tasks.workunit.client.1.smithi019.stdout:8/593: mkdir d6/d2a/d3c/dab/d30/d4a/dc7 0 2024-03-16T09:01:18.918 INFO:tasks.workunit.client.1.smithi019.stdout:8/594: chown d6/d2a/d45/d5d/c63 282389 1 2024-03-16T09:01:18.918 INFO:tasks.workunit.client.1.smithi019.stdout:8/595: write d6/d2a/d45/d57/d72/d79/d7b/fa6 [1601072,122030] 0 2024-03-16T09:01:18.918 INFO:tasks.workunit.client.1.smithi019.stdout:8/596: dread - d6/de/fb2 zero size 2024-03-16T09:01:18.918 INFO:tasks.workunit.client.1.smithi019.stdout:8/597: write f5 [283326,130658] 0 2024-03-16T09:01:18.918 INFO:tasks.workunit.client.1.smithi019.stdout:8/598: chown d6/d2a/d45/d5d/fb3 50877 1 2024-03-16T09:01:18.919 INFO:tasks.workunit.client.1.smithi019.stdout:2/648: mkdir d5/de/dd4 0 2024-03-16T09:01:18.919 INFO:tasks.workunit.client.1.smithi019.stdout:8/599: mknod d6/d2a/d45/d19/d74/cc8 0 2024-03-16T09:01:18.919 INFO:tasks.workunit.client.1.smithi019.stdout:8/600: chown d6/d2a/d45/d19/d74/f7e 422881 1 2024-03-16T09:01:18.919 INFO:tasks.workunit.client.1.smithi019.stdout:8/601: fsync d6/d2a/d3c/dab/faa 0 2024-03-16T09:01:18.920 INFO:tasks.workunit.client.1.smithi019.stdout:2/649: mknod d5/d6/d2b/d53/cd5 0 2024-03-16T09:01:18.923 INFO:tasks.workunit.client.1.smithi019.stdout:2/650: symlink d5/d6/d2b/d89/ld6 0 2024-03-16T09:01:18.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/868: rename d3/d6c/dbd/dd7/d7c/c9c to d3/d9/da/d18/d23/d36/c11d 0 2024-03-16T09:01:18.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/869: write d3/d9/da/d48/fbe [273431,83193] 0 2024-03-16T09:01:18.925 INFO:tasks.workunit.client.0.smithi007.stdout:0/870: write d3/d9/db4/d2c/fb8 [173128,68581] 0 2024-03-16T09:01:18.925 INFO:tasks.workunit.client.0.smithi007.stdout:2/842: link dc/df/d24/d28/d2e/d35/c37 dc/df/d24/d28/d2e/c118 0 2024-03-16T09:01:18.925 INFO:tasks.workunit.client.0.smithi007.stdout:2/843: write dc/d27/f8e [2911951,14999] 0 2024-03-16T09:01:18.926 INFO:tasks.workunit.client.1.smithi019.stdout:5/564: sync 2024-03-16T09:01:18.926 INFO:tasks.workunit.client.1.smithi019.stdout:5/565: symlink d6/dd/d47/d7a/lae 0 2024-03-16T09:01:18.927 INFO:tasks.workunit.client.1.smithi019.stdout:5/566: mkdir d6/dd/daf 0 2024-03-16T09:01:18.927 INFO:tasks.workunit.client.0.smithi007.stdout:0/871: truncate d3/d9/da/f4c 8020439 0 2024-03-16T09:01:18.929 INFO:tasks.workunit.client.1.smithi019.stdout:5/567: mknod d6/d1f/d78/cb0 0 2024-03-16T09:01:18.931 INFO:tasks.workunit.client.0.smithi007.stdout:2/844: rename dc/df/d24/d28/d2e/f85 to dc/df/d24/d28/d2e/d4a/f119 0 2024-03-16T09:01:18.932 INFO:tasks.workunit.client.0.smithi007.stdout:2/845: getdents dc/df/d24/d28 0 2024-03-16T09:01:18.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/846: symlink dc/df/d6c/l11a 0 2024-03-16T09:01:18.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/847: write dc/fd [2322,56056] 0 2024-03-16T09:01:18.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/848: fdatasync dc/df/d24/d28/d2e/d35/fb4 0 2024-03-16T09:01:18.934 INFO:tasks.workunit.client.0.smithi007.stdout:2/849: dread - dc/df/d24/d28/d4c/f64 zero size 2024-03-16T09:01:18.936 INFO:tasks.workunit.client.0.smithi007.stdout:2/850: unlink dc/df/d24/d28/d4c/d5e/c70 0 2024-03-16T09:01:18.936 INFO:tasks.workunit.client.0.smithi007.stdout:2/851: creat dc/df/d24/d28/d4c/dbc/dfe/f11b x:0 0 0 2024-03-16T09:01:18.936 INFO:tasks.workunit.client.0.smithi007.stdout:2/852: stat dc/df/d24/d28/d2e/d6e/d114 0 2024-03-16T09:01:18.936 INFO:tasks.workunit.client.0.smithi007.stdout:9/721: dwrite d24/d4a/d96/db9/db1/fce [0,4194304] 0 2024-03-16T09:01:18.937 INFO:tasks.workunit.client.1.smithi019.stdout:4/581: dwrite f2 [0,4194304] 0 2024-03-16T09:01:18.940 INFO:tasks.workunit.client.0.smithi007.stdout:2/853: creat dc/df/d24/d28/d2e/dc5/f11c x:0 0 0 2024-03-16T09:01:18.943 INFO:tasks.workunit.client.1.smithi019.stdout:1/536: write d16/f22 [1070758,122061] 0 2024-03-16T09:01:18.945 INFO:tasks.workunit.client.0.smithi007.stdout:2/854: creat dc/df/d24/d28/d2e/dad/f11d x:0 0 0 2024-03-16T09:01:18.949 INFO:tasks.workunit.client.1.smithi019.stdout:2/651: dwrite d5/d6/d2b/d2f/f46 [0,4194304] 0 2024-03-16T09:01:18.949 INFO:tasks.workunit.client.1.smithi019.stdout:2/652: write d5/d6/d64/d6f/da6/fa1 [4622460,20917] 0 2024-03-16T09:01:18.949 INFO:tasks.workunit.client.1.smithi019.stdout:8/602: dread d6/d2a/d45/d19/f1a [0,4194304] 0 2024-03-16T09:01:18.950 INFO:tasks.workunit.client.1.smithi019.stdout:2/653: mknod d5/de/da2/cd7 0 2024-03-16T09:01:18.953 INFO:tasks.workunit.client.1.smithi019.stdout:8/603: rename d6/d2a/d3c/dab/d30/d4a/d67/f70 to d6/de/d51/fc9 0 2024-03-16T09:01:18.957 INFO:tasks.workunit.client.1.smithi019.stdout:2/654: rename d5/d6/d2b/la8 to d5/d6/d2b/d89/ld8 0 2024-03-16T09:01:18.959 INFO:tasks.workunit.client.1.smithi019.stdout:2/655: dread d5/d6/d2b/d89/fd1 [0,4194304] 0 2024-03-16T09:01:18.962 INFO:tasks.workunit.client.1.smithi019.stdout:4/582: dwrite d4/d1b/d2a/d3d/f46 [0,4194304] 0 2024-03-16T09:01:18.962 INFO:tasks.workunit.client.1.smithi019.stdout:4/583: readlink d4/d1b/d49/d58/d80/d8e/lc2 0 2024-03-16T09:01:18.964 INFO:tasks.workunit.client.1.smithi019.stdout:4/584: link d4/d1b/d49/l4a d4/d64/d9b/da9/lc4 0 2024-03-16T09:01:18.964 INFO:tasks.workunit.client.1.smithi019.stdout:4/585: chown d4/d1b/d2a/la7 12790387 1 2024-03-16T09:01:18.966 INFO:tasks.workunit.client.1.smithi019.stdout:4/586: symlink d4/d6/lc5 0 2024-03-16T09:01:18.966 INFO:tasks.workunit.client.1.smithi019.stdout:4/587: write d4/d64/d99/f40 [366078,53788] 0 2024-03-16T09:01:18.968 INFO:tasks.workunit.client.1.smithi019.stdout:4/588: getdents d4 0 2024-03-16T09:01:18.971 INFO:tasks.workunit.client.0.smithi007.stdout:2/855: dwrite dc/df/d24/d28/d2e/f45 [0,4194304] 0 2024-03-16T09:01:18.971 INFO:tasks.workunit.client.0.smithi007.stdout:2/856: readlink dc/df/d24/d28/d2e/d35/l93 0 2024-03-16T09:01:18.971 INFO:tasks.workunit.client.0.smithi007.stdout:2/857: read dc/df/d6c/fec [139874,16709] 0 2024-03-16T09:01:18.971 INFO:tasks.workunit.client.1.smithi019.stdout:8/604: dwrite d6/d2a/d45/d19/f1a [0,4194304] 0 2024-03-16T09:01:18.972 INFO:tasks.workunit.client.1.smithi019.stdout:8/605: dread - d6/d2a/d3c/dab/d30/d4a/d67/fa9 zero size 2024-03-16T09:01:18.972 INFO:tasks.workunit.client.1.smithi019.stdout:8/606: chown d6/d2a/d45/d57/d96/l99 19455 1 2024-03-16T09:01:18.972 INFO:tasks.workunit.client.0.smithi007.stdout:2/858: symlink dc/df/d24/d28/d4c/d5e/l11e 0 2024-03-16T09:01:18.973 INFO:tasks.workunit.client.1.smithi019.stdout:8/607: readlink d6/d2a/d3c/dab/l8b 0 2024-03-16T09:01:18.974 INFO:tasks.workunit.client.1.smithi019.stdout:8/608: dread d6/f77 [0,4194304] 0 2024-03-16T09:01:18.976 INFO:tasks.workunit.client.1.smithi019.stdout:8/609: rename d6/d2a/d3c/dab/d30/f3b to d6/d2a/d3c/dab/d98/fca 0 2024-03-16T09:01:18.977 INFO:tasks.workunit.client.1.smithi019.stdout:8/610: symlink d6/d2a/d45/lcb 0 2024-03-16T09:01:18.989 INFO:tasks.workunit.client.0.smithi007.stdout:6/650: dwrite d2/d13/d3f/d50/d72/fb4 [0,4194304] 0 2024-03-16T09:01:18.989 INFO:tasks.workunit.client.0.smithi007.stdout:6/651: read - d2/d13/d4b/fa6 zero size 2024-03-16T09:01:18.993 INFO:tasks.workunit.client.0.smithi007.stdout:9/722: dread d24/d4a/d55/f5d [0,4194304] 0 2024-03-16T09:01:18.995 INFO:tasks.workunit.client.0.smithi007.stdout:6/652: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fd7 x:0 0 0 2024-03-16T09:01:18.995 INFO:tasks.workunit.client.0.smithi007.stdout:6/653: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/f35 [1272190,111713] 0 2024-03-16T09:01:18.997 INFO:tasks.workunit.client.0.smithi007.stdout:6/654: truncate d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/f63 999257 0 2024-03-16T09:01:19.004 INFO:tasks.workunit.client.0.smithi007.stdout:4/630: dwrite d4/d14/d64/d9f/f61 [4194304,4194304] 0 2024-03-16T09:01:19.007 INFO:tasks.workunit.client.1.smithi019.stdout:1/537: dread f10 [0,4194304] 0 2024-03-16T09:01:19.008 INFO:tasks.workunit.client.1.smithi019.stdout:1/538: read d16/d31/f72 [547060,3677] 0 2024-03-16T09:01:19.009 INFO:tasks.workunit.client.1.smithi019.stdout:1/539: fdatasync d16/d31/d76/f6b 0 2024-03-16T09:01:19.011 INFO:tasks.workunit.client.1.smithi019.stdout:1/540: creat d16/d31/d76/f95 x:0 0 0 2024-03-16T09:01:19.019 INFO:tasks.workunit.client.0.smithi007.stdout:6/655: dread d2/d13/d24/f6a [0,4194304] 0 2024-03-16T09:01:19.028 INFO:tasks.workunit.client.0.smithi007.stdout:4/631: dread d4/d14/d64/d9f/f61 [0,4194304] 0 2024-03-16T09:01:19.029 INFO:tasks.workunit.client.0.smithi007.stdout:4/632: truncate d4/d14/d1c/f3b 558850 0 2024-03-16T09:01:19.031 INFO:tasks.workunit.client.0.smithi007.stdout:4/633: rename d4/d14/d1c/d5b/d67/d8f/d90 to d4/d14/d1c/d5b/d67/d8f/da8/d9d/dd7 0 2024-03-16T09:01:19.032 INFO:tasks.workunit.client.0.smithi007.stdout:4/634: rmdir d4/d14/d64/d68/d82/dac 39 2024-03-16T09:01:19.034 INFO:tasks.workunit.client.0.smithi007.stdout:4/635: truncate d4/d14/d64/d68/f56 5458572 0 2024-03-16T09:01:19.044 INFO:tasks.workunit.client.1.smithi019.stdout:5/568: dwrite d6/d1f/d33/f4f [0,4194304] 0 2024-03-16T09:01:19.044 INFO:tasks.workunit.client.1.smithi019.stdout:4/589: dwrite d4/d1b/f7e [4194304,4194304] 0 2024-03-16T09:01:19.044 INFO:tasks.workunit.client.1.smithi019.stdout:5/569: creat d6/dd/d37/d54/fb1 x:0 0 0 2024-03-16T09:01:19.044 INFO:tasks.workunit.client.1.smithi019.stdout:5/570: chown d6/d19/c83 119233 1 2024-03-16T09:01:19.044 INFO:tasks.workunit.client.1.smithi019.stdout:9/450: dwrite d1/d7/de/d24/f6d [0,4194304] 0 2024-03-16T09:01:19.045 INFO:tasks.workunit.client.0.smithi007.stdout:2/859: dwrite dc/db7/f10c [0,4194304] 0 2024-03-16T09:01:19.045 INFO:tasks.workunit.client.0.smithi007.stdout:6/656: dwrite d2/d82/fa0 [0,4194304] 0 2024-03-16T09:01:19.045 INFO:tasks.workunit.client.0.smithi007.stdout:9/723: dwrite d24/f31 [0,4194304] 0 2024-03-16T09:01:19.047 INFO:tasks.workunit.client.1.smithi019.stdout:4/590: creat d4/d1b/d2a/d4f/da0/fc6 x:0 0 0 2024-03-16T09:01:19.051 INFO:tasks.workunit.client.0.smithi007.stdout:2/860: link dc/df/d24/d28/d2e/d35/da2/ff4 dc/df/d24/d28/dea/d10d/f11f 0 2024-03-16T09:01:19.051 INFO:tasks.workunit.client.1.smithi019.stdout:9/451: creat d1/d92/fa0 x:0 0 0 2024-03-16T09:01:19.052 INFO:tasks.workunit.client.1.smithi019.stdout:5/571: getdents d6/dd/d47/d56 0 2024-03-16T09:01:19.052 INFO:tasks.workunit.client.1.smithi019.stdout:5/572: stat d6/d1f/l3a 0 2024-03-16T09:01:19.054 INFO:tasks.workunit.client.1.smithi019.stdout:9/452: fdatasync d1/d7/fd 0 2024-03-16T09:01:19.056 INFO:tasks.workunit.client.0.smithi007.stdout:6/657: mknod d2/d29/cd8 0 2024-03-16T09:01:19.056 INFO:tasks.workunit.client.0.smithi007.stdout:6/658: read - d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d7a/d9a/fa7 zero size 2024-03-16T09:01:19.058 INFO:tasks.workunit.client.0.smithi007.stdout:2/861: read dc/df/d24/d28/d2e/d35/f98 [2567322,10717] 0 2024-03-16T09:01:19.058 INFO:tasks.workunit.client.0.smithi007.stdout:2/862: write dc/df/d24/fb1 [612957,64382] 0 2024-03-16T09:01:19.058 INFO:tasks.workunit.client.0.smithi007.stdout:4/636: dwrite d4/d14/d64/d68/f8b [0,4194304] 0 2024-03-16T09:01:19.059 INFO:tasks.workunit.client.1.smithi019.stdout:5/573: rename d6/d1f/d33 to d6/dd/d47/d70/db2 0 2024-03-16T09:01:19.059 INFO:tasks.workunit.client.1.smithi019.stdout:9/453: dread d1/d7/de/d24/d44/d60/d74/f91 [0,4194304] 0 2024-03-16T09:01:19.060 INFO:tasks.workunit.client.1.smithi019.stdout:5/574: symlink d6/dd/lb3 0 2024-03-16T09:01:19.060 INFO:tasks.workunit.client.0.smithi007.stdout:2/863: dread dc/df/d24/d28/f66 [0,4194304] 0 2024-03-16T09:01:19.060 INFO:tasks.workunit.client.1.smithi019.stdout:8/611: getdents d6/d2a/d3c/dab/d30/d4a/d67 0 2024-03-16T09:01:19.060 INFO:tasks.workunit.client.1.smithi019.stdout:8/612: chown d6/d2a/d3c/dab/d98/cac 0 1 2024-03-16T09:01:19.061 INFO:tasks.workunit.client.0.smithi007.stdout:9/724: mknod d24/d4a/d55/cd2 0 2024-03-16T09:01:19.062 INFO:tasks.workunit.client.1.smithi019.stdout:9/454: rename d1/d7/de/c19 to d1/d7/de/d24/d44/ca1 0 2024-03-16T09:01:19.062 INFO:tasks.workunit.client.1.smithi019.stdout:8/613: dread d6/d2a/d45/d57/fbb [0,4194304] 0 2024-03-16T09:01:19.063 INFO:tasks.workunit.client.0.smithi007.stdout:6/659: getdents d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/dc1 0 2024-03-16T09:01:19.064 INFO:tasks.workunit.client.1.smithi019.stdout:4/591: dwrite d4/d64/d99/f6b [4194304,4194304] 0 2024-03-16T09:01:19.064 INFO:tasks.workunit.client.1.smithi019.stdout:9/455: read d1/d7/de/d24/d44/d76/f5f [426354,109466] 0 2024-03-16T09:01:19.064 INFO:tasks.workunit.client.1.smithi019.stdout:5/575: truncate d6/dd/d47/d70/db2/d89/d9e/f8b 1435238 0 2024-03-16T09:01:19.065 INFO:tasks.workunit.client.1.smithi019.stdout:8/614: symlink d6/d2a/d45/d2c/lcc 0 2024-03-16T09:01:19.068 INFO:tasks.workunit.client.1.smithi019.stdout:5/576: truncate d6/dd/f1a 4520909 0 2024-03-16T09:01:19.068 INFO:tasks.workunit.client.1.smithi019.stdout:5/577: creat d6/dd/d37/fb4 x:0 0 0 2024-03-16T09:01:19.068 INFO:tasks.workunit.client.1.smithi019.stdout:5/578: fdatasync d6/dd/d47/d70/db2/d66/f48 0 2024-03-16T09:01:19.068 INFO:tasks.workunit.client.1.smithi019.stdout:5/579: stat d6/dd/d37/d54/f5f 0 2024-03-16T09:01:19.069 INFO:tasks.workunit.client.0.smithi007.stdout:2/864: link dc/df/d24/d28/d97/da4/fa6 dc/df/d24/d28/d2e/dad/f120 0 2024-03-16T09:01:19.070 INFO:tasks.workunit.client.0.smithi007.stdout:2/865: truncate dc/df/daa/fe2 119894 0 2024-03-16T09:01:19.070 INFO:tasks.workunit.client.1.smithi019.stdout:8/615: mkdir d6/d2a/d45/d19/d74/dcd 0 2024-03-16T09:01:19.071 INFO:tasks.workunit.client.0.smithi007.stdout:9/725: creat d24/d4a/d9e/fd3 x:0 0 0 2024-03-16T09:01:19.073 INFO:tasks.workunit.client.0.smithi007.stdout:9/726: creat d24/d4a/d96/db9/fd4 x:0 0 0 2024-03-16T09:01:19.073 INFO:tasks.workunit.client.1.smithi019.stdout:5/580: truncate d6/d19/f4e 7081476 0 2024-03-16T09:01:19.074 INFO:tasks.workunit.client.1.smithi019.stdout:9/456: rename d1/d7/de/d24/d44/d6b to d1/d7/d21/da2 0 2024-03-16T09:01:19.076 INFO:tasks.workunit.client.1.smithi019.stdout:4/592: read d4/d1b/d49/d6e/fb2 [557831,49424] 0 2024-03-16T09:01:19.076 INFO:tasks.workunit.client.0.smithi007.stdout:9/727: mkdir d24/d4a/da2/dd5 0 2024-03-16T09:01:19.078 INFO:tasks.workunit.client.0.smithi007.stdout:2/866: dread dc/df/d24/d28/d2e/d35/f40 [0,4194304] 0 2024-03-16T09:01:19.078 INFO:tasks.workunit.client.0.smithi007.stdout:2/867: fsync dc/df/d24/d28/d2e/dc5/fd5 0 2024-03-16T09:01:19.078 INFO:tasks.workunit.client.0.smithi007.stdout:6/660: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/fb6 [12233098,1756] 0 2024-03-16T09:01:19.078 INFO:tasks.workunit.client.0.smithi007.stdout:6/661: dread - d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/faf zero size 2024-03-16T09:01:19.078 INFO:tasks.workunit.client.0.smithi007.stdout:6/662: dread - d2/d13/d62/d91/f95 zero size 2024-03-16T09:01:19.079 INFO:tasks.workunit.client.1.smithi019.stdout:5/581: unlink d6/dd/d47/d70/db2/f4f 0 2024-03-16T09:01:19.079 INFO:tasks.workunit.client.0.smithi007.stdout:9/728: rename d24/d4c/d86/daa/d95/c7d to d24/d4a/d96/db9/cd6 0 2024-03-16T09:01:19.082 INFO:tasks.workunit.client.0.smithi007.stdout:2/868: unlink dc/df/d24/d28/d4c/dbc/dfe/f11b 0 2024-03-16T09:01:19.082 INFO:tasks.workunit.client.0.smithi007.stdout:2/869: fsync dc/df/d24/d28/d2e/dc5/f100 0 2024-03-16T09:01:19.083 INFO:tasks.workunit.client.1.smithi019.stdout:9/457: mkdir d1/d7/de/da3 0 2024-03-16T09:01:19.083 INFO:tasks.workunit.client.1.smithi019.stdout:8/616: rename d6/d2a/f80 to d6/d2a/d45/d57/d72/d9d/fce 0 2024-03-16T09:01:19.083 INFO:tasks.workunit.client.1.smithi019.stdout:8/617: dread - d6/d2a/d68/fba zero size 2024-03-16T09:01:19.083 INFO:tasks.workunit.client.1.smithi019.stdout:8/618: creat d6/d2a/d3c/dab/d30/d4a/fcf x:0 0 0 2024-03-16T09:01:19.083 INFO:tasks.workunit.client.1.smithi019.stdout:1/541: rmdir d16/d31 39 2024-03-16T09:01:19.083 INFO:tasks.workunit.client.1.smithi019.stdout:1/542: mkdir d16/d1f/d3c/d79/d96 0 2024-03-16T09:01:19.084 INFO:tasks.workunit.client.1.smithi019.stdout:1/543: fdatasync d16/d1f/f2e 0 2024-03-16T09:01:19.084 INFO:tasks.workunit.client.0.smithi007.stdout:6/663: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c 39 2024-03-16T09:01:19.084 INFO:tasks.workunit.client.0.smithi007.stdout:6/664: readlink d2/d10/l1c 0 2024-03-16T09:01:19.084 INFO:tasks.workunit.client.1.smithi019.stdout:9/458: dread d1/d7/de/d24/f39 [0,4194304] 0 2024-03-16T09:01:19.088 INFO:tasks.workunit.client.1.smithi019.stdout:9/459: link d1/d7/de/d24/d44/d60/d74/f91 d1/d7/d67/d77/fa4 0 2024-03-16T09:01:19.090 INFO:tasks.workunit.client.0.smithi007.stdout:6/665: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d7a/d9a/dc9/fd9 x:0 0 0 2024-03-16T09:01:19.093 INFO:tasks.workunit.client.1.smithi019.stdout:9/460: dread d1/d7/d21/d27/f58 [0,4194304] 0 2024-03-16T09:01:19.094 INFO:tasks.workunit.client.0.smithi007.stdout:9/729: unlink d24/f3d 0 2024-03-16T09:01:19.095 INFO:tasks.workunit.client.0.smithi007.stdout:9/730: readlink d24/d8e/l9b 0 2024-03-16T09:01:19.095 INFO:tasks.workunit.client.0.smithi007.stdout:9/731: fsync d24/d4a/d55/f5d 0 2024-03-16T09:01:19.095 INFO:tasks.workunit.client.0.smithi007.stdout:8/991: sync 2024-03-16T09:01:19.095 INFO:tasks.workunit.client.0.smithi007.stdout:7/714: sync 2024-03-16T09:01:19.095 INFO:tasks.workunit.client.0.smithi007.stdout:5/729: sync 2024-03-16T09:01:19.096 INFO:tasks.workunit.client.1.smithi019.stdout:4/593: dread d4/d6c/d89/f93 [0,4194304] 0 2024-03-16T09:01:19.096 INFO:tasks.workunit.client.1.smithi019.stdout:4/594: read - d4/d6/d15/da2/fae zero size 2024-03-16T09:01:19.097 INFO:tasks.workunit.client.0.smithi007.stdout:4/637: dwrite d4/ff [0,4194304] 0 2024-03-16T09:01:19.099 INFO:tasks.workunit.client.0.smithi007.stdout:6/666: mknod d2/cda 0 2024-03-16T09:01:19.099 INFO:tasks.workunit.client.0.smithi007.stdout:6/667: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fdb x:0 0 0 2024-03-16T09:01:19.099 INFO:tasks.workunit.client.0.smithi007.stdout:6/668: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/fcf [615485,75275] 0 2024-03-16T09:01:19.101 INFO:tasks.workunit.client.0.smithi007.stdout:9/732: creat d24/d4c/d86/fd7 x:0 0 0 2024-03-16T09:01:19.102 INFO:tasks.workunit.client.0.smithi007.stdout:8/992: mkdir d0/dc/dd/d8b/d9e/d14a/d15d 0 2024-03-16T09:01:19.102 INFO:tasks.workunit.client.0.smithi007.stdout:8/993: write d0/dc/f23 [7379620,57322] 0 2024-03-16T09:01:19.102 INFO:tasks.workunit.client.0.smithi007.stdout:7/715: getdents d0/dc/d5a/dba/dd9 0 2024-03-16T09:01:19.104 INFO:tasks.workunit.client.0.smithi007.stdout:4/638: creat d4/dcb/fd8 x:0 0 0 2024-03-16T09:01:19.105 INFO:tasks.workunit.client.0.smithi007.stdout:6/669: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d7a/ddc 0 2024-03-16T09:01:19.106 INFO:tasks.workunit.client.0.smithi007.stdout:9/733: fsync d24/f27 0 2024-03-16T09:01:19.106 INFO:tasks.workunit.client.0.smithi007.stdout:9/734: chown d24/d4a/d96/db9/fa3 209156349 1 2024-03-16T09:01:19.107 INFO:tasks.workunit.client.0.smithi007.stdout:8/994: getdents d0/d26 0 2024-03-16T09:01:19.107 INFO:tasks.workunit.client.0.smithi007.stdout:8/995: chown d0/d26/d2d/f120 2 1 2024-03-16T09:01:19.108 INFO:tasks.workunit.client.0.smithi007.stdout:8/996: dread d0/dc/d8a/f91 [0,4194304] 0 2024-03-16T09:01:19.109 INFO:tasks.workunit.client.0.smithi007.stdout:6/670: unlink d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/f52 0 2024-03-16T09:01:19.109 INFO:tasks.workunit.client.0.smithi007.stdout:6/671: dread - d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fd7 zero size 2024-03-16T09:01:19.112 INFO:tasks.workunit.client.0.smithi007.stdout:8/997: getdents d0/dc/d50/d8e/dd2/dea 0 2024-03-16T09:01:19.113 INFO:tasks.workunit.client.0.smithi007.stdout:6/672: symlink d2/d13/d3f/d50/d78/d81/ldd 0 2024-03-16T09:01:19.113 INFO:tasks.workunit.client.0.smithi007.stdout:6/673: chown d2/d13/d3f/d50/d78/dbe/dcb/c43 31 1 2024-03-16T09:01:19.114 INFO:tasks.workunit.client.1.smithi019.stdout:5/582: dwrite d6/d1f/d78/f9a [0,4194304] 0 2024-03-16T09:01:19.114 INFO:tasks.workunit.client.1.smithi019.stdout:8/619: dwrite d6/d2a/d45/d57/f60 [0,4194304] 0 2024-03-16T09:01:19.114 INFO:tasks.workunit.client.1.smithi019.stdout:8/620: chown d6/d2a 1964248025 1 2024-03-16T09:01:19.114 INFO:tasks.workunit.client.1.smithi019.stdout:8/621: creat d6/d2a/d68/fd0 x:0 0 0 2024-03-16T09:01:19.115 INFO:tasks.workunit.client.0.smithi007.stdout:6/674: write d2/d10/f70 [891336,76077] 0 2024-03-16T09:01:19.115 INFO:tasks.workunit.client.0.smithi007.stdout:6/675: chown d2/d10/f70 8200173 1 2024-03-16T09:01:19.115 INFO:tasks.workunit.client.0.smithi007.stdout:6/676: write d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d7a/fc5 [83511,22510] 0 2024-03-16T09:01:19.116 INFO:tasks.workunit.client.1.smithi019.stdout:4/595: dwrite d4/d6/d15/d1f/fb8 [0,4194304] 0 2024-03-16T09:01:19.116 INFO:tasks.workunit.client.0.smithi007.stdout:8/998: link d0/d26/d2d/d43/d72/l7a d0/dc/d50/d8e/dd2/dea/d18/da7/dc5/d130/l15e 0 2024-03-16T09:01:19.117 INFO:tasks.workunit.client.0.smithi007.stdout:8/999: stat d0/dc/d50/d8e/dd2/dea/d5f/lee 0 2024-03-16T09:01:19.118 INFO:tasks.workunit.client.1.smithi019.stdout:5/583: rename d6/d19/d34/c8c to d6/dd/d37/d54/cb5 0 2024-03-16T09:01:19.118 INFO:tasks.workunit.client.0.smithi007.stdout:6/677: getdents d2/d13/d4b/d94 0 2024-03-16T09:01:19.120 INFO:tasks.workunit.client.1.smithi019.stdout:4/596: mknod d4/d1b/d2a/d4f/cc7 0 2024-03-16T09:01:19.120 INFO:tasks.workunit.client.0.smithi007.stdout:6/678: mknod d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/cde 0 2024-03-16T09:01:19.120 INFO:tasks.workunit.client.0.smithi007.stdout:9/735: dread d24/f46 [0,4194304] 0 2024-03-16T09:01:19.121 INFO:tasks.workunit.client.0.smithi007.stdout:9/736: fsync d24/d4c/d86/daa/d7f/d8f/fb8 0 2024-03-16T09:01:19.121 INFO:tasks.workunit.client.0.smithi007.stdout:9/737: chown d24/d4c/d86/daa/d7f/d80/lc6 0 1 2024-03-16T09:01:19.121 INFO:tasks.workunit.client.1.smithi019.stdout:5/584: creat d6/dd/d47/d70/db2/d66/fb6 x:0 0 0 2024-03-16T09:01:19.121 INFO:tasks.workunit.client.1.smithi019.stdout:5/585: chown d6/dd/d37/d54/l68 156150 1 2024-03-16T09:01:19.121 INFO:tasks.workunit.client.1.smithi019.stdout:1/544: dwrite d16/d1f/f34 [0,4194304] 0 2024-03-16T09:01:19.123 INFO:tasks.workunit.client.0.smithi007.stdout:6/679: creat d2/d13/d3f/d50/d78/d81/fdf x:0 0 0 2024-03-16T09:01:19.124 INFO:tasks.workunit.client.0.smithi007.stdout:9/738: mknod d24/d4c/d86/daa/d7f/d8f/db5/cd8 0 2024-03-16T09:01:19.126 INFO:tasks.workunit.client.0.smithi007.stdout:2/870: rename dc/df/d24/d28/d2e/dad to dc/df/d121 0 2024-03-16T09:01:19.126 INFO:tasks.workunit.client.0.smithi007.stdout:2/871: readlink dc/df/d24/d28/l31 0 2024-03-16T09:01:19.126 INFO:tasks.workunit.client.0.smithi007.stdout:2/872: creat dc/df/d24/d28/dea/d102/f122 x:0 0 0 2024-03-16T09:01:19.127 INFO:tasks.workunit.client.0.smithi007.stdout:6/680: rmdir d2/d10/d3d/d6d 39 2024-03-16T09:01:19.128 INFO:tasks.workunit.client.1.smithi019.stdout:9/461: rmdir d1/d7/de/d24 39 2024-03-16T09:01:19.128 INFO:tasks.workunit.client.1.smithi019.stdout:9/462: write d1/d7/de/d24/d44/d60/f72 [209111,82473] 0 2024-03-16T09:01:19.128 INFO:tasks.workunit.client.1.smithi019.stdout:9/463: fsync d1/d7/de/d24/d44/d60/d74/f91 0 2024-03-16T09:01:19.129 INFO:tasks.workunit.client.0.smithi007.stdout:4/639: rename d4/d14/d64/d68/f8b to d4/d14/d1c/d5b/d67/d8f/da8/d9d/dbb/fd9 0 2024-03-16T09:01:19.129 INFO:tasks.workunit.client.1.smithi019.stdout:5/586: rename d6/dd/d37/f9d to d6/dd/d37/d76/fb7 0 2024-03-16T09:01:19.130 INFO:tasks.workunit.client.1.smithi019.stdout:1/545: getdents d16/d31/d7b 0 2024-03-16T09:01:19.131 INFO:tasks.workunit.client.1.smithi019.stdout:1/546: readlink d16/d1f/d4f/d68/l7a 0 2024-03-16T09:01:19.131 INFO:tasks.workunit.client.0.smithi007.stdout:5/730: dwrite d1/d7/d1c/d19/d34/d5e/f56 [0,4194304] 0 2024-03-16T09:01:19.131 INFO:tasks.workunit.client.0.smithi007.stdout:5/731: fsync d1/d7/d1c/d19/d34/f8c 0 2024-03-16T09:01:19.132 INFO:tasks.workunit.client.0.smithi007.stdout:5/732: stat d1/d7/d1c/d19/d34/d5e/d6f/d72/c82 0 2024-03-16T09:01:19.132 INFO:tasks.workunit.client.0.smithi007.stdout:5/733: chown d1/d7/d1c/d19/dcc/d102 191750933 1 2024-03-16T09:01:19.132 INFO:tasks.workunit.client.0.smithi007.stdout:5/734: getdents d1/d7/d1c/d88/da2/df4 0 2024-03-16T09:01:19.132 INFO:tasks.workunit.client.0.smithi007.stdout:5/735: creat d1/d7/d1c/d19/d34/d4d/f108 x:0 0 0 2024-03-16T09:01:19.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:18 smithi007 ceph-mon[29470]: pgmap v23: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 181 MiB/s rd, 224 MiB/s wr, 459 op/s 2024-03-16T09:01:19.132 INFO:tasks.workunit.client.0.smithi007.stdout:2/873: mknod dc/df/d24/d28/d97/da4/db8/c123 0 2024-03-16T09:01:19.133 INFO:tasks.workunit.client.0.smithi007.stdout:2/874: write dc/df/d24/d28/dea/d102/f113 [898155,122716] 0 2024-03-16T09:01:19.133 INFO:tasks.workunit.client.0.smithi007.stdout:2/875: chown dc/df/d6c/fe0 17 1 2024-03-16T09:01:19.133 INFO:tasks.workunit.client.0.smithi007.stdout:2/876: write dc/df/d24/d28/d2e/d35/da2/fe3 [3646374,14167] 0 2024-03-16T09:01:19.133 INFO:tasks.workunit.client.1.smithi019.stdout:9/464: creat d1/d7/de/d24/d44/d76/fa5 x:0 0 0 2024-03-16T09:01:19.134 INFO:tasks.workunit.client.1.smithi019.stdout:5/587: mknod d6/d19/cb8 0 2024-03-16T09:01:19.134 INFO:tasks.workunit.client.1.smithi019.stdout:5/588: fsync d6/dd/d37/f6b 0 2024-03-16T09:01:19.134 INFO:tasks.workunit.client.1.smithi019.stdout:4/597: dread d4/d1b/d49/d6e/f90 [0,4194304] 0 2024-03-16T09:01:19.134 INFO:tasks.workunit.client.1.smithi019.stdout:1/547: symlink d16/d1f/d4f/l97 0 2024-03-16T09:01:19.134 INFO:tasks.workunit.client.0.smithi007.stdout:7/716: dwrite d0/fa5 [0,4194304] 0 2024-03-16T09:01:19.134 INFO:tasks.workunit.client.1.smithi019.stdout:9/465: link d1/d7/d21/l65 d1/d7/de/da3/la6 0 2024-03-16T09:01:19.135 INFO:tasks.workunit.client.1.smithi019.stdout:8/622: dwrite d6/d2a/d3c/dab/d30/d4a/fcf [0,4194304] 0 2024-03-16T09:01:19.136 INFO:tasks.workunit.client.0.smithi007.stdout:6/681: creat d2/d10/fe0 x:0 0 0 2024-03-16T09:01:19.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/589: mkdir d6/dd/d47/d7a/db9 0 2024-03-16T09:01:19.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/590: chown d6/dd/d61/fa9 255 1 2024-03-16T09:01:19.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/591: readlink d6/l65 0 2024-03-16T09:01:19.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/592: write d6/d1f/d77/f9c [814763,39364] 0 2024-03-16T09:01:19.137 INFO:tasks.workunit.client.1.smithi019.stdout:8/623: dread d6/f94 [0,4194304] 0 2024-03-16T09:01:19.137 INFO:tasks.workunit.client.1.smithi019.stdout:4/598: unlink d4/d1b/d49/d6e/fb2 0 2024-03-16T09:01:19.139 INFO:tasks.workunit.client.1.smithi019.stdout:4/599: dread d4/fa5 [0,4194304] 0 2024-03-16T09:01:19.139 INFO:tasks.workunit.client.1.smithi019.stdout:1/548: unlink d16/d1f/c28 0 2024-03-16T09:01:19.141 INFO:tasks.workunit.client.0.smithi007.stdout:5/736: mknod d1/d7/d1c/d88/da2/d89/db3/d2a/c109 0 2024-03-16T09:01:19.141 INFO:tasks.workunit.client.0.smithi007.stdout:5/737: dread - d1/d7/d1c/d19/d34/d1d/da8/fba zero size 2024-03-16T09:01:19.143 INFO:tasks.workunit.client.1.smithi019.stdout:9/466: unlink d1/d7/de/d24/d44/d76/f52 0 2024-03-16T09:01:19.144 INFO:tasks.workunit.client.1.smithi019.stdout:5/593: creat d6/dd/d47/d70/db2/d89/d9e/fba x:0 0 0 2024-03-16T09:01:19.144 INFO:tasks.workunit.client.1.smithi019.stdout:5/594: chown d6/dd/f82 284415 1 2024-03-16T09:01:19.145 INFO:tasks.workunit.client.0.smithi007.stdout:2/877: rename dc/df/d24/d28/d2e/d6e/f6f to dc/df/d24/d28/d4c/ded/f124 0 2024-03-16T09:01:19.146 INFO:tasks.workunit.client.1.smithi019.stdout:8/624: mknod d6/d2a/d3c/dab/d30/d4a/d67/cd1 0 2024-03-16T09:01:19.147 INFO:tasks.workunit.client.1.smithi019.stdout:8/625: write d6/d2a/d45/d19/f43 [2699219,55394] 0 2024-03-16T09:01:19.147 INFO:tasks.workunit.client.1.smithi019.stdout:8/626: getdents d6/d2a/d45/d19/d74/dcd 0 2024-03-16T09:01:19.147 INFO:tasks.workunit.client.1.smithi019.stdout:8/627: chown d6/d2a/d68 110124161 1 2024-03-16T09:01:19.148 INFO:tasks.workunit.client.1.smithi019.stdout:4/600: rename d4/d6/l3c to d4/d9e/lc8 0 2024-03-16T09:01:19.148 INFO:tasks.workunit.client.1.smithi019.stdout:4/601: readlink d4/lb 0 2024-03-16T09:01:19.148 INFO:tasks.workunit.client.1.smithi019.stdout:4/602: stat d4/d6/d15/c95 0 2024-03-16T09:01:19.150 INFO:tasks.workunit.client.1.smithi019.stdout:1/549: mknod d16/d1f/d4f/d68/d8f/c98 0 2024-03-16T09:01:19.151 INFO:tasks.workunit.client.0.smithi007.stdout:5/738: dread d1/d7/d1c/d19/d34/d5e/d6f/d72/f92 [0,4194304] 0 2024-03-16T09:01:19.152 INFO:tasks.workunit.client.0.smithi007.stdout:6/682: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/dc1 39 2024-03-16T09:01:19.152 INFO:tasks.workunit.client.0.smithi007.stdout:6/683: fdatasync d2/d13/d24/f38 0 2024-03-16T09:01:19.152 INFO:tasks.workunit.client.0.smithi007.stdout:6/684: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fe1 x:0 0 0 2024-03-16T09:01:19.154 INFO:tasks.workunit.client.0.smithi007.stdout:7/717: rename d0/d19/d1f/d69/dc7/fef to d0/d19/d1f/d5d/d9d/df7/d3a/daf/f104 0 2024-03-16T09:01:19.154 INFO:tasks.workunit.client.0.smithi007.stdout:2/878: mkdir dc/df/d24/d28/d125 0 2024-03-16T09:01:19.155 INFO:tasks.workunit.client.0.smithi007.stdout:5/739: symlink d1/d7/d1c/d19/df7/l10a 0 2024-03-16T09:01:19.156 INFO:tasks.workunit.client.1.smithi019.stdout:3/579: sync 2024-03-16T09:01:19.156 INFO:tasks.workunit.client.1.smithi019.stdout:7/539: sync 2024-03-16T09:01:19.156 INFO:tasks.workunit.client.1.smithi019.stdout:2/656: sync 2024-03-16T09:01:19.156 INFO:tasks.workunit.client.1.smithi019.stdout:3/580: dread - d3/d9b/fa9 zero size 2024-03-16T09:01:19.156 INFO:tasks.workunit.client.1.smithi019.stdout:0/598: sync 2024-03-16T09:01:19.157 INFO:tasks.workunit.client.1.smithi019.stdout:6/692: sync 2024-03-16T09:01:19.157 INFO:tasks.workunit.client.1.smithi019.stdout:0/599: chown d7/d1d/d63/d6e/db0 1497 1 2024-03-16T09:01:19.157 INFO:tasks.workunit.client.1.smithi019.stdout:3/581: write d3/d8b/dae/d5a/f61 [661750,8825] 0 2024-03-16T09:01:19.157 INFO:tasks.workunit.client.1.smithi019.stdout:0/600: creat d7/d1d/d28/da1/da7/fcf x:0 0 0 2024-03-16T09:01:19.157 INFO:tasks.workunit.client.1.smithi019.stdout:6/693: write de/d26/d32/d97/dae/fdb [560236,116992] 0 2024-03-16T09:01:19.158 INFO:tasks.workunit.client.0.smithi007.stdout:6/685: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/fe2 x:0 0 0 2024-03-16T09:01:19.158 INFO:tasks.workunit.client.1.smithi019.stdout:2/657: chown d5/d6/d2b/f5b 10407 1 2024-03-16T09:01:19.159 INFO:tasks.workunit.client.1.smithi019.stdout:7/540: mkdir d9/da/d13/d18/d30/d3e/d5d/dad 0 2024-03-16T09:01:19.159 INFO:tasks.workunit.client.1.smithi019.stdout:7/541: fdatasync d9/db/f2c 0 2024-03-16T09:01:19.160 INFO:tasks.workunit.client.1.smithi019.stdout:7/542: stat d9/da/d13/d18/l66 0 2024-03-16T09:01:19.160 INFO:tasks.workunit.client.1.smithi019.stdout:4/603: dread d4/f35 [0,4194304] 0 2024-03-16T09:01:19.161 INFO:tasks.workunit.client.0.smithi007.stdout:7/718: link d0/d19/d1f/d5d/d9d/df7/d3a/fd3 d0/d19/d1f/d5d/d9d/df7/d3a/daf/dec/f105 0 2024-03-16T09:01:19.161 INFO:tasks.workunit.client.0.smithi007.stdout:7/719: chown d0/dc/d38/d4e/fb4 55415072 1 2024-03-16T09:01:19.162 INFO:tasks.workunit.client.0.smithi007.stdout:2/879: mkdir dc/df/d24/d126 0 2024-03-16T09:01:19.163 INFO:tasks.workunit.client.1.smithi019.stdout:8/628: dread d6/d2a/d45/d2c/f58 [0,4194304] 0 2024-03-16T09:01:19.163 INFO:tasks.workunit.client.1.smithi019.stdout:8/629: stat d6/f27 0 2024-03-16T09:01:19.163 INFO:tasks.workunit.client.1.smithi019.stdout:0/601: mkdir d7/d8/d32/d4a/d61/d68/dd0 0 2024-03-16T09:01:19.164 INFO:tasks.workunit.client.1.smithi019.stdout:6/694: symlink de/d26/d32/d97/da1/ldf 0 2024-03-16T09:01:19.164 INFO:tasks.workunit.client.1.smithi019.stdout:4/604: unlink d4/d1b/c5e 0 2024-03-16T09:01:19.164 INFO:tasks.workunit.client.1.smithi019.stdout:4/605: dread - d4/d1b/d49/f7d zero size 2024-03-16T09:01:19.165 INFO:tasks.workunit.client.0.smithi007.stdout:9/739: dwrite d24/d4c/d86/daa/d7f/d8f/db5/fad [0,4194304] 0 2024-03-16T09:01:19.165 INFO:tasks.workunit.client.0.smithi007.stdout:2/880: creat dc/df/d24/d28/d97/da4/f127 x:0 0 0 2024-03-16T09:01:19.166 INFO:tasks.workunit.client.0.smithi007.stdout:2/881: dread - dc/df/d24/d28/d2e/fdd zero size 2024-03-16T09:01:19.166 INFO:tasks.workunit.client.1.smithi019.stdout:8/630: symlink d6/d2a/d45/d57/d72/d79/db7/ld2 0 2024-03-16T09:01:19.167 INFO:tasks.workunit.client.1.smithi019.stdout:8/631: write d6/d2a/d3c/dab/d98/fca [610494,116222] 0 2024-03-16T09:01:19.167 INFO:tasks.workunit.client.1.smithi019.stdout:8/632: creat d6/d2a/d45/d19/fd3 x:0 0 0 2024-03-16T09:01:19.167 INFO:tasks.workunit.client.1.smithi019.stdout:8/633: creat d6/fd4 x:0 0 0 2024-03-16T09:01:19.167 INFO:tasks.workunit.client.1.smithi019.stdout:9/467: dwrite d1/d92/fa0 [0,4194304] 0 2024-03-16T09:01:19.168 INFO:tasks.workunit.client.0.smithi007.stdout:5/740: dread d1/d7/d1c/d19/d34/d3e/d41/f4c [0,4194304] 0 2024-03-16T09:01:19.168 INFO:tasks.workunit.client.0.smithi007.stdout:5/741: read - d1/d7/d1c/d88/da2/d89/db3/d2a/d79/fbe zero size 2024-03-16T09:01:19.168 INFO:tasks.workunit.client.0.smithi007.stdout:4/640: dwrite d4/d14/f1b [0,4194304] 0 2024-03-16T09:01:19.168 INFO:tasks.workunit.client.1.smithi019.stdout:6/695: mkdir de/d26/d32/d4e/d75/dcd/de0 0 2024-03-16T09:01:19.169 INFO:tasks.workunit.client.1.smithi019.stdout:1/550: dread f9 [0,4194304] 0 2024-03-16T09:01:19.171 INFO:tasks.workunit.client.0.smithi007.stdout:9/740: mknod d24/d4c/d86/daa/d7f/d8f/db5/cd9 0 2024-03-16T09:01:19.173 INFO:tasks.workunit.client.1.smithi019.stdout:4/606: rename d4/c42 to d4/d1b/d2a/d4f/cc9 0 2024-03-16T09:01:19.173 INFO:tasks.workunit.client.1.smithi019.stdout:4/607: readlink d4/d6/d15/d1f/l96 0 2024-03-16T09:01:19.176 INFO:tasks.workunit.client.0.smithi007.stdout:2/882: mkdir dc/df/daa/d128 0 2024-03-16T09:01:19.177 INFO:tasks.workunit.client.1.smithi019.stdout:0/602: read d7/d8/d32/d66/d6d/d8c/db8/d82/f41 [2904694,97813] 0 2024-03-16T09:01:19.177 INFO:tasks.workunit.client.0.smithi007.stdout:6/686: rename d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d7a to d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3 0 2024-03-16T09:01:19.179 INFO:tasks.workunit.client.0.smithi007.stdout:5/742: creat d1/d7/d1c/d19/df7/f10b x:0 0 0 2024-03-16T09:01:19.179 INFO:tasks.workunit.client.1.smithi019.stdout:8/634: link d6/d2a/d68/f86 d6/d2a/d45/d57/d72/d79/fd5 0 2024-03-16T09:01:19.179 INFO:tasks.workunit.client.1.smithi019.stdout:8/635: stat d6/d2a/c4b 0 2024-03-16T09:01:19.187 INFO:tasks.workunit.client.1.smithi019.stdout:9/468: truncate d1/d7/d21/d27/f7a 601175 0 2024-03-16T09:01:19.188 INFO:tasks.workunit.client.0.smithi007.stdout:9/741: read d24/f29 [3610290,130936] 0 2024-03-16T09:01:19.189 INFO:tasks.workunit.client.1.smithi019.stdout:4/608: read d4/d6c/f7c [1432585,47082] 0 2024-03-16T09:01:19.190 INFO:tasks.workunit.client.1.smithi019.stdout:0/603: mknod d7/d8/d32/d4a/d61/dbf/cd1 0 2024-03-16T09:01:19.190 INFO:tasks.workunit.client.0.smithi007.stdout:2/883: creat dc/df/d24/d28/d2e/f129 x:0 0 0 2024-03-16T09:01:19.191 INFO:tasks.workunit.client.1.smithi019.stdout:8/636: mknod d6/d2a/d3c/dab/d30/d4a/d67/d95/d9a/cd6 0 2024-03-16T09:01:19.191 INFO:tasks.workunit.client.1.smithi019.stdout:8/637: dread - d6/d2a/d45/d57/d96/fad zero size 2024-03-16T09:01:19.191 INFO:tasks.workunit.client.1.smithi019.stdout:5/595: rmdir d6/dd/d47/d70/db2/d89/d9e 39 2024-03-16T09:01:19.192 INFO:tasks.workunit.client.0.smithi007.stdout:5/743: truncate d1/d7/d1c/d19/d34/d1d/f5a 3036036 0 2024-03-16T09:01:19.192 INFO:tasks.workunit.client.1.smithi019.stdout:1/551: rmdir d16/d1f/d43 39 2024-03-16T09:01:19.194 INFO:tasks.workunit.client.0.smithi007.stdout:9/742: mkdir d24/d4c/d86/daa/d7f/d80/d87/dda 0 2024-03-16T09:01:19.196 INFO:tasks.workunit.client.0.smithi007.stdout:5/744: symlink d1/d7/d1c/d19/d34/d3e/d62/l10c 0 2024-03-16T09:01:19.197 INFO:tasks.workunit.client.1.smithi019.stdout:4/609: creat d4/d64/d99/fca x:0 0 0 2024-03-16T09:01:19.197 INFO:tasks.workunit.client.0.smithi007.stdout:4/641: getdents d4/d99 0 2024-03-16T09:01:19.198 INFO:tasks.workunit.client.1.smithi019.stdout:0/604: symlink d7/d1d/d28/d6c/ld2 0 2024-03-16T09:01:19.198 INFO:tasks.workunit.client.1.smithi019.stdout:0/605: fsync d7/d8/d32/d66/d6d/fc8 0 2024-03-16T09:01:19.198 INFO:tasks.workunit.client.1.smithi019.stdout:0/606: write d7/d1d/d28/d48/f65 [742699,31585] 0 2024-03-16T09:01:19.199 INFO:tasks.workunit.client.1.smithi019.stdout:8/638: creat d6/d2a/d45/fd7 x:0 0 0 2024-03-16T09:01:19.199 INFO:tasks.workunit.client.1.smithi019.stdout:5/596: dread d6/dd/f44 [0,4194304] 0 2024-03-16T09:01:19.200 INFO:tasks.workunit.client.1.smithi019.stdout:5/597: chown d6/dd/d37/d76/f94 336 1 2024-03-16T09:01:19.200 INFO:tasks.workunit.client.1.smithi019.stdout:4/610: symlink d4/d6c/d89/lcb 0 2024-03-16T09:01:19.200 INFO:tasks.workunit.client.0.smithi007.stdout:9/743: mkdir d24/d4c/d86/daa/d7f/d80/d87/dda/ddb 0 2024-03-16T09:01:19.200 INFO:tasks.workunit.client.0.smithi007.stdout:9/744: chown d24/d4a/d55/f5b 11206 1 2024-03-16T09:01:19.200 INFO:tasks.workunit.client.0.smithi007.stdout:9/745: stat d24/d4a/c6b 0 2024-03-16T09:01:19.201 INFO:tasks.workunit.client.1.smithi019.stdout:9/469: dread d1/d7/de/d24/f42 [0,4194304] 0 2024-03-16T09:01:19.201 INFO:tasks.workunit.client.1.smithi019.stdout:8/639: rename d6/d2a/d3c/dab/d30/d4a/d87/l8e to d6/d2a/d68/ld8 0 2024-03-16T09:01:19.202 INFO:tasks.workunit.client.0.smithi007.stdout:5/745: truncate d1/d7/d1c/d19/d34/d3e/d41/d48/fc3 342279 0 2024-03-16T09:01:19.202 INFO:tasks.workunit.client.0.smithi007.stdout:5/746: truncate d1/d7/d1c/d88/da2/dae/ff8 920493 0 2024-03-16T09:01:19.202 INFO:tasks.workunit.client.1.smithi019.stdout:8/640: write d6/d2a/d68/fba [663815,65027] 0 2024-03-16T09:01:19.203 INFO:tasks.workunit.client.1.smithi019.stdout:1/552: creat d16/d31/d76/f99 x:0 0 0 2024-03-16T09:01:19.204 INFO:tasks.workunit.client.0.smithi007.stdout:4/642: symlink d4/d14/d1c/d5b/d67/d8f/da8/lda 0 2024-03-16T09:01:19.204 INFO:tasks.workunit.client.1.smithi019.stdout:6/696: dwrite de/d26/d32/d97/d66/d72/d7f/fc9 [0,4194304] 0 2024-03-16T09:01:19.204 INFO:tasks.workunit.client.1.smithi019.stdout:6/697: chown de/d26/d32/d97/d66/d72/d7f/dd5 2 1 2024-03-16T09:01:19.204 INFO:tasks.workunit.client.1.smithi019.stdout:6/698: dread - de/d26/fc6 zero size 2024-03-16T09:01:19.205 INFO:tasks.workunit.client.0.smithi007.stdout:7/720: dwrite d0/f55 [0,4194304] 0 2024-03-16T09:01:19.205 INFO:tasks.workunit.client.1.smithi019.stdout:4/611: creat d4/d6/fcc x:0 0 0 2024-03-16T09:01:19.205 INFO:tasks.workunit.client.1.smithi019.stdout:7/543: dwrite d9/d25/f31 [4194304,4194304] 0 2024-03-16T09:01:19.205 INFO:tasks.workunit.client.1.smithi019.stdout:2/658: dwrite d5/d2e/f9a [0,4194304] 0 2024-03-16T09:01:19.206 INFO:tasks.workunit.client.1.smithi019.stdout:9/470: creat d1/d7/d21/d27/fa7 x:0 0 0 2024-03-16T09:01:19.209 INFO:tasks.workunit.client.0.smithi007.stdout:5/747: rename d1/d7/d1c/d88/da2/d89/db3/d2a/f57 to d1/d7/d1c/d19/d34/de/f10d 0 2024-03-16T09:01:19.210 INFO:tasks.workunit.client.1.smithi019.stdout:7/544: getdents d9/db/d12/d5b/d33 0 2024-03-16T09:01:19.210 INFO:tasks.workunit.client.1.smithi019.stdout:7/545: write d9/d25/d28/f57 [364845,122953] 0 2024-03-16T09:01:19.210 INFO:tasks.workunit.client.1.smithi019.stdout:5/598: getdents d6/dd/d37/d54 0 2024-03-16T09:01:19.210 INFO:tasks.workunit.client.1.smithi019.stdout:7/546: creat d9/da/fae x:0 0 0 2024-03-16T09:01:19.211 INFO:tasks.workunit.client.1.smithi019.stdout:7/547: getdents d9/da/d13/d18/d30/d3e/da5 0 2024-03-16T09:01:19.211 INFO:tasks.workunit.client.1.smithi019.stdout:4/612: dread d4/d1b/d2a/f4d [0,4194304] 0 2024-03-16T09:01:19.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:18 smithi019 ceph-mon[36407]: pgmap v23: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 181 MiB/s rd, 224 MiB/s wr, 459 op/s 2024-03-16T09:01:19.213 INFO:tasks.workunit.client.0.smithi007.stdout:4/643: dread d4/d14/d64/d9f/f4c [0,4194304] 0 2024-03-16T09:01:19.213 INFO:tasks.workunit.client.0.smithi007.stdout:4/644: dread - d4/d14/d1c/d5b/d67/d8f/da8/d9d/fb4 zero size 2024-03-16T09:01:19.214 INFO:tasks.workunit.client.1.smithi019.stdout:2/659: creat d5/d6/d64/fd9 x:0 0 0 2024-03-16T09:01:19.214 INFO:tasks.workunit.client.0.smithi007.stdout:5/748: mkdir d1/d7/d1c/d19/dcc/d102/d10e 0 2024-03-16T09:01:19.217 INFO:tasks.workunit.client.1.smithi019.stdout:6/699: read de/d26/d32/d97/d66/d72/d7f/f81 [2666309,75289] 0 2024-03-16T09:01:19.217 INFO:tasks.workunit.client.1.smithi019.stdout:6/700: readlink de/d26/d32/d97/d2c/l83 0 2024-03-16T09:01:19.218 INFO:tasks.workunit.client.1.smithi019.stdout:6/701: fsync de/f1a 0 2024-03-16T09:01:19.218 INFO:tasks.workunit.client.1.smithi019.stdout:3/582: dwrite d3/d1b/d88/fa5 [0,4194304] 0 2024-03-16T09:01:19.218 INFO:tasks.workunit.client.1.smithi019.stdout:3/583: truncate d3/d8b/dae/d3f/faf 203953 0 2024-03-16T09:01:19.218 INFO:tasks.workunit.client.1.smithi019.stdout:5/599: creat d6/dd/d47/d7a/db9/fbb x:0 0 0 2024-03-16T09:01:19.218 INFO:tasks.workunit.client.1.smithi019.stdout:1/553: link l13 d16/d31/d76/l9a 0 2024-03-16T09:01:19.219 INFO:tasks.workunit.client.1.smithi019.stdout:7/548: creat d9/da/d13/d9a/faf x:0 0 0 2024-03-16T09:01:19.220 INFO:tasks.workunit.client.1.smithi019.stdout:4/613: rename d4/d1b/d2a/l37 to d4/d64/d9b/da9/lcd 0 2024-03-16T09:01:19.221 INFO:tasks.workunit.client.0.smithi007.stdout:4/645: symlink d4/dcb/dcd/ldb 0 2024-03-16T09:01:19.223 INFO:tasks.workunit.client.0.smithi007.stdout:5/749: creat d1/d7/d1c/d88/da2/dae/df3/f10f x:0 0 0 2024-03-16T09:01:19.225 INFO:tasks.workunit.client.1.smithi019.stdout:6/702: symlink de/d26/d32/d97/d66/d72/d7f/le1 0 2024-03-16T09:01:19.226 INFO:tasks.workunit.client.1.smithi019.stdout:8/641: dread d6/d2a/d45/f26 [0,4194304] 0 2024-03-16T09:01:19.226 INFO:tasks.workunit.client.0.smithi007.stdout:4/646: creat d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/d89/fdc x:0 0 0 2024-03-16T09:01:19.226 INFO:tasks.workunit.client.0.smithi007.stdout:1/630: sync 2024-03-16T09:01:19.226 INFO:tasks.workunit.client.0.smithi007.stdout:3/745: sync 2024-03-16T09:01:19.229 INFO:tasks.workunit.client.1.smithi019.stdout:5/600: mknod d6/dd/d37/cbc 0 2024-03-16T09:01:19.229 INFO:tasks.workunit.client.0.smithi007.stdout:7/721: dread d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f7b [0,4194304] 0 2024-03-16T09:01:19.231 INFO:tasks.workunit.client.0.smithi007.stdout:1/631: symlink d9/dd/d1b/d28/d29/lc9 0 2024-03-16T09:01:19.231 INFO:tasks.workunit.client.0.smithi007.stdout:1/632: chown d9/d6b/fa0 1803 1 2024-03-16T09:01:19.231 INFO:tasks.workunit.client.1.smithi019.stdout:1/554: symlink d16/d31/d71/d89/l9b 0 2024-03-16T09:01:19.233 INFO:tasks.workunit.client.1.smithi019.stdout:6/703: mknod de/d26/d32/d97/ce2 0 2024-03-16T09:01:19.234 INFO:tasks.workunit.client.0.smithi007.stdout:4/647: symlink d4/d14/d64/d68/d82/dac/ldd 0 2024-03-16T09:01:19.234 INFO:tasks.workunit.client.1.smithi019.stdout:0/607: dwrite d7/d1d/d28/f34 [0,4194304] 0 2024-03-16T09:01:19.234 INFO:tasks.workunit.client.1.smithi019.stdout:1/555: truncate d16/d1f/d43/f70 116999 0 2024-03-16T09:01:19.235 INFO:tasks.workunit.client.1.smithi019.stdout:4/614: dread d4/d6/d15/f67 [4194304,4194304] 0 2024-03-16T09:01:19.235 INFO:tasks.workunit.client.1.smithi019.stdout:1/556: dread d16/d1f/d43/f70 [0,4194304] 0 2024-03-16T09:01:19.236 INFO:tasks.workunit.client.1.smithi019.stdout:6/704: creat de/d26/d32/d97/d66/d72/d7f/fe3 x:0 0 0 2024-03-16T09:01:19.236 INFO:tasks.workunit.client.0.smithi007.stdout:7/722: rename d0/d19/d1f/d5d/d9d/df7/d3a/fb0 to d0/d19/d1f/d5d/d9d/df7/d3a/daf/dec/f106 0 2024-03-16T09:01:19.240 INFO:tasks.workunit.client.1.smithi019.stdout:4/615: link d4/d1b/f7e d4/d1b/dc1/fce 0 2024-03-16T09:01:19.240 INFO:tasks.workunit.client.0.smithi007.stdout:4/648: creat d4/d14/d1c/d5b/d67/d8f/da8/d9d/dbb/fde x:0 0 0 2024-03-16T09:01:19.243 INFO:tasks.workunit.client.0.smithi007.stdout:5/750: dread d1/d7/d16/d1b/d3c/fa7 [0,4194304] 0 2024-03-16T09:01:19.245 INFO:tasks.workunit.client.1.smithi019.stdout:1/557: creat d16/f9c x:0 0 0 2024-03-16T09:01:19.246 INFO:tasks.workunit.client.0.smithi007.stdout:5/751: mkdir d1/d7/d1c/d19/df7/d110 0 2024-03-16T09:01:19.247 INFO:tasks.workunit.client.1.smithi019.stdout:8/642: dread d6/d2a/d45/f32 [0,4194304] 0 2024-03-16T09:01:19.249 INFO:tasks.workunit.client.1.smithi019.stdout:7/549: read d9/da/d13/d9a/dab/f6b [2289067,112701] 0 2024-03-16T09:01:19.249 INFO:tasks.workunit.client.1.smithi019.stdout:7/550: symlink d9/da/d13/d18/d30/d3e/da5/lb0 0 2024-03-16T09:01:19.250 INFO:tasks.workunit.client.1.smithi019.stdout:7/551: write f7 [1353961,74061] 0 2024-03-16T09:01:19.250 INFO:tasks.workunit.client.1.smithi019.stdout:5/601: dread d6/fb [0,4194304] 0 2024-03-16T09:01:19.250 INFO:tasks.workunit.client.0.smithi007.stdout:5/752: unlink d1/faa 0 2024-03-16T09:01:19.251 INFO:tasks.workunit.client.0.smithi007.stdout:4/649: dread d4/d14/d1c/d5b/f62 [0,4194304] 0 2024-03-16T09:01:19.251 INFO:tasks.workunit.client.0.smithi007.stdout:4/650: chown d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84 22 1 2024-03-16T09:01:19.251 INFO:tasks.workunit.client.0.smithi007.stdout:4/651: write d4/d99/fbc [82805,101071] 0 2024-03-16T09:01:19.252 INFO:tasks.workunit.client.0.smithi007.stdout:5/753: mkdir d1/d7/d1c/d19/d34/d1d/da8/d111 0 2024-03-16T09:01:19.252 INFO:tasks.workunit.client.1.smithi019.stdout:7/552: unlink d9/da/d13/l4f 0 2024-03-16T09:01:19.252 INFO:tasks.workunit.client.1.smithi019.stdout:7/553: chown d9/da/d13/d9a/f77 110650 1 2024-03-16T09:01:19.253 INFO:tasks.workunit.client.0.smithi007.stdout:5/754: write d1/d7/d1c/d88/f95 [358719,100351] 0 2024-03-16T09:01:19.254 INFO:tasks.workunit.client.0.smithi007.stdout:4/652: symlink d4/d14/d64/d9f/ldf 0 2024-03-16T09:01:19.254 INFO:tasks.workunit.client.1.smithi019.stdout:6/705: dread de/d26/d32/f5b [0,4194304] 0 2024-03-16T09:01:19.254 INFO:tasks.workunit.client.1.smithi019.stdout:6/706: write de/d26/d32/d97/d51/fba [1322554,21989] 0 2024-03-16T09:01:19.255 INFO:tasks.workunit.client.0.smithi007.stdout:1/633: write d9/f13 [4116290,50715] 0 2024-03-16T09:01:19.256 INFO:tasks.workunit.client.1.smithi019.stdout:5/602: mknod d6/d1f/d78/cbd 0 2024-03-16T09:01:19.256 INFO:tasks.workunit.client.1.smithi019.stdout:7/554: rmdir d9/d25/d28/d48 39 2024-03-16T09:01:19.256 INFO:tasks.workunit.client.1.smithi019.stdout:7/555: creat d9/db/fb1 x:0 0 0 2024-03-16T09:01:19.256 INFO:tasks.workunit.client.1.smithi019.stdout:7/556: dread - d9/d25/f42 zero size 2024-03-16T09:01:19.256 INFO:tasks.workunit.client.1.smithi019.stdout:7/557: truncate d9/da/d13/d18/d1a/fa8 312163 0 2024-03-16T09:01:19.257 INFO:tasks.workunit.client.1.smithi019.stdout:6/707: mkdir de/d26/d32/d4e/d75/de4 0 2024-03-16T09:01:19.258 INFO:tasks.workunit.client.0.smithi007.stdout:5/755: mknod d1/d7/d1c/d19/d34/d3e/d41/da6/de8/de9/c112 0 2024-03-16T09:01:19.258 INFO:tasks.workunit.client.0.smithi007.stdout:5/756: write d1/d7/d1c/d19/d34/d4d/f101 [354562,28620] 0 2024-03-16T09:01:19.261 INFO:tasks.workunit.client.1.smithi019.stdout:0/608: dread d7/d1d/d28/d48/f4e [4194304,4194304] 0 2024-03-16T09:01:19.262 INFO:tasks.workunit.client.0.smithi007.stdout:4/653: symlink d4/dcb/dcd/le0 0 2024-03-16T09:01:19.262 INFO:tasks.workunit.client.1.smithi019.stdout:7/558: rmdir d9/da/d13/d18/d1a 39 2024-03-16T09:01:19.263 INFO:tasks.workunit.client.0.smithi007.stdout:0/872: sync 2024-03-16T09:01:19.265 INFO:tasks.workunit.client.1.smithi019.stdout:5/603: dread d6/d19/f1b [0,4194304] 0 2024-03-16T09:01:19.265 INFO:tasks.workunit.client.1.smithi019.stdout:5/604: write d6/d1f/d77/f9c [1442888,7633] 0 2024-03-16T09:01:19.268 INFO:tasks.workunit.client.0.smithi007.stdout:4/654: creat d4/d14/d64/d68/d82/dac/fe1 x:0 0 0 2024-03-16T09:01:19.270 INFO:tasks.workunit.client.0.smithi007.stdout:0/873: creat d3/d6c/dbd/dd7/d7b/f11e x:0 0 0 2024-03-16T09:01:19.270 INFO:tasks.workunit.client.0.smithi007.stdout:0/874: truncate d3/d1f/f10c 179610 0 2024-03-16T09:01:19.270 INFO:tasks.workunit.client.0.smithi007.stdout:0/875: fdatasync d3/d9/db4/d2c/f90 0 2024-03-16T09:01:19.272 INFO:tasks.workunit.client.0.smithi007.stdout:1/634: rmdir d9/dd/d1b/d2d/d30 39 2024-03-16T09:01:19.276 INFO:tasks.workunit.client.0.smithi007.stdout:1/635: truncate d9/dd/d1b/d2d/d30/f4f 3048684 0 2024-03-16T09:01:19.276 INFO:tasks.workunit.client.0.smithi007.stdout:9/746: dwrite d24/d4a/d96/fa8 [0,4194304] 0 2024-03-16T09:01:19.276 INFO:tasks.workunit.client.0.smithi007.stdout:0/876: mkdir d3/d9/da/d48/d25/d11f 0 2024-03-16T09:01:19.282 INFO:tasks.workunit.client.0.smithi007.stdout:1/636: creat d9/fca x:0 0 0 2024-03-16T09:01:19.282 INFO:tasks.workunit.client.0.smithi007.stdout:1/637: write d9/dd/d1b/d28/d4c/f8c [958552,102670] 0 2024-03-16T09:01:19.283 INFO:tasks.workunit.client.0.smithi007.stdout:9/747: mkdir d24/d4a/d96/db9/dca/ddc 0 2024-03-16T09:01:19.283 INFO:tasks.workunit.client.0.smithi007.stdout:0/877: symlink d3/d9/da/d18/l120 0 2024-03-16T09:01:19.284 INFO:tasks.workunit.client.0.smithi007.stdout:1/638: symlink d9/d6b/d7e/dac/dbb/lcb 0 2024-03-16T09:01:19.285 INFO:tasks.workunit.client.0.smithi007.stdout:9/748: rmdir d24/d4c/d86/daa/d7f/d8f/db5 39 2024-03-16T09:01:19.292 INFO:tasks.workunit.client.0.smithi007.stdout:9/749: write d24/d4c/f63 [448652,85237] 0 2024-03-16T09:01:19.294 INFO:tasks.workunit.client.0.smithi007.stdout:9/750: symlink d24/d8e/ldd 0 2024-03-16T09:01:19.294 INFO:tasks.workunit.client.0.smithi007.stdout:9/751: getdents d24/d4a/d96/db9/dca/ddc 0 2024-03-16T09:01:19.295 INFO:tasks.workunit.client.1.smithi019.stdout:9/471: dwrite d1/d7/d21/d27/f22 [0,4194304] 0 2024-03-16T09:01:19.298 INFO:tasks.workunit.client.0.smithi007.stdout:0/878: dread d3/d9/da/d48/d4b/f51 [0,4194304] 0 2024-03-16T09:01:19.298 INFO:tasks.workunit.client.0.smithi007.stdout:2/884: dwrite dc/df/d24/d28/d2e/d35/da2/fe3 [0,4194304] 0 2024-03-16T09:01:19.299 INFO:tasks.workunit.client.1.smithi019.stdout:0/609: dwrite d7/d8/f50 [0,4194304] 0 2024-03-16T09:01:19.299 INFO:tasks.workunit.client.1.smithi019.stdout:0/610: dread - d7/d1d/d2e/fc5 zero size 2024-03-16T09:01:19.300 INFO:tasks.workunit.client.0.smithi007.stdout:0/879: rename d3/d9/da/d18/d23/f63 to d3/d9/da/d92/d99/f121 0 2024-03-16T09:01:19.303 INFO:tasks.workunit.client.0.smithi007.stdout:2/885: creat dc/df/d24/d28/dea/f12a x:0 0 0 2024-03-16T09:01:19.305 INFO:tasks.workunit.client.1.smithi019.stdout:9/472: dread d1/d7/f4c [0,4194304] 0 2024-03-16T09:01:19.307 INFO:tasks.workunit.client.1.smithi019.stdout:9/473: symlink d1/d7/de/la8 0 2024-03-16T09:01:19.309 INFO:tasks.workunit.client.1.smithi019.stdout:9/474: rename d1/d7/de/d24/d44/d76/l5d to d1/d7/la9 0 2024-03-16T09:01:19.316 INFO:tasks.workunit.client.0.smithi007.stdout:7/723: dwrite d0/d6c/f64 [4194304,4194304] 0 2024-03-16T09:01:19.316 INFO:tasks.workunit.client.0.smithi007.stdout:7/724: fdatasync d0/d19/f27 0 2024-03-16T09:01:19.316 INFO:tasks.workunit.client.0.smithi007.stdout:7/725: dread - d0/d19/d1f/d5d/d9d/df7/d3a/db3/fcc zero size 2024-03-16T09:01:19.324 INFO:tasks.workunit.client.0.smithi007.stdout:9/752: read d24/d4a/d55/f65 [2618730,119220] 0 2024-03-16T09:01:19.326 INFO:tasks.workunit.client.0.smithi007.stdout:3/746: dwrite d15/d18/f35 [0,4194304] 0 2024-03-16T09:01:19.327 INFO:tasks.workunit.client.0.smithi007.stdout:9/753: unlink d24/l67 0 2024-03-16T09:01:19.328 INFO:tasks.workunit.client.1.smithi019.stdout:2/660: dwrite d5/de/da2/fc6 [0,4194304] 0 2024-03-16T09:01:19.328 INFO:tasks.workunit.client.1.smithi019.stdout:3/584: dwrite d3/d8b/dae/d5a/f8c [0,4194304] 0 2024-03-16T09:01:19.328 INFO:tasks.workunit.client.1.smithi019.stdout:4/616: dwrite d4/d6/d15/f60 [0,4194304] 0 2024-03-16T09:01:19.328 INFO:tasks.workunit.client.1.smithi019.stdout:4/617: fsync d4/d1b/d2a/d4f/fbc 0 2024-03-16T09:01:19.328 INFO:tasks.workunit.client.1.smithi019.stdout:6/708: dwrite de/f49 [4194304,4194304] 0 2024-03-16T09:01:19.329 INFO:tasks.workunit.client.0.smithi007.stdout:3/747: creat d15/fe8 x:0 0 0 2024-03-16T09:01:19.329 INFO:tasks.workunit.client.0.smithi007.stdout:0/880: dread d3/d9/da/d48/d25/d43/d91/fba [0,4194304] 0 2024-03-16T09:01:19.330 INFO:tasks.workunit.client.0.smithi007.stdout:0/881: readlink d3/d9/da/d48/d9e/lb3 0 2024-03-16T09:01:19.330 INFO:tasks.workunit.client.0.smithi007.stdout:9/754: symlink d24/d4c/d86/daa/d7f/d80/lde 0 2024-03-16T09:01:19.331 INFO:tasks.workunit.client.1.smithi019.stdout:2/661: mkdir d5/d6/d64/dda 0 2024-03-16T09:01:19.331 INFO:tasks.workunit.client.0.smithi007.stdout:3/748: creat d15/d18/d6a/fe9 x:0 0 0 2024-03-16T09:01:19.334 INFO:tasks.workunit.client.1.smithi019.stdout:6/709: symlink de/d26/d32/d97/d66/d72/le5 0 2024-03-16T09:01:19.334 INFO:tasks.workunit.client.1.smithi019.stdout:6/710: fdatasync de/d26/d32/d97/d66/d72/d7f/fc9 0 2024-03-16T09:01:19.334 INFO:tasks.workunit.client.1.smithi019.stdout:6/711: readlink de/d26/l80 0 2024-03-16T09:01:19.334 INFO:tasks.workunit.client.1.smithi019.stdout:6/712: write de/d1c/f28 [5004186,31291] 0 2024-03-16T09:01:19.335 INFO:tasks.workunit.client.0.smithi007.stdout:7/726: dread d0/d19/d1f/d76/fa3 [0,4194304] 0 2024-03-16T09:01:19.335 INFO:tasks.workunit.client.0.smithi007.stdout:0/882: rename d3/d9/da/d92/fce to d3/d6c/f122 0 2024-03-16T09:01:19.337 INFO:tasks.workunit.client.0.smithi007.stdout:0/883: mkdir d3/d9/da/d92/d9f/de8/d123 0 2024-03-16T09:01:19.339 INFO:tasks.workunit.client.0.smithi007.stdout:7/727: dread d0/dc/d5a/f89 [0,4194304] 0 2024-03-16T09:01:19.340 INFO:tasks.workunit.client.1.smithi019.stdout:2/662: creat d5/d6/d2b/d2f/fdb x:0 0 0 2024-03-16T09:01:19.341 INFO:tasks.workunit.client.0.smithi007.stdout:4/655: dwrite d4/d14/d64/d9f/f42 [0,4194304] 0 2024-03-16T09:01:19.341 INFO:tasks.workunit.client.0.smithi007.stdout:7/728: rmdir d0/d19/d1f/d5d/d9d/df7/d3a/daf 39 2024-03-16T09:01:19.343 INFO:tasks.workunit.client.1.smithi019.stdout:5/605: dwrite d6/dd/d47/d70/db2/d89/f96 [0,4194304] 0 2024-03-16T09:01:19.343 INFO:tasks.workunit.client.1.smithi019.stdout:5/606: chown d6/d19/f29 6084 1 2024-03-16T09:01:19.344 INFO:tasks.workunit.client.1.smithi019.stdout:2/663: creat d5/d6/d2b/d2f/d35/d65/fdc x:0 0 0 2024-03-16T09:01:19.344 INFO:tasks.workunit.client.1.smithi019.stdout:2/664: stat d5/d5a/d80 0 2024-03-16T09:01:19.344 INFO:tasks.workunit.client.1.smithi019.stdout:2/665: chown d5/d6/d2b/d2f/d35/d4b/ca7 276 1 2024-03-16T09:01:19.344 INFO:tasks.workunit.client.0.smithi007.stdout:1/639: dwrite d9/dd/d1b/d28/f36 [0,4194304] 0 2024-03-16T09:01:19.344 INFO:tasks.workunit.client.0.smithi007.stdout:5/757: dwrite d1/d7/d1c/d19/df7/f10b [0,4194304] 0 2024-03-16T09:01:19.346 INFO:tasks.workunit.client.1.smithi019.stdout:0/611: dwrite d7/d1d/d63/f78 [4194304,4194304] 0 2024-03-16T09:01:19.346 INFO:tasks.workunit.client.1.smithi019.stdout:0/612: fdatasync d7/d8/d32/d66/d6d/d8c/db8/d82/fba 0 2024-03-16T09:01:19.347 INFO:tasks.workunit.client.0.smithi007.stdout:4/656: rmdir d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/d93 39 2024-03-16T09:01:19.347 INFO:tasks.workunit.client.0.smithi007.stdout:9/755: dread d24/d4c/d86/daa/d7f/d80/d87/d89/fae [0,4194304] 0 2024-03-16T09:01:19.347 INFO:tasks.workunit.client.0.smithi007.stdout:9/756: read - d24/d4a/d9e/fd3 zero size 2024-03-16T09:01:19.347 INFO:tasks.workunit.client.1.smithi019.stdout:2/666: symlink d5/d6/d64/d6f/d87/ldd 0 2024-03-16T09:01:19.350 INFO:tasks.workunit.client.1.smithi019.stdout:8/643: dwrite d6/d2a/d45/d57/d72/d79/d7b/f93 [0,4194304] 0 2024-03-16T09:01:19.350 INFO:tasks.workunit.client.1.smithi019.stdout:8/644: read - d6/de/fb2 zero size 2024-03-16T09:01:19.352 INFO:tasks.workunit.client.0.smithi007.stdout:7/729: link d0/d19/d1f/d5d/d9d/df7/d3a/db3/lfb d0/dc/d38/l107 0 2024-03-16T09:01:19.352 INFO:tasks.workunit.client.1.smithi019.stdout:9/475: dwrite d1/d7/d21/d27/f4e [0,4194304] 0 2024-03-16T09:01:19.354 INFO:tasks.workunit.client.1.smithi019.stdout:0/613: unlink d7/d8/d32/d66/d6d/d8c/db8/d82/l80 0 2024-03-16T09:01:19.354 INFO:tasks.workunit.client.1.smithi019.stdout:0/614: write d7/d1d/d2e/f3c [1571934,12216] 0 2024-03-16T09:01:19.354 INFO:tasks.workunit.client.0.smithi007.stdout:3/749: read d15/d18/d1f/f58 [5305473,38512] 0 2024-03-16T09:01:19.356 INFO:tasks.workunit.client.1.smithi019.stdout:1/558: rmdir d16/d31 39 2024-03-16T09:01:19.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/615: dread d7/d1d/d63/d89/d9e/fc7 [0,4194304] 0 2024-03-16T09:01:19.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/616: truncate d7/d8/f23 1191049 0 2024-03-16T09:01:19.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/617: dread - d7/d1d/d2e/fc5 zero size 2024-03-16T09:01:19.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/618: dread - d7/d1d/d2e/fc3 zero size 2024-03-16T09:01:19.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/619: write d7/d8/d32/d66/d6d/d8c/db8/d82/f41 [308696,62482] 0 2024-03-16T09:01:19.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/620: write d7/d8/d32/d66/d6d/d8c/db8/d82/fc2 [253762,107038] 0 2024-03-16T09:01:19.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/621: read - d7/d8/d32/d66/d6d/d8c/db8/d82/faa zero size 2024-03-16T09:01:19.357 INFO:tasks.workunit.client.1.smithi019.stdout:3/585: dwrite d3/d89/f47 [4194304,4194304] 0 2024-03-16T09:01:19.358 INFO:tasks.workunit.client.0.smithi007.stdout:5/758: symlink d1/d7/d1c/d88/da2/df4/l113 0 2024-03-16T09:01:19.359 INFO:tasks.workunit.client.0.smithi007.stdout:5/759: stat d1/d7/d1c/d88/da2/d89/db3/f3d 0 2024-03-16T09:01:19.359 INFO:tasks.workunit.client.0.smithi007.stdout:5/760: write f0 [58875,38430] 0 2024-03-16T09:01:19.359 INFO:tasks.workunit.client.0.smithi007.stdout:5/761: fsync d1/d7/d1c/d88/da2/d89/fb6 0 2024-03-16T09:01:19.359 INFO:tasks.workunit.client.0.smithi007.stdout:5/762: write d1/d7/d1c/d19/d34/d5e/d6f/fb4 [567499,58526] 0 2024-03-16T09:01:19.363 INFO:tasks.workunit.client.0.smithi007.stdout:3/750: mknod d15/d18/d1f/d29/d43/cea 0 2024-03-16T09:01:19.363 INFO:tasks.workunit.client.0.smithi007.stdout:3/751: creat d15/d18/d1f/d29/d3e/d85/feb x:0 0 0 2024-03-16T09:01:19.365 INFO:tasks.workunit.client.0.smithi007.stdout:5/763: symlink d1/d7/d1c/d19/dcc/l114 0 2024-03-16T09:01:19.365 INFO:tasks.workunit.client.1.smithi019.stdout:9/476: dread d1/d7/f68 [0,4194304] 0 2024-03-16T09:01:19.366 INFO:tasks.workunit.client.1.smithi019.stdout:0/622: truncate d7/d8/d32/d66/d6d/d8c/db8/f69 648891 0 2024-03-16T09:01:19.366 INFO:tasks.workunit.client.0.smithi007.stdout:1/640: rmdir d9/dd/d1b/d28/d4c 39 2024-03-16T09:01:19.367 INFO:tasks.workunit.client.1.smithi019.stdout:0/623: write d7/d8/d32/d66/d6d/d8c/db8/d82/f6f [424255,85489] 0 2024-03-16T09:01:19.367 INFO:tasks.workunit.client.1.smithi019.stdout:0/624: write d7/d1d/d63/d6e/fc9 [228213,23163] 0 2024-03-16T09:01:19.369 INFO:tasks.workunit.client.0.smithi007.stdout:9/757: truncate d24/f3f 3221284 0 2024-03-16T09:01:19.371 INFO:tasks.workunit.client.1.smithi019.stdout:2/667: mkdir d5/d6/d2b/dde 0 2024-03-16T09:01:19.371 INFO:tasks.workunit.client.1.smithi019.stdout:2/668: write d5/d6/d2b/d2f/d35/d65/fb4 [146581,36597] 0 2024-03-16T09:01:19.371 INFO:tasks.workunit.client.1.smithi019.stdout:8/645: getdents d6/d2a/d3c 0 2024-03-16T09:01:19.372 INFO:tasks.workunit.client.1.smithi019.stdout:0/625: dread d7/d1d/d28/d48/f4e [4194304,4194304] 0 2024-03-16T09:01:19.372 INFO:tasks.workunit.client.1.smithi019.stdout:3/586: rmdir d3/d1b/d40/d4a 39 2024-03-16T09:01:19.372 INFO:tasks.workunit.client.1.smithi019.stdout:3/587: stat d3/d9b/f9f 0 2024-03-16T09:01:19.374 INFO:tasks.workunit.client.0.smithi007.stdout:3/752: unlink d15/d18/d1f/d3b/d87/l99 0 2024-03-16T09:01:19.374 INFO:tasks.workunit.client.1.smithi019.stdout:2/669: unlink d5/d6/c38 0 2024-03-16T09:01:19.375 INFO:tasks.workunit.client.0.smithi007.stdout:9/758: dread ff [0,4194304] 0 2024-03-16T09:01:19.375 INFO:tasks.workunit.client.0.smithi007.stdout:5/764: mkdir d1/d7/d1c/d19/d115 0 2024-03-16T09:01:19.376 INFO:tasks.workunit.client.1.smithi019.stdout:8/646: truncate d6/d2a/d45/d57/f65 3017123 0 2024-03-16T09:01:19.376 INFO:tasks.workunit.client.0.smithi007.stdout:4/657: write d4/f4d [3469606,5121] 0 2024-03-16T09:01:19.376 INFO:tasks.workunit.client.0.smithi007.stdout:1/641: rmdir d9/dd/d1b/d58 39 2024-03-16T09:01:19.377 INFO:tasks.workunit.client.0.smithi007.stdout:1/642: getdents d9/dd/d1b/d28/d29/d7b/dba 0 2024-03-16T09:01:19.377 INFO:tasks.workunit.client.0.smithi007.stdout:1/643: dread - d9/dd/d1b/f96 zero size 2024-03-16T09:01:19.377 INFO:tasks.workunit.client.0.smithi007.stdout:1/644: chown d9/dd/d1b/d2d/d30/d45/f5c 193931838 1 2024-03-16T09:01:19.377 INFO:tasks.workunit.client.0.smithi007.stdout:1/645: read d9/dd/d1b/f49 [172,57984] 0 2024-03-16T09:01:19.378 INFO:tasks.workunit.client.1.smithi019.stdout:3/588: link d3/d8b/dae/d5a/f8c d3/d1b/d40/fb5 0 2024-03-16T09:01:19.378 INFO:tasks.workunit.client.1.smithi019.stdout:3/589: creat d3/d9b/fb6 x:0 0 0 2024-03-16T09:01:19.379 INFO:tasks.workunit.client.1.smithi019.stdout:2/670: symlink d5/d6/d2b/d89/ldf 0 2024-03-16T09:01:19.381 INFO:tasks.workunit.client.0.smithi007.stdout:5/765: symlink d1/d7/d1c/d88/da2/dae/l116 0 2024-03-16T09:01:19.381 INFO:tasks.workunit.client.1.smithi019.stdout:2/671: link d5/d5a/c6a d5/d6/d2b/d2f/d35/ce0 0 2024-03-16T09:01:19.381 INFO:tasks.workunit.client.1.smithi019.stdout:2/672: creat d5/d2e/d4f/fe1 x:0 0 0 2024-03-16T09:01:19.381 INFO:tasks.workunit.client.1.smithi019.stdout:2/673: creat d5/de/dab/fe2 x:0 0 0 2024-03-16T09:01:19.382 INFO:tasks.workunit.client.1.smithi019.stdout:8/647: getdents d6/d2a/d45/d2c 0 2024-03-16T09:01:19.382 INFO:tasks.workunit.client.1.smithi019.stdout:8/648: write d6/d2a/d45/d57/d72/d9f/fc4 [497153,38152] 0 2024-03-16T09:01:19.383 INFO:tasks.workunit.client.1.smithi019.stdout:3/590: getdents d3/d11 0 2024-03-16T09:01:19.384 INFO:tasks.workunit.client.0.smithi007.stdout:1/646: symlink d9/dd/d1b/d2d/d30/d45/lcc 0 2024-03-16T09:01:19.385 INFO:tasks.workunit.client.1.smithi019.stdout:8/649: mknod d6/de/cd9 0 2024-03-16T09:01:19.388 INFO:tasks.workunit.client.1.smithi019.stdout:3/591: getdents d3/d1b/d40/d4a 0 2024-03-16T09:01:19.388 INFO:tasks.workunit.client.1.smithi019.stdout:3/592: fsync d3/d8b/dae/d6a/f7e 0 2024-03-16T09:01:19.397 INFO:tasks.workunit.client.0.smithi007.stdout:2/886: rmdir dc/df 39 2024-03-16T09:01:19.399 INFO:tasks.workunit.client.1.smithi019.stdout:9/477: dread d1/d92/fa0 [0,4194304] 0 2024-03-16T09:01:19.402 INFO:tasks.workunit.client.0.smithi007.stdout:3/753: dread d15/d18/d1f/d29/d3e/d85/fe2 [0,4194304] 0 2024-03-16T09:01:19.404 INFO:tasks.workunit.client.1.smithi019.stdout:0/626: dread d7/d1d/d63/f78 [4194304,4194304] 0 2024-03-16T09:01:19.406 INFO:tasks.workunit.client.1.smithi019.stdout:0/627: symlink d7/d8/d32/d66/d6d/d8c/db8/ld3 0 2024-03-16T09:01:19.407 INFO:tasks.workunit.client.1.smithi019.stdout:0/628: truncate d7/d1d/d28/d92/da6/fbb 2224099 0 2024-03-16T09:01:19.408 INFO:tasks.workunit.client.1.smithi019.stdout:0/629: stat d7/d8/d32/d4a/d61/lb1 0 2024-03-16T09:01:19.412 INFO:tasks.workunit.client.0.smithi007.stdout:0/884: dwrite d3/d9/da/d18/d23/d36/fef [0,4194304] 0 2024-03-16T09:01:19.415 INFO:tasks.workunit.client.1.smithi019.stdout:3/593: write d3/d8b/dae/d6a/f7b [1453901,17443] 0 2024-03-16T09:01:19.415 INFO:tasks.workunit.client.1.smithi019.stdout:3/594: fsync d3/d8b/dae/d5a/d75/d7a/fa0 0 2024-03-16T09:01:19.415 INFO:tasks.workunit.client.1.smithi019.stdout:6/713: dwrite de/d1c/f5d [0,4194304] 0 2024-03-16T09:01:19.415 INFO:tasks.workunit.client.1.smithi019.stdout:6/714: chown de/d26/d32/d97/dae/db1 18043943 1 2024-03-16T09:01:19.415 INFO:tasks.workunit.client.1.smithi019.stdout:5/607: dwrite d6/f35 [0,4194304] 0 2024-03-16T09:01:19.415 INFO:tasks.workunit.client.1.smithi019.stdout:6/715: chown de/d26/l84 3 1 2024-03-16T09:01:19.416 INFO:tasks.workunit.client.0.smithi007.stdout:0/885: unlink d3/d9/da/d48/d25/d43/d1a/f20 0 2024-03-16T09:01:19.416 INFO:tasks.workunit.client.0.smithi007.stdout:0/886: fdatasync d3/d9/da/d18/d27/d34/f89 0 2024-03-16T09:01:19.417 INFO:tasks.workunit.client.0.smithi007.stdout:0/887: chown d3/d9/da/d18/d27/d34/d3a 52660 1 2024-03-16T09:01:19.420 INFO:tasks.workunit.client.1.smithi019.stdout:3/595: mkdir d3/d1b/d40/d4a/db7 0 2024-03-16T09:01:19.420 INFO:tasks.workunit.client.1.smithi019.stdout:6/716: creat de/fe6 x:0 0 0 2024-03-16T09:01:19.427 INFO:tasks.workunit.client.0.smithi007.stdout:7/730: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/fd3 [0,4194304] 0 2024-03-16T09:01:19.427 INFO:tasks.workunit.client.0.smithi007.stdout:7/731: chown d0/d19/d1f/d5d/d9d/df7/d3a/daf/dec/f105 15092 1 2024-03-16T09:01:19.435 INFO:tasks.workunit.client.1.smithi019.stdout:3/596: write d3/d8b/dae/d5a/f8c [1119118,76132] 0 2024-03-16T09:01:19.436 INFO:tasks.workunit.client.1.smithi019.stdout:3/597: chown d3/d8b/dae/d5a/d75/c76 166039 1 2024-03-16T09:01:19.440 INFO:tasks.workunit.client.1.smithi019.stdout:3/598: symlink d3/d1b/d88/d91/lb8 0 2024-03-16T09:01:19.441 INFO:tasks.workunit.client.1.smithi019.stdout:3/599: rename d3/d8b/dae/d5a/f73 to d3/d8b/dae/d6a/fb9 0 2024-03-16T09:01:19.441 INFO:tasks.workunit.client.1.smithi019.stdout:3/600: chown d3/l28 12 1 2024-03-16T09:01:19.448 INFO:tasks.workunit.client.0.smithi007.stdout:7/732: dread d0/dc/d38/d4e/fd5 [0,4194304] 0 2024-03-16T09:01:19.450 INFO:tasks.workunit.client.1.smithi019.stdout:3/601: dread d3/d8b/dae/d3f/fa4 [0,4194304] 0 2024-03-16T09:01:19.450 INFO:tasks.workunit.client.1.smithi019.stdout:3/602: readlink d3/d8b/dae/d6a/l99 0 2024-03-16T09:01:19.450 INFO:tasks.workunit.client.1.smithi019.stdout:3/603: read - d3/d8b/dae/d3f/fad zero size 2024-03-16T09:01:19.450 INFO:tasks.workunit.client.1.smithi019.stdout:3/604: fsync d3/d1b/f6e 0 2024-03-16T09:01:19.451 INFO:tasks.workunit.client.1.smithi019.stdout:6/717: read de/d1c/fbc [7152786,58225] 0 2024-03-16T09:01:19.451 INFO:tasks.workunit.client.1.smithi019.stdout:6/718: chown de/l69 7063780 1 2024-03-16T09:01:19.451 INFO:tasks.workunit.client.1.smithi019.stdout:6/719: fdatasync de/d1c/f5d 0 2024-03-16T09:01:19.451 INFO:tasks.workunit.client.1.smithi019.stdout:6/720: fsync de/d26/d32/d97/f5a 0 2024-03-16T09:01:19.451 INFO:tasks.workunit.client.1.smithi019.stdout:3/605: link d3/d8b/dae/d5a/d75/d7a/c86 d3/d8b/dae/d5a/d75/cba 0 2024-03-16T09:01:19.452 INFO:tasks.workunit.client.1.smithi019.stdout:6/721: rename de/d26/l84 to de/d26/le7 0 2024-03-16T09:01:19.453 INFO:tasks.workunit.client.1.smithi019.stdout:6/722: rmdir de/d26/d32/d4e/d75/dcd/de0 0 2024-03-16T09:01:19.454 INFO:tasks.workunit.client.1.smithi019.stdout:6/723: creat de/d26/d32/d4e/d75/dcd/fe8 x:0 0 0 2024-03-16T09:01:19.454 INFO:tasks.workunit.client.1.smithi019.stdout:6/724: write de/f1a [483387,5362] 0 2024-03-16T09:01:19.455 INFO:tasks.workunit.client.1.smithi019.stdout:6/725: write de/d26/d32/d97/fb8 [840249,69715] 0 2024-03-16T09:01:19.455 INFO:tasks.workunit.client.1.smithi019.stdout:6/726: fdatasync de/f15 0 2024-03-16T09:01:19.455 INFO:tasks.workunit.client.0.smithi007.stdout:7/733: write d0/dc/d5a/fdc [1956318,30464] 0 2024-03-16T09:01:19.455 INFO:tasks.workunit.client.0.smithi007.stdout:7/734: write d0/d19/d1f/d5d/fc4 [215976,3488] 0 2024-03-16T09:01:19.455 INFO:tasks.workunit.client.0.smithi007.stdout:7/735: dread - d0/d19/d1f/d5d/d9d/df7/d7a/fa8 zero size 2024-03-16T09:01:19.456 INFO:tasks.workunit.client.1.smithi019.stdout:6/727: dread de/f45 [0,4194304] 0 2024-03-16T09:01:19.458 INFO:tasks.workunit.client.1.smithi019.stdout:4/618: dwrite d4/d1b/d2a/f4e [0,4194304] 0 2024-03-16T09:01:19.459 INFO:tasks.workunit.client.1.smithi019.stdout:1/559: dwrite d16/f5d [0,4194304] 0 2024-03-16T09:01:19.459 INFO:tasks.workunit.client.1.smithi019.stdout:6/728: truncate de/d26/d32/d4e/fa5 1162003 0 2024-03-16T09:01:19.459 INFO:tasks.workunit.client.1.smithi019.stdout:6/729: chown de/d26/d32/f9c 144202248 1 2024-03-16T09:01:19.459 INFO:tasks.workunit.client.1.smithi019.stdout:4/619: write d4/f26 [4735729,17434] 0 2024-03-16T09:01:19.459 INFO:tasks.workunit.client.0.smithi007.stdout:9/759: dwrite d24/d4a/d55/f5b [0,4194304] 0 2024-03-16T09:01:19.459 INFO:tasks.workunit.client.0.smithi007.stdout:9/760: write d24/f29 [1778036,26320] 0 2024-03-16T09:01:19.460 INFO:tasks.workunit.client.1.smithi019.stdout:6/730: truncate de/d1c/f20 1820586 0 2024-03-16T09:01:19.461 INFO:tasks.workunit.client.1.smithi019.stdout:1/560: rename d16/d31/d76/l80 to d16/d1f/d4f/l9d 0 2024-03-16T09:01:19.461 INFO:tasks.workunit.client.1.smithi019.stdout:1/561: fdatasync d16/d1f/f34 0 2024-03-16T09:01:19.461 INFO:tasks.workunit.client.1.smithi019.stdout:1/562: dread - d16/d31/d71/f8d zero size 2024-03-16T09:01:19.461 INFO:tasks.workunit.client.1.smithi019.stdout:1/563: dread - d16/d31/d71/f75 zero size 2024-03-16T09:01:19.462 INFO:tasks.workunit.client.0.smithi007.stdout:9/761: mkdir d24/d4a/da2/ddf 0 2024-03-16T09:01:19.462 INFO:tasks.workunit.client.1.smithi019.stdout:1/564: mknod d16/d31/d71/d89/c9e 0 2024-03-16T09:01:19.462 INFO:tasks.workunit.client.0.smithi007.stdout:1/647: dwrite d9/d6b/d7e/dac/dbb/fbe [0,4194304] 0 2024-03-16T09:01:19.463 INFO:tasks.workunit.client.0.smithi007.stdout:7/736: dread d0/d19/d1f/d69/f6b [0,4194304] 0 2024-03-16T09:01:19.464 INFO:tasks.workunit.client.0.smithi007.stdout:2/887: dwrite dc/df/d24/d28/d2e/f5c [0,4194304] 0 2024-03-16T09:01:19.465 INFO:tasks.workunit.client.1.smithi019.stdout:6/731: rename de/d26/d32/d97/d66/d72/d7f/fc9 to de/d26/d32/d97/d66/d72/d7f/dd5/fe9 0 2024-03-16T09:01:19.465 INFO:tasks.workunit.client.1.smithi019.stdout:1/565: truncate d16/f42 2183945 0 2024-03-16T09:01:19.465 INFO:tasks.workunit.client.1.smithi019.stdout:3/606: dread d3/d1b/d40/f9d [4194304,4194304] 0 2024-03-16T09:01:19.466 INFO:tasks.workunit.client.1.smithi019.stdout:1/566: stat d16/f2b 0 2024-03-16T09:01:19.466 INFO:tasks.workunit.client.1.smithi019.stdout:1/567: write d16/d31/d71/f75 [342675,26392] 0 2024-03-16T09:01:19.466 INFO:tasks.workunit.client.0.smithi007.stdout:1/648: unlink d9/d6b/d7e/dac/cc5 0 2024-03-16T09:01:19.467 INFO:tasks.workunit.client.0.smithi007.stdout:9/762: mknod d24/d4c/ce0 0 2024-03-16T09:01:19.468 INFO:tasks.workunit.client.0.smithi007.stdout:5/766: dwrite d1/d7/d1c/d19/d34/d1d/f9e [0,4194304] 0 2024-03-16T09:01:19.468 INFO:tasks.workunit.client.0.smithi007.stdout:5/767: creat d1/d7/d1c/d19/d34/d3e/f117 x:0 0 0 2024-03-16T09:01:19.469 INFO:tasks.workunit.client.1.smithi019.stdout:5/608: dwrite d6/d1f/f5e [4194304,4194304] 0 2024-03-16T09:01:19.469 INFO:tasks.workunit.client.1.smithi019.stdout:0/630: write d7/d8/d32/d66/d6d/d8c/db8/f69 [1190185,21225] 0 2024-03-16T09:01:19.469 INFO:tasks.workunit.client.1.smithi019.stdout:0/631: chown d7/d1d/d63/la2 0 1 2024-03-16T09:01:19.469 INFO:tasks.workunit.client.0.smithi007.stdout:7/737: mkdir d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108 0 2024-03-16T09:01:19.469 INFO:tasks.workunit.client.0.smithi007.stdout:5/768: write d1/d7/d1c/d19/d34/d3e/d41/da6/fc9 [686893,7826] 0 2024-03-16T09:01:19.470 INFO:tasks.workunit.client.1.smithi019.stdout:9/478: dwrite d1/d7/d33/d59/f89 [0,4194304] 0 2024-03-16T09:01:19.470 INFO:tasks.workunit.client.1.smithi019.stdout:8/650: dwrite d6/d2a/d45/f32 [0,4194304] 0 2024-03-16T09:01:19.470 INFO:tasks.workunit.client.1.smithi019.stdout:4/620: read d4/d6/d15/d1f/fb8 [2091898,57800] 0 2024-03-16T09:01:19.470 INFO:tasks.workunit.client.1.smithi019.stdout:4/621: readlink d4/d1b/l45 0 2024-03-16T09:01:19.470 INFO:tasks.workunit.client.1.smithi019.stdout:4/622: write d4/d64/d99/f6b [8838059,93671] 0 2024-03-16T09:01:19.471 INFO:tasks.workunit.client.1.smithi019.stdout:4/623: readlink d4/lb 0 2024-03-16T09:01:19.471 INFO:tasks.workunit.client.1.smithi019.stdout:4/624: chown d4/d1b/d2a/d3d/l86 4418374 1 2024-03-16T09:01:19.471 INFO:tasks.workunit.client.1.smithi019.stdout:4/625: readlink d4/d6/d15/d1f/l94 0 2024-03-16T09:01:19.471 INFO:tasks.workunit.client.1.smithi019.stdout:4/626: fsync d4/d6/d15/d1f/fbd 0 2024-03-16T09:01:19.471 INFO:tasks.workunit.client.1.smithi019.stdout:4/627: creat d4/d1b/d2a/d4f/fcf x:0 0 0 2024-03-16T09:01:19.472 INFO:tasks.workunit.client.0.smithi007.stdout:2/888: rename dc/df/d24/d28/d2e/d35/c50 to dc/df/d24/d28/d2e/d6e/d114/c12b 0 2024-03-16T09:01:19.472 INFO:tasks.workunit.client.0.smithi007.stdout:2/889: stat dc/df/d6c/fcd 0 2024-03-16T09:01:19.472 INFO:tasks.workunit.client.1.smithi019.stdout:5/609: rename f4 to d6/dd/daf/fbe 0 2024-03-16T09:01:19.473 INFO:tasks.workunit.client.0.smithi007.stdout:9/763: mkdir d24/d4a/da2/de1 0 2024-03-16T09:01:19.473 INFO:tasks.workunit.client.0.smithi007.stdout:9/764: truncate d24/d4c/d86/daa/d7f/d8f/db5/fc9 727248 0 2024-03-16T09:01:19.474 INFO:tasks.workunit.client.1.smithi019.stdout:6/732: dread de/f49 [0,4194304] 0 2024-03-16T09:01:19.476 INFO:tasks.workunit.client.0.smithi007.stdout:9/765: write d24/d4a/da2/db7/fc3 [649181,124843] 0 2024-03-16T09:01:19.476 INFO:tasks.workunit.client.0.smithi007.stdout:9/766: creat d24/d4a/d96/db9/fe2 x:0 0 0 2024-03-16T09:01:19.476 INFO:tasks.workunit.client.1.smithi019.stdout:0/632: symlink d7/d1d/d28/da1/da7/ld4 0 2024-03-16T09:01:19.478 INFO:tasks.workunit.client.0.smithi007.stdout:5/769: creat d1/d7/d1c/d19/d34/d3e/d41/f118 x:0 0 0 2024-03-16T09:01:19.478 INFO:tasks.workunit.client.0.smithi007.stdout:5/770: stat d1/d7/cb9 0 2024-03-16T09:01:19.484 INFO:tasks.workunit.client.1.smithi019.stdout:9/479: dread d1/d7/f51 [0,4194304] 0 2024-03-16T09:01:19.485 INFO:tasks.workunit.client.1.smithi019.stdout:9/480: readlink d1/d7/de/l5c 0 2024-03-16T09:01:19.485 INFO:tasks.workunit.client.1.smithi019.stdout:9/481: write d1/d7/de/d24/d44/d76/f9a [400599,10246] 0 2024-03-16T09:01:19.487 INFO:tasks.workunit.client.1.smithi019.stdout:8/651: mknod d6/d2a/d45/d57/d72/d79/cda 0 2024-03-16T09:01:19.491 INFO:tasks.workunit.client.1.smithi019.stdout:5/610: symlink d6/dd/d47/d7a/lbf 0 2024-03-16T09:01:19.491 INFO:tasks.workunit.client.1.smithi019.stdout:5/611: chown d6/dd/d47/d70/db2/d66/c71 6706843 1 2024-03-16T09:01:19.492 INFO:tasks.workunit.client.1.smithi019.stdout:6/733: truncate de/d26/d32/d4e/d75/fa2 3506629 0 2024-03-16T09:01:19.497 INFO:tasks.workunit.client.0.smithi007.stdout:5/771: getdents d1/d7/d16/d1b 0 2024-03-16T09:01:19.497 INFO:tasks.workunit.client.0.smithi007.stdout:5/772: creat d1/d7/d1c/d19/d34/d3e/d41/da6/f119 x:0 0 0 2024-03-16T09:01:19.499 INFO:tasks.workunit.client.1.smithi019.stdout:8/652: truncate d6/d2a/d45/d57/f65 353993 0 2024-03-16T09:01:19.499 INFO:tasks.workunit.client.1.smithi019.stdout:8/653: fdatasync d6/d2a/d45/d19/f1a 0 2024-03-16T09:01:19.500 INFO:tasks.workunit.client.1.smithi019.stdout:6/734: creat de/d1c/fea x:0 0 0 2024-03-16T09:01:19.501 INFO:tasks.workunit.client.0.smithi007.stdout:5/773: rename d1/d7/d16/d1b/d3c/fa7 to d1/d7/d1c/d88/da2/d89/db3/d2a/d81/f11a 0 2024-03-16T09:01:19.501 INFO:tasks.workunit.client.0.smithi007.stdout:5/774: fsync d1/d7/d16/d1b/f60 0 2024-03-16T09:01:19.501 INFO:tasks.workunit.client.0.smithi007.stdout:5/775: stat d1/d7/d1c/d88/da2/d89/db3/l65 0 2024-03-16T09:01:19.501 INFO:tasks.workunit.client.0.smithi007.stdout:5/776: write d1/f2 [2581385,86858] 0 2024-03-16T09:01:19.502 INFO:tasks.workunit.client.1.smithi019.stdout:0/633: rmdir d7/d1d/d2e 39 2024-03-16T09:01:19.504 INFO:tasks.workunit.client.1.smithi019.stdout:6/735: rename de/d26/d32/d97/d51/d70/f91 to de/d26/d32/d97/d66/d72/db7/feb 0 2024-03-16T09:01:19.504 INFO:tasks.workunit.client.1.smithi019.stdout:5/612: dread d6/d19/f9b [0,4194304] 0 2024-03-16T09:01:19.506 INFO:tasks.workunit.client.1.smithi019.stdout:0/634: truncate d7/d1d/d28/d6c/f77 2592380 0 2024-03-16T09:01:19.509 INFO:tasks.workunit.client.1.smithi019.stdout:6/736: symlink de/d26/d32/d97/d66/d72/db7/lec 0 2024-03-16T09:01:19.511 INFO:tasks.workunit.client.1.smithi019.stdout:0/635: creat d7/d1d/d28/d92/da6/fd5 x:0 0 0 2024-03-16T09:01:19.511 INFO:tasks.workunit.client.1.smithi019.stdout:0/636: chown d7/d8/d32/d66/c7e 60071787 1 2024-03-16T09:01:19.513 INFO:tasks.workunit.client.1.smithi019.stdout:0/637: mknod d7/d1d/d28/d6c/d87/da3/cd6 0 2024-03-16T09:01:19.519 INFO:tasks.workunit.client.1.smithi019.stdout:8/654: dread d6/d2a/d45/d57/f60 [0,4194304] 0 2024-03-16T09:01:19.519 INFO:tasks.workunit.client.1.smithi019.stdout:8/655: chown d6/d2a/d3c/dab/d30/d4a/d67/d95/d9a 36285 1 2024-03-16T09:01:19.520 INFO:tasks.workunit.client.1.smithi019.stdout:8/656: rename d6/d2a/f8d to d6/fdb 0 2024-03-16T09:01:19.522 INFO:tasks.workunit.client.1.smithi019.stdout:8/657: symlink d6/d2a/d45/d57/d72/d79/db7/ldc 0 2024-03-16T09:01:19.523 INFO:tasks.workunit.client.1.smithi019.stdout:0/638: write d7/d8/f50 [2301331,117108] 0 2024-03-16T09:01:19.524 INFO:tasks.workunit.client.1.smithi019.stdout:0/639: readlink d7/l96 0 2024-03-16T09:01:19.524 INFO:tasks.workunit.client.1.smithi019.stdout:0/640: fsync d7/d1d/d2e/fc5 0 2024-03-16T09:01:19.526 INFO:tasks.workunit.client.1.smithi019.stdout:0/641: creat d7/d8/d32/d66/d6d/d8c/db5/fd7 x:0 0 0 2024-03-16T09:01:19.526 INFO:tasks.workunit.client.0.smithi007.stdout:7/738: dwrite d0/d19/d1f/f4b [0,4194304] 0 2024-03-16T09:01:19.526 INFO:tasks.workunit.client.0.smithi007.stdout:7/739: chown d0/d19/d1f/d5d/d9d/df7/d73/f5c 887 1 2024-03-16T09:01:19.528 INFO:tasks.workunit.client.1.smithi019.stdout:0/642: mknod d7/cd8 0 2024-03-16T09:01:19.528 INFO:tasks.workunit.client.1.smithi019.stdout:0/643: chown d7/d8/d32/d66/d6d/d7c/ca0 231906697 1 2024-03-16T09:01:19.528 INFO:tasks.workunit.client.1.smithi019.stdout:0/644: stat d7/d8/d32/d66/d6d/d8c/db8/d82/f81 0 2024-03-16T09:01:19.529 INFO:tasks.workunit.client.1.smithi019.stdout:0/645: creat d7/d1d/d28/d6c/d87/da3/fd9 x:0 0 0 2024-03-16T09:01:19.530 INFO:tasks.workunit.client.1.smithi019.stdout:0/646: mknod d7/d1d/d28/d6c/cda 0 2024-03-16T09:01:19.532 INFO:tasks.workunit.client.0.smithi007.stdout:7/740: write d0/f3c [958683,65278] 0 2024-03-16T09:01:19.534 INFO:tasks.workunit.client.1.smithi019.stdout:0/647: dread d7/d1d/d28/d48/f65 [0,4194304] 0 2024-03-16T09:01:19.534 INFO:tasks.workunit.client.0.smithi007.stdout:7/741: symlink d0/d19/d1f/d69/deb/l109 0 2024-03-16T09:01:19.534 INFO:tasks.workunit.client.0.smithi007.stdout:7/742: fdatasync d0/d19/d1f/d69/fc6 0 2024-03-16T09:01:19.534 INFO:tasks.workunit.client.1.smithi019.stdout:0/648: creat d7/d1d/d28/d92/dc6/fdb x:0 0 0 2024-03-16T09:01:19.538 INFO:tasks.workunit.client.1.smithi019.stdout:8/658: write d6/d2a/d3c/dab/d30/d4a/fcf [1684410,92067] 0 2024-03-16T09:01:19.541 INFO:tasks.workunit.client.1.smithi019.stdout:8/659: link d6/de/f49 d6/d2a/d3c/dab/d30/d4a/d67/fdd 0 2024-03-16T09:01:19.542 INFO:tasks.workunit.client.1.smithi019.stdout:8/660: chown d6/d2a/d45/d2c 31310 1 2024-03-16T09:01:19.542 INFO:tasks.workunit.client.1.smithi019.stdout:8/661: write d6/d2a/d45/fd7 [434207,98279] 0 2024-03-16T09:01:19.542 INFO:tasks.workunit.client.1.smithi019.stdout:3/607: dwrite d3/d8b/dae/f72 [4194304,4194304] 0 2024-03-16T09:01:19.542 INFO:tasks.workunit.client.1.smithi019.stdout:3/608: write d3/d11/f14 [8181375,21587] 0 2024-03-16T09:01:19.543 INFO:tasks.workunit.client.0.smithi007.stdout:5/777: dwrite d1/d7/d1c/d19/d34/d4d/f86 [0,4194304] 0 2024-03-16T09:01:19.543 INFO:tasks.workunit.client.0.smithi007.stdout:5/778: getdents d1/d7/d16/d1b/df1 0 2024-03-16T09:01:19.543 INFO:tasks.workunit.client.0.smithi007.stdout:5/779: dread - d1/d7/d1c/d88/da2/d89/db3/d2a/f61 zero size 2024-03-16T09:01:19.543 INFO:tasks.workunit.client.0.smithi007.stdout:3/754: dwrite d15/f2c [0,4194304] 0 2024-03-16T09:01:19.544 INFO:tasks.workunit.client.1.smithi019.stdout:9/482: dwrite d1/d7/d67/d77/fa4 [0,4194304] 0 2024-03-16T09:01:19.544 INFO:tasks.workunit.client.1.smithi019.stdout:9/483: write d1/d7/f45 [2462347,106457] 0 2024-03-16T09:01:19.544 INFO:tasks.workunit.client.1.smithi019.stdout:8/662: creat d6/d2a/d45/d57/d72/d79/db7/fde x:0 0 0 2024-03-16T09:01:19.544 INFO:tasks.workunit.client.0.smithi007.stdout:2/890: dwrite dc/df/d24/d28/d2e/d35/f91 [0,4194304] 0 2024-03-16T09:01:19.545 INFO:tasks.workunit.client.0.smithi007.stdout:2/891: write dc/df/d24/d28/dea/d102/f122 [581932,2480] 0 2024-03-16T09:01:19.545 INFO:tasks.workunit.client.0.smithi007.stdout:9/767: dwrite d24/f43 [0,4194304] 0 2024-03-16T09:01:19.545 INFO:tasks.workunit.client.1.smithi019.stdout:6/737: dwrite de/fe6 [0,4194304] 0 2024-03-16T09:01:19.547 INFO:tasks.workunit.client.1.smithi019.stdout:3/609: creat d3/d8b/dae/d5a/fbb x:0 0 0 2024-03-16T09:01:19.548 INFO:tasks.workunit.client.0.smithi007.stdout:3/755: creat d15/d18/d1f/d3b/dae/fec x:0 0 0 2024-03-16T09:01:19.549 INFO:tasks.workunit.client.0.smithi007.stdout:9/768: dread d24/d4a/d55/f5d [0,4194304] 0 2024-03-16T09:01:19.551 INFO:tasks.workunit.client.1.smithi019.stdout:9/484: creat d1/d7/d67/d77/faa x:0 0 0 2024-03-16T09:01:19.552 INFO:tasks.workunit.client.0.smithi007.stdout:9/769: mkdir d24/d4c/d86/daa/d7f/d80/de3 0 2024-03-16T09:01:19.552 INFO:tasks.workunit.client.0.smithi007.stdout:9/770: read d24/f27 [3765591,51938] 0 2024-03-16T09:01:19.552 INFO:tasks.workunit.client.0.smithi007.stdout:3/756: dread d15/d18/d1f/d3b/d63/fb7 [0,4194304] 0 2024-03-16T09:01:19.553 INFO:tasks.workunit.client.1.smithi019.stdout:8/663: creat d6/d2a/d45/d57/d96/fdf x:0 0 0 2024-03-16T09:01:19.555 INFO:tasks.workunit.client.1.smithi019.stdout:6/738: mkdir de/d26/d73/ded 0 2024-03-16T09:01:19.557 INFO:tasks.workunit.client.1.smithi019.stdout:6/739: dread de/f45 [0,4194304] 0 2024-03-16T09:01:19.557 INFO:tasks.workunit.client.1.smithi019.stdout:3/610: unlink d3/d1b/d40/c82 0 2024-03-16T09:01:19.557 INFO:tasks.workunit.client.1.smithi019.stdout:3/611: chown d3/d1b/d40/d4a/f98 71698393 1 2024-03-16T09:01:19.557 INFO:tasks.workunit.client.1.smithi019.stdout:3/612: getdents d3/d11 0 2024-03-16T09:01:19.558 INFO:tasks.workunit.client.0.smithi007.stdout:2/892: dwrite dc/df/fca [0,4194304] 0 2024-03-16T09:01:19.558 INFO:tasks.workunit.client.0.smithi007.stdout:2/893: fsync dc/f11 0 2024-03-16T09:01:19.559 INFO:tasks.workunit.client.1.smithi019.stdout:9/485: link d1/d7/d67/d77/fa4 d1/d7/d21/fab 0 2024-03-16T09:01:19.559 INFO:tasks.workunit.client.1.smithi019.stdout:9/486: readlink d1/d7/de/l1b 0 2024-03-16T09:01:19.560 INFO:tasks.workunit.client.1.smithi019.stdout:8/664: mkdir d6/d2a/d3c/dab/d30/d4a/d67/d95/de0 0 2024-03-16T09:01:19.560 INFO:tasks.workunit.client.1.smithi019.stdout:8/665: creat d6/d2a/d45/d57/d72/d79/db7/fe1 x:0 0 0 2024-03-16T09:01:19.560 INFO:tasks.workunit.client.1.smithi019.stdout:8/666: chown d6/d2a/d45/d19/d74/f7e 1015744837 1 2024-03-16T09:01:19.561 INFO:tasks.workunit.client.1.smithi019.stdout:6/740: stat de/d26/d32/d4e/d57/c8c 0 2024-03-16T09:01:19.561 INFO:tasks.workunit.client.1.smithi019.stdout:6/741: readlink de/d26/d32/d97/d66/lad 0 2024-03-16T09:01:19.562 INFO:tasks.workunit.client.1.smithi019.stdout:3/613: creat d3/d89/fbc x:0 0 0 2024-03-16T09:01:19.563 INFO:tasks.workunit.client.1.smithi019.stdout:3/614: fdatasync d3/d9b/f9f 0 2024-03-16T09:01:19.563 INFO:tasks.workunit.client.0.smithi007.stdout:2/894: unlink dc/df/daa/fe2 0 2024-03-16T09:01:19.564 INFO:tasks.workunit.client.0.smithi007.stdout:2/895: mkdir dc/df/daa/d12c 0 2024-03-16T09:01:19.564 INFO:tasks.workunit.client.0.smithi007.stdout:2/896: chown dc/db7 27689 1 2024-03-16T09:01:19.566 INFO:tasks.workunit.client.1.smithi019.stdout:8/667: creat d6/d2a/d45/d57/d72/d9d/fe2 x:0 0 0 2024-03-16T09:01:19.566 INFO:tasks.workunit.client.0.smithi007.stdout:2/897: dread dc/df/d6c/fcd [0,4194304] 0 2024-03-16T09:01:19.567 INFO:tasks.workunit.client.1.smithi019.stdout:0/649: dwrite d7/d8/d32/d4a/d61/f27 [4194304,4194304] 0 2024-03-16T09:01:19.567 INFO:tasks.workunit.client.1.smithi019.stdout:0/650: readlink d7/d8/d32/d66/d6d/d8c/db8/l73 0 2024-03-16T09:01:19.567 INFO:tasks.workunit.client.1.smithi019.stdout:0/651: chown d7/d1d/d63/d89/d9e/fc7 75817 1 2024-03-16T09:01:19.567 INFO:tasks.workunit.client.1.smithi019.stdout:0/652: write d7/d8/d32/d66/d6d/d7c/fcd [389200,106807] 0 2024-03-16T09:01:19.567 INFO:tasks.workunit.client.1.smithi019.stdout:6/742: link de/d26/d32/d97/d66/lad de/d26/d73/ded/lee 0 2024-03-16T09:01:19.567 INFO:tasks.workunit.client.1.smithi019.stdout:6/743: chown de/d26/d32/d97/d2c/c48 21051701 1 2024-03-16T09:01:19.568 INFO:tasks.workunit.client.0.smithi007.stdout:3/757: write d15/d18/d1f/d3b/f57 [3327734,115751] 0 2024-03-16T09:01:19.569 INFO:tasks.workunit.client.0.smithi007.stdout:7/743: dwrite d0/d19/d1f/d5d/d9d/df7/fcf [0,4194304] 0 2024-03-16T09:01:19.569 INFO:tasks.workunit.client.0.smithi007.stdout:7/744: dread - d0/dc/fca zero size 2024-03-16T09:01:19.569 INFO:tasks.workunit.client.0.smithi007.stdout:7/745: write d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d9c/fe2 [597983,88861] 0 2024-03-16T09:01:19.570 INFO:tasks.workunit.client.1.smithi019.stdout:8/668: rmdir d6/d2a/d45/d57/d72/d79 39 2024-03-16T09:01:19.572 INFO:tasks.workunit.client.0.smithi007.stdout:3/758: unlink d15/d18/d1f/d3b/f57 0 2024-03-16T09:01:19.574 INFO:tasks.workunit.client.0.smithi007.stdout:2/898: rename dc/df/c41 to dc/df/d24/d28/d97/d99/c12d 0 2024-03-16T09:01:19.575 INFO:tasks.workunit.client.0.smithi007.stdout:2/899: fdatasync dc/df/d24/d28/d2e/f78 0 2024-03-16T09:01:19.575 INFO:tasks.workunit.client.0.smithi007.stdout:9/771: dread d24/d4c/d86/daa/d95/f7b [0,4194304] 0 2024-03-16T09:01:19.579 INFO:tasks.workunit.client.0.smithi007.stdout:3/759: getdents d15/d18/d1f/d29/d3e/d4c 0 2024-03-16T09:01:19.579 INFO:tasks.workunit.client.0.smithi007.stdout:3/760: creat d15/d18/d1f/d29/d43/fed x:0 0 0 2024-03-16T09:01:19.582 INFO:tasks.workunit.client.0.smithi007.stdout:2/900: creat dc/df/f12e x:0 0 0 2024-03-16T09:01:19.582 INFO:tasks.workunit.client.0.smithi007.stdout:2/901: chown dc/df/f5a 449 1 2024-03-16T09:01:19.582 INFO:tasks.workunit.client.0.smithi007.stdout:2/902: chown dc/df/d24/d28/d97/da4/db8 1921 1 2024-03-16T09:01:19.583 INFO:tasks.workunit.client.0.smithi007.stdout:7/746: write d0/d19/f48 [351800,34061] 0 2024-03-16T09:01:19.583 INFO:tasks.workunit.client.0.smithi007.stdout:7/747: chown d0/d19/d1f/d5d/d9d/df7/d3a/daf/dc2/cff 3 1 2024-03-16T09:01:19.584 INFO:tasks.workunit.client.0.smithi007.stdout:3/761: mkdir d15/d18/d1f/d3b/d87/dee 0 2024-03-16T09:01:19.587 INFO:tasks.workunit.client.0.smithi007.stdout:5/780: write d1/d7/d1c/d19/d34/d3e/d41/f71 [669534,95654] 0 2024-03-16T09:01:19.590 INFO:tasks.workunit.client.1.smithi019.stdout:8/669: getdents d6/d2a/d3c/dab/d30/d4a/d67/d95 0 2024-03-16T09:01:19.590 INFO:tasks.workunit.client.1.smithi019.stdout:8/670: chown d6/d2a/d3c/f44 220 1 2024-03-16T09:01:19.590 INFO:tasks.workunit.client.1.smithi019.stdout:8/671: write d6/f77 [440272,47811] 0 2024-03-16T09:01:19.591 INFO:tasks.workunit.client.1.smithi019.stdout:3/615: fsync d3/d89/fbc 0 2024-03-16T09:01:19.591 INFO:tasks.workunit.client.1.smithi019.stdout:8/672: mkdir d6/d2a/d45/d2c/de3 0 2024-03-16T09:01:19.592 INFO:tasks.workunit.client.0.smithi007.stdout:3/762: mkdir d15/d18/d1f/d29/d3e/d85/def 0 2024-03-16T09:01:19.592 INFO:tasks.workunit.client.1.smithi019.stdout:3/616: mknod d3/d1b/d40/d4a/db7/cbd 0 2024-03-16T09:01:19.594 INFO:tasks.workunit.client.1.smithi019.stdout:8/673: creat d6/d2a/d3c/dab/d30/d4a/d67/d95/fe4 x:0 0 0 2024-03-16T09:01:19.595 INFO:tasks.workunit.client.1.smithi019.stdout:8/674: symlink d6/d2a/d45/d19/d74/le5 0 2024-03-16T09:01:19.596 INFO:tasks.workunit.client.1.smithi019.stdout:8/675: symlink d6/d2a/d3c/dab/d30/d4a/d67/d95/de0/le6 0 2024-03-16T09:01:19.600 INFO:tasks.workunit.client.1.smithi019.stdout:8/676: rename d6/de/f49 to d6/d2a/d45/d57/d72/d79/db7/fe7 0 2024-03-16T09:01:19.629 INFO:tasks.workunit.client.1.smithi019.stdout:0/653: dwrite d7/d1d/d2e/f39 [0,4194304] 0 2024-03-16T09:01:19.629 INFO:tasks.workunit.client.1.smithi019.stdout:3/617: dwrite d3/d8b/dae/d6a/f7b [0,4194304] 0 2024-03-16T09:01:19.629 INFO:tasks.workunit.client.0.smithi007.stdout:9/772: dwrite d24/d4c/d86/daa/d7f/d8f/db5/fc7 [0,4194304] 0 2024-03-16T09:01:19.630 INFO:tasks.workunit.client.0.smithi007.stdout:7/748: dwrite d0/dc/d5a/dba/dd9/fb9 [0,4194304] 0 2024-03-16T09:01:19.630 INFO:tasks.workunit.client.0.smithi007.stdout:7/749: readlink d0/d6c/ldf 0 2024-03-16T09:01:19.630 INFO:tasks.workunit.client.0.smithi007.stdout:9/773: read - d24/d4c/d86/daa/d7f/d8f/fb8 zero size 2024-03-16T09:01:19.630 INFO:tasks.workunit.client.0.smithi007.stdout:7/750: write d0/d19/d1f/d5d/d9d/df7/da1/fab [291335,90191] 0 2024-03-16T09:01:19.630 INFO:tasks.workunit.client.0.smithi007.stdout:7/751: creat d0/dc/d5a/dba/dd9/d93/f10a x:0 0 0 2024-03-16T09:01:19.630 INFO:tasks.workunit.client.0.smithi007.stdout:3/763: dwrite d15/d18/d1f/d3b/fca [0,4194304] 0 2024-03-16T09:01:19.630 INFO:tasks.workunit.client.1.smithi019.stdout:8/677: dwrite d6/d2a/d45/d57/d72/d79/db7/fe7 [0,4194304] 0 2024-03-16T09:01:19.632 INFO:tasks.workunit.client.1.smithi019.stdout:3/618: mkdir d3/d1b/d88/d91/dbe 0 2024-03-16T09:01:19.633 INFO:tasks.workunit.client.1.smithi019.stdout:8/678: symlink d6/de/le8 0 2024-03-16T09:01:19.637 INFO:tasks.workunit.client.1.smithi019.stdout:3/619: link d3/d8b/dae/d5a/l6d d3/d1b/d40/lbf 0 2024-03-16T09:01:19.637 INFO:tasks.workunit.client.1.smithi019.stdout:3/620: fsync d3/d8b/dae/d6a/f70 0 2024-03-16T09:01:19.643 INFO:tasks.workunit.client.0.smithi007.stdout:7/752: mknod d0/d19/d1f/d5d/d9d/df8/c10b 0 2024-03-16T09:01:19.643 INFO:tasks.workunit.client.0.smithi007.stdout:7/753: write d0/dc/d38/d4e/f68 [1132591,60711] 0 2024-03-16T09:01:19.643 INFO:tasks.workunit.client.1.smithi019.stdout:2/674: sync 2024-03-16T09:01:19.644 INFO:tasks.workunit.client.1.smithi019.stdout:7/559: sync 2024-03-16T09:01:19.644 INFO:tasks.workunit.client.1.smithi019.stdout:7/560: fdatasync d9/db/d12/f55 0 2024-03-16T09:01:19.644 INFO:tasks.workunit.client.1.smithi019.stdout:7/561: write d9/da/d13/d18/fac [4699572,98197] 0 2024-03-16T09:01:19.645 INFO:tasks.workunit.client.0.smithi007.stdout:9/774: dread d24/f54 [4194304,4194304] 0 2024-03-16T09:01:19.646 INFO:tasks.workunit.client.1.smithi019.stdout:2/675: symlink d5/d4e/d78/le3 0 2024-03-16T09:01:19.646 INFO:tasks.workunit.client.0.smithi007.stdout:3/764: mkdir d15/d18/d1f/d56/d98/df0 0 2024-03-16T09:01:19.648 INFO:tasks.workunit.client.1.smithi019.stdout:7/562: mknod d9/da/d13/d18/d30/d75/cb2 0 2024-03-16T09:01:19.649 INFO:tasks.workunit.client.0.smithi007.stdout:9/775: link d24/d4a/d96/fa8 d24/d4c/d86/fe4 0 2024-03-16T09:01:19.649 INFO:tasks.workunit.client.0.smithi007.stdout:9/776: chown d24/f43 321 1 2024-03-16T09:01:19.649 INFO:tasks.workunit.client.0.smithi007.stdout:9/777: write d24/d4c/d86/daa/d7f/d80/f8c [3443426,15367] 0 2024-03-16T09:01:19.649 INFO:tasks.workunit.client.1.smithi019.stdout:2/676: symlink d5/d6/d2b/d53/le4 0 2024-03-16T09:01:19.650 INFO:tasks.workunit.client.1.smithi019.stdout:2/677: mknod d5/d6/d64/d6f/d87/ce5 0 2024-03-16T09:01:19.651 INFO:tasks.workunit.client.0.smithi007.stdout:9/778: creat d24/d4c/d86/daa/fe5 x:0 0 0 2024-03-16T09:01:19.652 INFO:tasks.workunit.client.1.smithi019.stdout:2/678: mkdir d5/d5a/de6 0 2024-03-16T09:01:19.652 INFO:tasks.workunit.client.0.smithi007.stdout:9/779: symlink d24/d4a/da2/dd5/le6 0 2024-03-16T09:01:19.654 INFO:tasks.workunit.client.1.smithi019.stdout:2/679: creat d5/d6/d64/d6f/d87/fe7 x:0 0 0 2024-03-16T09:01:19.654 INFO:tasks.workunit.client.0.smithi007.stdout:9/780: mknod d24/d4c/d86/daa/d7f/d80/d87/d89/ce7 0 2024-03-16T09:01:19.656 INFO:tasks.workunit.client.1.smithi019.stdout:2/680: unlink d5/d6/d2b/d2f/f3a 0 2024-03-16T09:01:19.656 INFO:tasks.workunit.client.1.smithi019.stdout:0/654: dwrite d7/d1d/d28/d48/f4e [0,4194304] 0 2024-03-16T09:01:19.656 INFO:tasks.workunit.client.1.smithi019.stdout:0/655: fsync d7/d1d/d63/d6e/fc9 0 2024-03-16T09:01:19.657 INFO:tasks.workunit.client.1.smithi019.stdout:2/681: mkdir d5/d5a/d96/dba/de8 0 2024-03-16T09:01:19.657 INFO:tasks.workunit.client.1.smithi019.stdout:0/656: creat d7/d1d/d28/d92/dcc/fdc x:0 0 0 2024-03-16T09:01:19.658 INFO:tasks.workunit.client.1.smithi019.stdout:0/657: chown d7/d1d/d28/d92/da6/fbb 1879950 1 2024-03-16T09:01:19.661 INFO:tasks.workunit.client.0.smithi007.stdout:9/781: dread d24/d4a/f74 [0,4194304] 0 2024-03-16T09:01:19.664 INFO:tasks.workunit.client.0.smithi007.stdout:9/782: mkdir d24/d4a/d96/db9/dca/ddc/de8 0 2024-03-16T09:01:19.665 INFO:tasks.workunit.client.1.smithi019.stdout:7/563: dread f4 [0,4194304] 0 2024-03-16T09:01:19.665 INFO:tasks.workunit.client.1.smithi019.stdout:8/679: dwrite d6/fdb [0,4194304] 0 2024-03-16T09:01:19.666 INFO:tasks.workunit.client.1.smithi019.stdout:7/564: truncate d9/da/d13/d18/d1a/f5f 3501887 0 2024-03-16T09:01:19.668 INFO:tasks.workunit.client.1.smithi019.stdout:8/680: link d6/d2a/d3c/dab/f90 d6/d2a/d55/fe9 0 2024-03-16T09:01:19.668 INFO:tasks.workunit.client.1.smithi019.stdout:8/681: creat d6/de/fea x:0 0 0 2024-03-16T09:01:19.671 INFO:tasks.workunit.client.1.smithi019.stdout:7/565: rename d9/d25/f42 to d9/da/d13/d18/d30/d75/fb3 0 2024-03-16T09:01:19.671 INFO:tasks.workunit.client.1.smithi019.stdout:7/566: readlink d9/da/d13/d18/d30/d3e/d5d/l99 0 2024-03-16T09:01:19.673 INFO:tasks.workunit.client.1.smithi019.stdout:8/682: dread d6/d2a/d45/d5d/fb3 [0,4194304] 0 2024-03-16T09:01:19.673 INFO:tasks.workunit.client.1.smithi019.stdout:8/683: chown d6/d2a/d3c/dab/d30/d4a/c5a 24136 1 2024-03-16T09:01:19.674 INFO:tasks.workunit.client.1.smithi019.stdout:7/567: link d9/da/d13/d18/fac d9/db/d12/d5b/fb4 0 2024-03-16T09:01:19.675 INFO:tasks.workunit.client.1.smithi019.stdout:7/568: write d9/d25/d28/f88 [1003497,75209] 0 2024-03-16T09:01:19.675 INFO:tasks.workunit.client.1.smithi019.stdout:7/569: dread - d9/d25/d28/fa6 zero size 2024-03-16T09:01:19.675 INFO:tasks.workunit.client.1.smithi019.stdout:7/570: creat d9/d25/fb5 x:0 0 0 2024-03-16T09:01:19.675 INFO:tasks.workunit.client.1.smithi019.stdout:8/684: unlink d6/d2a/d45/d57/d72/d79/d7b/f93 0 2024-03-16T09:01:19.677 INFO:tasks.workunit.client.1.smithi019.stdout:8/685: rename d6/d2a/d45/d57/d72/d9d/fe2 to d6/d2a/d3c/dab/d30/d4a/d67/feb 0 2024-03-16T09:01:19.680 INFO:tasks.workunit.client.1.smithi019.stdout:8/686: rename d6/d2a/d45/d57/d72/d79/dc1/fc0 to d6/d2a/d45/d57/d72/d79/da5/fec 0 2024-03-16T09:01:19.680 INFO:tasks.workunit.client.1.smithi019.stdout:8/687: creat d6/d2a/d45/d57/d96/fed x:0 0 0 2024-03-16T09:01:19.682 INFO:tasks.workunit.client.1.smithi019.stdout:8/688: unlink d6/d2a/d68/l8c 0 2024-03-16T09:01:19.682 INFO:tasks.workunit.client.1.smithi019.stdout:8/689: readlink d6/d2a/d45/d57/d72/d79/d7b/lc2 0 2024-03-16T09:01:19.682 INFO:tasks.workunit.client.1.smithi019.stdout:8/690: mkdir d6/d2a/d3c/dab/d30/d4a/d67/dee 0 2024-03-16T09:01:19.682 INFO:tasks.workunit.client.1.smithi019.stdout:8/691: write d6/d2a/d45/f42 [903998,44106] 0 2024-03-16T09:01:19.683 INFO:tasks.workunit.client.1.smithi019.stdout:8/692: link d6/d2a/d45/d57/d72/d79/db7/fe1 d6/d2a/d3c/dab/d30/d4a/fef 0 2024-03-16T09:01:19.685 INFO:tasks.workunit.client.0.smithi007.stdout:3/765: dwrite d15/d18/d1f/d29/d7c/f94 [4194304,4194304] 0 2024-03-16T09:01:19.686 INFO:tasks.workunit.client.0.smithi007.stdout:3/766: chown d15/d18/d1f/d29/f5b 3935043 1 2024-03-16T09:01:19.689 INFO:tasks.workunit.client.0.smithi007.stdout:3/767: dread d15/fa4 [0,4194304] 0 2024-03-16T09:01:19.691 INFO:tasks.workunit.client.0.smithi007.stdout:3/768: rename d15/d18/d1f/d29/d3e/c61 to d15/d18/d1f/d29/d3e/dda/cf1 0 2024-03-16T09:01:19.691 INFO:tasks.workunit.client.0.smithi007.stdout:3/769: truncate d15/d18/d1f/d56/f65 1231128 0 2024-03-16T09:01:19.691 INFO:tasks.workunit.client.0.smithi007.stdout:3/770: chown d15/d18/d1f/d3b/d63/fb7 3 1 2024-03-16T09:01:19.691 INFO:tasks.workunit.client.0.smithi007.stdout:3/771: mknod d15/d18/d1f/d77/cf2 0 2024-03-16T09:01:19.694 INFO:tasks.workunit.client.0.smithi007.stdout:7/754: dwrite d0/dc/d5a/f74 [0,4194304] 0 2024-03-16T09:01:19.697 INFO:tasks.workunit.client.1.smithi019.stdout:0/658: dwrite d7/d1d/d63/d89/d9e/fc7 [0,4194304] 0 2024-03-16T09:01:19.697 INFO:tasks.workunit.client.0.smithi007.stdout:7/755: mknod d0/dc/d38/d4e/c10c 0 2024-03-16T09:01:19.697 INFO:tasks.workunit.client.0.smithi007.stdout:7/756: readlink d0/d19/d1f/d5d/d9d/df7/l40 0 2024-03-16T09:01:19.697 INFO:tasks.workunit.client.0.smithi007.stdout:7/757: chown d0/dc/d38/d4e/fd5 890 1 2024-03-16T09:01:19.697 INFO:tasks.workunit.client.0.smithi007.stdout:7/758: truncate d0/d19/d1f/d5d/d9d/df7/f7f 239621 0 2024-03-16T09:01:19.698 INFO:tasks.workunit.client.0.smithi007.stdout:9/783: dwrite d24/d4a/d96/db9/f60 [0,4194304] 0 2024-03-16T09:01:19.698 INFO:tasks.workunit.client.1.smithi019.stdout:2/682: dwrite d5/d6/d2b/f91 [0,4194304] 0 2024-03-16T09:01:19.701 INFO:tasks.workunit.client.1.smithi019.stdout:0/659: mknod d7/d8/d32/d66/d6d/d8c/db8/cdd 0 2024-03-16T09:01:19.701 INFO:tasks.workunit.client.1.smithi019.stdout:0/660: fdatasync d7/f19 0 2024-03-16T09:01:19.701 INFO:tasks.workunit.client.1.smithi019.stdout:0/661: creat d7/d1d/d28/d48/fde x:0 0 0 2024-03-16T09:01:19.701 INFO:tasks.workunit.client.1.smithi019.stdout:0/662: dread - d7/d1d/d2e/fc3 zero size 2024-03-16T09:01:19.702 INFO:tasks.workunit.client.0.smithi007.stdout:9/784: mkdir d24/d4a/d55/dbf/de9 0 2024-03-16T09:01:19.702 INFO:tasks.workunit.client.0.smithi007.stdout:9/785: truncate d24/d4c/d86/daa/d7f/d80/d87/fc5 878470 0 2024-03-16T09:01:19.703 INFO:tasks.workunit.client.0.smithi007.stdout:9/786: chown d24/d4a/d96/db9/dca/ddc 96179 1 2024-03-16T09:01:19.703 INFO:tasks.workunit.client.0.smithi007.stdout:9/787: readlink d24/d4c/l51 0 2024-03-16T09:01:19.704 INFO:tasks.workunit.client.1.smithi019.stdout:2/683: link d5/d6/d64/d6f/da6/c85 d5/d6/d64/dda/ce9 0 2024-03-16T09:01:19.704 INFO:tasks.workunit.client.1.smithi019.stdout:2/684: dread d5/d5a/f9d [0,4194304] 0 2024-03-16T09:01:19.704 INFO:tasks.workunit.client.1.smithi019.stdout:2/685: getdents d5/d5a/de6 0 2024-03-16T09:01:19.704 INFO:tasks.workunit.client.1.smithi019.stdout:2/686: creat d5/d6/d2b/d89/fea x:0 0 0 2024-03-16T09:01:19.707 INFO:tasks.workunit.client.0.smithi007.stdout:9/788: rename d24/d4c/d86/daa/d95/f7b to d24/d4c/d86/daa/d7f/d8f/dd0/fea 0 2024-03-16T09:01:19.707 INFO:tasks.workunit.client.1.smithi019.stdout:2/687: rename d5/fb2 to d5/d6/d64/dda/feb 0 2024-03-16T09:01:19.709 INFO:tasks.workunit.client.1.smithi019.stdout:0/663: dread d7/d1d/d28/d6c/f77 [0,4194304] 0 2024-03-16T09:01:19.712 INFO:tasks.workunit.client.1.smithi019.stdout:0/664: mkdir d7/d1d/d28/da1/ddf 0 2024-03-16T09:01:19.712 INFO:tasks.workunit.client.1.smithi019.stdout:0/665: read d7/d1d/d28/d6c/f99 [3271410,36303] 0 2024-03-16T09:01:19.715 INFO:tasks.workunit.client.1.smithi019.stdout:7/571: truncate d9/da/d13/d18/d30/d75/f8c 4110384 0 2024-03-16T09:01:19.718 INFO:tasks.workunit.client.1.smithi019.stdout:2/688: dread d5/d2e/f9a [0,4194304] 0 2024-03-16T09:01:19.720 INFO:tasks.workunit.client.1.smithi019.stdout:2/689: link d5/d6/d2b/l70 d5/d6/d64/dda/lec 0 2024-03-16T09:01:19.721 INFO:tasks.workunit.client.0.smithi007.stdout:3/772: dwrite d15/d18/f35 [0,4194304] 0 2024-03-16T09:01:19.722 INFO:tasks.workunit.client.1.smithi019.stdout:0/666: dread d7/d8/f88 [0,4194304] 0 2024-03-16T09:01:19.722 INFO:tasks.workunit.client.1.smithi019.stdout:2/690: creat d5/d6/d64/dda/fed x:0 0 0 2024-03-16T09:01:19.723 INFO:tasks.workunit.client.0.smithi007.stdout:7/759: dwrite d0/d19/d1f/d5d/d9d/df8/f103 [0,4194304] 0 2024-03-16T09:01:19.723 INFO:tasks.workunit.client.0.smithi007.stdout:7/760: readlink d0/d19/l60 0 2024-03-16T09:01:19.723 INFO:tasks.workunit.client.0.smithi007.stdout:3/773: unlink d15/d18/d1f/d56/c73 0 2024-03-16T09:01:19.723 INFO:tasks.workunit.client.0.smithi007.stdout:3/774: chown d15/d18/d1f/d3b/f44 127177653 1 2024-03-16T09:01:19.723 INFO:tasks.workunit.client.0.smithi007.stdout:3/775: truncate d15/d18/d1f/d29/f5b 766009 0 2024-03-16T09:01:19.724 INFO:tasks.workunit.client.0.smithi007.stdout:9/789: dwrite d24/f5c [0,4194304] 0 2024-03-16T09:01:19.724 INFO:tasks.workunit.client.1.smithi019.stdout:2/691: truncate d5/de/f5d 59275 0 2024-03-16T09:01:19.727 INFO:tasks.workunit.client.1.smithi019.stdout:8/693: rmdir d6/d2a 39 2024-03-16T09:01:19.729 INFO:tasks.workunit.client.1.smithi019.stdout:8/694: creat d6/d2a/d3c/dab/d30/d4a/d67/ff0 x:0 0 0 2024-03-16T09:01:19.732 INFO:tasks.workunit.client.1.smithi019.stdout:0/667: dread d7/d1d/d28/f44 [0,4194304] 0 2024-03-16T09:01:19.733 INFO:tasks.workunit.client.1.smithi019.stdout:0/668: write d7/d1d/d63/d6e/fc9 [498586,55769] 0 2024-03-16T09:01:19.733 INFO:tasks.workunit.client.1.smithi019.stdout:2/692: dread d5/d2e/f9a [0,4194304] 0 2024-03-16T09:01:19.733 INFO:tasks.workunit.client.1.smithi019.stdout:2/693: dread - d5/fac zero size 2024-03-16T09:01:19.733 INFO:tasks.workunit.client.1.smithi019.stdout:2/694: fdatasync d5/d5a/d80/fb3 0 2024-03-16T09:01:19.733 INFO:tasks.workunit.client.0.smithi007.stdout:7/761: mknod d0/d19/d1f/d5d/d9d/df7/d7a/d9a/c10d 0 2024-03-16T09:01:19.734 INFO:tasks.workunit.client.1.smithi019.stdout:2/695: mknod d5/d6/d2b/d53/cee 0 2024-03-16T09:01:19.735 INFO:tasks.workunit.client.1.smithi019.stdout:1/568: sync 2024-03-16T09:01:19.735 INFO:tasks.workunit.client.1.smithi019.stdout:1/569: write fb [3955874,44393] 0 2024-03-16T09:01:19.739 INFO:tasks.workunit.client.1.smithi019.stdout:1/570: mkdir d16/d31/d76/d92/d9f 0 2024-03-16T09:01:19.740 INFO:tasks.workunit.client.0.smithi007.stdout:6/687: sync 2024-03-16T09:01:19.740 INFO:tasks.workunit.client.0.smithi007.stdout:4/658: sync 2024-03-16T09:01:19.740 INFO:tasks.workunit.client.0.smithi007.stdout:0/888: sync 2024-03-16T09:01:19.743 INFO:tasks.workunit.client.0.smithi007.stdout:4/659: fsync d4/fa3 0 2024-03-16T09:01:19.745 INFO:tasks.workunit.client.0.smithi007.stdout:0/889: rmdir d3/d9/da/d18/d27/d34/df5 0 2024-03-16T09:01:19.746 INFO:tasks.workunit.client.0.smithi007.stdout:6/688: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/fe4 x:0 0 0 2024-03-16T09:01:19.746 INFO:tasks.workunit.client.0.smithi007.stdout:6/689: stat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/l9d 0 2024-03-16T09:01:19.746 INFO:tasks.workunit.client.0.smithi007.stdout:6/690: rename d2/d13/d3f/d50/d78/dbe/dcb to d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/de5 22 2024-03-16T09:01:19.746 INFO:tasks.workunit.client.0.smithi007.stdout:7/762: dread d0/f55 [0,4194304] 0 2024-03-16T09:01:19.746 INFO:tasks.workunit.client.0.smithi007.stdout:7/763: stat d0/d19/d1f/d5d/d9d/cf1 0 2024-03-16T09:01:19.747 INFO:tasks.workunit.client.0.smithi007.stdout:6/691: mknod d2/d82/ce6 0 2024-03-16T09:01:19.748 INFO:tasks.workunit.client.0.smithi007.stdout:7/764: mkdir d0/df5/d10e 0 2024-03-16T09:01:19.750 INFO:tasks.workunit.client.0.smithi007.stdout:0/890: dread d3/d9/da/d48/d4b/f51 [0,4194304] 0 2024-03-16T09:01:19.750 INFO:tasks.workunit.client.0.smithi007.stdout:0/891: dread - d3/d9/da/d18/d23/f10e zero size 2024-03-16T09:01:19.751 INFO:tasks.workunit.client.0.smithi007.stdout:0/892: rmdir d3/d6c 39 2024-03-16T09:01:19.751 INFO:tasks.workunit.client.0.smithi007.stdout:0/893: dread - d3/d9/db4/d1c/ff2 zero size 2024-03-16T09:01:19.754 INFO:tasks.workunit.client.1.smithi019.stdout:1/571: dread f12 [0,4194304] 0 2024-03-16T09:01:19.755 INFO:tasks.workunit.client.0.smithi007.stdout:4/660: read d4/d14/d64/d68/f6f [114321,84365] 0 2024-03-16T09:01:19.757 INFO:tasks.workunit.client.0.smithi007.stdout:4/661: rmdir d4/d14/d1c/d5b/d67/d8f/da8/d9d 39 2024-03-16T09:01:19.757 INFO:tasks.workunit.client.0.smithi007.stdout:4/662: truncate d4/d14/d64/d9f/f4b 4881693 0 2024-03-16T09:01:19.757 INFO:tasks.workunit.client.1.smithi019.stdout:8/695: dwrite d6/d2a/d3c/dab/d30/d4a/d67/fdd [0,4194304] 0 2024-03-16T09:01:19.758 INFO:tasks.workunit.client.1.smithi019.stdout:2/696: dwrite d5/d4e/d74/f95 [0,4194304] 0 2024-03-16T09:01:19.761 INFO:tasks.workunit.client.1.smithi019.stdout:2/697: link d5/d6/d2b/d2f/d35/d4b/ca7 d5/d6/d2b/d89/cef 0 2024-03-16T09:01:19.762 INFO:tasks.workunit.client.1.smithi019.stdout:2/698: truncate d5/d6/d2b/d2f/f46 4800829 0 2024-03-16T09:01:19.762 INFO:tasks.workunit.client.1.smithi019.stdout:2/699: write d5/d6/fb0 [490696,47993] 0 2024-03-16T09:01:19.768 INFO:tasks.workunit.client.1.smithi019.stdout:1/572: write d16/d31/d76/f6b [3559624,28953] 0 2024-03-16T09:01:19.768 INFO:tasks.workunit.client.0.smithi007.stdout:4/663: write d4/d14/d1c/f37 [1049129,80794] 0 2024-03-16T09:01:19.768 INFO:tasks.workunit.client.0.smithi007.stdout:4/664: chown d4/d14/d64/d68/d82/dac/fe1 1017237 1 2024-03-16T09:01:19.769 INFO:tasks.workunit.client.1.smithi019.stdout:8/696: dread d6/f27 [0,4194304] 0 2024-03-16T09:01:19.770 INFO:tasks.workunit.client.0.smithi007.stdout:4/665: unlink d4/d14/d64/d68/d82/dac/ldd 0 2024-03-16T09:01:19.771 INFO:tasks.workunit.client.1.smithi019.stdout:8/697: rename d6/d2a/d45/d2c/f58 to d6/ff1 0 2024-03-16T09:01:19.774 INFO:tasks.workunit.client.1.smithi019.stdout:8/698: dread d6/d2a/d45/d57/fbb [0,4194304] 0 2024-03-16T09:01:19.774 INFO:tasks.workunit.client.1.smithi019.stdout:8/699: creat d6/ff2 x:0 0 0 2024-03-16T09:01:19.775 INFO:tasks.workunit.client.1.smithi019.stdout:8/700: creat d6/d2a/d45/d57/d72/d79/d7b/ff3 x:0 0 0 2024-03-16T09:01:19.775 INFO:tasks.workunit.client.1.smithi019.stdout:1/573: dread f12 [0,4194304] 0 2024-03-16T09:01:19.776 INFO:tasks.workunit.client.1.smithi019.stdout:1/574: getdents d16/d1f/d4f/d68 0 2024-03-16T09:01:19.776 INFO:tasks.workunit.client.1.smithi019.stdout:1/575: chown d16/d1f/d4f/f58 4256 1 2024-03-16T09:01:19.776 INFO:tasks.workunit.client.1.smithi019.stdout:1/576: creat d16/d1f/d3c/d79/d96/fa0 x:0 0 0 2024-03-16T09:01:19.777 INFO:tasks.workunit.client.1.smithi019.stdout:1/577: mkdir d16/da1 0 2024-03-16T09:01:19.778 INFO:tasks.workunit.client.1.smithi019.stdout:1/578: write d16/f2f [1263941,3114] 0 2024-03-16T09:01:19.778 INFO:tasks.workunit.client.1.smithi019.stdout:1/579: fsync d16/d1f/d4f/f50 0 2024-03-16T09:01:19.779 INFO:tasks.workunit.client.1.smithi019.stdout:1/580: truncate d16/f20 187671 0 2024-03-16T09:01:19.780 INFO:tasks.workunit.client.1.smithi019.stdout:1/581: creat d16/d1f/d4f/fa2 x:0 0 0 2024-03-16T09:01:19.780 INFO:tasks.workunit.client.1.smithi019.stdout:1/582: write d16/d1f/d3c/d79/f91 [466432,107910] 0 2024-03-16T09:01:19.781 INFO:tasks.workunit.client.0.smithi007.stdout:7/765: dwrite d0/d19/d1f/d5d/d9d/df7/f7f [0,4194304] 0 2024-03-16T09:01:19.782 INFO:tasks.workunit.client.0.smithi007.stdout:0/894: dwrite d3/d9/da/d48/d25/d43/fdc [0,4194304] 0 2024-03-16T09:01:19.783 INFO:tasks.workunit.client.0.smithi007.stdout:0/895: readlink d3/d9/db4/d60/lb6 0 2024-03-16T09:01:19.790 INFO:tasks.workunit.client.0.smithi007.stdout:6/692: dwrite d2/d13/d3f/d50/d72/f69 [0,4194304] 0 2024-03-16T09:01:19.791 INFO:tasks.workunit.client.1.smithi019.stdout:8/701: write d6/d2a/d45/f32 [3580602,114349] 0 2024-03-16T09:01:19.792 INFO:tasks.workunit.client.1.smithi019.stdout:8/702: mknod d6/d2a/d3c/dab/d30/cf4 0 2024-03-16T09:01:19.817 INFO:tasks.workunit.client.0.smithi007.stdout:7/766: dwrite d0/dc/d5a/fdc [4194304,4194304] 0 2024-03-16T09:01:19.822 INFO:tasks.workunit.client.1.smithi019.stdout:2/700: dwrite d5/d6/fb0 [0,4194304] 0 2024-03-16T09:01:19.823 INFO:tasks.workunit.client.0.smithi007.stdout:0/896: dwrite d3/d9/da/d48/d25/d43/d1a/d58/d61/f110 [0,4194304] 0 2024-03-16T09:01:19.823 INFO:tasks.workunit.client.0.smithi007.stdout:6/693: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/f64 [0,4194304] 0 2024-03-16T09:01:19.823 INFO:tasks.workunit.client.0.smithi007.stdout:6/694: chown d2/d13/d3f/d50/d78/fcd 31 1 2024-03-16T09:01:19.823 INFO:tasks.workunit.client.0.smithi007.stdout:6/695: write d2/fa [225599,20485] 0 2024-03-16T09:01:19.823 INFO:tasks.workunit.client.0.smithi007.stdout:6/696: fdatasync d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/f5c 0 2024-03-16T09:01:19.823 INFO:tasks.workunit.client.0.smithi007.stdout:6/697: creat d2/d29/fe7 x:0 0 0 2024-03-16T09:01:19.823 INFO:tasks.workunit.client.0.smithi007.stdout:6/698: dread - d2/d13/d4b/f55 zero size 2024-03-16T09:01:19.825 INFO:tasks.workunit.client.1.smithi019.stdout:8/703: dwrite d6/d2a/d45/d19/f97 [0,4194304] 0 2024-03-16T09:01:19.825 INFO:tasks.workunit.client.0.smithi007.stdout:0/897: mknod d3/d9/da/d18/d27/d34/d3a/c124 0 2024-03-16T09:01:19.829 INFO:tasks.workunit.client.1.smithi019.stdout:2/701: read d5/d6/d2b/f98 [5533,126612] 0 2024-03-16T09:01:19.829 INFO:tasks.workunit.client.1.smithi019.stdout:2/702: write d5/d6/fcc [754915,65907] 0 2024-03-16T09:01:19.830 INFO:tasks.workunit.client.1.smithi019.stdout:2/703: mknod d5/d6/d2b/dde/cf0 0 2024-03-16T09:01:19.831 INFO:tasks.workunit.client.0.smithi007.stdout:0/898: mknod d3/d9/da/d48/d25/d43/c125 0 2024-03-16T09:01:19.831 INFO:tasks.workunit.client.1.smithi019.stdout:2/704: truncate d5/d6/d2b/f91 694723 0 2024-03-16T09:01:19.839 INFO:tasks.workunit.client.1.smithi019.stdout:8/704: dread d6/ff1 [0,4194304] 0 2024-03-16T09:01:19.841 INFO:tasks.workunit.client.1.smithi019.stdout:8/705: mkdir d6/d2a/d45/d57/d96/df5 0 2024-03-16T09:01:19.841 INFO:tasks.workunit.client.1.smithi019.stdout:8/706: write d6/d2a/d45/d19/d74/f7e [4350157,39964] 0 2024-03-16T09:01:19.842 INFO:tasks.workunit.client.1.smithi019.stdout:8/707: readlink d6/d2a/d68/ld8 0 2024-03-16T09:01:19.843 INFO:tasks.workunit.client.0.smithi007.stdout:7/767: dwrite d0/f55 [0,4194304] 0 2024-03-16T09:01:19.843 INFO:tasks.workunit.client.0.smithi007.stdout:6/699: dwrite d2/f7 [4194304,4194304] 0 2024-03-16T09:01:19.843 INFO:tasks.workunit.client.0.smithi007.stdout:7/768: read - d0/d19/fc1 zero size 2024-03-16T09:01:19.844 INFO:tasks.workunit.client.1.smithi019.stdout:2/705: dread d5/d6/d2b/d2f/f46 [0,4194304] 0 2024-03-16T09:01:19.844 INFO:tasks.workunit.client.1.smithi019.stdout:2/706: chown d5/d5a/c92 4 1 2024-03-16T09:01:19.844 INFO:tasks.workunit.client.1.smithi019.stdout:2/707: dread - d5/d5a/d80/fb3 zero size 2024-03-16T09:01:19.844 INFO:tasks.workunit.client.1.smithi019.stdout:2/708: creat d5/d2e/ff1 x:0 0 0 2024-03-16T09:01:19.844 INFO:tasks.workunit.client.1.smithi019.stdout:2/709: chown d5/d5a/d96/cd2 5120 1 2024-03-16T09:01:19.846 INFO:tasks.workunit.client.0.smithi007.stdout:6/700: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/d84/fe8 x:0 0 0 2024-03-16T09:01:19.846 INFO:tasks.workunit.client.0.smithi007.stdout:6/701: stat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/f5c 0 2024-03-16T09:01:19.846 INFO:tasks.workunit.client.0.smithi007.stdout:6/702: readlink d2/d13/l7e 0 2024-03-16T09:01:19.847 INFO:tasks.workunit.client.0.smithi007.stdout:7/769: creat d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d9c/f10f x:0 0 0 2024-03-16T09:01:19.847 INFO:tasks.workunit.client.0.smithi007.stdout:6/703: write d2/d13/d3f/d50/d78/d81/f8c [94590,34816] 0 2024-03-16T09:01:19.847 INFO:tasks.workunit.client.0.smithi007.stdout:6/704: read - d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fe1 zero size 2024-03-16T09:01:19.848 INFO:tasks.workunit.client.1.smithi019.stdout:4/628: sync 2024-03-16T09:01:19.848 INFO:tasks.workunit.client.1.smithi019.stdout:4/629: dread - d4/d1b/d2a/f74 zero size 2024-03-16T09:01:19.848 INFO:tasks.workunit.client.1.smithi019.stdout:4/630: dread - d4/d64/d9b/fb7 zero size 2024-03-16T09:01:19.848 INFO:tasks.workunit.client.1.smithi019.stdout:4/631: fdatasync d4/d1b/d2a/d4f/f8b 0 2024-03-16T09:01:19.848 INFO:tasks.workunit.client.1.smithi019.stdout:4/632: chown d4/d6/d15/d1f 0 1 2024-03-16T09:01:19.848 INFO:tasks.workunit.client.1.smithi019.stdout:5/613: sync 2024-03-16T09:01:19.848 INFO:tasks.workunit.client.1.smithi019.stdout:3/621: sync 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.1.smithi019.stdout:6/744: sync 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.1.smithi019.stdout:7/572: sync 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.1.smithi019.stdout:5/614: chown d6/dd/d47 137417099 1 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.1.smithi019.stdout:5/615: fdatasync d6/dd/d37/f3d 0 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.1.smithi019.stdout:7/573: dread - d9/db/d12/f55 zero size 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.1.smithi019.stdout:9/487: sync 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.1.smithi019.stdout:9/488: write d1/d7/de/d24/d44/d60/f97 [997628,50628] 0 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.0.smithi007.stdout:7/770: mknod d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d9c/c110 0 2024-03-16T09:01:19.849 INFO:tasks.workunit.client.0.smithi007.stdout:6/705: symlink d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/d84/le9 0 2024-03-16T09:01:19.850 INFO:tasks.workunit.client.1.smithi019.stdout:2/710: creat d5/d6/d2b/d2f/d35/d4b/ff2 x:0 0 0 2024-03-16T09:01:19.851 INFO:tasks.workunit.client.1.smithi019.stdout:6/745: truncate de/d26/d32/f8a 221915 0 2024-03-16T09:01:19.852 INFO:tasks.workunit.client.1.smithi019.stdout:2/711: write d5/d6/d2b/d2f/d35/d4b/f5c [1066189,12643] 0 2024-03-16T09:01:19.852 INFO:tasks.workunit.client.0.smithi007.stdout:7/771: rename d0/d19/d1f/d5d/d9d/fa4 to d0/d19/d1f/d5d/d9d/df7/d3a/f111 0 2024-03-16T09:01:19.852 INFO:tasks.workunit.client.0.smithi007.stdout:7/772: chown d0/d19/d1f/d5d/d9d/df7/d73/ded/lf4 4282 1 2024-03-16T09:01:19.853 INFO:tasks.workunit.client.0.smithi007.stdout:6/706: mkdir d2/d13/d4b/d94/dea 0 2024-03-16T09:01:19.854 INFO:tasks.workunit.client.1.smithi019.stdout:7/574: rename d9/db/d12/d5b/f9f to d9/da/d13/d18/d30/d3e/d5d/fb6 0 2024-03-16T09:01:19.854 INFO:tasks.workunit.client.1.smithi019.stdout:8/708: dread d6/d2a/d3c/dab/d30/d4a/f4e [0,4194304] 0 2024-03-16T09:01:19.854 INFO:tasks.workunit.client.1.smithi019.stdout:8/709: stat d6/d2a/d45/d5d/faf 0 2024-03-16T09:01:19.854 INFO:tasks.workunit.client.1.smithi019.stdout:8/710: readlink l1 0 2024-03-16T09:01:19.857 INFO:tasks.workunit.client.1.smithi019.stdout:2/712: dread d5/de/da2/faa [0,4194304] 0 2024-03-16T09:01:19.857 INFO:tasks.workunit.client.0.smithi007.stdout:6/707: link d2/d13/l3c d2/d10/leb 0 2024-03-16T09:01:19.857 INFO:tasks.workunit.client.0.smithi007.stdout:6/708: readlink d2/d29/l5d 0 2024-03-16T09:01:19.857 INFO:tasks.workunit.client.0.smithi007.stdout:6/709: chown d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/fc4 10271 1 2024-03-16T09:01:19.857 INFO:tasks.workunit.client.0.smithi007.stdout:6/710: creat d2/d13/d24/fec x:0 0 0 2024-03-16T09:01:19.858 INFO:tasks.workunit.client.0.smithi007.stdout:7/773: dread d0/f3c [0,4194304] 0 2024-03-16T09:01:19.858 INFO:tasks.workunit.client.0.smithi007.stdout:7/774: chown d0/dc/d38 118203 1 2024-03-16T09:01:19.858 INFO:tasks.workunit.client.1.smithi019.stdout:6/746: unlink de/d26/f5c 0 2024-03-16T09:01:19.858 INFO:tasks.workunit.client.1.smithi019.stdout:6/747: write de/d26/d32/fd1 [362012,60716] 0 2024-03-16T09:01:19.859 INFO:tasks.workunit.client.1.smithi019.stdout:7/575: mknod d9/da/d13/d9a/dab/cb7 0 2024-03-16T09:01:19.859 INFO:tasks.workunit.client.1.smithi019.stdout:7/576: read d9/da/d13/d18/d1a/f2b [2412793,693] 0 2024-03-16T09:01:19.859 INFO:tasks.workunit.client.0.smithi007.stdout:6/711: dread d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/d8f/fb1 [0,4194304] 0 2024-03-16T09:01:19.860 INFO:tasks.workunit.client.1.smithi019.stdout:8/711: getdents d6/d2a/d45/d19/d74 0 2024-03-16T09:01:19.861 INFO:tasks.workunit.client.1.smithi019.stdout:2/713: symlink d5/d6/d64/d6f/da6/d83/lf3 0 2024-03-16T09:01:19.861 INFO:tasks.workunit.client.1.smithi019.stdout:2/714: stat d5/d6/c6c 0 2024-03-16T09:01:19.861 INFO:tasks.workunit.client.1.smithi019.stdout:6/748: rmdir de/d26/d32/d97/d66 39 2024-03-16T09:01:19.862 INFO:tasks.workunit.client.0.smithi007.stdout:7/775: creat d0/d19/d1f/f112 x:0 0 0 2024-03-16T09:01:19.862 INFO:tasks.workunit.client.1.smithi019.stdout:5/616: write d6/f8 [2689185,29079] 0 2024-03-16T09:01:19.863 INFO:tasks.workunit.client.1.smithi019.stdout:8/712: mkdir d6/de/d51/df6 0 2024-03-16T09:01:19.863 INFO:tasks.workunit.client.1.smithi019.stdout:8/713: write d6/d2a/d45/f32 [2475186,88650] 0 2024-03-16T09:01:19.863 INFO:tasks.workunit.client.0.smithi007.stdout:1/649: sync 2024-03-16T09:01:19.863 INFO:tasks.workunit.client.0.smithi007.stdout:1/650: readlink d9/dd/d1b/d28/l75 0 2024-03-16T09:01:19.863 INFO:tasks.workunit.client.0.smithi007.stdout:1/651: chown d9/l9c 28 1 2024-03-16T09:01:19.866 INFO:tasks.workunit.client.1.smithi019.stdout:2/715: creat d5/d4e/d74/ff4 x:0 0 0 2024-03-16T09:01:19.871 INFO:tasks.workunit.client.1.smithi019.stdout:5/617: mkdir d6/dd/d47/d70/db2/d66/dc0 0 2024-03-16T09:01:19.873 INFO:tasks.workunit.client.1.smithi019.stdout:7/577: dread d9/da/d13/d18/d1a/f2b [4194304,4194304] 0 2024-03-16T09:01:19.873 INFO:tasks.workunit.client.1.smithi019.stdout:7/578: creat d9/da/d13/fb8 x:0 0 0 2024-03-16T09:01:19.873 INFO:tasks.workunit.client.1.smithi019.stdout:7/579: fsync f8 0 2024-03-16T09:01:19.873 INFO:tasks.workunit.client.1.smithi019.stdout:2/716: mknod d5/d6/d64/d6f/d87/cf5 0 2024-03-16T09:01:19.873 INFO:tasks.workunit.client.1.smithi019.stdout:2/717: creat d5/d6/d64/d6f/d87/ff6 x:0 0 0 2024-03-16T09:01:19.873 INFO:tasks.workunit.client.1.smithi019.stdout:2/718: dread - d5/de/faf zero size 2024-03-16T09:01:19.874 INFO:tasks.workunit.client.1.smithi019.stdout:6/749: getdents de/d26/d32/d97/d66 0 2024-03-16T09:01:19.874 INFO:tasks.workunit.client.1.smithi019.stdout:6/750: chown de/f1a 3102711 1 2024-03-16T09:01:19.874 INFO:tasks.workunit.client.1.smithi019.stdout:5/618: symlink d6/dd/d47/d7a/lc1 0 2024-03-16T09:01:19.875 INFO:tasks.workunit.client.1.smithi019.stdout:5/619: fdatasync d6/dd/f82 0 2024-03-16T09:01:19.875 INFO:tasks.workunit.client.1.smithi019.stdout:5/620: stat d6/c6d 0 2024-03-16T09:01:19.875 INFO:tasks.workunit.client.0.smithi007.stdout:7/776: write d0/dc/d5a/fdc [4813030,33660] 0 2024-03-16T09:01:19.878 INFO:tasks.workunit.client.1.smithi019.stdout:8/714: dread f4 [4194304,4194304] 0 2024-03-16T09:01:19.878 INFO:tasks.workunit.client.1.smithi019.stdout:8/715: chown d6/d2a/d45/d57/d72/d79/d7b/ff3 46765174 1 2024-03-16T09:01:19.878 INFO:tasks.workunit.client.1.smithi019.stdout:8/716: chown d6/de/d51/df6 113424503 1 2024-03-16T09:01:19.878 INFO:tasks.workunit.client.0.smithi007.stdout:7/777: rename d0/d6c/f64 to d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/f113 0 2024-03-16T09:01:19.882 INFO:tasks.workunit.client.1.smithi019.stdout:4/633: dwrite d4/d1b/f1c [4194304,4194304] 0 2024-03-16T09:01:19.882 INFO:tasks.workunit.client.1.smithi019.stdout:4/634: creat d4/d1b/d49/d58/d80/d91/fd0 x:0 0 0 2024-03-16T09:01:19.885 INFO:tasks.workunit.client.1.smithi019.stdout:9/489: dwrite d1/d7/f81 [0,4194304] 0 2024-03-16T09:01:19.888 INFO:tasks.workunit.client.0.smithi007.stdout:7/778: write d0/dc/d5a/f74 [3773493,37223] 0 2024-03-16T09:01:19.888 INFO:tasks.workunit.client.0.smithi007.stdout:1/652: dwrite d9/dd/d5e/f79 [0,4194304] 0 2024-03-16T09:01:19.888 INFO:tasks.workunit.client.0.smithi007.stdout:1/653: chown d9/dd/d1b/d28/f46 36164 1 2024-03-16T09:01:19.888 INFO:tasks.workunit.client.0.smithi007.stdout:1/654: write d9/dd/d1b/d28/f50 [335223,55508] 0 2024-03-16T09:01:19.895 INFO:tasks.workunit.client.1.smithi019.stdout:7/580: getdents d9/d25 0 2024-03-16T09:01:19.897 INFO:tasks.workunit.client.1.smithi019.stdout:7/581: write d9/db/d12/f2f [742814,62309] 0 2024-03-16T09:01:19.904 INFO:tasks.workunit.client.1.smithi019.stdout:8/717: creat d6/d2a/d45/d57/ff7 x:0 0 0 2024-03-16T09:01:19.904 INFO:tasks.workunit.client.1.smithi019.stdout:2/719: dwrite d5/d6/d2b/d2f/f67 [4194304,4194304] 0 2024-03-16T09:01:19.911 INFO:tasks.workunit.client.1.smithi019.stdout:6/751: dwrite de/d26/d32/d97/f46 [0,4194304] 0 2024-03-16T09:01:19.916 INFO:tasks.workunit.client.1.smithi019.stdout:4/635: mknod d4/d1b/d49/cd1 0 2024-03-16T09:01:19.916 INFO:tasks.workunit.client.1.smithi019.stdout:4/636: chown d4/d6c/f7c 2970 1 2024-03-16T09:01:19.916 INFO:tasks.workunit.client.1.smithi019.stdout:4/637: dread - d4/d6/fcc zero size 2024-03-16T09:01:19.917 INFO:tasks.workunit.client.0.smithi007.stdout:1/655: dwrite d9/dd/d1b/d28/d29/f47 [4194304,4194304] 0 2024-03-16T09:01:19.917 INFO:tasks.workunit.client.0.smithi007.stdout:7/779: dwrite d0/d19/d1f/d5d/d9d/df7/d73/f58 [4194304,4194304] 0 2024-03-16T09:01:19.919 INFO:tasks.workunit.client.0.smithi007.stdout:1/656: unlink d9/dd/d1b/d2d/d30/d45/lc6 0 2024-03-16T09:01:19.928 INFO:tasks.workunit.client.0.smithi007.stdout:7/780: write d0/dc/f3d [331685,46050] 0 2024-03-16T09:01:19.929 INFO:tasks.workunit.client.1.smithi019.stdout:9/490: unlink d1/d7/de/d24/d44/f7d 0 2024-03-16T09:01:19.935 INFO:tasks.workunit.client.0.smithi007.stdout:7/781: link d0/d19/d1f/d5d/d9d/df7/d3a/db3/lfb d0/d19/d1f/d5d/d9d/df7/d73/l114 0 2024-03-16T09:01:19.937 INFO:tasks.workunit.client.0.smithi007.stdout:7/782: symlink d0/dc/d5a/dba/dd9/l115 0 2024-03-16T09:01:19.937 INFO:tasks.workunit.client.0.smithi007.stdout:7/783: write d0/d19/d1f/f54 [4055335,52403] 0 2024-03-16T09:01:19.937 INFO:tasks.workunit.client.0.smithi007.stdout:7/784: fsync d0/f11 0 2024-03-16T09:01:19.939 INFO:tasks.workunit.client.0.smithi007.stdout:7/785: getdents d0/dc/d5a 0 2024-03-16T09:01:19.939 INFO:tasks.workunit.client.0.smithi007.stdout:7/786: creat d0/d19/d1f/f116 x:0 0 0 2024-03-16T09:01:19.947 INFO:tasks.workunit.client.0.smithi007.stdout:1/657: dwrite d9/dd/d1b/f49 [0,4194304] 0 2024-03-16T09:01:19.947 INFO:tasks.workunit.client.0.smithi007.stdout:1/658: chown d9/dd/d1b/d28/d29/l41 868595 1 2024-03-16T09:01:19.949 INFO:tasks.workunit.client.1.smithi019.stdout:5/621: rename d6/dd/d47/d70/db2/d66/dc0 to d6/d19/d34/dc2 0 2024-03-16T09:01:19.949 INFO:tasks.workunit.client.1.smithi019.stdout:5/622: fsync d6/dd/d47/d70/db2/d66/fb6 0 2024-03-16T09:01:19.952 INFO:tasks.workunit.client.0.smithi007.stdout:1/659: symlink d9/d6b/lcd 0 2024-03-16T09:01:19.965 INFO:tasks.workunit.client.0.smithi007.stdout:3/776: sync 2024-03-16T09:01:19.965 INFO:tasks.workunit.client.0.smithi007.stdout:2/903: sync 2024-03-16T09:01:19.965 INFO:tasks.workunit.client.0.smithi007.stdout:2/904: chown dc/df/d121/f11d 48808535 1 2024-03-16T09:01:19.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/781: sync 2024-03-16T09:01:19.967 INFO:tasks.workunit.client.0.smithi007.stdout:5/782: chown d1/d7/d1c/d19/d34/d5e/d6f/f51 0 1 2024-03-16T09:01:19.967 INFO:tasks.workunit.client.0.smithi007.stdout:3/777: creat d15/d18/d1f/d3b/ff3 x:0 0 0 2024-03-16T09:01:19.967 INFO:tasks.workunit.client.0.smithi007.stdout:3/778: truncate d15/d18/d1f/d3b/d87/fb8 75384 0 2024-03-16T09:01:19.967 INFO:tasks.workunit.client.0.smithi007.stdout:3/779: dread - d15/d18/d1f/d29/d7c/fba zero size 2024-03-16T09:01:19.967 INFO:tasks.workunit.client.0.smithi007.stdout:3/780: write d15/d18/d1f/d29/d7c/fba [298532,119182] 0 2024-03-16T09:01:19.968 INFO:tasks.workunit.client.0.smithi007.stdout:2/905: creat dc/df/d121/f12f x:0 0 0 2024-03-16T09:01:19.968 INFO:tasks.workunit.client.1.smithi019.stdout:6/752: creat de/d26/d73/ded/fef x:0 0 0 2024-03-16T09:01:19.968 INFO:tasks.workunit.client.1.smithi019.stdout:6/753: write de/d1c/f5d [6121500,6982] 0 2024-03-16T09:01:19.968 INFO:tasks.workunit.client.1.smithi019.stdout:6/754: read - de/d26/d73/ded/fef zero size 2024-03-16T09:01:19.968 INFO:tasks.workunit.client.1.smithi019.stdout:6/755: write de/f1a [3866653,10900] 0 2024-03-16T09:01:19.969 INFO:tasks.workunit.client.1.smithi019.stdout:4/638: symlink d4/d73/ld2 0 2024-03-16T09:01:19.969 INFO:tasks.workunit.client.1.smithi019.stdout:4/639: chown d4/d6/d15/l51 2 1 2024-03-16T09:01:19.969 INFO:tasks.workunit.client.1.smithi019.stdout:4/640: write d4/d6/fcc [840014,9132] 0 2024-03-16T09:01:19.972 INFO:tasks.workunit.client.0.smithi007.stdout:5/783: symlink d1/d7/d16/d1b/l11b 0 2024-03-16T09:01:19.972 INFO:tasks.workunit.client.1.smithi019.stdout:9/491: symlink d1/d7/de/da3/lac 0 2024-03-16T09:01:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:7/787: dwrite d0/d19/fc1 [0,4194304] 0 2024-03-16T09:01:19.977 INFO:tasks.workunit.client.1.smithi019.stdout:5/623: rename d6/dd/d37/d76/f8e to d6/dd/d47/d56/d87/fc3 0 2024-03-16T09:01:19.977 INFO:tasks.workunit.client.1.smithi019.stdout:5/624: write d6/dd/f16 [502743,106041] 0 2024-03-16T09:01:19.977 INFO:tasks.workunit.client.1.smithi019.stdout:5/625: chown d6/d1f 175 1 2024-03-16T09:01:19.977 INFO:tasks.workunit.client.1.smithi019.stdout:5/626: truncate d6/dd/d37/d54/f80 310767 0 2024-03-16T09:01:19.981 INFO:tasks.workunit.client.0.smithi007.stdout:2/906: dread dc/df/d24/d28/d2e/fc7 [0,4194304] 0 2024-03-16T09:01:19.983 INFO:tasks.workunit.client.0.smithi007.stdout:5/784: mknod d1/d7/d16/c11c 0 2024-03-16T09:01:19.985 INFO:tasks.workunit.client.0.smithi007.stdout:7/788: mkdir d0/d19/d1f/d5d/d9d/df7/d3a/daf/d117 0 2024-03-16T09:01:19.985 INFO:tasks.workunit.client.0.smithi007.stdout:7/789: creat d0/d19/d1f/d69/f118 x:0 0 0 2024-03-16T09:01:19.987 INFO:tasks.workunit.client.0.smithi007.stdout:2/907: symlink dc/df/d24/d28/d97/da4/l130 0 2024-03-16T09:01:19.987 INFO:tasks.workunit.client.0.smithi007.stdout:2/908: fsync dc/df/d24/fb1 0 2024-03-16T09:01:19.989 INFO:tasks.workunit.client.0.smithi007.stdout:7/790: mknod d0/d19/d1f/d5d/d9d/df7/d3a/c119 0 2024-03-16T09:01:19.991 INFO:tasks.workunit.client.0.smithi007.stdout:5/785: getdents d1/d7/d1c/d19/d34/d3e/d41/da6 0 2024-03-16T09:01:19.992 INFO:tasks.workunit.client.0.smithi007.stdout:5/786: rmdir d1/d7/d1c/d19/dcc/d102 39 2024-03-16T09:01:19.995 INFO:tasks.workunit.client.0.smithi007.stdout:5/787: creat d1/d7/d1c/d19/d34/d1d/f11d x:0 0 0 2024-03-16T09:01:19.998 INFO:tasks.workunit.client.0.smithi007.stdout:5/788: write d1/d7/d1c/d88/f95 [174704,78598] 0 2024-03-16T09:01:20.000 INFO:tasks.workunit.client.0.smithi007.stdout:5/789: mkdir d1/d7/d1c/d88/d11e 0 2024-03-16T09:01:20.000 INFO:tasks.workunit.client.0.smithi007.stdout:1/660: dwrite d9/dd/d1b/d28/f32 [0,4194304] 0 2024-03-16T09:01:20.000 INFO:tasks.workunit.client.0.smithi007.stdout:1/661: fsync d9/dd/d5e/f9f 0 2024-03-16T09:01:20.002 INFO:tasks.workunit.client.0.smithi007.stdout:1/662: creat d9/dd/d1b/d28/d29/d7b/db4/fce x:0 0 0 2024-03-16T09:01:20.002 INFO:tasks.workunit.client.0.smithi007.stdout:1/663: write d9/f13 [4264756,85902] 0 2024-03-16T09:01:20.002 INFO:tasks.workunit.client.0.smithi007.stdout:1/664: creat d9/dd/d1b/d28/d29/d7b/db4/fcf x:0 0 0 2024-03-16T09:01:20.005 INFO:tasks.workunit.client.0.smithi007.stdout:1/665: creat d9/dd/d5e/d89/fd0 x:0 0 0 2024-03-16T09:01:20.005 INFO:tasks.workunit.client.0.smithi007.stdout:1/666: chown d9/d6b/d7e/dac/dbb/fbe 1039691 1 2024-03-16T09:01:20.024 INFO:tasks.workunit.client.0.smithi007.stdout:3/781: dwrite d15/d18/d1f/d29/d7c/fb0 [0,4194304] 0 2024-03-16T09:01:20.025 INFO:tasks.workunit.client.0.smithi007.stdout:3/782: read d15/d18/d1f/d29/d7c/f8c [239098,90537] 0 2024-03-16T09:01:20.025 INFO:tasks.workunit.client.0.smithi007.stdout:3/783: chown d15/c38 2786 1 2024-03-16T09:01:20.025 INFO:tasks.workunit.client.1.smithi019.stdout:1/583: sync 2024-03-16T09:01:20.025 INFO:tasks.workunit.client.1.smithi019.stdout:0/669: sync 2024-03-16T09:01:20.025 INFO:tasks.workunit.client.1.smithi019.stdout:1/584: fsync d16/d1f/f41 0 2024-03-16T09:01:20.025 INFO:tasks.workunit.client.1.smithi019.stdout:6/756: dwrite de/d26/d32/d97/f46 [0,4194304] 0 2024-03-16T09:01:20.026 INFO:tasks.workunit.client.1.smithi019.stdout:9/492: rename d1/d7/de/d24/f8a to d1/d7/de/d24/d44/d60/d74/fad 0 2024-03-16T09:01:20.026 INFO:tasks.workunit.client.0.smithi007.stdout:2/909: dwrite dc/df/d6c/fcd [0,4194304] 0 2024-03-16T09:01:20.027 INFO:tasks.workunit.client.0.smithi007.stdout:7/791: dwrite d0/d19/d1f/d5d/d9d/df7/d7a/fe8 [0,4194304] 0 2024-03-16T09:01:20.027 INFO:tasks.workunit.client.0.smithi007.stdout:7/792: write d0/d19/d1f/d5d/d9d/df7/d7a/fa8 [113044,100984] 0 2024-03-16T09:01:20.027 INFO:tasks.workunit.client.0.smithi007.stdout:3/784: getdents d15/d18/d1f/dce/dd6/ddf 0 2024-03-16T09:01:20.027 INFO:tasks.workunit.client.1.smithi019.stdout:5/627: dwrite d6/d19/f29 [4194304,4194304] 0 2024-03-16T09:01:20.027 INFO:tasks.workunit.client.1.smithi019.stdout:4/641: dwrite d4/d1b/d2a/f4e [0,4194304] 0 2024-03-16T09:01:20.027 INFO:tasks.workunit.client.1.smithi019.stdout:4/642: stat d4/f12 0 2024-03-16T09:01:20.028 INFO:tasks.workunit.client.1.smithi019.stdout:5/628: chown d6/d19/d34/l81 1510009 1 2024-03-16T09:01:20.028 INFO:tasks.workunit.client.1.smithi019.stdout:5/629: fsync d6/dd/d37/d54/f80 0 2024-03-16T09:01:20.028 INFO:tasks.workunit.client.0.smithi007.stdout:3/785: read d15/d18/d1f/d3b/d87/fb8 [43039,40703] 0 2024-03-16T09:01:20.028 INFO:tasks.workunit.client.0.smithi007.stdout:5/790: dwrite d1/d7/d1c/d88/da2/dae/fde [0,4194304] 0 2024-03-16T09:01:20.028 INFO:tasks.workunit.client.1.smithi019.stdout:1/585: symlink d16/d1f/d4f/la3 0 2024-03-16T09:01:20.030 INFO:tasks.workunit.client.1.smithi019.stdout:6/757: mkdir de/d1c/d9a/d9e/df0 0 2024-03-16T09:01:20.030 INFO:tasks.workunit.client.1.smithi019.stdout:6/758: write de/f14 [1769648,59398] 0 2024-03-16T09:01:20.032 INFO:tasks.workunit.client.1.smithi019.stdout:9/493: truncate d1/d7/d21/f7b 1810980 0 2024-03-16T09:01:20.033 INFO:tasks.workunit.client.1.smithi019.stdout:0/670: dread d7/d8/f50 [0,4194304] 0 2024-03-16T09:01:20.033 INFO:tasks.workunit.client.1.smithi019.stdout:0/671: chown d7/d8/d32/d66/d6d/d8c/db8/c53 126673985 1 2024-03-16T09:01:20.034 INFO:tasks.workunit.client.0.smithi007.stdout:7/793: mknod d0/d19/d1f/c11a 0 2024-03-16T09:01:20.034 INFO:tasks.workunit.client.0.smithi007.stdout:7/794: truncate d0/dc/d5a/dba/dd9/d93/f10a 118849 0 2024-03-16T09:01:20.035 INFO:tasks.workunit.client.1.smithi019.stdout:0/672: write d7/f97 [655509,14736] 0 2024-03-16T09:01:20.038 INFO:tasks.workunit.client.1.smithi019.stdout:5/630: creat d6/d19/d34/dc2/fc4 x:0 0 0 2024-03-16T09:01:20.038 INFO:tasks.workunit.client.1.smithi019.stdout:1/586: rename d16/d1f/d4f/d68/d8f/c98 to d16/d1f/d3c/d79/d96/ca4 0 2024-03-16T09:01:20.040 INFO:tasks.workunit.client.1.smithi019.stdout:5/631: creat d6/dd/d47/d70/db2/d89/fc5 x:0 0 0 2024-03-16T09:01:20.041 INFO:tasks.workunit.client.0.smithi007.stdout:3/786: creat d15/d18/d1f/d29/d3e/dc0/ff4 x:0 0 0 2024-03-16T09:01:20.049 INFO:tasks.workunit.client.1.smithi019.stdout:9/494: rename d1/d7/d21/d27 to d1/d98/dae 0 2024-03-16T09:01:20.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/910: dread dc/d27/f4f [0,4194304] 0 2024-03-16T09:01:20.049 INFO:tasks.workunit.client.0.smithi007.stdout:2/911: chown dc/fa5 41610 1 2024-03-16T09:01:20.050 INFO:tasks.workunit.client.1.smithi019.stdout:9/495: getdents d1/d7/de/da3 0 2024-03-16T09:01:20.050 INFO:tasks.workunit.client.1.smithi019.stdout:9/496: chown d1/d7/de/d24/f39 25486718 1 2024-03-16T09:01:20.051 INFO:tasks.workunit.client.0.smithi007.stdout:5/791: rename d1/d7/d1c/d19/d34/d3e/f117 to d1/d7/d1c/d19/d34/d5e/d6f/da9/f11f 0 2024-03-16T09:01:20.052 INFO:tasks.workunit.client.0.smithi007.stdout:7/795: mkdir d0/d19/d1f/d5d/d9d/df7/d3a/d11b 0 2024-03-16T09:01:20.056 INFO:tasks.workunit.client.0.smithi007.stdout:3/787: symlink d15/d18/d1f/dce/dd6/ddf/lf5 0 2024-03-16T09:01:20.056 INFO:tasks.workunit.client.0.smithi007.stdout:3/788: fdatasync d15/d18/d1f/d3b/dae/fec 0 2024-03-16T09:01:20.056 INFO:tasks.workunit.client.0.smithi007.stdout:2/912: readlink dc/df/l21 0 2024-03-16T09:01:20.056 INFO:tasks.workunit.client.0.smithi007.stdout:2/913: write dc/df/d24/d28/d2e/d4a/f119 [5208087,90846] 0 2024-03-16T09:01:20.058 INFO:tasks.workunit.client.0.smithi007.stdout:3/789: rmdir d15/d18/de1 0 2024-03-16T09:01:20.059 INFO:tasks.workunit.client.0.smithi007.stdout:3/790: dread d15/d18/d1f/d56/fb2 [0,4194304] 0 2024-03-16T09:01:20.059 INFO:tasks.workunit.client.0.smithi007.stdout:2/914: rmdir dc/df/d24/d28/dea/d10d 39 2024-03-16T09:01:20.061 INFO:tasks.workunit.client.0.smithi007.stdout:3/791: mknod d15/d18/d1f/dce/dd6/cf6 0 2024-03-16T09:01:20.061 INFO:tasks.workunit.client.0.smithi007.stdout:3/792: chown d15/f16 0 1 2024-03-16T09:01:20.066 INFO:tasks.workunit.client.0.smithi007.stdout:0/899: sync 2024-03-16T09:01:20.066 INFO:tasks.workunit.client.0.smithi007.stdout:4/666: sync 2024-03-16T09:01:20.066 INFO:tasks.workunit.client.0.smithi007.stdout:9/790: sync 2024-03-16T09:01:20.066 INFO:tasks.workunit.client.0.smithi007.stdout:9/791: write d24/d4c/d91/fa5 [1161406,24638] 0 2024-03-16T09:01:20.066 INFO:tasks.workunit.client.0.smithi007.stdout:0/900: creat d3/d9/da/d48/d25/d87/dbb/f126 x:0 0 0 2024-03-16T09:01:20.066 INFO:tasks.workunit.client.0.smithi007.stdout:9/792: read - d24/d4c/d86/daa/d95/f68 zero size 2024-03-16T09:01:20.066 INFO:tasks.workunit.client.0.smithi007.stdout:0/901: fsync d3/d9/db4/de3/fe9 0 2024-03-16T09:01:20.069 INFO:tasks.workunit.client.0.smithi007.stdout:9/793: unlink d24/d4c/d86/daa/d7f/d8f/db5/cd8 0 2024-03-16T09:01:20.069 INFO:tasks.workunit.client.0.smithi007.stdout:9/794: dread - d24/d8e/fa7 zero size 2024-03-16T09:01:20.070 INFO:tasks.workunit.client.1.smithi019.stdout:5/632: dwrite d6/dd/d37/f93 [0,4194304] 0 2024-03-16T09:01:20.072 INFO:tasks.workunit.client.0.smithi007.stdout:2/915: write dc/df/d121/f120 [1254698,61904] 0 2024-03-16T09:01:20.073 INFO:tasks.workunit.client.0.smithi007.stdout:3/793: dread d15/d18/d6a/db1/fa7 [0,4194304] 0 2024-03-16T09:01:20.074 INFO:tasks.workunit.client.0.smithi007.stdout:3/794: chown d15/d18/d6a/db1/fc1 4099465 1 2024-03-16T09:01:20.074 INFO:tasks.workunit.client.1.smithi019.stdout:5/633: rename d6/dd/d37/f3d to d6/dd/d47/d70/db2/d4c/fc6 0 2024-03-16T09:01:20.075 INFO:tasks.workunit.client.1.smithi019.stdout:5/634: rename d6/d19/l2c to d6/dd/d61/lc7 0 2024-03-16T09:01:20.075 INFO:tasks.workunit.client.1.smithi019.stdout:5/635: mknod d6/dd/daf/cc8 0 2024-03-16T09:01:20.076 INFO:tasks.workunit.client.1.smithi019.stdout:5/636: symlink d6/d19/d34/dc2/lc9 0 2024-03-16T09:01:20.077 INFO:tasks.workunit.client.1.smithi019.stdout:5/637: symlink d6/dd/d47/lca 0 2024-03-16T09:01:20.078 INFO:tasks.workunit.client.1.smithi019.stdout:0/673: dwrite d7/d8/d32/d66/f91 [0,4194304] 0 2024-03-16T09:01:20.079 INFO:tasks.workunit.client.1.smithi019.stdout:5/638: mknod d6/dd/d47/d70/db2/d4c/ccb 0 2024-03-16T09:01:20.080 INFO:tasks.workunit.client.0.smithi007.stdout:0/902: creat d3/d9/da/d18/f127 x:0 0 0 2024-03-16T09:01:20.081 INFO:tasks.workunit.client.0.smithi007.stdout:4/667: dread d4/d14/d64/d68/f6f [0,4194304] 0 2024-03-16T09:01:20.081 INFO:tasks.workunit.client.0.smithi007.stdout:4/668: chown d4/d14/d1c/d5b/d67/d8f/da8/d9d/dd7/fb8 1252 1 2024-03-16T09:01:20.081 INFO:tasks.workunit.client.0.smithi007.stdout:4/669: fdatasync d4/dcb/fd8 0 2024-03-16T09:01:20.084 INFO:tasks.workunit.client.1.smithi019.stdout:0/674: rename d7/d1d/d28/d92/dcc/fdc to d7/d1d/d28/d48/fe0 0 2024-03-16T09:01:20.085 INFO:tasks.workunit.client.1.smithi019.stdout:0/675: creat d7/d1d/fe1 x:0 0 0 2024-03-16T09:01:20.092 INFO:tasks.workunit.client.0.smithi007.stdout:2/916: symlink dc/df/d24/d28/d2e/d6e/d114/l131 0 2024-03-16T09:01:20.093 INFO:tasks.workunit.client.0.smithi007.stdout:3/795: truncate d15/d18/d1f/d29/d43/f55 3824695 0 2024-03-16T09:01:20.095 INFO:tasks.workunit.client.0.smithi007.stdout:0/903: creat d3/d9/da/d48/d25/d87/f128 x:0 0 0 2024-03-16T09:01:20.096 INFO:tasks.workunit.client.0.smithi007.stdout:0/904: write d3/d9/da/d48/d4b/f52 [831490,78693] 0 2024-03-16T09:01:20.096 INFO:tasks.workunit.client.0.smithi007.stdout:0/905: creat d3/d9/db4/d60/f129 x:0 0 0 2024-03-16T09:01:20.096 INFO:tasks.workunit.client.0.smithi007.stdout:4/670: rename d4/d14/d1c/l21 to d4/d14/d64/d68/dbf/le2 0 2024-03-16T09:01:20.097 INFO:tasks.workunit.client.0.smithi007.stdout:9/795: rmdir d24/d4a/d96/db9/db1 39 2024-03-16T09:01:20.098 INFO:tasks.workunit.client.0.smithi007.stdout:3/796: rmdir d15/d18/d1f/d56/d98 39 2024-03-16T09:01:20.102 INFO:tasks.workunit.client.0.smithi007.stdout:4/671: mkdir d4/d14/d64/d9f/d9a/de3 0 2024-03-16T09:01:20.103 INFO:tasks.workunit.client.0.smithi007.stdout:9/796: mknod d24/d4a/d55/dbf/de9/ceb 0 2024-03-16T09:01:20.104 INFO:tasks.workunit.client.0.smithi007.stdout:3/797: mkdir d15/d18/d1f/d29/d43/df7 0 2024-03-16T09:01:20.104 INFO:tasks.workunit.client.0.smithi007.stdout:3/798: creat d15/d18/d1f/d29/d7c/ff8 x:0 0 0 2024-03-16T09:01:20.105 INFO:tasks.workunit.client.0.smithi007.stdout:4/672: link d4/d14/d64/d68/d82/dac/fe1 d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/fe4 0 2024-03-16T09:01:20.109 INFO:tasks.workunit.client.0.smithi007.stdout:9/797: dread d24/f5c [0,4194304] 0 2024-03-16T09:01:20.109 INFO:tasks.workunit.client.0.smithi007.stdout:9/798: chown d24/d4c/d86/daa/d95/c9f 62 1 2024-03-16T09:01:20.110 INFO:tasks.workunit.client.0.smithi007.stdout:9/799: stat d24/d4c/d86/daa/d7f/d80/d87/dda/ddb 0 2024-03-16T09:01:20.116 INFO:tasks.workunit.client.0.smithi007.stdout:3/799: rename fa to d15/d18/d1f/d29/d43/ff9 0 2024-03-16T09:01:20.117 INFO:tasks.workunit.client.0.smithi007.stdout:4/673: rename d4/f2f to d4/dcb/fe5 0 2024-03-16T09:01:20.119 INFO:tasks.workunit.client.0.smithi007.stdout:4/674: unlink d4/d14/d64/d68/d82/dac/lb0 0 2024-03-16T09:01:20.119 INFO:tasks.workunit.client.0.smithi007.stdout:4/675: creat d4/d14/fe6 x:0 0 0 2024-03-16T09:01:20.120 INFO:tasks.workunit.client.0.smithi007.stdout:4/676: creat d4/dcb/dcd/fe7 x:0 0 0 2024-03-16T09:01:20.121 INFO:tasks.workunit.client.0.smithi007.stdout:4/677: mkdir d4/d14/d1c/d5b/d67/de8 0 2024-03-16T09:01:20.121 INFO:tasks.workunit.client.0.smithi007.stdout:4/678: write d4/d14/d64/fb5 [126475,68390] 0 2024-03-16T09:01:20.123 INFO:tasks.workunit.client.0.smithi007.stdout:4/679: mkdir d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/de9 0 2024-03-16T09:01:20.148 INFO:tasks.workunit.client.0.smithi007.stdout:3/800: dwrite d15/d18/d6a/f7d [0,4194304] 0 2024-03-16T09:01:20.150 INFO:tasks.workunit.client.1.smithi019.stdout:4/643: dwrite d4/d1b/f7e [0,4194304] 0 2024-03-16T09:01:20.150 INFO:tasks.workunit.client.1.smithi019.stdout:1/587: dwrite d16/d1f/f74 [0,4194304] 0 2024-03-16T09:01:20.150 INFO:tasks.workunit.client.0.smithi007.stdout:3/801: mkdir d15/d18/d1f/d29/dfa 0 2024-03-16T09:01:20.152 INFO:tasks.workunit.client.0.smithi007.stdout:3/802: creat d15/d18/d6a/d78/ffb x:0 0 0 2024-03-16T09:01:20.158 INFO:tasks.workunit.client.0.smithi007.stdout:4/680: fsync d4/dcb/fe5 0 2024-03-16T09:01:20.158 INFO:tasks.workunit.client.1.smithi019.stdout:9/497: dread d1/d7/de/d24/d5b/f6e [0,4194304] 0 2024-03-16T09:01:20.158 INFO:tasks.workunit.client.1.smithi019.stdout:9/498: chown d1/d7/de/d24/f42 103 1 2024-03-16T09:01:20.159 INFO:tasks.workunit.client.1.smithi019.stdout:6/759: dwrite de/d26/d32/d97/d66/d72/d7f/dd5/fe9 [0,4194304] 0 2024-03-16T09:01:20.160 INFO:tasks.workunit.client.0.smithi007.stdout:4/681: mknod d4/d14/d1c/d5b/d67/d8f/da8/d9d/dbb/dd2/cea 0 2024-03-16T09:01:20.161 INFO:tasks.workunit.client.0.smithi007.stdout:0/906: dwrite d3/d9/da/d48/f13 [4194304,4194304] 0 2024-03-16T09:01:20.162 INFO:tasks.workunit.client.0.smithi007.stdout:0/907: chown d3/d9/db4/d2c/fc1 951953522 1 2024-03-16T09:01:20.162 INFO:tasks.workunit.client.0.smithi007.stdout:0/908: creat d3/d9/da/d48/d25/d87/dbb/f12a x:0 0 0 2024-03-16T09:01:20.162 INFO:tasks.workunit.client.0.smithi007.stdout:4/682: link d4/d14/d1c/f6a d4/dcb/feb 0 2024-03-16T09:01:20.162 INFO:tasks.workunit.client.0.smithi007.stdout:0/909: dread d3/d9/f93 [0,4194304] 0 2024-03-16T09:01:20.163 INFO:tasks.workunit.client.1.smithi019.stdout:4/644: mkdir d4/d1b/d2a/d3d/dd3 0 2024-03-16T09:01:20.164 INFO:tasks.workunit.client.1.smithi019.stdout:8/718: dread d6/d2a/d3f/f7d [0,4194304] 0 2024-03-16T09:01:20.165 INFO:tasks.workunit.client.0.smithi007.stdout:9/800: dread d24/d4c/d86/daa/d7f/d80/d87/d89/fb3 [0,4194304] 0 2024-03-16T09:01:20.165 INFO:tasks.workunit.client.1.smithi019.stdout:1/588: fsync d16/d18/f6c 0 2024-03-16T09:01:20.168 INFO:tasks.workunit.client.1.smithi019.stdout:6/760: mknod de/d26/d32/d97/cf1 0 2024-03-16T09:01:20.169 INFO:tasks.workunit.client.1.smithi019.stdout:4/645: mknod d4/d6/d15/cd4 0 2024-03-16T09:01:20.175 INFO:tasks.workunit.client.1.smithi019.stdout:8/719: getdents d6 0 2024-03-16T09:01:20.178 INFO:tasks.workunit.client.0.smithi007.stdout:0/910: mkdir d3/d9/da/d48/d25/d11f/d12b 0 2024-03-16T09:01:20.178 INFO:tasks.workunit.client.0.smithi007.stdout:0/911: chown d3/d6c/dbd/dd7/cf4 32460595 1 2024-03-16T09:01:20.212 INFO:tasks.workunit.client.1.smithi019.stdout:8/720: dread d6/d2a/d45/d19/f1c [0,4194304] 0 2024-03-16T09:01:20.212 INFO:tasks.workunit.client.1.smithi019.stdout:8/721: write d6/fd4 [187860,51971] 0 2024-03-16T09:01:20.229 INFO:tasks.workunit.client.1.smithi019.stdout:0/676: dwrite d7/f5d [0,4194304] 0 2024-03-16T09:01:20.229 INFO:tasks.workunit.client.1.smithi019.stdout:9/499: dwrite d1/d98/dae/f55 [0,4194304] 0 2024-03-16T09:01:20.230 INFO:tasks.workunit.client.0.smithi007.stdout:7/796: dwrite d0/d19/d1f/f116 [0,4194304] 0 2024-03-16T09:01:20.231 INFO:tasks.workunit.client.1.smithi019.stdout:0/677: symlink d7/d1d/d63/d6e/db0/le2 0 2024-03-16T09:01:20.231 INFO:tasks.workunit.client.1.smithi019.stdout:0/678: stat d7/d8/d32/d66/d6d/d8c/db8/d82/fba 0 2024-03-16T09:01:20.233 INFO:tasks.workunit.client.1.smithi019.stdout:9/500: mknod d1/d98/dae/caf 0 2024-03-16T09:01:20.235 INFO:tasks.workunit.client.1.smithi019.stdout:9/501: rename d1/d7/f4c to d1/d7/de/da3/fb0 0 2024-03-16T09:01:20.237 INFO:tasks.workunit.client.0.smithi007.stdout:4/683: dwrite d4/f17 [0,4194304] 0 2024-03-16T09:01:20.237 INFO:tasks.workunit.client.1.smithi019.stdout:9/502: link d1/d7/de/d24/d5b/c71 d1/d7/d67/d77/cb1 0 2024-03-16T09:01:20.237 INFO:tasks.workunit.client.1.smithi019.stdout:9/503: truncate d1/d7/d86/f8c 996285 0 2024-03-16T09:01:20.238 INFO:tasks.workunit.client.1.smithi019.stdout:1/589: dwrite d16/d31/f69 [0,4194304] 0 2024-03-16T09:01:20.238 INFO:tasks.workunit.client.0.smithi007.stdout:0/912: dwrite d3/d9/db4/d2c/f5e [0,4194304] 0 2024-03-16T09:01:20.238 INFO:tasks.workunit.client.0.smithi007.stdout:3/803: dwrite d15/d18/d1f/d29/d3e/d85/fe2 [0,4194304] 0 2024-03-16T09:01:20.239 INFO:tasks.workunit.client.1.smithi019.stdout:8/722: dwrite d6/d2a/d45/d19/d74/f8a [0,4194304] 0 2024-03-16T09:01:20.240 INFO:tasks.workunit.client.0.smithi007.stdout:9/801: dwrite d24/f4f [0,4194304] 0 2024-03-16T09:01:20.240 INFO:tasks.workunit.client.1.smithi019.stdout:4/646: rmdir d4/d1b/d2a/d3d 39 2024-03-16T09:01:20.246 INFO:tasks.workunit.client.0.smithi007.stdout:4/684: unlink d4/d14/d1c/f2c 0 2024-03-16T09:01:20.249 INFO:tasks.workunit.client.0.smithi007.stdout:6/712: sync 2024-03-16T09:01:20.253 INFO:tasks.workunit.client.1.smithi019.stdout:1/590: dread d16/d18/f24 [0,4194304] 0 2024-03-16T09:01:20.259 INFO:tasks.workunit.client.1.smithi019.stdout:0/679: dwrite d7/d8/d32/d66/d6d/d8c/db8/f8f [0,4194304] 0 2024-03-16T09:01:20.259 INFO:tasks.workunit.client.1.smithi019.stdout:0/680: dread - d7/f6a zero size 2024-03-16T09:01:20.259 INFO:tasks.workunit.client.1.smithi019.stdout:0/681: creat d7/d8/d32/d66/d6d/d8c/db8/d82/fe3 x:0 0 0 2024-03-16T09:01:20.265 INFO:tasks.workunit.client.0.smithi007.stdout:0/913: dwrite d3/d9/da/d48/f38 [4194304,4194304] 0 2024-03-16T09:01:20.269 INFO:tasks.workunit.client.1.smithi019.stdout:8/723: rename d6/d2a/d45/d2c/lcc to d6/d2a/d45/d2c/de3/lf8 0 2024-03-16T09:01:20.270 INFO:tasks.workunit.client.1.smithi019.stdout:4/647: dwrite d4/f35 [0,4194304] 0 2024-03-16T09:01:20.270 INFO:tasks.workunit.client.1.smithi019.stdout:4/648: fsync d4/d1b/f57 0 2024-03-16T09:01:20.271 INFO:tasks.workunit.client.1.smithi019.stdout:4/649: write d4/d1b/f28 [1077139,108878] 0 2024-03-16T09:01:20.271 INFO:tasks.workunit.client.1.smithi019.stdout:4/650: creat d4/d1b/d49/d6e/fd5 x:0 0 0 2024-03-16T09:01:20.272 INFO:tasks.workunit.client.0.smithi007.stdout:3/804: mknod d15/d18/d1f/d3b/d63/cfc 0 2024-03-16T09:01:20.275 INFO:tasks.workunit.client.1.smithi019.stdout:0/682: creat d7/fe4 x:0 0 0 2024-03-16T09:01:20.276 INFO:tasks.workunit.client.1.smithi019.stdout:1/591: rename d16/d18/l4a to d16/d1f/d4f/la5 0 2024-03-16T09:01:20.277 INFO:tasks.workunit.client.1.smithi019.stdout:9/504: truncate d1/d98/dae/f55 3253084 0 2024-03-16T09:01:20.278 INFO:tasks.workunit.client.1.smithi019.stdout:4/651: mkdir d4/d73/d8a/dd6 0 2024-03-16T09:01:20.279 INFO:tasks.workunit.client.1.smithi019.stdout:0/683: symlink d7/d1d/d63/le5 0 2024-03-16T09:01:20.279 INFO:tasks.workunit.client.1.smithi019.stdout:0/684: dread - d7/d1d/d28/d48/fde zero size 2024-03-16T09:01:20.281 INFO:tasks.workunit.client.0.smithi007.stdout:9/802: mkdir d24/d4a/d55/dbf/de9/dec 0 2024-03-16T09:01:20.281 INFO:tasks.workunit.client.0.smithi007.stdout:9/803: write d24/d4c/f7a [852969,87327] 0 2024-03-16T09:01:20.282 INFO:tasks.workunit.client.1.smithi019.stdout:1/592: symlink d16/d1f/d43/d55/la6 0 2024-03-16T09:01:20.282 INFO:tasks.workunit.client.1.smithi019.stdout:1/593: write d16/d1f/f41 [692238,121625] 0 2024-03-16T09:01:20.283 INFO:tasks.workunit.client.0.smithi007.stdout:7/797: rmdir d0/d19/d1f/d69/dc7 39 2024-03-16T09:01:20.283 INFO:tasks.workunit.client.1.smithi019.stdout:7/582: sync 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:3/622: sync 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:7/583: write d9/da/d13/d9a/faf [67316,13261] 0 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:3/623: write d3/d9b/fa9 [460647,29176] 0 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:7/584: chown d9/da/d13/d18/f83 254951251 1 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:3/624: chown d3/d8b/dae/d6a/lb4 45297027 1 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:7/585: truncate d9/da/d13/d18/d30/f97 579611 0 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:3/625: chown d3/d1b/d40/d4a/f98 9 1 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:9/505: unlink d1/d7/de/d24/f47 0 2024-03-16T09:01:20.284 INFO:tasks.workunit.client.1.smithi019.stdout:9/506: chown d1/d7/d33/d59/f73 143903177 1 2024-03-16T09:01:20.285 INFO:tasks.workunit.client.0.smithi007.stdout:4/685: symlink d4/d14/d64/d9f/lec 0 2024-03-16T09:01:20.287 INFO:tasks.workunit.client.1.smithi019.stdout:4/652: unlink d4/d1b/d49/d6e/fd5 0 2024-03-16T09:01:20.288 INFO:tasks.workunit.client.0.smithi007.stdout:6/713: rename d2/d13/d4b/d94/lb0 to d2/d13/d3f/d50/led 0 2024-03-16T09:01:20.288 INFO:tasks.workunit.client.0.smithi007.stdout:6/714: write d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/faf [717029,63699] 0 2024-03-16T09:01:20.288 INFO:tasks.workunit.client.0.smithi007.stdout:6/715: chown d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/f97 15842107 1 2024-03-16T09:01:20.288 INFO:tasks.workunit.client.0.smithi007.stdout:6/716: fdatasync d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/f80 0 2024-03-16T09:01:20.288 INFO:tasks.workunit.client.0.smithi007.stdout:6/717: read - d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/fd9 zero size 2024-03-16T09:01:20.289 INFO:tasks.workunit.client.1.smithi019.stdout:0/685: link d7/d1d/d63/f7b d7/d8/d32/d4a/d61/dbf/fe6 0 2024-03-16T09:01:20.289 INFO:tasks.workunit.client.1.smithi019.stdout:8/724: getdents d6/d2a/d45/d5d 0 2024-03-16T09:01:20.289 INFO:tasks.workunit.client.1.smithi019.stdout:8/725: chown d6/d2a/d45/d2c/de3 69 1 2024-03-16T09:01:20.290 INFO:tasks.workunit.client.1.smithi019.stdout:8/726: chown d6/d2a/d3c/dab/d30/d4a/d67/d95/fb9 50786943 1 2024-03-16T09:01:20.290 INFO:tasks.workunit.client.0.smithi007.stdout:9/804: dread d24/f46 [0,4194304] 0 2024-03-16T09:01:20.290 INFO:tasks.workunit.client.0.smithi007.stdout:0/914: creat d3/d9/da/d48/d25/d43/d1a/f12c x:0 0 0 2024-03-16T09:01:20.290 INFO:tasks.workunit.client.0.smithi007.stdout:0/915: truncate d3/d9/da/d18/f127 218417 0 2024-03-16T09:01:20.290 INFO:tasks.workunit.client.0.smithi007.stdout:0/916: read d3/d1f/f10c [98841,98121] 0 2024-03-16T09:01:20.290 INFO:tasks.workunit.client.0.smithi007.stdout:0/917: chown d3/d6c/dbd/dd7/d7c/dec 0 1 2024-03-16T09:01:20.291 INFO:tasks.workunit.client.0.smithi007.stdout:0/918: truncate d3/d9/da/d48/d9e/f117 661377 0 2024-03-16T09:01:20.291 INFO:tasks.workunit.client.0.smithi007.stdout:0/919: chown d3/d9/da/d48/d9e/lda 1 1 2024-03-16T09:01:20.291 INFO:tasks.workunit.client.0.smithi007.stdout:0/920: stat d3/d1f/f71 0 2024-03-16T09:01:20.291 INFO:tasks.workunit.client.0.smithi007.stdout:7/798: rmdir d0/d19/d1f/d5d/d9d/df7/da1 39 2024-03-16T09:01:20.291 INFO:tasks.workunit.client.0.smithi007.stdout:0/921: dread d3/d9/da/d18/d27/d34/fea [0,4194304] 0 2024-03-16T09:01:20.291 INFO:tasks.workunit.client.0.smithi007.stdout:0/922: write d3/d9/da/d18/d23/d36/f100 [3745895,69337] 0 2024-03-16T09:01:20.291 INFO:tasks.workunit.client.1.smithi019.stdout:1/594: creat d16/d1f/d73/fa7 x:0 0 0 2024-03-16T09:01:20.292 INFO:tasks.workunit.client.0.smithi007.stdout:4/686: symlink d4/d14/d1c/d5b/led 0 2024-03-16T09:01:20.293 INFO:tasks.workunit.client.1.smithi019.stdout:7/586: mkdir d9/da/d13/d18/d30/d3e/d5d/dad/db9 0 2024-03-16T09:01:20.294 INFO:tasks.workunit.client.0.smithi007.stdout:6/718: mknod d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/cee 0 2024-03-16T09:01:20.294 INFO:tasks.workunit.client.0.smithi007.stdout:6/719: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/f87 [2208153,4981] 0 2024-03-16T09:01:20.294 INFO:tasks.workunit.client.0.smithi007.stdout:9/805: symlink d24/d4a/d96/db9/db1/led 0 2024-03-16T09:01:20.296 INFO:tasks.workunit.client.0.smithi007.stdout:7/799: creat d0/d19/d1f/d5d/d9d/df7/f11c x:0 0 0 2024-03-16T09:01:20.297 INFO:tasks.workunit.client.1.smithi019.stdout:1/595: symlink d16/d1f/d4f/d68/d8f/la8 0 2024-03-16T09:01:20.298 INFO:tasks.workunit.client.0.smithi007.stdout:4/687: truncate d4/d14/f2d 165447 0 2024-03-16T09:01:20.298 INFO:tasks.workunit.client.0.smithi007.stdout:3/805: rename d15/d18/d1f/d29/f1b to d15/d18/d1f/ffd 0 2024-03-16T09:01:20.298 INFO:tasks.workunit.client.0.smithi007.stdout:3/806: readlink d15/d18/d1f/d3b/d63/ld2 0 2024-03-16T09:01:20.298 INFO:tasks.workunit.client.0.smithi007.stdout:9/806: mknod d24/d4a/d55/dbf/cee 0 2024-03-16T09:01:20.299 INFO:tasks.workunit.client.1.smithi019.stdout:4/653: dread d4/d1b/d49/f72 [0,4194304] 0 2024-03-16T09:01:20.299 INFO:tasks.workunit.client.1.smithi019.stdout:4/654: chown d4/d1b/f1c 194129873 1 2024-03-16T09:01:20.300 INFO:tasks.workunit.client.1.smithi019.stdout:0/686: read d7/d8/d32/d66/d6d/d8c/db8/f69 [306074,78477] 0 2024-03-16T09:01:20.300 INFO:tasks.workunit.client.1.smithi019.stdout:5/639: sync 2024-03-16T09:01:20.300 INFO:tasks.workunit.client.1.smithi019.stdout:0/687: stat d7/f19 0 2024-03-16T09:01:20.300 INFO:tasks.workunit.client.0.smithi007.stdout:7/800: creat d0/d19/d1f/d5d/d9d/df7/d73/ded/f11d x:0 0 0 2024-03-16T09:01:20.300 INFO:tasks.workunit.client.0.smithi007.stdout:7/801: truncate d0/d19/d1f/d5d/d9d/df7/d73/f91 5467990 0 2024-03-16T09:01:20.300 INFO:tasks.workunit.client.0.smithi007.stdout:0/923: write d3/d9/da/d48/d25/d43/d1a/d58/d61/f110 [3655872,43608] 0 2024-03-16T09:01:20.300 INFO:tasks.workunit.client.0.smithi007.stdout:0/924: creat d3/d9/db4/d1c/f12d x:0 0 0 2024-03-16T09:01:20.301 INFO:tasks.workunit.client.0.smithi007.stdout:0/925: read d3/d9/da/d18/d23/f81 [229873,103371] 0 2024-03-16T09:01:20.301 INFO:tasks.workunit.client.1.smithi019.stdout:1/596: getdents d16/d31/d76 0 2024-03-16T09:01:20.301 INFO:tasks.workunit.client.0.smithi007.stdout:4/688: unlink d4/d14/d1c/d5b/d67/d8f/da8/d9d/cc8 0 2024-03-16T09:01:20.302 INFO:tasks.workunit.client.1.smithi019.stdout:4/655: creat d4/d1b/d49/d58/d80/fd7 x:0 0 0 2024-03-16T09:01:20.303 INFO:tasks.workunit.client.1.smithi019.stdout:5/640: chown d6/dd/d47/d70/db2/d4c/fa8 3829 1 2024-03-16T09:01:20.303 INFO:tasks.workunit.client.1.smithi019.stdout:5/641: fsync d6/dd/d47/d70/db2/d66/f45 0 2024-03-16T09:01:20.303 INFO:tasks.workunit.client.1.smithi019.stdout:5/642: chown d6/dd/d37/f53 4 1 2024-03-16T09:01:20.303 INFO:tasks.workunit.client.1.smithi019.stdout:5/643: truncate d6/d1f/d77/d84/f92 965002 0 2024-03-16T09:01:20.303 INFO:tasks.workunit.client.1.smithi019.stdout:5/644: truncate d6/dd/d37/d76/fb7 348685 0 2024-03-16T09:01:20.303 INFO:tasks.workunit.client.1.smithi019.stdout:5/645: stat d6/d1f/f3f 0 2024-03-16T09:01:20.304 INFO:tasks.workunit.client.0.smithi007.stdout:6/720: rename d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/f56 to d2/d13/d3f/d50/d78/d81/fef 0 2024-03-16T09:01:20.304 INFO:tasks.workunit.client.0.smithi007.stdout:6/721: readlink d2/d13/d3f/d50/d78/d81/lb2 0 2024-03-16T09:01:20.305 INFO:tasks.workunit.client.1.smithi019.stdout:1/597: mknod d16/d1f/d4f/d8a/ca9 0 2024-03-16T09:01:20.305 INFO:tasks.workunit.client.1.smithi019.stdout:1/598: truncate d16/f2b 5086114 0 2024-03-16T09:01:20.305 INFO:tasks.workunit.client.1.smithi019.stdout:1/599: readlink d16/d1f/d4f/la5 0 2024-03-16T09:01:20.305 INFO:tasks.workunit.client.1.smithi019.stdout:1/600: dread - d16/d1f/d73/fa7 zero size 2024-03-16T09:01:20.305 INFO:tasks.workunit.client.1.smithi019.stdout:1/601: write d16/d1f/d3c/d79/f91 [1464308,90596] 0 2024-03-16T09:01:20.305 INFO:tasks.workunit.client.1.smithi019.stdout:4/656: truncate d4/f12 1249306 0 2024-03-16T09:01:20.305 INFO:tasks.workunit.client.1.smithi019.stdout:1/602: dread - d16/f9c zero size 2024-03-16T09:01:20.306 INFO:tasks.workunit.client.0.smithi007.stdout:3/807: creat d15/d18/d1f/d3b/d87/ffe x:0 0 0 2024-03-16T09:01:20.306 INFO:tasks.workunit.client.0.smithi007.stdout:3/808: truncate d15/d18/d1f/f8d 4542346 0 2024-03-16T09:01:20.307 INFO:tasks.workunit.client.1.smithi019.stdout:8/727: dread f4 [0,4194304] 0 2024-03-16T09:01:20.307 INFO:tasks.workunit.client.1.smithi019.stdout:1/603: getdents d16 0 2024-03-16T09:01:20.307 INFO:tasks.workunit.client.1.smithi019.stdout:8/728: chown d6/d2a/d45/f6b 236 1 2024-03-16T09:01:20.308 INFO:tasks.workunit.client.0.smithi007.stdout:7/802: write d0/d19/d1f/d5d/d9d/df7/d3a/f96 [1433549,86184] 0 2024-03-16T09:01:20.308 INFO:tasks.workunit.client.0.smithi007.stdout:9/807: mkdir d24/d4c/d86/daa/d7f/d80/d87/d89/def 0 2024-03-16T09:01:20.311 INFO:tasks.workunit.client.0.smithi007.stdout:0/926: mknod d3/d9/da/d18/d23/c12e 0 2024-03-16T09:01:20.311 INFO:tasks.workunit.client.1.smithi019.stdout:5/646: dread d6/f2f [0,4194304] 0 2024-03-16T09:01:20.312 INFO:tasks.workunit.client.1.smithi019.stdout:2/720: rename d5/d6/d64 to d5/d5a/d96/df7 0 2024-03-16T09:01:20.312 INFO:tasks.workunit.client.1.smithi019.stdout:5/647: truncate d6/dd/d47/d70/db2/d4c/f6c 585559 0 2024-03-16T09:01:20.312 INFO:tasks.workunit.client.1.smithi019.stdout:5/648: chown d6/d1f/l3a 3 1 2024-03-16T09:01:20.313 INFO:tasks.workunit.client.0.smithi007.stdout:4/689: creat d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/d93/fee x:0 0 0 2024-03-16T09:01:20.313 INFO:tasks.workunit.client.0.smithi007.stdout:4/690: write d4/d14/d1c/d5b/fa0 [748648,74949] 0 2024-03-16T09:01:20.314 INFO:tasks.workunit.client.1.smithi019.stdout:3/626: rename d3/d8b/dae/d6a/lb4 to d3/d1b/d40/d4a/db7/lc0 0 2024-03-16T09:01:20.314 INFO:tasks.workunit.client.1.smithi019.stdout:3/627: read - d3/d8b/dae/d5a/fbb zero size 2024-03-16T09:01:20.314 INFO:tasks.workunit.client.1.smithi019.stdout:5/649: mkdir d6/dd/d47/d70/db2/d89/dcc 0 2024-03-16T09:01:20.316 INFO:tasks.workunit.client.1.smithi019.stdout:4/657: read f2 [2045158,50048] 0 2024-03-16T09:01:20.317 INFO:tasks.workunit.client.1.smithi019.stdout:5/650: symlink d6/dd/d47/d7a/lcd 0 2024-03-16T09:01:20.318 INFO:tasks.workunit.client.0.smithi007.stdout:6/722: link d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/f35 d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c/ff0 0 2024-03-16T09:01:20.319 INFO:tasks.workunit.client.1.smithi019.stdout:8/729: write d6/de/ff [292671,40316] 0 2024-03-16T09:01:20.319 INFO:tasks.workunit.client.1.smithi019.stdout:7/587: rename d9/da/d13/d18/d30/d3e/d5d to d9/d25/d28/d48/d8d/dba 0 2024-03-16T09:01:20.321 INFO:tasks.workunit.client.0.smithi007.stdout:3/809: rename d15/d18/d6a/db1/fa3 to d15/d18/d1f/d3b/d87/dee/fff 0 2024-03-16T09:01:20.322 INFO:tasks.workunit.client.1.smithi019.stdout:9/507: dwrite d1/d7/f34 [4194304,4194304] 0 2024-03-16T09:01:20.322 INFO:tasks.workunit.client.1.smithi019.stdout:7/588: mknod d9/d25/d28/d48/d8d/dba/cbb 0 2024-03-16T09:01:20.325 INFO:tasks.workunit.client.0.smithi007.stdout:7/803: truncate d0/d19/d1f/d5d/d9d/df7/f7f 3140542 0 2024-03-16T09:01:20.325 INFO:tasks.workunit.client.0.smithi007.stdout:7/804: creat d0/d19/d1f/d69/f11e x:0 0 0 2024-03-16T09:01:20.327 INFO:tasks.workunit.client.0.smithi007.stdout:9/808: chown d24/f3c 2704 1 2024-03-16T09:01:20.327 INFO:tasks.workunit.client.0.smithi007.stdout:9/809: write d24/d4c/d86/daa/d7f/d80/d87/d89/fc2 [476523,45961] 0 2024-03-16T09:01:20.329 INFO:tasks.workunit.client.0.smithi007.stdout:0/927: truncate d3/d9/da/d18/d27/f83 163128 0 2024-03-16T09:01:20.329 INFO:tasks.workunit.client.0.smithi007.stdout:0/928: creat d3/d9/da/d48/d25/d43/d1a/d58/d61/f12f x:0 0 0 2024-03-16T09:01:20.329 INFO:tasks.workunit.client.0.smithi007.stdout:0/929: write d3/d9/da/d18/d23/d36/fef [4281726,75959] 0 2024-03-16T09:01:20.334 INFO:tasks.workunit.client.0.smithi007.stdout:4/691: link d4/d14/d1c/f3e d4/d14/d1c/d5b/d67/d8f/da8/dbe/fef 0 2024-03-16T09:01:20.335 INFO:tasks.workunit.client.1.smithi019.stdout:0/688: rename d7/c11 to d7/d8/d32/d4a/d61/d68/ce7 0 2024-03-16T09:01:20.335 INFO:tasks.workunit.client.1.smithi019.stdout:9/508: write d1/d98/dae/f3c [1730051,115181] 0 2024-03-16T09:01:20.335 INFO:tasks.workunit.client.1.smithi019.stdout:0/689: creat d7/d1d/d28/d6c/d87/fe8 x:0 0 0 2024-03-16T09:01:20.335 INFO:tasks.workunit.client.1.smithi019.stdout:9/509: write d1/d7/d33/d59/f85 [192970,86319] 0 2024-03-16T09:01:20.336 INFO:tasks.workunit.client.0.smithi007.stdout:3/810: read d15/f16 [1922425,128247] 0 2024-03-16T09:01:20.336 INFO:tasks.workunit.client.0.smithi007.stdout:3/811: stat d15/d18/d1f/d29/d7c/fb0 0 2024-03-16T09:01:20.336 INFO:tasks.workunit.client.0.smithi007.stdout:6/723: truncate d2/d13/d3f/d50/d78/dbe/dcb/f79 1173735 0 2024-03-16T09:01:20.337 INFO:tasks.workunit.client.0.smithi007.stdout:7/805: write d0/d19/d1f/d5d/d9d/df7/d3a/daf/dec/f105 [1096886,48607] 0 2024-03-16T09:01:20.338 INFO:tasks.workunit.client.1.smithi019.stdout:9/510: creat d1/d7/fb2 x:0 0 0 2024-03-16T09:01:20.339 INFO:tasks.workunit.client.1.smithi019.stdout:1/604: rename d16/d84/c88 to d16/d31/d71/d89/caa 0 2024-03-16T09:01:20.340 INFO:tasks.workunit.client.1.smithi019.stdout:4/658: rename d4/f12 to d4/d73/d8a/fd8 0 2024-03-16T09:01:20.342 INFO:tasks.workunit.client.1.smithi019.stdout:1/605: link d16/d1f/l5a d16/da1/lab 0 2024-03-16T09:01:20.342 INFO:tasks.workunit.client.1.smithi019.stdout:1/606: stat d16/d18/f6c 0 2024-03-16T09:01:20.342 INFO:tasks.workunit.client.1.smithi019.stdout:9/511: getdents d1/d7/de/d24 0 2024-03-16T09:01:20.343 INFO:tasks.workunit.client.1.smithi019.stdout:9/512: fsync d1/d98/dae/fa7 0 2024-03-16T09:01:20.343 INFO:tasks.workunit.client.1.smithi019.stdout:9/513: truncate d1/f6a 740033 0 2024-03-16T09:01:20.343 INFO:tasks.workunit.client.1.smithi019.stdout:9/514: chown d1/d7/de/d24/d44/f53 828 1 2024-03-16T09:01:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:3/812: dread f5 [0,4194304] 0 2024-03-16T09:01:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:3/813: creat d15/d18/d6a/f100 x:0 0 0 2024-03-16T09:01:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:3/814: creat d15/d18/d1f/d56/f101 x:0 0 0 2024-03-16T09:01:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:3/815: chown d15/d18/d1f/d29/d3e/dda/fe5 825 1 2024-03-16T09:01:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:3/816: creat d15/d18/d6a/db1/f102 x:0 0 0 2024-03-16T09:01:20.345 INFO:tasks.workunit.client.0.smithi007.stdout:3/817: dread - d15/d18/d1f/d29/d43/fe6 zero size 2024-03-16T09:01:20.346 INFO:tasks.workunit.client.0.smithi007.stdout:0/930: rename d3/l113 to d3/d9/da/d48/d25/d11f/d12b/l130 0 2024-03-16T09:01:20.346 INFO:tasks.workunit.client.1.smithi019.stdout:0/690: write d7/d1d/d63/d6e/f93 [807917,121754] 0 2024-03-16T09:01:20.346 INFO:tasks.workunit.client.1.smithi019.stdout:4/659: unlink d4/d64/faa 0 2024-03-16T09:01:20.347 INFO:tasks.workunit.client.1.smithi019.stdout:1/607: mkdir d16/d1f/d3c/d79/d96/dac 0 2024-03-16T09:01:20.349 INFO:tasks.workunit.client.1.smithi019.stdout:9/515: unlink d1/d98/dae/f14 0 2024-03-16T09:01:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:4/692: rmdir d4/d14/d64/d9f/d9a/de3 0 2024-03-16T09:01:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:4/693: write d4/d14/d64/d9f/f53 [84964,120441] 0 2024-03-16T09:01:20.350 INFO:tasks.workunit.client.0.smithi007.stdout:4/694: fdatasync d4/d14/d64/d9f/f42 0 2024-03-16T09:01:20.351 INFO:tasks.workunit.client.1.smithi019.stdout:0/691: creat d7/d1d/d28/d6c/fe9 x:0 0 0 2024-03-16T09:01:20.351 INFO:tasks.workunit.client.1.smithi019.stdout:2/721: mknod d5/d5a/d96/dba/de8/cf8 0 2024-03-16T09:01:20.351 INFO:tasks.workunit.client.1.smithi019.stdout:2/722: fdatasync d5/d6/d2b/d2f/d35/d65/fb4 0 2024-03-16T09:01:20.351 INFO:tasks.workunit.client.1.smithi019.stdout:2/723: write d5/d6/d2b/d2f/d35/f56 [5692,104580] 0 2024-03-16T09:01:20.353 INFO:tasks.workunit.client.1.smithi019.stdout:4/660: rename d4/d1b/d2a/d4f/f82 to d4/d1b/d49/d58/d80/d91/fd9 0 2024-03-16T09:01:20.353 INFO:tasks.workunit.client.1.smithi019.stdout:4/661: truncate d4/d1b/d2a/d4f/fcf 470390 0 2024-03-16T09:01:20.356 INFO:tasks.workunit.client.1.smithi019.stdout:8/730: mknod d6/d2a/d45/d57/d72/d9d/cf9 0 2024-03-16T09:01:20.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/692: unlink d7/d1d/d28/d6c/d87/da3/fd9 0 2024-03-16T09:01:20.356 INFO:tasks.workunit.client.1.smithi019.stdout:0/693: chown d7/d1d/d63/d6e/db0/cc0 260 1 2024-03-16T09:01:20.359 INFO:tasks.workunit.client.1.smithi019.stdout:5/651: dwrite d6/dd/d61/fa9 [0,4194304] 0 2024-03-16T09:01:20.360 INFO:tasks.workunit.client.1.smithi019.stdout:5/652: fsync d6/dd/d47/d70/db2/d89/fc5 0 2024-03-16T09:01:20.360 INFO:tasks.workunit.client.1.smithi019.stdout:5/653: chown d6/dd/d61/l73 166384 1 2024-03-16T09:01:20.360 INFO:tasks.workunit.client.1.smithi019.stdout:9/516: dread d1/d7/f45 [0,4194304] 0 2024-03-16T09:01:20.360 INFO:tasks.workunit.client.1.smithi019.stdout:5/654: write d6/dd/d47/d70/db2/d89/fc5 [576083,96853] 0 2024-03-16T09:01:20.360 INFO:tasks.workunit.client.1.smithi019.stdout:2/724: rename d5/d6/cb8 to d5/de/dab/cf9 0 2024-03-16T09:01:20.360 INFO:tasks.workunit.client.1.smithi019.stdout:4/662: rename d4/d1b/d49 to d4/d1b/d49/d58/d80/d91/dda 22 2024-03-16T09:01:20.362 INFO:tasks.workunit.client.1.smithi019.stdout:5/655: creat d6/dd/d47/d70/db2/d4c/fce x:0 0 0 2024-03-16T09:01:20.363 INFO:tasks.workunit.client.1.smithi019.stdout:7/589: dwrite d9/da/d13/d18/d1a/f5f [0,4194304] 0 2024-03-16T09:01:20.363 INFO:tasks.workunit.client.1.smithi019.stdout:4/663: stat d4/d1b/d2a/d3d/l86 0 2024-03-16T09:01:20.363 INFO:tasks.workunit.client.0.smithi007.stdout:6/724: mknod d2/d13/d24/cf1 0 2024-03-16T09:01:20.363 INFO:tasks.workunit.client.0.smithi007.stdout:6/725: chown d2/c6 28 1 2024-03-16T09:01:20.364 INFO:tasks.workunit.client.1.smithi019.stdout:6/761: sync 2024-03-16T09:01:20.364 INFO:tasks.workunit.client.1.smithi019.stdout:2/725: getdents d5/d2e 0 2024-03-16T09:01:20.367 INFO:tasks.workunit.client.0.smithi007.stdout:7/806: creat d0/d19/d1f/d76/f11f x:0 0 0 2024-03-16T09:01:20.367 INFO:tasks.workunit.client.1.smithi019.stdout:8/731: rename d6/d2a/d45/d19/d74/le5 to d6/d2a/d3f/lfa 0 2024-03-16T09:01:20.368 INFO:tasks.workunit.client.1.smithi019.stdout:5/656: creat d6/dd/d37/d76/fcf x:0 0 0 2024-03-16T09:01:20.370 INFO:tasks.workunit.client.1.smithi019.stdout:2/726: creat d5/d2e/d4f/ffa x:0 0 0 2024-03-16T09:01:20.372 INFO:tasks.workunit.client.1.smithi019.stdout:6/762: mknod de/d1c/d9a/d9e/cf2 0 2024-03-16T09:01:20.372 INFO:tasks.workunit.client.1.smithi019.stdout:8/732: dread d6/d2a/d3c/dab/faa [0,4194304] 0 2024-03-16T09:01:20.373 INFO:tasks.workunit.client.0.smithi007.stdout:4/695: mknod d4/d14/d1c/d5b/d67/d8f/da8/d9d/dbb/cf0 0 2024-03-16T09:01:20.376 INFO:tasks.workunit.client.1.smithi019.stdout:6/763: rename de/d26/d32/d97/d2c/c98 to de/d26/d32/d97/d66/d72/d7f/cf3 0 2024-03-16T09:01:20.376 INFO:tasks.workunit.client.1.smithi019.stdout:6/764: read de/d26/d32/d97/d66/d72/d7f/f81 [2586375,96010] 0 2024-03-16T09:01:20.379 INFO:tasks.workunit.client.0.smithi007.stdout:4/696: getdents d4/d14/d1c/d5b 0 2024-03-16T09:01:20.380 INFO:tasks.workunit.client.1.smithi019.stdout:8/733: unlink d6/d2a/d45/d57/d96/fed 0 2024-03-16T09:01:20.381 INFO:tasks.workunit.client.1.smithi019.stdout:9/517: read d1/d7/f34 [3614951,119839] 0 2024-03-16T09:01:20.381 INFO:tasks.workunit.client.1.smithi019.stdout:9/518: write d1/d7/f51 [4782485,28232] 0 2024-03-16T09:01:20.384 INFO:tasks.workunit.client.0.smithi007.stdout:0/931: dwrite d3/d9/db4/d60/f129 [0,4194304] 0 2024-03-16T09:01:20.384 INFO:tasks.workunit.client.0.smithi007.stdout:4/697: creat d4/d14/d1c/d5b/d67/dd3/ff1 x:0 0 0 2024-03-16T09:01:20.385 INFO:tasks.workunit.client.0.smithi007.stdout:3/818: dwrite d15/d18/d1f/d29/d3e/d85/fa6 [0,4194304] 0 2024-03-16T09:01:20.385 INFO:tasks.workunit.client.0.smithi007.stdout:3/819: stat d15/c36 0 2024-03-16T09:01:20.387 INFO:tasks.workunit.client.1.smithi019.stdout:8/734: getdents d6/d2a/d45 0 2024-03-16T09:01:20.387 INFO:tasks.workunit.client.1.smithi019.stdout:8/735: write d6/d2a/d45/d19/fb8 [327035,106005] 0 2024-03-16T09:01:20.392 INFO:tasks.workunit.client.1.smithi019.stdout:1/608: getdents d16/d84 0 2024-03-16T09:01:20.393 INFO:tasks.workunit.client.1.smithi019.stdout:5/657: write d6/dd/d47/d70/db2/d4c/f62 [167114,86740] 0 2024-03-16T09:01:20.393 INFO:tasks.workunit.client.1.smithi019.stdout:8/736: symlink d6/d2a/d45/d57/d72/d79/d7b/lfb 0 2024-03-16T09:01:20.395 INFO:tasks.workunit.client.0.smithi007.stdout:0/932: rmdir d3/d9/da/d18/d27 39 2024-03-16T09:01:20.396 INFO:tasks.workunit.client.1.smithi019.stdout:1/609: link d16/d1f/d4f/c83 d16/d1f/d43/d55/cad 0 2024-03-16T09:01:20.397 INFO:tasks.workunit.client.1.smithi019.stdout:8/737: symlink d6/d2a/d68/lfc 0 2024-03-16T09:01:20.398 INFO:tasks.workunit.client.1.smithi019.stdout:5/658: mkdir d6/dd/d47/dd0 0 2024-03-16T09:01:20.398 INFO:tasks.workunit.client.1.smithi019.stdout:5/659: write d6/dd/d37/d76/f85 [943008,15107] 0 2024-03-16T09:01:20.399 INFO:tasks.workunit.client.0.smithi007.stdout:4/698: unlink d4/d14/f1b 0 2024-03-16T09:01:20.399 INFO:tasks.workunit.client.1.smithi019.stdout:1/610: creat d16/d31/d7b/fae x:0 0 0 2024-03-16T09:01:20.402 INFO:tasks.workunit.client.1.smithi019.stdout:5/660: rename d6/dd/d47/d7a/f95 to d6/dd/d47/d70/db2/d66/fd1 0 2024-03-16T09:01:20.402 INFO:tasks.workunit.client.1.smithi019.stdout:5/661: readlink d6/dd/d47/d7a/lae 0 2024-03-16T09:01:20.403 INFO:tasks.workunit.client.1.smithi019.stdout:8/738: creat d6/d2a/d3c/dab/d30/d4a/d67/ffd x:0 0 0 2024-03-16T09:01:20.403 INFO:tasks.workunit.client.1.smithi019.stdout:8/739: write d6/d2a/d45/d19/f43 [1615383,115432] 0 2024-03-16T09:01:20.404 INFO:tasks.workunit.client.1.smithi019.stdout:8/740: read d6/d2a/d45/d57/fbb [577016,104070] 0 2024-03-16T09:01:20.406 INFO:tasks.workunit.client.0.smithi007.stdout:4/699: dread d4/d14/d1c/d5b/fa0 [0,4194304] 0 2024-03-16T09:01:20.407 INFO:tasks.workunit.client.0.smithi007.stdout:3/820: unlink d15/d18/d1f/d3b/d87/f92 0 2024-03-16T09:01:20.410 INFO:tasks.workunit.client.0.smithi007.stdout:0/933: symlink d3/d9/da/d92/d9f/de8/d123/l131 0 2024-03-16T09:01:20.410 INFO:tasks.workunit.client.0.smithi007.stdout:0/934: creat d3/d1f/dca/f132 x:0 0 0 2024-03-16T09:01:20.413 INFO:tasks.workunit.client.1.smithi019.stdout:5/662: write d6/dd/d47/d70/db2/f4b [1439269,58766] 0 2024-03-16T09:01:20.413 INFO:tasks.workunit.client.1.smithi019.stdout:5/663: chown d6/dd/d47/f86 428 1 2024-03-16T09:01:20.413 INFO:tasks.workunit.client.1.smithi019.stdout:5/664: write d6/dd/daf/fbe [2060954,48447] 0 2024-03-16T09:01:20.416 INFO:tasks.workunit.client.1.smithi019.stdout:5/665: mkdir d6/d19/d34/dd2 0 2024-03-16T09:01:20.417 INFO:tasks.workunit.client.1.smithi019.stdout:5/666: symlink d6/dd/d47/d70/ld3 0 2024-03-16T09:01:20.425 INFO:tasks.workunit.client.1.smithi019.stdout:2/727: dwrite d5/d2e/f3d [0,4194304] 0 2024-03-16T09:01:20.426 INFO:tasks.workunit.client.0.smithi007.stdout:6/726: dwrite d2/d13/d4b/f7c [0,4194304] 0 2024-03-16T09:01:20.426 INFO:tasks.workunit.client.0.smithi007.stdout:6/727: chown d2/d13/d3f/d50/d78/dbe/dcb/d1e 17913 1 2024-03-16T09:01:20.426 INFO:tasks.workunit.client.0.smithi007.stdout:6/728: readlink d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/d84/le9 0 2024-03-16T09:01:20.427 INFO:tasks.workunit.client.1.smithi019.stdout:0/694: dwrite d7/d1d/d28/d6c/d87/f9f [0,4194304] 0 2024-03-16T09:01:20.427 INFO:tasks.workunit.client.0.smithi007.stdout:7/807: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/db3/fce [0,4194304] 0 2024-03-16T09:01:20.428 INFO:tasks.workunit.client.0.smithi007.stdout:7/808: fdatasync d0/d19/d1f/d69/f87 0 2024-03-16T09:01:20.428 INFO:tasks.workunit.client.0.smithi007.stdout:7/809: truncate d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f98 181217 0 2024-03-16T09:01:20.428 INFO:tasks.workunit.client.0.smithi007.stdout:7/810: creat d0/d19/d1f/d69/deb/f120 x:0 0 0 2024-03-16T09:01:20.432 INFO:tasks.workunit.client.1.smithi019.stdout:0/695: mkdir d7/d8/d32/d4a/d61/d68/dea 0 2024-03-16T09:01:20.432 INFO:tasks.workunit.client.1.smithi019.stdout:0/696: truncate d7/d1d/fe1 48633 0 2024-03-16T09:01:20.432 INFO:tasks.workunit.client.0.smithi007.stdout:4/700: read d4/d14/d1c/f6a [97113,123843] 0 2024-03-16T09:01:20.432 INFO:tasks.workunit.client.0.smithi007.stdout:4/701: readlink d4/d14/d1c/d5b/led 0 2024-03-16T09:01:20.432 INFO:tasks.workunit.client.0.smithi007.stdout:4/702: read d4/d14/d1c/f30 [1960448,94690] 0 2024-03-16T09:01:20.432 INFO:tasks.workunit.client.0.smithi007.stdout:6/729: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/f87 2072743 0 2024-03-16T09:01:20.436 INFO:tasks.workunit.client.0.smithi007.stdout:6/730: read d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/d8f/fb1 [621208,3439] 0 2024-03-16T09:01:20.436 INFO:tasks.workunit.client.0.smithi007.stdout:6/731: write d2/d13/d3f/d50/f53 [697673,94661] 0 2024-03-16T09:01:20.436 INFO:tasks.workunit.client.0.smithi007.stdout:6/732: write d2/d13/d24/fec [597907,40691] 0 2024-03-16T09:01:20.439 INFO:tasks.workunit.client.0.smithi007.stdout:7/811: mkdir d0/d19/d1f/d5d/d9d/df8/d121 0 2024-03-16T09:01:20.439 INFO:tasks.workunit.client.0.smithi007.stdout:7/812: stat d0/d19/cb2 0 2024-03-16T09:01:20.439 INFO:tasks.workunit.client.0.smithi007.stdout:7/813: truncate d0/f3c 5087517 0 2024-03-16T09:01:20.441 INFO:tasks.workunit.client.0.smithi007.stdout:6/733: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/ff2 x:0 0 0 2024-03-16T09:01:20.441 INFO:tasks.workunit.client.0.smithi007.stdout:6/734: creat d2/d29/ff3 x:0 0 0 2024-03-16T09:01:20.441 INFO:tasks.workunit.client.0.smithi007.stdout:6/735: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/ff4 x:0 0 0 2024-03-16T09:01:20.443 INFO:tasks.workunit.client.0.smithi007.stdout:6/736: symlink d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/lf5 0 2024-03-16T09:01:20.444 INFO:tasks.workunit.client.0.smithi007.stdout:6/737: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/df6 0 2024-03-16T09:01:20.445 INFO:tasks.workunit.client.0.smithi007.stdout:6/738: truncate d2/f20 1817718 0 2024-03-16T09:01:20.446 INFO:tasks.workunit.client.1.smithi019.stdout:6/765: dwrite de/d1c/f28 [4194304,4194304] 0 2024-03-16T09:01:20.446 INFO:tasks.workunit.client.1.smithi019.stdout:6/766: write de/d26/d32/f5b [1662310,112618] 0 2024-03-16T09:01:20.447 INFO:tasks.workunit.client.0.smithi007.stdout:7/814: dread d0/f3 [0,4194304] 0 2024-03-16T09:01:20.447 INFO:tasks.workunit.client.0.smithi007.stdout:6/739: mkdir d2/d13/d4b/d94/dea/df7 0 2024-03-16T09:01:20.447 INFO:tasks.workunit.client.1.smithi019.stdout:9/519: dwrite d1/d7/f34 [4194304,4194304] 0 2024-03-16T09:01:20.447 INFO:tasks.workunit.client.1.smithi019.stdout:6/767: symlink de/d26/lf4 0 2024-03-16T09:01:20.448 INFO:tasks.workunit.client.1.smithi019.stdout:9/520: rmdir d1/d7/d67 39 2024-03-16T09:01:20.448 INFO:tasks.workunit.client.1.smithi019.stdout:9/521: readlink d1/d7/d33/d59/l83 0 2024-03-16T09:01:20.448 INFO:tasks.workunit.client.0.smithi007.stdout:7/815: mknod d0/d19/d1f/d5d/d9d/df7/c122 0 2024-03-16T09:01:20.450 INFO:tasks.workunit.client.1.smithi019.stdout:0/697: dread d7/d8/f88 [0,4194304] 0 2024-03-16T09:01:20.451 INFO:tasks.workunit.client.1.smithi019.stdout:6/768: rename de/d26/d32/d4e/l96 to de/d26/d32/d4e/d57/lf5 0 2024-03-16T09:01:20.451 INFO:tasks.workunit.client.0.smithi007.stdout:6/740: link d2/d13/d3f/d50/d78/dbe/dcb/d3b/lc7 d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/lf8 0 2024-03-16T09:01:20.451 INFO:tasks.workunit.client.0.smithi007.stdout:4/703: dread d4/d14/f19 [0,4194304] 0 2024-03-16T09:01:20.452 INFO:tasks.workunit.client.1.smithi019.stdout:6/769: mknod de/d1c/d9a/d9e/cf6 0 2024-03-16T09:01:20.452 INFO:tasks.workunit.client.1.smithi019.stdout:6/770: stat de/d26/d32/d97/d51/lbe 0 2024-03-16T09:01:20.453 INFO:tasks.workunit.client.0.smithi007.stdout:7/816: rename d0/c16 to d0/df5/c123 0 2024-03-16T09:01:20.453 INFO:tasks.workunit.client.0.smithi007.stdout:7/817: chown d0/d19/d1f/d5d/d9d/df7/d7a/f6e 6867 1 2024-03-16T09:01:20.453 INFO:tasks.workunit.client.1.smithi019.stdout:9/522: dread d1/d7/d21/f7b [0,4194304] 0 2024-03-16T09:01:20.454 INFO:tasks.workunit.client.0.smithi007.stdout:4/704: rename d4/d14/d1c/d5b/d67/d8f/da8/dbe/fc2 to d4/d14/d1c/d5b/d67/d8f/ff2 0 2024-03-16T09:01:20.455 INFO:tasks.workunit.client.0.smithi007.stdout:4/705: truncate d4/d14/d1c/d5b/d67/d8f/fc9 430278 0 2024-03-16T09:01:20.455 INFO:tasks.workunit.client.1.smithi019.stdout:9/523: symlink d1/d7/lb3 0 2024-03-16T09:01:20.455 INFO:tasks.workunit.client.1.smithi019.stdout:9/524: readlink d1/d7/de/l5c 0 2024-03-16T09:01:20.455 INFO:tasks.workunit.client.1.smithi019.stdout:9/525: write d1/d98/dae/f55 [3260406,121751] 0 2024-03-16T09:01:20.456 INFO:tasks.workunit.client.0.smithi007.stdout:7/818: creat d0/d19/d1f/d5d/d9d/df7/d73/f124 x:0 0 0 2024-03-16T09:01:20.456 INFO:tasks.workunit.client.0.smithi007.stdout:7/819: fdatasync d0/d19/f6a 0 2024-03-16T09:01:20.456 INFO:tasks.workunit.client.0.smithi007.stdout:7/820: stat d0/dc/d38/d4e/fb4 0 2024-03-16T09:01:20.456 INFO:tasks.workunit.client.0.smithi007.stdout:4/706: getdents d4/d14/d64/d9f/d9a 0 2024-03-16T09:01:20.456 INFO:tasks.workunit.client.0.smithi007.stdout:4/707: creat d4/d14/d64/d68/ff3 x:0 0 0 2024-03-16T09:01:20.456 INFO:tasks.workunit.client.0.smithi007.stdout:4/708: fdatasync d4/d14/d1c/d5b/d67/d8f/da8/d9d/dd7/fb8 0 2024-03-16T09:01:20.459 INFO:tasks.workunit.client.1.smithi019.stdout:6/771: dread de/d26/d32/fd1 [0,4194304] 0 2024-03-16T09:01:20.460 INFO:tasks.workunit.client.1.smithi019.stdout:6/772: stat de/d1c/l2d 0 2024-03-16T09:01:20.460 INFO:tasks.workunit.client.0.smithi007.stdout:0/935: dwrite d3/d9/db4/d1c/d72/fd1 [0,4194304] 0 2024-03-16T09:01:20.460 INFO:tasks.workunit.client.0.smithi007.stdout:4/709: rename d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/fe4 to d4/d14/ff4 0 2024-03-16T09:01:20.461 INFO:tasks.workunit.client.0.smithi007.stdout:4/710: truncate d4/d14/d1c/f3b 706269 0 2024-03-16T09:01:20.461 INFO:tasks.workunit.client.0.smithi007.stdout:4/711: write d4/d14/f19 [4945602,38786] 0 2024-03-16T09:01:20.461 INFO:tasks.workunit.client.1.smithi019.stdout:6/773: rmdir de/d26/d32/d4e/d75/dcd 39 2024-03-16T09:01:20.461 INFO:tasks.workunit.client.1.smithi019.stdout:8/741: dwrite d6/d2a/d3c/dab/d30/d4a/f81 [0,4194304] 0 2024-03-16T09:01:20.461 INFO:tasks.workunit.client.1.smithi019.stdout:2/728: dwrite d5/d6/d2b/d2f/f67 [0,4194304] 0 2024-03-16T09:01:20.461 INFO:tasks.workunit.client.1.smithi019.stdout:2/729: dread - d5/d2e/ff1 zero size 2024-03-16T09:01:20.462 INFO:tasks.workunit.client.1.smithi019.stdout:0/698: dread d7/d1d/d28/f34 [0,4194304] 0 2024-03-16T09:01:20.462 INFO:tasks.workunit.client.1.smithi019.stdout:0/699: chown d7/fe4 53002469 1 2024-03-16T09:01:20.462 INFO:tasks.workunit.client.1.smithi019.stdout:6/774: truncate de/d26/d32/d97/d51/d70/f88 4010858 0 2024-03-16T09:01:20.462 INFO:tasks.workunit.client.1.smithi019.stdout:5/667: dwrite d6/dd/d47/d56/d87/fc3 [0,4194304] 0 2024-03-16T09:01:20.463 INFO:tasks.workunit.client.1.smithi019.stdout:0/700: truncate d7/d1d/f76 430899 0 2024-03-16T09:01:20.463 INFO:tasks.workunit.client.1.smithi019.stdout:0/701: chown d7/d1d/f76 1890793 1 2024-03-16T09:01:20.463 INFO:tasks.workunit.client.1.smithi019.stdout:0/702: stat d7/d1d/d2e/c55 0 2024-03-16T09:01:20.466 INFO:tasks.workunit.client.1.smithi019.stdout:0/703: rename d7/d8/d32/d66/d6d/d8c/db8/d82/l62 to d7/d1d/d28/d6c/d87/leb 0 2024-03-16T09:01:20.469 INFO:tasks.workunit.client.1.smithi019.stdout:0/704: mkdir d7/d8/dec 0 2024-03-16T09:01:20.473 INFO:tasks.workunit.client.0.smithi007.stdout:6/741: dread d2/d13/d3f/d50/d78/dbe/dcb/f25 [0,4194304] 0 2024-03-16T09:01:20.476 INFO:tasks.workunit.client.1.smithi019.stdout:9/526: dwrite d1/d7/de/d24/d44/d76/f9a [0,4194304] 0 2024-03-16T09:01:20.483 INFO:tasks.workunit.client.0.smithi007.stdout:4/712: dread d4/fa3 [0,4194304] 0 2024-03-16T09:01:20.483 INFO:tasks.workunit.client.0.smithi007.stdout:7/821: dread d0/dc/d5a/fdc [4194304,4194304] 0 2024-03-16T09:01:20.483 INFO:tasks.workunit.client.0.smithi007.stdout:4/713: chown d4/d14/d1c/d5b/f5e 91535 1 2024-03-16T09:01:20.483 INFO:tasks.workunit.client.0.smithi007.stdout:4/714: getdents d4/d14/d64/d9f/d9a 0 2024-03-16T09:01:20.518 INFO:tasks.workunit.client.0.smithi007.stdout:6/742: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/f63 [0,4194304] 0 2024-03-16T09:01:20.518 INFO:tasks.workunit.client.0.smithi007.stdout:6/743: stat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/fe4 0 2024-03-16T09:01:20.519 INFO:tasks.workunit.client.1.smithi019.stdout:5/668: dwrite d6/dd/d47/d70/db2/d4c/fc6 [4194304,4194304] 0 2024-03-16T09:01:20.519 INFO:tasks.workunit.client.1.smithi019.stdout:0/705: dwrite d7/d8/d32/d66/d6d/d8c/db8/f69 [0,4194304] 0 2024-03-16T09:01:20.519 INFO:tasks.workunit.client.1.smithi019.stdout:6/775: dwrite de/d26/d32/d97/d51/d70/f88 [4194304,4194304] 0 2024-03-16T09:01:20.525 INFO:tasks.workunit.client.1.smithi019.stdout:0/706: symlink d7/d8/d32/d66/d6d/d8c/db8/led 0 2024-03-16T09:01:20.525 INFO:tasks.workunit.client.1.smithi019.stdout:0/707: creat d7/d1d/d63/fee x:0 0 0 2024-03-16T09:01:20.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/936: mknod d3/d9/da/d18/d27/d34/c133 0 2024-03-16T09:01:20.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/937: readlink d3/d9/da/d18/d23/lc8 0 2024-03-16T09:01:20.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/938: fsync d3/d9/da/d48/fa5 0 2024-03-16T09:01:20.526 INFO:tasks.workunit.client.0.smithi007.stdout:0/939: dread d3/d9/da/d48/d9e/f117 [0,4194304] 0 2024-03-16T09:01:20.528 INFO:tasks.workunit.client.1.smithi019.stdout:8/742: symlink d6/d2a/d45/d57/d72/d79/lfe 0 2024-03-16T09:01:20.528 INFO:tasks.workunit.client.1.smithi019.stdout:6/776: rmdir de/d1c/d9a 39 2024-03-16T09:01:20.528 INFO:tasks.workunit.client.1.smithi019.stdout:6/777: dread - de/d26/d32/d97/d66/d72/d7f/dcb/fd9 zero size 2024-03-16T09:01:20.534 INFO:tasks.workunit.client.1.smithi019.stdout:5/669: rmdir d6 39 2024-03-16T09:01:20.534 INFO:tasks.workunit.client.1.smithi019.stdout:5/670: readlink d6/dd/d37/l50 0 2024-03-16T09:01:20.536 INFO:tasks.workunit.client.1.smithi019.stdout:5/671: creat d6/dd/d61/fd4 x:0 0 0 2024-03-16T09:01:20.539 INFO:tasks.workunit.client.1.smithi019.stdout:8/743: symlink d6/d2a/d3c/lff 0 2024-03-16T09:01:20.540 INFO:tasks.workunit.client.0.smithi007.stdout:7/822: symlink d0/d19/d1f/d5d/d9d/df7/d3a/db3/l125 0 2024-03-16T09:01:20.540 INFO:tasks.workunit.client.1.smithi019.stdout:0/708: rename d7/d1d/d28 to d7/d8/dec/def 0 2024-03-16T09:01:20.540 INFO:tasks.workunit.client.1.smithi019.stdout:0/709: readlink d7/d8/d32/d4a/lbe 0 2024-03-16T09:01:20.541 INFO:tasks.workunit.client.1.smithi019.stdout:2/730: mkdir d5/d6/d2b/dfb 0 2024-03-16T09:01:20.541 INFO:tasks.workunit.client.1.smithi019.stdout:5/672: unlink d6/dd/d47/f4a 0 2024-03-16T09:01:20.542 INFO:tasks.workunit.client.1.smithi019.stdout:5/673: fdatasync d6/dd/d37/d54/f80 0 2024-03-16T09:01:20.542 INFO:tasks.workunit.client.0.smithi007.stdout:4/715: symlink d4/d14/d64/d68/d82/dac/lf5 0 2024-03-16T09:01:20.542 INFO:tasks.workunit.client.0.smithi007.stdout:4/716: stat d4/d14/d64/f7d 0 2024-03-16T09:01:20.542 INFO:tasks.workunit.client.0.smithi007.stdout:4/717: write d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/d93/fee [74957,98481] 0 2024-03-16T09:01:20.543 INFO:tasks.workunit.client.1.smithi019.stdout:0/710: truncate d7/d8/d32/d66/d6d/d8c/db8/f8f 2480165 0 2024-03-16T09:01:20.543 INFO:tasks.workunit.client.1.smithi019.stdout:0/711: write d7/d8/d32/d66/d6d/fc8 [1900289,116115] 0 2024-03-16T09:01:20.543 INFO:tasks.workunit.client.1.smithi019.stdout:0/712: creat d7/d8/d32/d66/d6d/d8c/db8/d82/ff0 x:0 0 0 2024-03-16T09:01:20.543 INFO:tasks.workunit.client.1.smithi019.stdout:5/674: creat d6/d1f/d77/fd5 x:0 0 0 2024-03-16T09:01:20.545 INFO:tasks.workunit.client.1.smithi019.stdout:8/744: rename d6/d2a/d45/d19/f1c to d6/d2a/d45/d57/f100 0 2024-03-16T09:01:20.545 INFO:tasks.workunit.client.1.smithi019.stdout:5/675: dread d6/dd/d37/d54/f5f [0,4194304] 0 2024-03-16T09:01:20.545 INFO:tasks.workunit.client.1.smithi019.stdout:8/745: fsync d6/fdb 0 2024-03-16T09:01:20.545 INFO:tasks.workunit.client.1.smithi019.stdout:8/746: stat d6/d2a/d45/d57/d72/d79/d7b/fa6 0 2024-03-16T09:01:20.546 INFO:tasks.workunit.client.1.smithi019.stdout:8/747: truncate d6/d2a/d3c/dab/d30/d4a/d67/fa9 810695 0 2024-03-16T09:01:20.546 INFO:tasks.workunit.client.1.smithi019.stdout:0/713: unlink d7/d1d/d63/d6e/db0/cc0 0 2024-03-16T09:01:20.547 INFO:tasks.workunit.client.1.smithi019.stdout:6/778: dwrite de/d26/d32/d97/fb8 [0,4194304] 0 2024-03-16T09:01:20.548 INFO:tasks.workunit.client.0.smithi007.stdout:6/744: creat d2/d13/d4b/d94/dea/df7/ff9 x:0 0 0 2024-03-16T09:01:20.548 INFO:tasks.workunit.client.1.smithi019.stdout:5/676: truncate d6/dd/f67 4469870 0 2024-03-16T09:01:20.548 INFO:tasks.workunit.client.1.smithi019.stdout:2/731: fsync d5/d6/d2b/f1e 0 2024-03-16T09:01:20.549 INFO:tasks.workunit.client.1.smithi019.stdout:0/714: mkdir d7/d1d/d63/df1 0 2024-03-16T09:01:20.552 INFO:tasks.workunit.client.0.smithi007.stdout:0/940: symlink d3/d9/da/d48/d25/d43/d1a/d58/l134 0 2024-03-16T09:01:20.552 INFO:tasks.workunit.client.0.smithi007.stdout:0/941: creat d3/d9/f135 x:0 0 0 2024-03-16T09:01:20.552 INFO:tasks.workunit.client.1.smithi019.stdout:0/715: mknod d7/d8/d32/d4a/d61/d68/dd0/cf2 0 2024-03-16T09:01:20.554 INFO:tasks.workunit.client.1.smithi019.stdout:0/716: symlink d7/d1d/d63/df1/lf3 0 2024-03-16T09:01:20.555 INFO:tasks.workunit.client.1.smithi019.stdout:8/748: creat d6/de/d51/f101 x:0 0 0 2024-03-16T09:01:20.557 INFO:tasks.workunit.client.1.smithi019.stdout:0/717: mknod d7/d1d/d63/d6e/db0/cf4 0 2024-03-16T09:01:20.558 INFO:tasks.workunit.client.1.smithi019.stdout:0/718: write d7/d1d/d63/d6e/fc9 [383518,76041] 0 2024-03-16T09:01:20.575 INFO:tasks.workunit.client.1.smithi019.stdout:2/732: rename d5/d4e to d5/d5a/d96/dba/dfc 0 2024-03-16T09:01:20.579 INFO:tasks.workunit.client.1.smithi019.stdout:6/779: dwrite de/d26/d32/d97/d66/d72/d7f/dd5/fe9 [0,4194304] 0 2024-03-16T09:01:20.582 INFO:tasks.workunit.client.1.smithi019.stdout:6/780: mknod de/d26/d32/d4e/cf7 0 2024-03-16T09:01:20.583 INFO:tasks.workunit.client.1.smithi019.stdout:6/781: mkdir de/d1c/df8 0 2024-03-16T09:01:20.593 INFO:tasks.workunit.client.1.smithi019.stdout:6/782: dread de/d26/d32/d97/d2c/f3a [4194304,4194304] 0 2024-03-16T09:01:20.602 INFO:tasks.workunit.client.1.smithi019.stdout:6/783: write de/d26/d32/d97/d66/d72/d7f/dd5/fe9 [2285771,110441] 0 2024-03-16T09:01:20.603 INFO:tasks.workunit.client.0.smithi007.stdout:7/823: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/f4f [0,4194304] 0 2024-03-16T09:01:20.603 INFO:tasks.workunit.client.0.smithi007.stdout:0/942: dwrite d3/d9/db4/d2c/f5e [0,4194304] 0 2024-03-16T09:01:20.603 INFO:tasks.workunit.client.0.smithi007.stdout:4/718: dwrite d4/d14/d1c/d5b/d67/d8f/da8/d9d/dbb/fde [0,4194304] 0 2024-03-16T09:01:20.604 INFO:tasks.workunit.client.0.smithi007.stdout:6/745: dwrite d2/f20 [0,4194304] 0 2024-03-16T09:01:20.604 INFO:tasks.workunit.client.1.smithi019.stdout:5/677: dwrite d6/dd/d47/d70/db2/d4c/fce [0,4194304] 0 2024-03-16T09:01:20.605 INFO:tasks.workunit.client.1.smithi019.stdout:5/678: fdatasync d6/dd/d47/d70/db2/d4c/fc6 0 2024-03-16T09:01:20.605 INFO:tasks.workunit.client.1.smithi019.stdout:2/733: dwrite d5/d6/d2b/d89/fd1 [0,4194304] 0 2024-03-16T09:01:20.608 INFO:tasks.workunit.client.1.smithi019.stdout:0/719: getdents d7/d1d/d63/d6e/db0 0 2024-03-16T09:01:20.608 INFO:tasks.workunit.client.1.smithi019.stdout:0/720: chown d7/d8/dec/def/da1/ddf 7042 1 2024-03-16T09:01:20.610 INFO:tasks.workunit.client.1.smithi019.stdout:5/679: symlink d6/d19/d34/ld6 0 2024-03-16T09:01:20.610 INFO:tasks.workunit.client.1.smithi019.stdout:8/749: rename d6/d2a/c4c to d6/d2a/d45/d19/c102 0 2024-03-16T09:01:20.613 INFO:tasks.workunit.client.1.smithi019.stdout:5/680: creat d6/d1f/d78/fd7 x:0 0 0 2024-03-16T09:01:20.613 INFO:tasks.workunit.client.1.smithi019.stdout:3/628: sync 2024-03-16T09:01:20.613 INFO:tasks.workunit.client.1.smithi019.stdout:3/629: creat d3/d1b/d88/d91/fc1 x:0 0 0 2024-03-16T09:01:20.613 INFO:tasks.workunit.client.1.smithi019.stdout:3/630: creat d3/d1b/d88/fc2 x:0 0 0 2024-03-16T09:01:20.613 INFO:tasks.workunit.client.1.smithi019.stdout:3/631: stat d3/d1b/d88/d91/fc1 0 2024-03-16T09:01:20.614 INFO:tasks.workunit.client.1.smithi019.stdout:2/734: creat d5/d5a/d96/df7/d6f/d87/ffd x:0 0 0 2024-03-16T09:01:20.615 INFO:tasks.workunit.client.1.smithi019.stdout:6/784: rename de/d26/d32/d4e/d75/cb4 to de/d1c/df8/cf9 0 2024-03-16T09:01:20.615 INFO:tasks.workunit.client.0.smithi007.stdout:5/792: sync 2024-03-16T09:01:20.615 INFO:tasks.workunit.client.0.smithi007.stdout:2/917: sync 2024-03-16T09:01:20.615 INFO:tasks.workunit.client.0.smithi007.stdout:1/667: sync 2024-03-16T09:01:20.615 INFO:tasks.workunit.client.0.smithi007.stdout:9/810: sync 2024-03-16T09:01:20.615 INFO:tasks.workunit.client.0.smithi007.stdout:2/918: chown dc/d27/l6d 139361 1 2024-03-16T09:01:20.616 INFO:tasks.workunit.client.1.smithi019.stdout:8/750: mkdir d6/d2a/d45/d103 0 2024-03-16T09:01:20.617 INFO:tasks.workunit.client.1.smithi019.stdout:2/735: rename d5/d5a/d96/dba/dfc/d78/le3 to d5/d5a/d96/lfe 0 2024-03-16T09:01:20.619 INFO:tasks.workunit.client.1.smithi019.stdout:6/785: creat de/d26/d32/d4e/d57/ffa x:0 0 0 2024-03-16T09:01:20.621 INFO:tasks.workunit.client.0.smithi007.stdout:4/719: symlink d4/d14/d64/lf6 0 2024-03-16T09:01:20.622 INFO:tasks.workunit.client.1.smithi019.stdout:6/786: creat de/d26/d32/d97/d66/d72/db7/ffb x:0 0 0 2024-03-16T09:01:20.622 INFO:tasks.workunit.client.1.smithi019.stdout:6/787: dread - de/d26/d32/f62 zero size 2024-03-16T09:01:20.623 INFO:tasks.workunit.client.1.smithi019.stdout:6/788: creat de/d26/d32/d97/d66/ddc/ffc x:0 0 0 2024-03-16T09:01:20.623 INFO:tasks.workunit.client.1.smithi019.stdout:5/681: dread d6/d19/f4e [0,4194304] 0 2024-03-16T09:01:20.623 INFO:tasks.workunit.client.0.smithi007.stdout:6/746: unlink d2/d13/d3f/d50/d78/dbe/dcb/ca9 0 2024-03-16T09:01:20.624 INFO:tasks.workunit.client.1.smithi019.stdout:6/789: unlink de/d26/d32/d97/dae/fdb 0 2024-03-16T09:01:20.624 INFO:tasks.workunit.client.1.smithi019.stdout:6/790: creat de/d26/d32/d97/d66/ddc/ffd x:0 0 0 2024-03-16T09:01:20.625 INFO:tasks.workunit.client.1.smithi019.stdout:6/791: write de/d26/fc6 [281979,16751] 0 2024-03-16T09:01:20.625 INFO:tasks.workunit.client.1.smithi019.stdout:5/682: symlink d6/d1f/d77/ld8 0 2024-03-16T09:01:20.626 INFO:tasks.workunit.client.1.smithi019.stdout:3/632: dread d3/d8b/dae/f50 [0,4194304] 0 2024-03-16T09:01:20.626 INFO:tasks.workunit.client.1.smithi019.stdout:2/736: link d5/d6/d2b/f3c d5/d6/d2b/fff 0 2024-03-16T09:01:20.626 INFO:tasks.workunit.client.1.smithi019.stdout:3/633: fdatasync d3/d89/fbc 0 2024-03-16T09:01:20.626 INFO:tasks.workunit.client.1.smithi019.stdout:5/683: truncate d6/dd/d47/d70/db2/d4c/f98 728674 0 2024-03-16T09:01:20.627 INFO:tasks.workunit.client.1.smithi019.stdout:5/684: truncate d6/dd/d37/fb4 955745 0 2024-03-16T09:01:20.627 INFO:tasks.workunit.client.1.smithi019.stdout:5/685: stat d6/dd/d47/d56/c57 0 2024-03-16T09:01:20.627 INFO:tasks.workunit.client.1.smithi019.stdout:5/686: chown d6/dd/d47/d56/faa 217 1 2024-03-16T09:01:20.632 INFO:tasks.workunit.client.1.smithi019.stdout:3/634: mkdir d3/d1b/d40/dc3 0 2024-03-16T09:01:20.632 INFO:tasks.workunit.client.1.smithi019.stdout:2/737: mknod d5/d6/d2b/d2f/d35/d65/c100 0 2024-03-16T09:01:20.632 INFO:tasks.workunit.client.1.smithi019.stdout:2/738: getdents d5/de/dd4 0 2024-03-16T09:01:20.633 INFO:tasks.workunit.client.1.smithi019.stdout:2/739: write d5/d5a/d96/df7/d6f/da6/fc2 [1237196,119791] 0 2024-03-16T09:01:20.633 INFO:tasks.workunit.client.1.smithi019.stdout:2/740: fsync d5/d6/d2b/f91 0 2024-03-16T09:01:20.633 INFO:tasks.workunit.client.1.smithi019.stdout:2/741: chown d5/d6/d2b/fff 571 1 2024-03-16T09:01:20.633 INFO:tasks.workunit.client.1.smithi019.stdout:2/742: chown d5/d5a/d80/l94 2 1 2024-03-16T09:01:20.633 INFO:tasks.workunit.client.1.smithi019.stdout:5/687: creat d6/d1f/fd9 x:0 0 0 2024-03-16T09:01:20.634 INFO:tasks.workunit.client.1.smithi019.stdout:3/635: mknod d3/d8b/dae/d3f/cc4 0 2024-03-16T09:01:20.634 INFO:tasks.workunit.client.1.smithi019.stdout:3/636: fsync d3/d11/f14 0 2024-03-16T09:01:20.634 INFO:tasks.workunit.client.1.smithi019.stdout:5/688: creat d6/dd/fda x:0 0 0 2024-03-16T09:01:20.634 INFO:tasks.workunit.client.1.smithi019.stdout:2/743: mknod d5/de/dab/c101 0 2024-03-16T09:01:20.634 INFO:tasks.workunit.client.1.smithi019.stdout:5/689: truncate d6/dd/d37/d76/fb7 5436 0 2024-03-16T09:01:20.634 INFO:tasks.workunit.client.0.smithi007.stdout:0/943: truncate d3/d9/da/d18/d23/f4d 890545 0 2024-03-16T09:01:20.635 INFO:tasks.workunit.client.0.smithi007.stdout:0/944: dread - d3/d9/da/d92/d9f/fdd zero size 2024-03-16T09:01:20.635 INFO:tasks.workunit.client.0.smithi007.stdout:0/945: chown d3/d9/db4/d1c/d72/f82 50 1 2024-03-16T09:01:20.635 INFO:tasks.workunit.client.0.smithi007.stdout:9/811: creat d24/d4a/d55/dbf/de9/ff0 x:0 0 0 2024-03-16T09:01:20.635 INFO:tasks.workunit.client.0.smithi007.stdout:1/668: creat d9/d6b/fd1 x:0 0 0 2024-03-16T09:01:20.635 INFO:tasks.workunit.client.0.smithi007.stdout:1/669: stat d9/dd/d1b/d2d/ca6 0 2024-03-16T09:01:20.635 INFO:tasks.workunit.client.0.smithi007.stdout:1/670: write d9/dd/d1b/d2d/d30/f8e [533573,37062] 0 2024-03-16T09:01:20.636 INFO:tasks.workunit.client.0.smithi007.stdout:5/793: dread d1/d7/d1c/d88/da2/d89/db3/d2a/d81/f90 [0,4194304] 0 2024-03-16T09:01:20.636 INFO:tasks.workunit.client.0.smithi007.stdout:5/794: truncate d1/d7/d1c/d88/da2/d89/db3/fa0 515770 0 2024-03-16T09:01:20.636 INFO:tasks.workunit.client.0.smithi007.stdout:2/919: creat dc/df/d24/d28/d4c/dbc/dfe/f132 x:0 0 0 2024-03-16T09:01:20.636 INFO:tasks.workunit.client.0.smithi007.stdout:2/920: stat dc/df/d24/d28/d2e/c43 0 2024-03-16T09:01:20.636 INFO:tasks.workunit.client.0.smithi007.stdout:2/921: chown dc/df/d24/d28/d4c/f67 1 1 2024-03-16T09:01:20.636 INFO:tasks.workunit.client.0.smithi007.stdout:4/720: symlink d4/d14/d1c/d5b/d67/d8f/da8/d78/lf7 0 2024-03-16T09:01:20.638 INFO:tasks.workunit.client.1.smithi019.stdout:0/721: dwrite d7/d8/dec/def/d6c/fe9 [0,4194304] 0 2024-03-16T09:01:20.638 INFO:tasks.workunit.client.1.smithi019.stdout:2/744: dread d5/d6/d2b/d2f/d35/f39 [0,4194304] 0 2024-03-16T09:01:20.638 INFO:tasks.workunit.client.0.smithi007.stdout:6/747: rename d2/d13/d3f/l8d to d2/d13/d3f/d50/d78/d81/lfa 0 2024-03-16T09:01:20.639 INFO:tasks.workunit.client.0.smithi007.stdout:7/824: dwrite d0/d19/d1f/d5d/d9d/df7/d73/f58 [4194304,4194304] 0 2024-03-16T09:01:20.639 INFO:tasks.workunit.client.0.smithi007.stdout:9/812: getdents d24/d4a/d96 0 2024-03-16T09:01:20.639 INFO:tasks.workunit.client.0.smithi007.stdout:7/825: dread - d0/d19/d1f/d69/f118 zero size 2024-03-16T09:01:20.639 INFO:tasks.workunit.client.0.smithi007.stdout:1/671: symlink d9/ld2 0 2024-03-16T09:01:20.639 INFO:tasks.workunit.client.0.smithi007.stdout:1/672: creat d9/dd/d5e/fd3 x:0 0 0 2024-03-16T09:01:20.640 INFO:tasks.workunit.client.0.smithi007.stdout:2/922: getdents dc/df/d24/d28/d2e/d35 0 2024-03-16T09:01:20.644 INFO:tasks.workunit.client.1.smithi019.stdout:7/590: sync 2024-03-16T09:01:20.644 INFO:tasks.workunit.client.0.smithi007.stdout:0/946: dread d3/d9/da/d48/d25/d43/d1a/f1b [0,4194304] 0 2024-03-16T09:01:20.647 INFO:tasks.workunit.client.1.smithi019.stdout:7/591: rename d9/db/d12/d5b/d33/f58 to d9/d25/d93/fbc 0 2024-03-16T09:01:20.649 INFO:tasks.workunit.client.0.smithi007.stdout:6/748: mknod d2/d13/d3f/d50/d78/d81/cfb 0 2024-03-16T09:01:20.649 INFO:tasks.workunit.client.0.smithi007.stdout:3/821: sync 2024-03-16T09:01:20.651 INFO:tasks.workunit.client.1.smithi019.stdout:8/751: getdents d6/d2a/d45 0 2024-03-16T09:01:20.651 INFO:tasks.workunit.client.1.smithi019.stdout:8/752: readlink d6/d2a/d45/lb4 0 2024-03-16T09:01:20.652 INFO:tasks.workunit.client.1.smithi019.stdout:7/592: mknod d9/db/d12/d5b/d33/cbd 0 2024-03-16T09:01:20.652 INFO:tasks.workunit.client.0.smithi007.stdout:6/749: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/fcf [247229,87391] 0 2024-03-16T09:01:20.652 INFO:tasks.workunit.client.0.smithi007.stdout:6/750: write d2/d10/d3d/d6d/dbf/fd5 [6005827,27229] 0 2024-03-16T09:01:20.654 INFO:tasks.workunit.client.1.smithi019.stdout:7/593: unlink d9/d25/fb5 0 2024-03-16T09:01:20.654 INFO:tasks.workunit.client.0.smithi007.stdout:6/751: write d2/d13/d3f/d50/d78/d81/f8c [1028879,85074] 0 2024-03-16T09:01:20.654 INFO:tasks.workunit.client.1.smithi019.stdout:7/594: write d9/da/d13/d18/d1a/f91 [300403,66604] 0 2024-03-16T09:01:20.655 INFO:tasks.workunit.client.1.smithi019.stdout:5/690: dwrite d6/dd/fda [0,4194304] 0 2024-03-16T09:01:20.655 INFO:tasks.workunit.client.1.smithi019.stdout:7/595: mknod d9/da/d13/d18/d1a/cbe 0 2024-03-16T09:01:20.656 INFO:tasks.workunit.client.1.smithi019.stdout:7/596: chown d9/db/d12/c1e 83578 1 2024-03-16T09:01:20.656 INFO:tasks.workunit.client.1.smithi019.stdout:7/597: readlink d9/db/d12/l7e 0 2024-03-16T09:01:20.656 INFO:tasks.workunit.client.0.smithi007.stdout:7/826: symlink d0/d19/d1f/d5d/d9d/df7/da1/l126 0 2024-03-16T09:01:20.659 INFO:tasks.workunit.client.1.smithi019.stdout:5/691: truncate d6/dd/d47/d70/db2/d66/f9f 2500746 0 2024-03-16T09:01:20.660 INFO:tasks.workunit.client.1.smithi019.stdout:7/598: truncate f7 2273621 0 2024-03-16T09:01:20.660 INFO:tasks.workunit.client.0.smithi007.stdout:4/721: getdents d4/d14/d64/d68/d82/dac 0 2024-03-16T09:01:20.661 INFO:tasks.workunit.client.0.smithi007.stdout:4/722: write d4/dcb/fd8 [176646,109261] 0 2024-03-16T09:01:20.661 INFO:tasks.workunit.client.0.smithi007.stdout:4/723: chown d4/d14/d1c/d5b/d67/d8f/da8/d78/d80/d84/fa9 482453 1 2024-03-16T09:01:20.661 INFO:tasks.workunit.client.1.smithi019.stdout:5/692: mknod d6/d1f/d77/d84/cdb 0 2024-03-16T09:01:20.661 INFO:tasks.workunit.client.1.smithi019.stdout:7/599: mknod d9/da/d13/d18/d30/d9e/cbf 0 2024-03-16T09:01:20.662 INFO:tasks.workunit.client.1.smithi019.stdout:8/753: dread d6/d2a/d45/d19/f97 [0,4194304] 0 2024-03-16T09:01:20.662 INFO:tasks.workunit.client.0.smithi007.stdout:3/822: link d15/d18/f35 d15/d18/d1f/d29/d3e/f103 0 2024-03-16T09:01:20.662 INFO:tasks.workunit.client.1.smithi019.stdout:5/693: symlink d6/dd/daf/ldc 0 2024-03-16T09:01:20.663 INFO:tasks.workunit.client.1.smithi019.stdout:5/694: creat d6/d1f/fdd x:0 0 0 2024-03-16T09:01:20.663 INFO:tasks.workunit.client.1.smithi019.stdout:7/600: creat d9/da/d13/d18/d30/d3e/da5/fc0 x:0 0 0 2024-03-16T09:01:20.663 INFO:tasks.workunit.client.1.smithi019.stdout:8/754: mknod d6/d2a/d45/d19/d74/c104 0 2024-03-16T09:01:20.663 INFO:tasks.workunit.client.1.smithi019.stdout:8/755: write d6/d2a/d45/d57/d96/f9c [1011318,129578] 0 2024-03-16T09:01:20.664 INFO:tasks.workunit.client.1.smithi019.stdout:5/695: mkdir d6/dd/dde 0 2024-03-16T09:01:20.664 INFO:tasks.workunit.client.1.smithi019.stdout:7/601: getdents d9/db 0 2024-03-16T09:01:20.665 INFO:tasks.workunit.client.0.smithi007.stdout:7/827: dread d0/d19/d1f/f2c [0,4194304] 0 2024-03-16T09:01:20.665 INFO:tasks.workunit.client.0.smithi007.stdout:6/752: link d2/d82/ce6 d2/d10/cfc 0 2024-03-16T09:01:20.666 INFO:tasks.workunit.client.1.smithi019.stdout:7/602: creat d9/da/d13/d18/d30/d9e/fc1 x:0 0 0 2024-03-16T09:01:20.666 INFO:tasks.workunit.client.1.smithi019.stdout:7/603: readlink d9/db/l3b 0 2024-03-16T09:01:20.666 INFO:tasks.workunit.client.0.smithi007.stdout:1/673: rmdir d9/d6b/d7e/d91 39 2024-03-16T09:01:20.666 INFO:tasks.workunit.client.0.smithi007.stdout:1/674: creat d9/d6b/d7e/dac/dbb/fd4 x:0 0 0 2024-03-16T09:01:20.667 INFO:tasks.workunit.client.0.smithi007.stdout:4/724: mknod d4/d14/d64/d68/cf8 0 2024-03-16T09:01:20.668 INFO:tasks.workunit.client.1.smithi019.stdout:5/696: dread d6/d19/f1b [0,4194304] 0 2024-03-16T09:01:20.668 INFO:tasks.workunit.client.1.smithi019.stdout:5/697: dread - d6/d1f/fdd zero size 2024-03-16T09:01:20.668 INFO:tasks.workunit.client.1.smithi019.stdout:8/756: dread d6/d2a/d45/d57/fbb [0,4194304] 0 2024-03-16T09:01:20.668 INFO:tasks.workunit.client.0.smithi007.stdout:3/823: getdents d15/d18/d1f/d56/d98 0 2024-03-16T09:01:20.670 INFO:tasks.workunit.client.0.smithi007.stdout:6/753: symlink d2/d13/d4b/d94/dea/df7/lfd 0 2024-03-16T09:01:20.671 INFO:tasks.workunit.client.1.smithi019.stdout:8/757: mknod d6/d2a/d45/d19/c105 0 2024-03-16T09:01:20.672 INFO:tasks.workunit.client.1.smithi019.stdout:8/758: stat d6/d2a/d45/d19/d74/f7e 0 2024-03-16T09:01:20.672 INFO:tasks.workunit.client.1.smithi019.stdout:8/759: chown d6/d2a/d3c/dab/d30/d4a/d67/d95/de0 745717 1 2024-03-16T09:01:20.674 INFO:tasks.workunit.client.0.smithi007.stdout:1/675: write d9/dd/f59 [3601444,98884] 0 2024-03-16T09:01:20.676 INFO:tasks.workunit.client.1.smithi019.stdout:8/760: unlink d6/d2a/d45/d19/fc5 0 2024-03-16T09:01:20.676 INFO:tasks.workunit.client.1.smithi019.stdout:8/761: chown d6/d2a/d45/d19/c102 6304735 1 2024-03-16T09:01:20.677 INFO:tasks.workunit.client.0.smithi007.stdout:6/754: getdents d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49 0 2024-03-16T09:01:20.679 INFO:tasks.workunit.client.0.smithi007.stdout:1/676: symlink d9/dd/d1b/d28/d29/ld5 0 2024-03-16T09:01:20.679 INFO:tasks.workunit.client.0.smithi007.stdout:1/677: read d9/f12 [552193,58981] 0 2024-03-16T09:01:20.680 INFO:tasks.workunit.client.1.smithi019.stdout:8/762: symlink d6/d2a/d68/l106 0 2024-03-16T09:01:20.682 INFO:tasks.workunit.client.1.smithi019.stdout:8/763: link d6/d2a/d45/d57/d96/fdf d6/d2a/d3c/dab/d30/d4a/dc7/f107 0 2024-03-16T09:01:20.682 INFO:tasks.workunit.client.1.smithi019.stdout:8/764: rename d6/d2a/d3c/dab/d30/d4a to d6/d2a/d3c/dab/d30/d4a/d67/d95/d108 22 2024-03-16T09:01:20.682 INFO:tasks.workunit.client.0.smithi007.stdout:7/828: write d0/d19/d1f/d5d/d9d/df7/d7a/f71 [7037886,83974] 0 2024-03-16T09:01:20.682 INFO:tasks.workunit.client.0.smithi007.stdout:7/829: creat d0/d19/d1f/d5d/d9d/df7/d73/f127 x:0 0 0 2024-03-16T09:01:20.684 INFO:tasks.workunit.client.0.smithi007.stdout:1/678: creat d9/dd/d1b/fd6 x:0 0 0 2024-03-16T09:01:20.684 INFO:tasks.workunit.client.0.smithi007.stdout:2/923: dwrite dc/db7/fde [0,4194304] 0 2024-03-16T09:01:20.684 INFO:tasks.workunit.client.1.smithi019.stdout:5/698: dread d6/dd/d47/f64 [0,4194304] 0 2024-03-16T09:01:20.684 INFO:tasks.workunit.client.1.smithi019.stdout:8/765: mkdir d6/d2a/d3f/d109 0 2024-03-16T09:01:20.688 INFO:tasks.workunit.client.1.smithi019.stdout:8/766: mknod d6/d2a/d3c/dab/d30/d4a/c10a 0 2024-03-16T09:01:20.691 INFO:tasks.workunit.client.1.smithi019.stdout:8/767: rename d6/d2a/d3c/dab/d30/d4a/f81 to d6/d2a/d3c/f10b 0 2024-03-16T09:01:20.691 INFO:tasks.workunit.client.1.smithi019.stdout:3/637: dwrite d3/d1b/d40/d4a/f98 [0,4194304] 0 2024-03-16T09:01:20.693 INFO:tasks.workunit.client.0.smithi007.stdout:1/679: rename d9/dd/c17 to d9/dd/d1b/d2d/cd7 0 2024-03-16T09:01:20.694 INFO:tasks.workunit.client.1.smithi019.stdout:8/768: unlink d6/de/ca0 0 2024-03-16T09:01:20.695 INFO:tasks.workunit.client.0.smithi007.stdout:0/947: dwrite d3/d1f/f10c [0,4194304] 0 2024-03-16T09:01:20.697 INFO:tasks.workunit.client.0.smithi007.stdout:2/924: rename dc/d27/fd1 to dc/df/d24/d28/d4c/daf/f133 0 2024-03-16T09:01:20.699 INFO:tasks.workunit.client.1.smithi019.stdout:8/769: creat d6/d2a/d3c/dab/d30/d4a/dc7/f10c x:0 0 0 2024-03-16T09:01:20.699 INFO:tasks.workunit.client.1.smithi019.stdout:8/770: readlink d6/l17 0 2024-03-16T09:01:20.701 INFO:tasks.workunit.client.1.smithi019.stdout:7/604: dwrite d9/da/d13/d9a/f77 [0,4194304] 0 2024-03-16T09:01:20.701 INFO:tasks.workunit.client.1.smithi019.stdout:7/605: dread - d9/db/d12/f55 zero size 2024-03-16T09:01:20.703 INFO:tasks.workunit.client.0.smithi007.stdout:0/948: mknod d3/d1f/c136 0 2024-03-16T09:01:20.703 INFO:tasks.workunit.client.0.smithi007.stdout:0/949: readlink d3/d9/da/d48/d25/d43/lcc 0 2024-03-16T09:01:20.703 INFO:tasks.workunit.client.0.smithi007.stdout:0/950: chown d3/d9/da/d48/d25/d87/dbb/f12a 9679 1 2024-03-16T09:01:20.704 INFO:tasks.workunit.client.1.smithi019.stdout:7/606: creat d9/da/d13/d18/d30/d75/da4/fc2 x:0 0 0 2024-03-16T09:01:20.705 INFO:tasks.workunit.client.1.smithi019.stdout:7/607: mkdir d9/db/d12/d5b/dc3 0 2024-03-16T09:01:20.705 INFO:tasks.workunit.client.0.smithi007.stdout:0/951: mkdir d3/d9/da/d48/d25/d87/d137 0 2024-03-16T09:01:20.706 INFO:tasks.workunit.client.0.smithi007.stdout:0/952: chown d3/d9/da/d48/d25/d43/d1a/d58/d61 164169 1 2024-03-16T09:01:20.708 INFO:tasks.workunit.client.1.smithi019.stdout:7/608: rmdir d9/db/d12/d5b/d33 39 2024-03-16T09:01:20.708 INFO:tasks.workunit.client.1.smithi019.stdout:7/609: fdatasync d9/d25/d28/f57 0 2024-03-16T09:01:20.708 INFO:tasks.workunit.client.1.smithi019.stdout:7/610: write d9/da/d13/f84 [2005985,8602] 0 2024-03-16T09:01:20.708 INFO:tasks.workunit.client.1.smithi019.stdout:7/611: fsync d9/f74 0 2024-03-16T09:01:20.709 INFO:tasks.workunit.client.0.smithi007.stdout:4/725: dwrite d4/d14/d64/d9f/f42 [4194304,4194304] 0 2024-03-16T09:01:20.709 INFO:tasks.workunit.client.0.smithi007.stdout:4/726: write d4/d14/d1c/d5b/f74 [1265902,46086] 0 2024-03-16T09:01:20.709 INFO:tasks.workunit.client.0.smithi007.stdout:4/727: chown d4/d14/f19 972709 1 2024-03-16T09:01:20.710 INFO:tasks.workunit.client.0.smithi007.stdout:1/680: dread d9/dd/d1b/d2d/f44 [0,4194304] 0 2024-03-16T09:01:20.711 INFO:tasks.workunit.client.1.smithi019.stdout:3/638: write d3/d1b/d40/d4a/f98 [1053770,127535] 0 2024-03-16T09:01:20.712 INFO:tasks.workunit.client.0.smithi007.stdout:2/925: dread dc/d27/f4f [0,4194304] 0 2024-03-16T09:01:20.712 INFO:tasks.workunit.client.0.smithi007.stdout:2/926: chown dc/df/d24/d28/d4c/f58 1751923509 1 2024-03-16T09:01:20.713 INFO:tasks.workunit.client.1.smithi019.stdout:3/639: mknod d3/d1b/d88/d91/dbe/cc5 0 2024-03-16T09:01:20.717 INFO:tasks.workunit.client.0.smithi007.stdout:0/953: write d3/d9/da/d48/f13 [5784592,3133] 0 2024-03-16T09:01:20.717 INFO:tasks.workunit.client.0.smithi007.stdout:0/954: read - d3/d9/da/d92/d9f/de8/f111 zero size 2024-03-16T09:01:20.717 INFO:tasks.workunit.client.0.smithi007.stdout:0/955: chown d3/d1f/fb7 1 1 2024-03-16T09:01:20.717 INFO:tasks.workunit.client.0.smithi007.stdout:0/956: chown d3/d9/db4/cd8 38026 1 2024-03-16T09:01:20.717 INFO:tasks.workunit.client.0.smithi007.stdout:4/728: link d4/d14/d64/d68/d82/dac/fe1 d4/d14/d1c/d5b/d67/de8/ff9 0 2024-03-16T09:01:20.717 INFO:tasks.workunit.client.0.smithi007.stdout:4/729: dread - d4/d14/d64/d68/fc4 zero size 2024-03-16T09:01:20.717 INFO:tasks.workunit.client.0.smithi007.stdout:4/730: write d4/d14/d64/d9f/fbd [63290,41785] 0 2024-03-16T09:01:20.718 INFO:tasks.workunit.client.1.smithi019.stdout:8/771: write d6/d2a/d45/d19/d74/f7e [349224,23127] 0 2024-03-16T09:01:20.718 INFO:tasks.workunit.client.1.smithi019.stdout:8/772: dread - d6/d2a/d45/d57/d96/fad zero size 2024-03-16T09:01:20.719 INFO:tasks.workunit.client.0.smithi007.stdout:2/927: truncate dc/df/d24/d28/d2e/f78 381972 0 2024-03-16T09:01:20.719 INFO:tasks.workunit.client.1.smithi019.stdout:8/773: mkdir d6/d2a/d3c/dab/d30/d4a/d67/dee/d10d 0 2024-03-16T09:01:20.720 INFO:tasks.workunit.client.1.smithi019.stdout:2/745: symlink d5/d2e/l102 0 2024-03-16T09:01:20.721 INFO:tasks.workunit.client.1.smithi019.stdout:2/746: chown d5/de/f4a 11 1 2024-03-16T09:01:20.722 INFO:tasks.workunit.client.1.smithi019.stdout:3/640: dread d3/d11/f1f [0,4194304] 0 2024-03-16T09:01:20.722 INFO:tasks.workunit.client.1.smithi019.stdout:3/641: fsync d3/d8b/dae/d3f/fa4 0 2024-03-16T09:01:20.722 INFO:tasks.workunit.client.0.smithi007.stdout:0/957: symlink d3/d9/da/d48/d25/l138 0 2024-03-16T09:01:20.722 INFO:tasks.workunit.client.0.smithi007.stdout:0/958: write d3/d9/da/d48/d25/d43/d1a/d58/d61/f12f [893389,97085] 0 2024-03-16T09:01:20.723 INFO:tasks.workunit.client.0.smithi007.stdout:4/731: mknod d4/dcb/dcd/cfa 0 2024-03-16T09:01:20.723 INFO:tasks.workunit.client.0.smithi007.stdout:4/732: write d4/d14/d1c/f32 [44092,73419] 0 2024-03-16T09:01:20.723 INFO:tasks.workunit.client.1.smithi019.stdout:3/642: creat d3/d1b/d40/fc6 x:0 0 0 2024-03-16T09:01:20.724 INFO:tasks.workunit.client.1.smithi019.stdout:2/747: creat d5/d6/d2b/f103 x:0 0 0 2024-03-16T09:01:20.724 INFO:tasks.workunit.client.1.smithi019.stdout:2/748: readlink d5/d5a/d96/dba/dfc/l77 0 2024-03-16T09:01:20.725 INFO:tasks.workunit.client.0.smithi007.stdout:1/681: mkdir d9/dd/d1b/dd8 0 2024-03-16T09:01:20.725 INFO:tasks.workunit.client.1.smithi019.stdout:3/643: unlink d3/d1b/f8e 0 2024-03-16T09:01:20.725 INFO:tasks.workunit.client.1.smithi019.stdout:3/644: chown d3/d8b/dae/l8a 1707438642 1 2024-03-16T09:01:20.726 INFO:tasks.workunit.client.0.smithi007.stdout:2/928: symlink dc/df/d24/d28/dea/d102/l134 0 2024-03-16T09:01:20.726 INFO:tasks.workunit.client.0.smithi007.stdout:2/929: dread - dc/df/d24/d28/d4c/d5e/f96 zero size 2024-03-16T09:01:20.726 INFO:tasks.workunit.client.0.smithi007.stdout:2/930: stat dc/df/d24/d28/d2e/fc7 0 2024-03-16T09:01:20.726 INFO:tasks.workunit.client.0.smithi007.stdout:2/931: chown dc/df/d24/d28/d4c/dbc/l109 164661785 1 2024-03-16T09:01:20.726 INFO:tasks.workunit.client.0.smithi007.stdout:2/932: read dc/f10 [2857907,66789] 0 2024-03-16T09:01:20.727 INFO:tasks.workunit.client.0.smithi007.stdout:2/933: unlink dc/df/d24/d28/d4c/dbc/l109 0 2024-03-16T09:01:20.728 INFO:tasks.workunit.client.1.smithi019.stdout:3/645: mkdir d3/d1b/dc7 0 2024-03-16T09:01:20.728 INFO:tasks.workunit.client.1.smithi019.stdout:3/646: write d3/d1b/d40/d4a/f98 [325758,43589] 0 2024-03-16T09:01:20.729 INFO:tasks.workunit.client.1.smithi019.stdout:3/647: creat d3/fc8 x:0 0 0 2024-03-16T09:01:20.730 INFO:tasks.workunit.client.1.smithi019.stdout:3/648: chown d3/d1b/d40/fb5 100 1 2024-03-16T09:01:20.731 INFO:tasks.workunit.client.1.smithi019.stdout:3/649: symlink d3/d1b/d88/d91/lc9 0 2024-03-16T09:01:20.732 INFO:tasks.workunit.client.1.smithi019.stdout:3/650: rmdir d3/d8b/dae/d3f 39 2024-03-16T09:01:20.734 INFO:tasks.workunit.client.1.smithi019.stdout:3/651: mkdir d3/d8b/dca 0 2024-03-16T09:01:20.735 INFO:tasks.workunit.client.1.smithi019.stdout:8/774: write d6/d2a/d45/d5d/faf [602379,2052] 0 2024-03-16T09:01:20.735 INFO:tasks.workunit.client.1.smithi019.stdout:8/775: fdatasync d6/d2a/d45/d19/d74/f7f 0 2024-03-16T09:01:20.735 INFO:tasks.workunit.client.1.smithi019.stdout:5/699: dwrite d6/dd/d37/d54/f5f [0,4194304] 0 2024-03-16T09:01:20.736 INFO:tasks.workunit.client.0.smithi007.stdout:1/682: write d9/dd/d1b/d28/d4c/f63 [329309,30645] 0 2024-03-16T09:01:20.737 INFO:tasks.workunit.client.1.smithi019.stdout:3/652: read d3/d11/f6f [18385,122280] 0 2024-03-16T09:01:20.737 INFO:tasks.workunit.client.1.smithi019.stdout:3/653: truncate d3/d1b/f9e 976097 0 2024-03-16T09:01:20.737 INFO:tasks.workunit.client.1.smithi019.stdout:3/654: write d3/f6c [1659356,98962] 0 2024-03-16T09:01:20.737 INFO:tasks.workunit.client.1.smithi019.stdout:3/655: creat d3/d89/fcb x:0 0 0 2024-03-16T09:01:20.741 INFO:tasks.workunit.client.1.smithi019.stdout:5/700: creat d6/d19/d34/fdf x:0 0 0 2024-03-16T09:01:20.741 INFO:tasks.workunit.client.1.smithi019.stdout:5/701: read d6/f8 [3301286,41960] 0 2024-03-16T09:01:20.742 INFO:tasks.workunit.client.0.smithi007.stdout:3/824: dwrite d15/d18/d1f/d29/d3e/d4c/f64 [0,4194304] 0 2024-03-16T09:01:20.742 INFO:tasks.workunit.client.0.smithi007.stdout:7/830: dwrite d0/d19/f2d [0,4194304] 0 2024-03-16T09:01:20.742 INFO:tasks.workunit.client.0.smithi007.stdout:7/831: fsync d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f7b 0 2024-03-16T09:01:20.742 INFO:tasks.workunit.client.0.smithi007.stdout:7/832: chown d0/df5/d10e 1960980 1 2024-03-16T09:01:20.743 INFO:tasks.workunit.client.0.smithi007.stdout:7/833: fsync d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f98 0 2024-03-16T09:01:20.744 INFO:tasks.workunit.client.1.smithi019.stdout:2/749: write d5/d5a/d96/dba/dfc/d74/f95 [3018125,110030] 0 2024-03-16T09:01:20.744 INFO:tasks.workunit.client.1.smithi019.stdout:3/656: mkdir d3/d8b/da8/dcc 0 2024-03-16T09:01:20.748 INFO:tasks.workunit.client.0.smithi007.stdout:6/755: dwrite d2/d13/d3f/d50/f53 [0,4194304] 0 2024-03-16T09:01:20.752 INFO:tasks.workunit.client.1.smithi019.stdout:7/612: dwrite d9/db/fb1 [0,4194304] 0 2024-03-16T09:01:20.752 INFO:tasks.workunit.client.1.smithi019.stdout:7/613: write d9/da/d13/f84 [2483195,34246] 0 2024-03-16T09:01:20.752 INFO:tasks.workunit.client.1.smithi019.stdout:7/614: truncate d9/da/d13/d18/f5a 493756 0 2024-03-16T09:01:20.753 INFO:tasks.workunit.client.1.smithi019.stdout:3/657: unlink d3/d1b/d40/f74 0 2024-03-16T09:01:20.753 INFO:tasks.workunit.client.1.smithi019.stdout:3/658: chown d3/d8b/dae/d5a/fbb 2240 1 2024-03-16T09:01:20.753 INFO:tasks.workunit.client.1.smithi019.stdout:3/659: chown d3/d11/c58 0 1 2024-03-16T09:01:20.756 INFO:tasks.workunit.client.1.smithi019.stdout:3/660: mkdir d3/d8b/dae/d5a/d75/d7a/dcd 0 2024-03-16T09:01:20.756 INFO:tasks.workunit.client.1.smithi019.stdout:3/661: chown d3/d1b/d88/fc2 249349 1 2024-03-16T09:01:20.756 INFO:tasks.workunit.client.1.smithi019.stdout:3/662: dread - d3/fc8 zero size 2024-03-16T09:01:20.758 INFO:tasks.workunit.client.0.smithi007.stdout:9/813: dwrite d24/d4c/d86/daa/d95/f62 [0,4194304] 0 2024-03-16T09:01:20.758 INFO:tasks.workunit.client.0.smithi007.stdout:9/814: fdatasync d24/f39 0 2024-03-16T09:01:20.758 INFO:tasks.workunit.client.1.smithi019.stdout:3/663: mknod d3/d8b/cce 0 2024-03-16T09:01:20.758 INFO:tasks.workunit.client.1.smithi019.stdout:3/664: write d3/d8b/dae/f72 [8831013,115515] 0 2024-03-16T09:01:20.762 INFO:tasks.workunit.client.1.smithi019.stdout:3/665: mknod d3/d8b/dae/d5a/d75/ccf 0 2024-03-16T09:01:20.764 INFO:tasks.workunit.client.1.smithi019.stdout:3/666: mkdir d3/d8b/dae/d5a/d75/d7a/dd0 0 2024-03-16T09:01:20.767 INFO:tasks.workunit.client.0.smithi007.stdout:9/815: dread d24/f26 [0,4194304] 0 2024-03-16T09:01:20.767 INFO:tasks.workunit.client.0.smithi007.stdout:9/816: stat d24/d4c/d86/daa/d7f/d8f/fab 0 2024-03-16T09:01:20.780 INFO:tasks.workunit.client.1.smithi019.stdout:8/776: symlink d6/d2a/d45/d57/d96/df5/l10e 0 2024-03-16T09:01:20.780 INFO:tasks.workunit.client.1.smithi019.stdout:8/777: truncate d6/d2a/d3c/dab/d30/f4f 4447150 0 2024-03-16T09:01:20.780 INFO:tasks.workunit.client.1.smithi019.stdout:8/778: dread - d6/d2a/d68/fd0 zero size 2024-03-16T09:01:20.781 INFO:tasks.workunit.client.0.smithi007.stdout:0/959: dwrite d3/d9/db4/d1c/f2a [4194304,4194304] 0 2024-03-16T09:01:20.799 INFO:tasks.workunit.client.0.smithi007.stdout:1/683: truncate d9/f12 162582 0 2024-03-16T09:01:20.800 INFO:tasks.workunit.client.0.smithi007.stdout:1/684: read - d9/fca zero size 2024-03-16T09:01:20.800 INFO:tasks.workunit.client.0.smithi007.stdout:1/685: write d9/dd/d1b/d2d/d30/f61 [1978869,48909] 0 2024-03-16T09:01:20.800 INFO:tasks.workunit.client.0.smithi007.stdout:1/686: write d9/dd/d1b/d28/d29/fab [387159,70177] 0 2024-03-16T09:01:20.800 INFO:tasks.workunit.client.0.smithi007.stdout:1/687: creat d9/dd/d1b/d28/d29/d7b/db4/fd9 x:0 0 0 2024-03-16T09:01:20.804 INFO:tasks.workunit.client.0.smithi007.stdout:3/825: dwrite d15/d18/d1f/d29/d3e/fc8 [0,4194304] 0 2024-03-16T09:01:20.805 INFO:tasks.workunit.client.0.smithi007.stdout:2/934: dwrite dc/df/d24/d28/d2e/d4a/fef [0,4194304] 0 2024-03-16T09:01:20.807 INFO:tasks.workunit.client.1.smithi019.stdout:2/750: rename d5/d5a/d96 to d5/d6/d104 0 2024-03-16T09:01:20.808 INFO:tasks.workunit.client.0.smithi007.stdout:4/733: dwrite d4/d14/d1c/d5b/f74 [0,4194304] 0 2024-03-16T09:01:20.810 INFO:tasks.workunit.client.1.smithi019.stdout:5/702: dwrite d6/dd/d47/d70/db2/d66/fd1 [0,4194304] 0 2024-03-16T09:01:20.810 INFO:tasks.workunit.client.1.smithi019.stdout:5/703: chown d6/d19/d34/dd2 3626 1 2024-03-16T09:01:20.810 INFO:tasks.workunit.client.1.smithi019.stdout:7/615: dwrite d9/f74 [0,4194304] 0 2024-03-16T09:01:20.811 INFO:tasks.workunit.client.1.smithi019.stdout:7/616: creat d9/db/d12/fc4 x:0 0 0 2024-03-16T09:01:20.811 INFO:tasks.workunit.client.1.smithi019.stdout:7/617: truncate d9/db/f2c 2920785 0 2024-03-16T09:01:20.811 INFO:tasks.workunit.client.1.smithi019.stdout:7/618: write d9/da/d13/f94 [419680,80757] 0 2024-03-16T09:01:20.811 INFO:tasks.workunit.client.1.smithi019.stdout:7/619: chown d9/da/d13/d18/d30/d3e/da5 31 1 2024-03-16T09:01:20.811 INFO:tasks.workunit.client.0.smithi007.stdout:9/817: dwrite d24/f48 [4194304,4194304] 0 2024-03-16T09:01:20.811 INFO:tasks.workunit.client.1.smithi019.stdout:3/667: dwrite d3/d8b/dae/d5a/f61 [0,4194304] 0 2024-03-16T09:01:20.812 INFO:tasks.workunit.client.1.smithi019.stdout:8/779: dwrite d6/fd4 [0,4194304] 0 2024-03-16T09:01:20.812 INFO:tasks.workunit.client.1.smithi019.stdout:8/780: readlink d6/d2a/d45/d57/d72/d79/db7/ld2 0 2024-03-16T09:01:20.818 INFO:tasks.workunit.client.1.smithi019.stdout:7/620: mknod d9/d25/d28/d48/d8d/dba/cc5 0 2024-03-16T09:01:20.819 INFO:tasks.workunit.client.1.smithi019.stdout:7/621: write d9/db/fa7 [222647,37070] 0 2024-03-16T09:01:20.822 INFO:tasks.workunit.client.1.smithi019.stdout:3/668: rmdir d3/d1b/d40/d4a/db7 39 2024-03-16T09:01:20.822 INFO:tasks.workunit.client.1.smithi019.stdout:5/704: dread d6/fb [0,4194304] 0 2024-03-16T09:01:20.825 INFO:tasks.workunit.client.0.smithi007.stdout:9/818: dwrite d24/d4a/d96/db9/fe2 [0,4194304] 0 2024-03-16T09:01:20.829 INFO:tasks.workunit.client.1.smithi019.stdout:3/669: symlink d3/d9b/ld1 0 2024-03-16T09:01:20.838 INFO:tasks.workunit.client.1.smithi019.stdout:8/781: unlink d6/d2a/d45/d57/d96/f9c 0 2024-03-16T09:01:20.842 INFO:tasks.workunit.client.1.smithi019.stdout:2/751: mkdir d5/d105 0 2024-03-16T09:01:20.843 INFO:tasks.workunit.client.1.smithi019.stdout:2/752: fdatasync d5/d5a/f9d 0 2024-03-16T09:01:20.843 INFO:tasks.workunit.client.1.smithi019.stdout:7/622: dwrite d9/da/d13/d18/d30/d75/fb3 [0,4194304] 0 2024-03-16T09:01:20.843 INFO:tasks.workunit.client.1.smithi019.stdout:5/705: dwrite d6/d19/f4e [0,4194304] 0 2024-03-16T09:01:20.844 INFO:tasks.workunit.client.0.smithi007.stdout:7/834: mkdir d0/dc/d5a/dba/dd9/d93/d128 0 2024-03-16T09:01:20.844 INFO:tasks.workunit.client.0.smithi007.stdout:7/835: dread - d0/d19/d1f/d5d/d9d/df7/d3a/f111 zero size 2024-03-16T09:01:20.844 INFO:tasks.workunit.client.1.smithi019.stdout:3/670: read d3/d8b/dae/d6a/fb9 [3397090,118742] 0 2024-03-16T09:01:20.845 INFO:tasks.workunit.client.1.smithi019.stdout:3/671: fdatasync d3/d8b/dae/d5a/d80/f9c 0 2024-03-16T09:01:20.845 INFO:tasks.workunit.client.1.smithi019.stdout:7/623: symlink d9/da/d13/d18/d1a/lc6 0 2024-03-16T09:01:20.847 INFO:tasks.workunit.client.1.smithi019.stdout:5/706: getdents d6/dd/d37 0 2024-03-16T09:01:20.847 INFO:tasks.workunit.client.1.smithi019.stdout:8/782: mkdir d6/d2a/d45/d57/d72/d10f 0 2024-03-16T09:01:20.847 INFO:tasks.workunit.client.1.smithi019.stdout:8/783: stat d6/d2a/d45/d57/d72/d79/cda 0 2024-03-16T09:01:20.847 INFO:tasks.workunit.client.1.smithi019.stdout:8/784: write d6/d2a/d45/f6b [1752384,102174] 0 2024-03-16T09:01:20.847 INFO:tasks.workunit.client.1.smithi019.stdout:3/672: mknod d3/d8b/dae/d5a/d75/d7a/dd0/cd2 0 2024-03-16T09:01:20.850 INFO:tasks.workunit.client.1.smithi019.stdout:2/753: creat d5/d6/d104/dba/dfc/d74/f106 x:0 0 0 2024-03-16T09:01:20.860 INFO:tasks.workunit.client.0.smithi007.stdout:9/819: dwrite d24/d4c/d86/daa/d95/f62 [0,4194304] 0 2024-03-16T09:01:20.860 INFO:tasks.workunit.client.0.smithi007.stdout:9/820: write d24/d4a/d55/f5d [751347,31666] 0 2024-03-16T09:01:20.860 INFO:tasks.workunit.client.1.smithi019.stdout:8/785: rename d6/d2a/d45/d19/d74/c104 to d6/d2a/d45/d57/d72/d79/db7/c110 0 2024-03-16T09:01:20.881 INFO:tasks.workunit.client.1.smithi019.stdout:7/624: dwrite d9/d25/f54 [0,4194304] 0 2024-03-16T09:01:20.884 INFO:tasks.workunit.client.1.smithi019.stdout:7/625: rename d9/db/d12/d5b/l35 to d9/da/d13/d18/lc7 0 2024-03-16T09:01:20.884 INFO:tasks.workunit.client.0.smithi007.stdout:7/836: dwrite d0/d19/f2b [0,4194304] 0 2024-03-16T09:01:20.885 INFO:tasks.workunit.client.1.smithi019.stdout:8/786: dwrite d6/fd4 [0,4194304] 0 2024-03-16T09:01:20.885 INFO:tasks.workunit.client.1.smithi019.stdout:8/787: readlink d6/d2a/d3c/dab/d30/d4a/l5e 0 2024-03-16T09:01:20.885 INFO:tasks.workunit.client.1.smithi019.stdout:8/788: write d6/d2a/d45/d57/d72/d9d/fce [192876,60659] 0 2024-03-16T09:01:20.886 INFO:tasks.workunit.client.1.smithi019.stdout:5/707: dwrite d6/dd/d47/d70/db2/d89/fc5 [0,4194304] 0 2024-03-16T09:01:20.889 INFO:tasks.workunit.client.1.smithi019.stdout:5/708: creat d6/dd/d47/d70/db2/d89/fe0 x:0 0 0 2024-03-16T09:01:20.889 INFO:tasks.workunit.client.1.smithi019.stdout:5/709: dread - d6/dd/d37/d76/fcf zero size 2024-03-16T09:01:20.891 INFO:tasks.workunit.client.1.smithi019.stdout:8/789: link d6/d2a/d45/d19/d74/cbf d6/de/c111 0 2024-03-16T09:01:20.893 INFO:tasks.workunit.client.0.smithi007.stdout:7/837: dread d0/d19/d1f/d5d/d9d/df7/d3a/db3/fce [0,4194304] 0 2024-03-16T09:01:20.895 INFO:tasks.workunit.client.1.smithi019.stdout:7/626: write d9/da/d13/d18/d30/d75/fb3 [133450,128926] 0 2024-03-16T09:01:20.895 INFO:tasks.workunit.client.1.smithi019.stdout:7/627: readlink d9/d25/d28/l7f 0 2024-03-16T09:01:20.896 INFO:tasks.workunit.client.1.smithi019.stdout:7/628: write d9/db/fa7 [85768,49659] 0 2024-03-16T09:01:20.897 INFO:tasks.workunit.client.1.smithi019.stdout:5/710: dread d6/dd/d37/f93 [0,4194304] 0 2024-03-16T09:01:20.898 INFO:tasks.workunit.client.1.smithi019.stdout:5/711: truncate d6/d19/f4e 8563734 0 2024-03-16T09:01:20.898 INFO:tasks.workunit.client.1.smithi019.stdout:5/712: dread d6/dd/d37/fb4 [0,4194304] 0 2024-03-16T09:01:20.900 INFO:tasks.workunit.client.1.smithi019.stdout:5/713: rmdir d6/dd/d47/d56/d87 39 2024-03-16T09:01:20.901 INFO:tasks.workunit.client.1.smithi019.stdout:5/714: truncate d6/dd/f16 679659 0 2024-03-16T09:01:20.901 INFO:tasks.workunit.client.1.smithi019.stdout:5/715: chown d6/d19/d34/dc2/fc4 70642997 1 2024-03-16T09:01:20.901 INFO:tasks.workunit.client.1.smithi019.stdout:5/716: mknod d6/dd/d47/dd0/ce1 0 2024-03-16T09:01:20.902 INFO:tasks.workunit.client.0.smithi007.stdout:7/838: dread d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f62 [0,4194304] 0 2024-03-16T09:01:20.902 INFO:tasks.workunit.client.0.smithi007.stdout:7/839: write d0/d19/d1f/d5d/d9d/df7/d7a/dc9/ff0 [415476,95004] 0 2024-03-16T09:01:20.903 INFO:tasks.workunit.client.1.smithi019.stdout:5/717: creat d6/dd/daf/fe2 x:0 0 0 2024-03-16T09:01:20.917 INFO:tasks.workunit.client.0.smithi007.stdout:7/840: write d0/d19/d1f/d5d/d9d/df7/d3a/db3/fee [241385,115370] 0 2024-03-16T09:01:20.923 INFO:tasks.workunit.client.1.smithi019.stdout:7/629: dwrite d9/da/f59 [0,4194304] 0 2024-03-16T09:01:20.926 INFO:tasks.workunit.client.1.smithi019.stdout:7/630: creat d9/d25/d28/d48/d8d/dba/fc8 x:0 0 0 2024-03-16T09:01:20.926 INFO:tasks.workunit.client.1.smithi019.stdout:4/664: sync 2024-03-16T09:01:20.926 INFO:tasks.workunit.client.1.smithi019.stdout:9/527: sync 2024-03-16T09:01:20.926 INFO:tasks.workunit.client.1.smithi019.stdout:1/611: sync 2024-03-16T09:01:20.927 INFO:tasks.workunit.client.1.smithi019.stdout:1/612: dread - d16/d1f/d3c/f67 zero size 2024-03-16T09:01:20.927 INFO:tasks.workunit.client.1.smithi019.stdout:6/792: sync 2024-03-16T09:01:20.927 INFO:tasks.workunit.client.1.smithi019.stdout:9/528: symlink d1/d7/de/d24/d44/d60/lb4 0 2024-03-16T09:01:20.928 INFO:tasks.workunit.client.1.smithi019.stdout:5/718: dwrite d6/dd/d47/f64 [0,4194304] 0 2024-03-16T09:01:20.928 INFO:tasks.workunit.client.1.smithi019.stdout:6/793: mknod de/d26/d32/d97/d66/ddc/cfe 0 2024-03-16T09:01:20.931 INFO:tasks.workunit.client.1.smithi019.stdout:9/529: unlink d1/d7/de/d24/c7c 0 2024-03-16T09:01:20.933 INFO:tasks.workunit.client.0.smithi007.stdout:0/960: mkdir d3/d1f/d139 0 2024-03-16T09:01:20.935 INFO:tasks.workunit.client.1.smithi019.stdout:1/613: dread f9 [0,4194304] 0 2024-03-16T09:01:20.935 INFO:tasks.workunit.client.1.smithi019.stdout:6/794: link de/d26/d32/d97/d66/d72/db7/fbb de/d26/fff 0 2024-03-16T09:01:20.937 INFO:tasks.workunit.client.0.smithi007.stdout:7/841: read d0/d19/d1f/d5d/d9d/df8/f103 [238300,86774] 0 2024-03-16T09:01:20.937 INFO:tasks.workunit.client.0.smithi007.stdout:7/842: write d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d9c/f10f [27769,18245] 0 2024-03-16T09:01:20.939 INFO:tasks.workunit.client.0.smithi007.stdout:1/688: mkdir d9/dd/d1b/d28/d29/d7b/dba/dda 0 2024-03-16T09:01:20.942 INFO:tasks.workunit.client.1.smithi019.stdout:4/665: read d4/d6/f83 [4164896,124520] 0 2024-03-16T09:01:20.942 INFO:tasks.workunit.client.1.smithi019.stdout:4/666: dread - d4/d1b/d49/d58/d80/d91/fba zero size 2024-03-16T09:01:20.943 INFO:tasks.workunit.client.0.smithi007.stdout:3/826: truncate d15/f16 4416773 0 2024-03-16T09:01:20.943 INFO:tasks.workunit.client.1.smithi019.stdout:5/719: dread d6/d1f/d78/f9a [0,4194304] 0 2024-03-16T09:01:20.945 INFO:tasks.workunit.client.1.smithi019.stdout:4/667: creat d4/d1b/d2a/fdb x:0 0 0 2024-03-16T09:01:20.947 INFO:tasks.workunit.client.0.smithi007.stdout:2/935: mkdir dc/dba/d135 0 2024-03-16T09:01:20.948 INFO:tasks.workunit.client.1.smithi019.stdout:5/720: mknod d6/dd/daf/ce3 0 2024-03-16T09:01:20.948 INFO:tasks.workunit.client.1.smithi019.stdout:5/721: creat d6/d19/d34/fe4 x:0 0 0 2024-03-16T09:01:20.948 INFO:tasks.workunit.client.1.smithi019.stdout:7/631: dwrite d9/db/d12/f26 [0,4194304] 0 2024-03-16T09:01:20.948 INFO:tasks.workunit.client.1.smithi019.stdout:7/632: creat d9/d25/d28/d48/d8d/dba/fc9 x:0 0 0 2024-03-16T09:01:20.948 INFO:tasks.workunit.client.1.smithi019.stdout:7/633: creat d9/d25/d28/d48/d8d/dba/fca x:0 0 0 2024-03-16T09:01:20.949 INFO:tasks.workunit.client.1.smithi019.stdout:7/634: write d9/faa [611626,80308] 0 2024-03-16T09:01:20.950 INFO:tasks.workunit.client.1.smithi019.stdout:9/530: dread d1/d98/dae/f57 [0,4194304] 0 2024-03-16T09:01:20.951 INFO:tasks.workunit.client.0.smithi007.stdout:4/734: unlink d4/d14/d64/d9f/l5c 0 2024-03-16T09:01:20.951 INFO:tasks.workunit.client.0.smithi007.stdout:4/735: chown d4/f40 5498 1 2024-03-16T09:01:20.951 INFO:tasks.workunit.client.1.smithi019.stdout:4/668: mknod d4/d6/cdc 0 2024-03-16T09:01:20.959 INFO:tasks.workunit.client.1.smithi019.stdout:5/722: symlink d6/dd/d47/d70/db2/d89/dcc/le5 0 2024-03-16T09:01:20.959 INFO:tasks.workunit.client.1.smithi019.stdout:5/723: creat d6/d1f/d78/fe6 x:0 0 0 2024-03-16T09:01:20.959 INFO:tasks.workunit.client.1.smithi019.stdout:5/724: write d6/d1f/d77/d84/f92 [1190400,45905] 0 2024-03-16T09:01:20.959 INFO:tasks.workunit.client.1.smithi019.stdout:5/725: chown d6/dd/l31 7852 1 2024-03-16T09:01:20.959 INFO:tasks.workunit.client.1.smithi019.stdout:6/795: getdents de/d26 0 2024-03-16T09:01:20.962 INFO:tasks.workunit.client.0.smithi007.stdout:9/821: link d24/d4c/d86/daa/d7f/d8f/fab d24/d8e/ff1 0 2024-03-16T09:01:20.962 INFO:tasks.workunit.client.0.smithi007.stdout:9/822: truncate d24/d4c/d86/fd7 33171 0 2024-03-16T09:01:20.962 INFO:tasks.workunit.client.0.smithi007.stdout:9/823: write d24/d4c/d86/daa/d7f/d8f/fb8 [392623,44837] 0 2024-03-16T09:01:20.963 INFO:tasks.workunit.client.1.smithi019.stdout:7/635: rmdir d9/db 39 2024-03-16T09:01:20.965 INFO:tasks.workunit.client.1.smithi019.stdout:7/636: read d9/d25/d28/d48/d8d/dba/f7c [219818,69239] 0 2024-03-16T09:01:20.965 INFO:tasks.workunit.client.1.smithi019.stdout:4/669: unlink d4/d1b/c87 0 2024-03-16T09:01:20.968 INFO:tasks.workunit.client.0.smithi007.stdout:7/843: rename d0/d19/d1f/d5d/d9d/df7/da1/fab to d0/d19/d1f/d69/de1/f129 0 2024-03-16T09:01:20.968 INFO:tasks.workunit.client.1.smithi019.stdout:5/726: creat d6/dd/d47/d70/db2/d89/dcc/fe7 x:0 0 0 2024-03-16T09:01:20.971 INFO:tasks.workunit.client.1.smithi019.stdout:5/727: link d6/dd/d37/f6b d6/d19/d34/fe8 0 2024-03-16T09:01:20.973 INFO:tasks.workunit.client.0.smithi007.stdout:3/827: mknod d15/d18/d6a/db1/c104 0 2024-03-16T09:01:20.973 INFO:tasks.workunit.client.0.smithi007.stdout:3/828: stat d15/d18/d1f/d29/d3e/d4c/lc9 0 2024-03-16T09:01:20.973 INFO:tasks.workunit.client.0.smithi007.stdout:2/936: symlink dc/df/d24/d28/d97/d99/l136 0 2024-03-16T09:01:20.973 INFO:tasks.workunit.client.0.smithi007.stdout:2/937: creat dc/df/d24/d28/d4c/dbc/dfe/f137 x:0 0 0 2024-03-16T09:01:20.974 INFO:tasks.workunit.client.1.smithi019.stdout:5/728: rename d6/dd/ce to d6/ce9 0 2024-03-16T09:01:20.975 INFO:tasks.workunit.client.1.smithi019.stdout:5/729: write d6/dd/d37/d54/f80 [1113919,35233] 0 2024-03-16T09:01:20.975 INFO:tasks.workunit.client.0.smithi007.stdout:3/829: write d15/d18/d1f/d29/f25 [55109,73485] 0 2024-03-16T09:01:20.975 INFO:tasks.workunit.client.0.smithi007.stdout:3/830: chown d15/d18/d1f/d29/d3e/f103 21159296 1 2024-03-16T09:01:20.978 INFO:tasks.workunit.client.0.smithi007.stdout:7/844: symlink d0/d19/d1f/d5d/d9d/df7/l12a 0 2024-03-16T09:01:20.979 INFO:tasks.workunit.client.0.smithi007.stdout:7/845: chown d0/d19/d1f/d5d/lbb 696191 1 2024-03-16T09:01:20.981 INFO:tasks.workunit.client.0.smithi007.stdout:3/831: link d15/d18/d1f/d29/d3e/l53 d15/d18/d1f/d3b/d63/l105 0 2024-03-16T09:01:20.982 INFO:tasks.workunit.client.1.smithi019.stdout:2/754: getdents d5/d6/d104/dba/dfc/d74 0 2024-03-16T09:01:20.984 INFO:tasks.workunit.client.1.smithi019.stdout:5/730: dread d6/dd/f1d [0,4194304] 0 2024-03-16T09:01:20.984 INFO:tasks.workunit.client.0.smithi007.stdout:3/832: mknod d15/d18/d1f/d29/d3e/dc0/da5/c106 0 2024-03-16T09:01:20.985 INFO:tasks.workunit.client.1.smithi019.stdout:5/731: mknod d6/dd/d47/d70/db2/d89/dcc/cea 0 2024-03-16T09:01:20.987 INFO:tasks.workunit.client.1.smithi019.stdout:5/732: rename d6/d19/d34/c51 to d6/dd/d47/ceb 0 2024-03-16T09:01:20.987 INFO:tasks.workunit.client.1.smithi019.stdout:5/733: fdatasync d6/d19/f7f 0 2024-03-16T09:01:20.987 INFO:tasks.workunit.client.1.smithi019.stdout:0/722: sync 2024-03-16T09:01:20.988 INFO:tasks.workunit.client.1.smithi019.stdout:0/723: chown d7/d8/dec/def/d6c/f77 1039 1 2024-03-16T09:01:20.988 INFO:tasks.workunit.client.1.smithi019.stdout:0/724: dread - d7/d8/dec/def/d48/fde zero size 2024-03-16T09:01:20.988 INFO:tasks.workunit.client.1.smithi019.stdout:0/725: fdatasync d7/f5d 0 2024-03-16T09:01:20.988 INFO:tasks.workunit.client.1.smithi019.stdout:0/726: fsync d7/d8/d32/d66/d6d/d8c/db8/d82/fe3 0 2024-03-16T09:01:20.988 INFO:tasks.workunit.client.0.smithi007.stdout:3/833: creat d15/d18/d1f/d29/d43/df7/f107 x:0 0 0 2024-03-16T09:01:20.988 INFO:tasks.workunit.client.1.smithi019.stdout:5/734: symlink d6/dd/lec 0 2024-03-16T09:01:20.990 INFO:tasks.workunit.client.1.smithi019.stdout:5/735: dread d6/d19/f3c [0,4194304] 0 2024-03-16T09:01:20.990 INFO:tasks.workunit.client.0.smithi007.stdout:4/736: creat d4/d14/d1c/d5b/d67/d8f/da8/d78/ffb x:0 0 0 2024-03-16T09:01:20.990 INFO:tasks.workunit.client.0.smithi007.stdout:3/834: write d15/d18/d6a/d78/ffb [103531,31206] 0 2024-03-16T09:01:20.990 INFO:tasks.workunit.client.0.smithi007.stdout:3/835: truncate d15/d18/d6a/fe9 320456 0 2024-03-16T09:01:20.991 INFO:tasks.workunit.client.1.smithi019.stdout:5/736: creat d6/dd/d47/d70/fed x:0 0 0 2024-03-16T09:01:20.991 INFO:tasks.workunit.client.1.smithi019.stdout:5/737: fdatasync d6/dd/d37/d76/f94 0 2024-03-16T09:01:20.993 INFO:tasks.workunit.client.0.smithi007.stdout:3/836: truncate d15/d18/d6a/db1/fa7 3678771 0 2024-03-16T09:01:20.993 INFO:tasks.workunit.client.1.smithi019.stdout:5/738: unlink d6/d19/d34/fe4 0 2024-03-16T09:01:20.994 INFO:tasks.workunit.client.0.smithi007.stdout:1/689: dwrite f7 [0,4194304] 0 2024-03-16T09:01:20.994 INFO:tasks.workunit.client.0.smithi007.stdout:2/938: dread dc/df/d121/f120 [0,4194304] 0 2024-03-16T09:01:20.996 INFO:tasks.workunit.client.1.smithi019.stdout:5/739: link d6/d1f/d77/d84/c8f d6/d1f/cee 0 2024-03-16T09:01:20.997 INFO:tasks.workunit.client.1.smithi019.stdout:5/740: unlink d6/dd/d61/l73 0 2024-03-16T09:01:20.997 INFO:tasks.workunit.client.1.smithi019.stdout:5/741: write d6/dd/f1a [3953063,108408] 0 2024-03-16T09:01:20.997 INFO:tasks.workunit.client.1.smithi019.stdout:5/742: stat d6/dd/fda 0 2024-03-16T09:01:20.998 INFO:tasks.workunit.client.1.smithi019.stdout:0/727: dread d7/d8/d32/d4a/d61/dbf/fe6 [0,4194304] 0 2024-03-16T09:01:20.999 INFO:tasks.workunit.client.1.smithi019.stdout:0/728: stat d7/d1d/d63/d89/d9e 0 2024-03-16T09:01:20.999 INFO:tasks.workunit.client.1.smithi019.stdout:0/729: getdents d7/d8/d32/d4a/d61/d68/dd0 0 2024-03-16T09:01:21.000 INFO:tasks.workunit.client.1.smithi019.stdout:0/730: fsync d7/d8/dec/def/f34 0 2024-03-16T09:01:21.006 INFO:tasks.workunit.client.1.smithi019.stdout:1/614: dwrite d16/d1f/d3c/f67 [0,4194304] 0 2024-03-16T09:01:21.006 INFO:tasks.workunit.client.1.smithi019.stdout:1/615: readlink d16/d1f/d4f/d68/l93 0 2024-03-16T09:01:21.008 INFO:tasks.workunit.client.0.smithi007.stdout:7/846: dread d0/fa5 [0,4194304] 0 2024-03-16T09:01:21.012 INFO:tasks.workunit.client.1.smithi019.stdout:0/731: dread d7/d1d/d63/d6e/f93 [0,4194304] 0 2024-03-16T09:01:21.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/690: dread d9/f8b [0,4194304] 0 2024-03-16T09:01:21.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/691: write d9/dd/d1b/d2d/f44 [3956681,43657] 0 2024-03-16T09:01:21.014 INFO:tasks.workunit.client.1.smithi019.stdout:1/616: write d16/f37 [1248046,95129] 0 2024-03-16T09:01:21.016 INFO:tasks.workunit.client.1.smithi019.stdout:0/732: dread d7/d8/d32/d66/d6d/d8c/db8/d82/f71 [0,4194304] 0 2024-03-16T09:01:21.016 INFO:tasks.workunit.client.0.smithi007.stdout:0/961: dwrite d3/d9/da/d48/f44 [0,4194304] 0 2024-03-16T09:01:21.017 INFO:tasks.workunit.client.1.smithi019.stdout:0/733: stat d7/l51 0 2024-03-16T09:01:21.017 INFO:tasks.workunit.client.1.smithi019.stdout:0/734: fsync d7/d8/f23 0 2024-03-16T09:01:21.017 INFO:tasks.workunit.client.1.smithi019.stdout:0/735: chown d7/d1d/f4b 134806201 1 2024-03-16T09:01:21.017 INFO:tasks.workunit.client.1.smithi019.stdout:0/736: mkdir d7/d8/d32/df5 0 2024-03-16T09:01:21.017 INFO:tasks.workunit.client.1.smithi019.stdout:0/737: chown d7/d1d/d63/d6e/f93 37885 1 2024-03-16T09:01:21.018 INFO:tasks.workunit.client.1.smithi019.stdout:0/738: unlink d7/d8/d32/d66/d6d/d8c/db8/d82/c43 0 2024-03-16T09:01:21.019 INFO:tasks.workunit.client.1.smithi019.stdout:0/739: write d7/d1d/d63/fee [101945,73875] 0 2024-03-16T09:01:21.019 INFO:tasks.workunit.client.1.smithi019.stdout:0/740: write d7/d8/dec/def/d6c/d87/fe8 [1044323,12995] 0 2024-03-16T09:01:21.019 INFO:tasks.workunit.client.1.smithi019.stdout:0/741: write d7/d8/f23 [1097427,16701] 0 2024-03-16T09:01:21.019 INFO:tasks.workunit.client.1.smithi019.stdout:0/742: write d7/d1d/d63/d89/d9e/fc7 [4739901,114855] 0 2024-03-16T09:01:21.019 INFO:tasks.workunit.client.1.smithi019.stdout:0/743: chown d7/d8/dec/def/d92/c9d 0 1 2024-03-16T09:01:21.021 INFO:tasks.workunit.client.1.smithi019.stdout:0/744: rename d7/d8/dec/def/d6c/cda to d7/d8/d32/d4a/cf6 0 2024-03-16T09:01:21.023 INFO:tasks.workunit.client.1.smithi019.stdout:9/531: dwrite d1/d98/dae/f58 [0,4194304] 0 2024-03-16T09:01:21.023 INFO:tasks.workunit.client.1.smithi019.stdout:6/796: dwrite de/d26/f2a [0,4194304] 0 2024-03-16T09:01:21.024 INFO:tasks.workunit.client.1.smithi019.stdout:4/670: dwrite d4/d1b/d49/d58/d80/d91/fd9 [0,4194304] 0 2024-03-16T09:01:21.024 INFO:tasks.workunit.client.0.smithi007.stdout:3/837: rmdir d15/d18/d1f/dce/dd6/dcc 0 2024-03-16T09:01:21.024 INFO:tasks.workunit.client.0.smithi007.stdout:3/838: fdatasync d15/d18/d6a/f100 0 2024-03-16T09:01:21.024 INFO:tasks.workunit.client.0.smithi007.stdout:3/839: stat d15/c30 0 2024-03-16T09:01:21.025 INFO:tasks.workunit.client.0.smithi007.stdout:3/840: chown d15/c17 486 1 2024-03-16T09:01:21.027 INFO:tasks.workunit.client.1.smithi019.stdout:7/637: dwrite d9/d25/d28/f88 [0,4194304] 0 2024-03-16T09:01:21.027 INFO:tasks.workunit.client.1.smithi019.stdout:2/755: dwrite d5/d6/d104/df7/d6f/d87/ff6 [0,4194304] 0 2024-03-16T09:01:21.028 INFO:tasks.workunit.client.0.smithi007.stdout:7/847: mkdir d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d9c/d12b 0 2024-03-16T09:01:21.030 INFO:tasks.workunit.client.1.smithi019.stdout:1/617: dread d16/f2f [0,4194304] 0 2024-03-16T09:01:21.030 INFO:tasks.workunit.client.1.smithi019.stdout:1/618: write d16/f20 [1192163,40291] 0 2024-03-16T09:01:21.031 INFO:tasks.workunit.client.1.smithi019.stdout:9/532: symlink d1/d7/de/da3/lb5 0 2024-03-16T09:01:21.032 INFO:tasks.workunit.client.0.smithi007.stdout:1/692: mknod d9/dd/d1b/d28/d29/d7b/dba/cdb 0 2024-03-16T09:01:21.034 INFO:tasks.workunit.client.0.smithi007.stdout:0/962: mknod d3/d9/da/d48/d25/d43/c13a 0 2024-03-16T09:01:21.034 INFO:tasks.workunit.client.0.smithi007.stdout:0/963: chown d3/d6c/dbd/dd7/cf4 87698703 1 2024-03-16T09:01:21.034 INFO:tasks.workunit.client.0.smithi007.stdout:0/964: truncate d3/d9/da/d92/d9f/fdd 757551 0 2024-03-16T09:01:21.037 INFO:tasks.workunit.client.0.smithi007.stdout:6/756: truncate d2/d13/d3f/d50/f53 3299348 0 2024-03-16T09:01:21.037 INFO:tasks.workunit.client.0.smithi007.stdout:6/757: chown d2/d13/d4b/d94/lc3 137575 1 2024-03-16T09:01:21.037 INFO:tasks.workunit.client.0.smithi007.stdout:6/758: readlink d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/lb7 0 2024-03-16T09:01:21.038 INFO:tasks.workunit.client.1.smithi019.stdout:2/756: write d5/d6/d2b/d2f/d35/d65/fb4 [1269886,54765] 0 2024-03-16T09:01:21.038 INFO:tasks.workunit.client.1.smithi019.stdout:2/757: readlink d5/d6/d2b/l70 0 2024-03-16T09:01:21.039 INFO:tasks.workunit.client.0.smithi007.stdout:7/848: rmdir d0/d19/d1f/d5d/d9d/df8 39 2024-03-16T09:01:21.041 INFO:tasks.workunit.client.1.smithi019.stdout:7/638: dread d9/db/d12/f26 [0,4194304] 0 2024-03-16T09:01:21.041 INFO:tasks.workunit.client.1.smithi019.stdout:7/639: fdatasync d9/f23 0 2024-03-16T09:01:21.043 INFO:tasks.workunit.client.0.smithi007.stdout:6/759: getdents d2/d13/d3f/d50/d78/dbe/dcb/d3b 0 2024-03-16T09:01:21.045 INFO:tasks.workunit.client.1.smithi019.stdout:1/619: mkdir d16/d31/d76/d92/d9f/daf 0 2024-03-16T09:01:21.048 INFO:tasks.workunit.client.1.smithi019.stdout:9/533: creat d1/d7/de/d24/d44/d76/d9c/fb6 x:0 0 0 2024-03-16T09:01:21.048 INFO:tasks.workunit.client.1.smithi019.stdout:4/671: dwrite d4/d6/f5f [0,4194304] 0 2024-03-16T09:01:21.049 INFO:tasks.workunit.client.1.smithi019.stdout:4/672: stat d4/d1b/d49 0 2024-03-16T09:01:21.049 INFO:tasks.workunit.client.1.smithi019.stdout:4/673: stat d4/d64/d99/f3f 0 2024-03-16T09:01:21.051 INFO:tasks.workunit.client.0.smithi007.stdout:1/693: write d9/f22 [665269,82925] 0 2024-03-16T09:01:21.052 INFO:tasks.workunit.client.1.smithi019.stdout:7/640: mknod d9/da/d13/d9a/ccb 0 2024-03-16T09:01:21.052 INFO:tasks.workunit.client.1.smithi019.stdout:7/641: write d9/d25/d28/d48/f7b [5237957,36743] 0 2024-03-16T09:01:21.057 INFO:tasks.workunit.client.1.smithi019.stdout:6/797: dwrite de/d26/d32/d97/d66/d72/db7/fc0 [0,4194304] 0 2024-03-16T09:01:21.062 INFO:tasks.workunit.client.0.smithi007.stdout:2/939: creat dc/df/d24/d28/d2e/f138 x:0 0 0 2024-03-16T09:01:21.062 INFO:tasks.workunit.client.0.smithi007.stdout:2/940: write dc/f17 [1973172,99702] 0 2024-03-16T09:01:21.063 INFO:tasks.workunit.client.1.smithi019.stdout:4/674: unlink d4/d6/f83 0 2024-03-16T09:01:21.063 INFO:tasks.workunit.client.1.smithi019.stdout:4/675: truncate d4/d6c/d89/f93 4538438 0 2024-03-16T09:01:21.063 INFO:tasks.workunit.client.1.smithi019.stdout:4/676: stat d4/d1b/f1c 0 2024-03-16T09:01:21.063 INFO:tasks.workunit.client.1.smithi019.stdout:4/677: dread d4/d1b/f5d [0,4194304] 0 2024-03-16T09:01:21.065 INFO:tasks.workunit.client.1.smithi019.stdout:7/642: symlink d9/da/d13/d18/d30/d3e/lcc 0 2024-03-16T09:01:21.065 INFO:tasks.workunit.client.0.smithi007.stdout:2/941: creat dc/df/daa/d12c/f139 x:0 0 0 2024-03-16T09:01:21.067 INFO:tasks.workunit.client.1.smithi019.stdout:1/620: getdents d16/d31/d7b 0 2024-03-16T09:01:21.067 INFO:tasks.workunit.client.0.smithi007.stdout:1/694: write d9/d6b/d7e/dac/dbb/fbe [3531927,65033] 0 2024-03-16T09:01:21.068 INFO:tasks.workunit.client.0.smithi007.stdout:2/942: mknod dc/df/d24/d28/d2e/dc5/c13a 0 2024-03-16T09:01:21.071 INFO:tasks.workunit.client.1.smithi019.stdout:7/643: creat d9/d25/fcd x:0 0 0 2024-03-16T09:01:21.072 INFO:tasks.workunit.client.0.smithi007.stdout:2/943: truncate dc/df/d24/d28/d2e/d4a/f7a 206638 0 2024-03-16T09:01:21.073 INFO:tasks.workunit.client.1.smithi019.stdout:6/798: dread de/f49 [4194304,4194304] 0 2024-03-16T09:01:21.074 INFO:tasks.workunit.client.0.smithi007.stdout:2/944: symlink dc/df/d24/l13b 0 2024-03-16T09:01:21.074 INFO:tasks.workunit.client.0.smithi007.stdout:2/945: write dc/df/d24/d28/d4c/d5e/f8b [5879254,64691] 0 2024-03-16T09:01:21.076 INFO:tasks.workunit.client.1.smithi019.stdout:1/621: link d16/d18/f1e d16/da1/fb0 0 2024-03-16T09:01:21.077 INFO:tasks.workunit.client.0.smithi007.stdout:2/946: unlink dc/df/d24/d28/d2e/lac 0 2024-03-16T09:01:21.078 INFO:tasks.workunit.client.1.smithi019.stdout:0/745: dwrite d7/d8/d32/d66/d6d/d8c/db8/f35 [0,4194304] 0 2024-03-16T09:01:21.078 INFO:tasks.workunit.client.1.smithi019.stdout:0/746: chown d7/d1d/c5c 0 1 2024-03-16T09:01:21.078 INFO:tasks.workunit.client.1.smithi019.stdout:7/644: unlink d9/da/d13/d18/d30/d9e/cbf 0 2024-03-16T09:01:21.080 INFO:tasks.workunit.client.0.smithi007.stdout:2/947: creat dc/df/d24/d28/f13c x:0 0 0 2024-03-16T09:01:21.080 INFO:tasks.workunit.client.0.smithi007.stdout:2/948: write dc/df/d24/d28/d4c/dbc/dfe/f137 [907274,34107] 0 2024-03-16T09:01:21.081 INFO:tasks.workunit.client.1.smithi019.stdout:6/799: mkdir de/d26/d100 0 2024-03-16T09:01:21.081 INFO:tasks.workunit.client.0.smithi007.stdout:2/949: symlink dc/df/d24/d28/d97/da4/db8/df1/l13d 0 2024-03-16T09:01:21.083 INFO:tasks.workunit.client.1.smithi019.stdout:7/645: symlink d9/d25/d28/d48/lce 0 2024-03-16T09:01:21.083 INFO:tasks.workunit.client.1.smithi019.stdout:7/646: chown d9/da/d13/d18/f5a 3712073 1 2024-03-16T09:01:21.084 INFO:tasks.workunit.client.1.smithi019.stdout:6/800: truncate de/d26/d32/d97/d66/f90 1986556 0 2024-03-16T09:01:21.085 INFO:tasks.workunit.client.1.smithi019.stdout:6/801: stat de/d1c/fea 0 2024-03-16T09:01:21.087 INFO:tasks.workunit.client.1.smithi019.stdout:0/747: dread d7/d1d/d63/fee [0,4194304] 0 2024-03-16T09:01:21.087 INFO:tasks.workunit.client.1.smithi019.stdout:6/802: mkdir de/d26/d32/d97/d51/d101 0 2024-03-16T09:01:21.087 INFO:tasks.workunit.client.1.smithi019.stdout:9/534: dwrite d1/d7/d21/f7b [0,4194304] 0 2024-03-16T09:01:21.090 INFO:tasks.workunit.client.1.smithi019.stdout:0/748: mknod d7/d8/dec/cf7 0 2024-03-16T09:01:21.091 INFO:tasks.workunit.client.1.smithi019.stdout:2/758: dwrite d5/d6/fcc [0,4194304] 0 2024-03-16T09:01:21.091 INFO:tasks.workunit.client.1.smithi019.stdout:2/759: dread - d5/de/dab/fc7 zero size 2024-03-16T09:01:21.091 INFO:tasks.workunit.client.1.smithi019.stdout:6/803: symlink de/d1c/l102 0 2024-03-16T09:01:21.091 INFO:tasks.workunit.client.1.smithi019.stdout:6/804: creat de/d26/d32/d97/d66/d72/d7f/f103 x:0 0 0 2024-03-16T09:01:21.092 INFO:tasks.workunit.client.1.smithi019.stdout:1/622: dread d16/f65 [0,4194304] 0 2024-03-16T09:01:21.093 INFO:tasks.workunit.client.0.smithi007.stdout:7/849: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/f111 [0,4194304] 0 2024-03-16T09:01:21.093 INFO:tasks.workunit.client.0.smithi007.stdout:6/760: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fd7 [0,4194304] 0 2024-03-16T09:01:21.093 INFO:tasks.workunit.client.0.smithi007.stdout:6/761: stat d2/d13/d3f/d50/f53 0 2024-03-16T09:01:21.096 INFO:tasks.workunit.client.1.smithi019.stdout:2/760: unlink d5/de/da2/ccd 0 2024-03-16T09:01:21.096 INFO:tasks.workunit.client.1.smithi019.stdout:2/761: chown d5/d6/d2b/d2f/d35/d4b/cbb 1568592 1 2024-03-16T09:01:21.096 INFO:tasks.workunit.client.1.smithi019.stdout:9/535: rename d1/d7/de/da3/la6 to d1/d7/d21/da2/lb7 0 2024-03-16T09:01:21.096 INFO:tasks.workunit.client.1.smithi019.stdout:9/536: chown d1/d7/d86 261390675 1 2024-03-16T09:01:21.097 INFO:tasks.workunit.client.1.smithi019.stdout:6/805: getdents de/d1c/d9a/d9e 0 2024-03-16T09:01:21.100 INFO:tasks.workunit.client.1.smithi019.stdout:1/623: rmdir d16/d31/d7b 39 2024-03-16T09:01:21.102 INFO:tasks.workunit.client.0.smithi007.stdout:6/762: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9 39 2024-03-16T09:01:21.104 INFO:tasks.workunit.client.0.smithi007.stdout:4/737: rename d4/d14/d1c/d5b/d67 to d4/d14/d64/d68/d82/dac/dfc 0 2024-03-16T09:01:21.107 INFO:tasks.workunit.client.0.smithi007.stdout:7/850: write d0/fa5 [3996130,5086] 0 2024-03-16T09:01:21.108 INFO:tasks.workunit.client.1.smithi019.stdout:0/749: rename d7/d8/dec/def/c1b to d7/d8/cf8 0 2024-03-16T09:01:21.109 INFO:tasks.workunit.client.1.smithi019.stdout:2/762: write d5/d6/d2b/d2f/f67 [3665963,43491] 0 2024-03-16T09:01:21.110 INFO:tasks.workunit.client.1.smithi019.stdout:9/537: getdents d1/d7/de/d24/d44/d60/d74 0 2024-03-16T09:01:21.111 INFO:tasks.workunit.client.1.smithi019.stdout:1/624: mkdir d16/d1f/d4f/d68/d78/db1 0 2024-03-16T09:01:21.112 INFO:tasks.workunit.client.0.smithi007.stdout:2/950: dwrite dc/df/d24/d28/d4c/d5e/fdc [0,4194304] 0 2024-03-16T09:01:21.112 INFO:tasks.workunit.client.1.smithi019.stdout:7/647: dwrite d9/da/d13/fb8 [0,4194304] 0 2024-03-16T09:01:21.112 INFO:tasks.workunit.client.1.smithi019.stdout:7/648: truncate d9/da/d13/d18/d1a/fa8 791444 0 2024-03-16T09:01:21.112 INFO:tasks.workunit.client.1.smithi019.stdout:7/649: fdatasync d9/db/f47 0 2024-03-16T09:01:21.113 INFO:tasks.workunit.client.1.smithi019.stdout:0/750: mknod d7/d8/dec/def/d48/cf9 0 2024-03-16T09:01:21.113 INFO:tasks.workunit.client.1.smithi019.stdout:0/751: creat d7/d8/d32/d66/d6d/d8c/db5/ffa x:0 0 0 2024-03-16T09:01:21.115 INFO:tasks.workunit.client.0.smithi007.stdout:3/841: rename d15/d18/d1f/d3b/c51 to d15/d18/d1f/d29/d43/c108 0 2024-03-16T09:01:21.115 INFO:tasks.workunit.client.1.smithi019.stdout:9/538: creat d1/d7/d21/da2/fb8 x:0 0 0 2024-03-16T09:01:21.115 INFO:tasks.workunit.client.1.smithi019.stdout:9/539: fdatasync d1/d7/d33/d59/f73 0 2024-03-16T09:01:21.116 INFO:tasks.workunit.client.1.smithi019.stdout:6/806: dread f7 [0,4194304] 0 2024-03-16T09:01:21.120 INFO:tasks.workunit.client.0.smithi007.stdout:6/763: dwrite d2/d13/d24/fec [0,4194304] 0 2024-03-16T09:01:21.120 INFO:tasks.workunit.client.0.smithi007.stdout:6/764: stat d2/d13/d3f/d50/d72/f69 0 2024-03-16T09:01:21.120 INFO:tasks.workunit.client.1.smithi019.stdout:2/763: rename d5/d6/d2b/d2f/d35/d4b/f93 to d5/d6/d104/dba/dfc/d78/f107 0 2024-03-16T09:01:21.122 INFO:tasks.workunit.client.0.smithi007.stdout:5/795: sync 2024-03-16T09:01:21.124 INFO:tasks.workunit.client.1.smithi019.stdout:0/752: unlink d7/d8/d32/c84 0 2024-03-16T09:01:21.124 INFO:tasks.workunit.client.0.smithi007.stdout:2/951: mknod dc/df/d24/d28/d2e/d35/da2/c13e 0 2024-03-16T09:01:21.125 INFO:tasks.workunit.client.0.smithi007.stdout:5/796: read d1/d7/d1c/d19/d34/d4d/f101 [332140,66293] 0 2024-03-16T09:01:21.125 INFO:tasks.workunit.client.0.smithi007.stdout:9/824: mkdir d24/d4a/d96/db9/df2 0 2024-03-16T09:01:21.126 INFO:tasks.workunit.client.1.smithi019.stdout:9/540: mknod d1/d7/de/da3/cb9 0 2024-03-16T09:01:21.126 INFO:tasks.workunit.client.1.smithi019.stdout:9/541: stat d1/d98/dae/caf 0 2024-03-16T09:01:21.126 INFO:tasks.workunit.client.1.smithi019.stdout:9/542: fsync d1/d7/de/d24/d44/d60/d74/fad 0 2024-03-16T09:01:21.132 INFO:tasks.workunit.client.0.smithi007.stdout:0/965: rename d3/d9/da/d48/d25/d87/c5d to d3/d9/da/d18/d27/d34/db9/c13b 0 2024-03-16T09:01:21.132 INFO:tasks.workunit.client.0.smithi007.stdout:0/966: dread - d3/d1f/dca/f132 zero size 2024-03-16T09:01:21.135 INFO:tasks.workunit.client.1.smithi019.stdout:1/625: getdents d16/d1f/d4f/d68/d78 0 2024-03-16T09:01:21.137 INFO:tasks.workunit.client.1.smithi019.stdout:6/807: dread de/d26/d32/d97/d66/f94 [0,4194304] 0 2024-03-16T09:01:21.138 INFO:tasks.workunit.client.0.smithi007.stdout:2/952: unlink dc/dba/lbf 0 2024-03-16T09:01:21.141 INFO:tasks.workunit.client.0.smithi007.stdout:2/953: mkdir dc/df/d24/d28/d97/d99/d13f 0 2024-03-16T09:01:21.147 INFO:tasks.workunit.client.0.smithi007.stdout:5/797: dread d1/d7/d1c/d19/d34/d4d/d99/fd7 [0,4194304] 0 2024-03-16T09:01:21.149 INFO:tasks.workunit.client.1.smithi019.stdout:1/626: write d16/d1f/f34 [2998842,53509] 0 2024-03-16T09:01:21.149 INFO:tasks.workunit.client.1.smithi019.stdout:1/627: chown d16/d1f/d43/f4c 464862 1 2024-03-16T09:01:21.154 INFO:tasks.workunit.client.0.smithi007.stdout:2/954: dread dc/db7/f10c [0,4194304] 0 2024-03-16T09:01:21.156 INFO:tasks.workunit.client.0.smithi007.stdout:2/955: mknod dc/df/d24/d28/d2e/d4a/d108/c140 0 2024-03-16T09:01:21.167 INFO:tasks.workunit.client.1.smithi019.stdout:1/628: write d16/f2b [1066886,65881] 0 2024-03-16T09:01:21.187 INFO:tasks.workunit.client.0.smithi007.stdout:9/825: dread d24/d4c/d86/daa/d7f/d8f/db5/fad [0,4194304] 0 2024-03-16T09:01:21.188 INFO:tasks.workunit.client.0.smithi007.stdout:9/826: getdents d24/d4c/d91 0 2024-03-16T09:01:21.191 INFO:tasks.workunit.client.0.smithi007.stdout:9/827: link d24/d4a/d96/db9/fa3 d24/d4a/d55/dbf/de9/ff3 0 2024-03-16T09:01:21.193 INFO:tasks.workunit.client.0.smithi007.stdout:9/828: rename f16 to d24/d4a/d96/db9/db1/dbb/ff4 0 2024-03-16T09:01:21.194 INFO:tasks.workunit.client.0.smithi007.stdout:9/829: mknod d24/d4a/d96/db9/dca/ddc/cf5 0 2024-03-16T09:01:21.210 INFO:tasks.workunit.client.0.smithi007.stdout:9/830: dread d24/d4a/d96/db9/db1/fce [0,4194304] 0 2024-03-16T09:01:21.212 INFO:tasks.workunit.client.0.smithi007.stdout:9/831: mkdir d24/d4c/d91/df6 0 2024-03-16T09:01:21.212 INFO:tasks.workunit.client.0.smithi007.stdout:6/765: dwrite d2/fa [0,4194304] 0 2024-03-16T09:01:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:20 smithi019 ceph-mon[36407]: pgmap v24: 65 pgs: 65 active+clean; 2.9 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 323 MiB/s rd, 399 MiB/s wr, 695 op/s 2024-03-16T09:01:21.214 INFO:tasks.workunit.client.1.smithi019.stdout:6/808: dwrite de/d26/d32/fd1 [0,4194304] 0 2024-03-16T09:01:21.215 INFO:tasks.workunit.client.0.smithi007.stdout:2/956: dwrite dc/df/d24/d28/d2e/d6e/f107 [0,4194304] 0 2024-03-16T09:01:21.215 INFO:tasks.workunit.client.0.smithi007.stdout:2/957: read - dc/df/d121/fb6 zero size 2024-03-16T09:01:21.215 INFO:tasks.workunit.client.0.smithi007.stdout:5/798: dwrite d1/d7/d16/d1b/fd9 [0,4194304] 0 2024-03-16T09:01:21.216 INFO:tasks.workunit.client.0.smithi007.stdout:9/832: dread d24/d4c/fac [0,4194304] 0 2024-03-16T09:01:21.216 INFO:tasks.workunit.client.0.smithi007.stdout:9/833: write d24/f46 [4779639,52543] 0 2024-03-16T09:01:21.216 INFO:tasks.workunit.client.0.smithi007.stdout:9/834: truncate d24/d4a/d9e/fd3 469165 0 2024-03-16T09:01:21.216 INFO:tasks.workunit.client.0.smithi007.stdout:3/842: dwrite d15/d18/d1f/d29/d43/df7/f107 [0,4194304] 0 2024-03-16T09:01:21.216 INFO:tasks.workunit.client.0.smithi007.stdout:6/766: link d2/d13/d24/c7f d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/cfe 0 2024-03-16T09:01:21.216 INFO:tasks.workunit.client.1.smithi019.stdout:0/753: dwrite d7/d8/d32/d66/d6d/fc8 [4194304,4194304] 0 2024-03-16T09:01:21.216 INFO:tasks.workunit.client.1.smithi019.stdout:0/754: write d7/d1d/d63/f78 [1557005,28937] 0 2024-03-16T09:01:21.216 INFO:tasks.workunit.client.1.smithi019.stdout:0/755: truncate d7/d8/dec/def/d48/fe0 301934 0 2024-03-16T09:01:21.217 INFO:tasks.workunit.client.1.smithi019.stdout:6/809: getdents de/d26/d32/d97/da1 0 2024-03-16T09:01:21.220 INFO:tasks.workunit.client.1.smithi019.stdout:6/810: creat de/d26/f104 x:0 0 0 2024-03-16T09:01:21.220 INFO:tasks.workunit.client.1.smithi019.stdout:6/811: chown de/d1c/l102 4 1 2024-03-16T09:01:21.221 INFO:tasks.workunit.client.1.smithi019.stdout:2/764: rename d5/de/fd to d5/d5a/d80/f108 0 2024-03-16T09:01:21.221 INFO:tasks.workunit.client.1.smithi019.stdout:2/765: readlink d5/d6/d104/dba/dfc/l77 0 2024-03-16T09:01:21.221 INFO:tasks.workunit.client.1.smithi019.stdout:2/766: read - d5/d6/d2b/d89/fcf zero size 2024-03-16T09:01:21.222 INFO:tasks.workunit.client.1.smithi019.stdout:6/812: unlink de/d26/d32/d4e/d75/cab 0 2024-03-16T09:01:21.222 INFO:tasks.workunit.client.1.smithi019.stdout:6/813: dread - de/d26/d32/d97/fc5 zero size 2024-03-16T09:01:21.223 INFO:tasks.workunit.client.1.smithi019.stdout:0/756: dread d7/d8/d32/d66/d6d/d8c/db8/f8f [0,4194304] 0 2024-03-16T09:01:21.225 INFO:tasks.workunit.client.1.smithi019.stdout:2/767: creat d5/d6/d2b/d2f/d35/d65/f109 x:0 0 0 2024-03-16T09:01:21.225 INFO:tasks.workunit.client.1.smithi019.stdout:2/768: fsync d5/fc5 0 2024-03-16T09:01:21.226 INFO:tasks.workunit.client.1.smithi019.stdout:0/757: dread d7/d8/d32/d66/d6d/d8c/db8/d82/f71 [0,4194304] 0 2024-03-16T09:01:21.226 INFO:tasks.workunit.client.1.smithi019.stdout:0/758: write d7/d8/dec/def/d92/dc6/fdb [726108,109706] 0 2024-03-16T09:01:21.226 INFO:tasks.workunit.client.1.smithi019.stdout:0/759: fsync d7/fe4 0 2024-03-16T09:01:21.227 INFO:tasks.workunit.client.1.smithi019.stdout:1/629: rename d16/d1f/d4f/d68 to d16/d31/d76/d92/d9f/daf/db2 0 2024-03-16T09:01:21.230 INFO:tasks.workunit.client.0.smithi007.stdout:5/799: truncate d1/d7/d16/d1b/f8d 207983 0 2024-03-16T09:01:21.230 INFO:tasks.workunit.client.1.smithi019.stdout:6/814: mkdir de/d26/d32/d97/da1/d105 0 2024-03-16T09:01:21.230 INFO:tasks.workunit.client.1.smithi019.stdout:6/815: readlink de/d26/d73/ded/lee 0 2024-03-16T09:01:21.231 INFO:tasks.workunit.client.1.smithi019.stdout:2/769: getdents d5/de/da2 0 2024-03-16T09:01:21.232 INFO:tasks.workunit.client.1.smithi019.stdout:0/760: mknod d7/d1d/d63/d89/cfb 0 2024-03-16T09:01:21.232 INFO:tasks.workunit.client.1.smithi019.stdout:2/770: mkdir d5/d6/d104/df7/d10a 0 2024-03-16T09:01:21.234 INFO:tasks.workunit.client.1.smithi019.stdout:1/630: truncate d16/d1f/f34 1716013 0 2024-03-16T09:01:21.236 INFO:tasks.workunit.client.0.smithi007.stdout:9/835: dwrite d24/d4a/d96/db9/f60 [4194304,4194304] 0 2024-03-16T09:01:21.239 INFO:tasks.workunit.client.0.smithi007.stdout:3/843: unlink d15/d18/d1f/d29/d43/d72/c4b 0 2024-03-16T09:01:21.239 INFO:tasks.workunit.client.1.smithi019.stdout:1/631: mknod d16/d1f/d3c/d79/d96/dac/cb3 0 2024-03-16T09:01:21.241 INFO:tasks.workunit.client.0.smithi007.stdout:6/767: unlink d2/d13/d4b/d94/dea/df7/ff9 0 2024-03-16T09:01:21.242 INFO:tasks.workunit.client.0.smithi007.stdout:6/768: readlink d2/d29/l5d 0 2024-03-16T09:01:21.242 INFO:tasks.workunit.client.1.smithi019.stdout:1/632: mkdir d16/d1f/d3c/d79/db4 0 2024-03-16T09:01:21.243 INFO:tasks.workunit.client.0.smithi007.stdout:5/800: getdents d1/d7/d1c/d19/d34/d3e/d41/da6/de8/de9 0 2024-03-16T09:01:21.244 INFO:tasks.workunit.client.1.smithi019.stdout:1/633: rename d16/d31/f72 to d16/d84/fb5 0 2024-03-16T09:01:21.246 INFO:tasks.workunit.client.0.smithi007.stdout:6/769: rename d2/d13/d3f/d50/d78/c83 to d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/ddc/cff 0 2024-03-16T09:01:21.247 INFO:tasks.workunit.client.0.smithi007.stdout:5/801: mknod d1/d7/d1c/d19/d34/de/c120 0 2024-03-16T09:01:21.247 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:20 smithi007 ceph-mon[29470]: pgmap v24: 65 pgs: 65 active+clean; 2.9 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 323 MiB/s rd, 399 MiB/s wr, 695 op/s 2024-03-16T09:01:21.249 INFO:tasks.workunit.client.0.smithi007.stdout:6/770: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/ddc 39 2024-03-16T09:01:21.249 INFO:tasks.workunit.client.0.smithi007.stdout:6/771: stat d2/d13/d62/f7b 0 2024-03-16T09:01:21.249 INFO:tasks.workunit.client.0.smithi007.stdout:6/772: truncate d2/d13/d3f/d50/d72/fa5 702079 0 2024-03-16T09:01:21.249 INFO:tasks.workunit.client.0.smithi007.stdout:6/773: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/ff4 983266 0 2024-03-16T09:01:21.249 INFO:tasks.workunit.client.0.smithi007.stdout:6/774: getdents d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/dc1 0 2024-03-16T09:01:21.250 INFO:tasks.workunit.client.0.smithi007.stdout:6/775: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/d84/f9e [2111785,117561] 0 2024-03-16T09:01:21.250 INFO:tasks.workunit.client.0.smithi007.stdout:5/802: symlink d1/d7/d1c/d88/d11e/l121 0 2024-03-16T09:01:21.252 INFO:tasks.workunit.client.0.smithi007.stdout:6/776: unlink d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/faf 0 2024-03-16T09:01:21.254 INFO:tasks.workunit.client.0.smithi007.stdout:5/803: creat d1/d7/d1c/f122 x:0 0 0 2024-03-16T09:01:21.255 INFO:tasks.workunit.client.0.smithi007.stdout:3/844: write d15/d18/d1f/d29/d7c/fb0 [172187,81067] 0 2024-03-16T09:01:21.255 INFO:tasks.workunit.client.0.smithi007.stdout:3/845: dread - d15/fe8 zero size 2024-03-16T09:01:21.255 INFO:tasks.workunit.client.0.smithi007.stdout:3/846: chown d15/d18/d1f/d29/d43/d72/f50 50725033 1 2024-03-16T09:01:21.255 INFO:tasks.workunit.client.0.smithi007.stdout:5/804: rmdir d1/d7/d1c/d88/da2/d89/db3/d2a/d81 39 2024-03-16T09:01:21.256 INFO:tasks.workunit.client.0.smithi007.stdout:2/958: write dc/df/f5a [1308653,104316] 0 2024-03-16T09:01:21.256 INFO:tasks.workunit.client.0.smithi007.stdout:2/959: write dc/df/d24/d28/dea/f12a [981978,108824] 0 2024-03-16T09:01:21.257 INFO:tasks.workunit.client.0.smithi007.stdout:3/847: unlink d15/d18/d6a/db1/f102 0 2024-03-16T09:01:21.257 INFO:tasks.workunit.client.0.smithi007.stdout:3/848: readlink d15/d18/d1f/d29/d3e/d85/daf/lbd 0 2024-03-16T09:01:21.259 INFO:tasks.workunit.client.1.smithi019.stdout:0/761: dwrite d7/f6a [0,4194304] 0 2024-03-16T09:01:21.259 INFO:tasks.workunit.client.0.smithi007.stdout:3/849: creat d15/d18/d1f/d29/d7c/f109 x:0 0 0 2024-03-16T09:01:21.266 INFO:tasks.workunit.client.1.smithi019.stdout:3/673: sync 2024-03-16T09:01:21.267 INFO:tasks.workunit.client.1.smithi019.stdout:8/790: sync 2024-03-16T09:01:21.267 INFO:tasks.workunit.client.1.smithi019.stdout:8/791: readlink d6/d2a/d68/l6f 0 2024-03-16T09:01:21.268 INFO:tasks.workunit.client.1.smithi019.stdout:8/792: creat d6/d2a/d55/f112 x:0 0 0 2024-03-16T09:01:21.270 INFO:tasks.workunit.client.1.smithi019.stdout:8/793: symlink d6/d2a/d45/d57/d72/d79/db7/l113 0 2024-03-16T09:01:21.271 INFO:tasks.workunit.client.1.smithi019.stdout:8/794: mknod d6/d2a/d45/d57/d72/d79/dc1/c114 0 2024-03-16T09:01:21.272 INFO:tasks.workunit.client.1.smithi019.stdout:8/795: mknod d6/d2a/d3c/dab/d30/d4a/c115 0 2024-03-16T09:01:21.272 INFO:tasks.workunit.client.1.smithi019.stdout:8/796: fsync d6/de/fb2 0 2024-03-16T09:01:21.299 INFO:tasks.workunit.client.0.smithi007.stdout:5/805: dwrite d1/d7/d1c/d19/d34/d4d/f101 [0,4194304] 0 2024-03-16T09:01:21.299 INFO:tasks.workunit.client.0.smithi007.stdout:2/960: dwrite dc/f68 [4194304,4194304] 0 2024-03-16T09:01:21.301 INFO:tasks.workunit.client.0.smithi007.stdout:5/806: creat d1/d7/d1c/d19/df7/d105/f123 x:0 0 0 2024-03-16T09:01:21.302 INFO:tasks.workunit.client.0.smithi007.stdout:5/807: creat d1/d7/d1c/d19/df7/d110/f124 x:0 0 0 2024-03-16T09:01:21.303 INFO:tasks.workunit.client.0.smithi007.stdout:5/808: rename d1/d7/d1c/d88/da2/dae/l116 to d1/d7/d1c/d88/d11e/l125 0 2024-03-16T09:01:21.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/809: mknod d1/d7/d1c/d19/d34/d1d/dee/c126 0 2024-03-16T09:01:21.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/810: creat d1/d7/d1c/d19/df7/d105/f127 x:0 0 0 2024-03-16T09:01:21.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/811: fdatasync d1/d7/d1c/d88/da2/d89/db3/fea 0 2024-03-16T09:01:21.304 INFO:tasks.workunit.client.0.smithi007.stdout:5/812: write d1/d7/d1c/d19/d34/d1d/f9e [4961355,110943] 0 2024-03-16T09:01:21.308 INFO:tasks.workunit.client.1.smithi019.stdout:8/797: dwrite d6/de/fea [0,4194304] 0 2024-03-16T09:01:21.311 INFO:tasks.workunit.client.1.smithi019.stdout:8/798: mkdir d6/d2a/d45/d116 0 2024-03-16T09:01:21.312 INFO:tasks.workunit.client.1.smithi019.stdout:8/799: creat d6/d2a/d3c/dab/d30/d4a/d67/d95/de0/f117 x:0 0 0 2024-03-16T09:01:21.312 INFO:tasks.workunit.client.1.smithi019.stdout:8/800: write d6/d2a/d45/d57/f100 [5135986,93069] 0 2024-03-16T09:01:21.313 INFO:tasks.workunit.client.1.smithi019.stdout:8/801: mkdir d6/d2a/d3c/d118 0 2024-03-16T09:01:21.313 INFO:tasks.workunit.client.1.smithi019.stdout:8/802: chown d6/d2a/d45/c23 1587 1 2024-03-16T09:01:21.315 INFO:tasks.workunit.client.0.smithi007.stdout:2/961: dread dc/df/d24/d28/d4c/d5e/f5b [0,4194304] 0 2024-03-16T09:01:21.317 INFO:tasks.workunit.client.0.smithi007.stdout:2/962: unlink dc/df/d24/d28/d4c/dbc/dfe/f137 0 2024-03-16T09:01:21.318 INFO:tasks.workunit.client.0.smithi007.stdout:5/813: write d1/d7/d1c/d88/da2/d89/db3/d2a/d81/f11a [2468485,113693] 0 2024-03-16T09:01:21.318 INFO:tasks.workunit.client.0.smithi007.stdout:2/963: getdents dc/df/d24 0 2024-03-16T09:01:21.320 INFO:tasks.workunit.client.0.smithi007.stdout:5/814: symlink d1/d7/d1c/d88/da2/d89/db3/d2a/d79/l128 0 2024-03-16T09:01:21.320 INFO:tasks.workunit.client.0.smithi007.stdout:2/964: creat dc/df/d24/d28/d97/d99/f141 x:0 0 0 2024-03-16T09:01:21.323 INFO:tasks.workunit.client.0.smithi007.stdout:2/965: rename dc/df/d24/f80 to dc/d27/f142 0 2024-03-16T09:01:21.331 INFO:tasks.workunit.client.0.smithi007.stdout:2/966: dread dc/df/d24/ff0 [0,4194304] 0 2024-03-16T09:01:21.331 INFO:tasks.workunit.client.0.smithi007.stdout:2/967: fsync dc/dba/ff6 0 2024-03-16T09:01:21.332 INFO:tasks.workunit.client.0.smithi007.stdout:2/968: mkdir dc/d27/da3/d143 0 2024-03-16T09:01:21.332 INFO:tasks.workunit.client.0.smithi007.stdout:2/969: truncate dc/df/d24/d28/f82 26981 0 2024-03-16T09:01:21.332 INFO:tasks.workunit.client.0.smithi007.stdout:2/970: readlink dc/df/d24/l8d 0 2024-03-16T09:01:21.333 INFO:tasks.workunit.client.1.smithi019.stdout:8/803: dwrite d6/d2a/d45/d19/f43 [0,4194304] 0 2024-03-16T09:01:21.334 INFO:tasks.workunit.client.1.smithi019.stdout:8/804: creat d6/d2a/d3f/f119 x:0 0 0 2024-03-16T09:01:21.334 INFO:tasks.workunit.client.1.smithi019.stdout:8/805: dread - d6/d2a/d45/d57/d96/fad zero size 2024-03-16T09:01:21.335 INFO:tasks.workunit.client.1.smithi019.stdout:8/806: link d6/d2a/d45/d19/d74/cc8 d6/d2a/d68/c11a 0 2024-03-16T09:01:21.335 INFO:tasks.workunit.client.1.smithi019.stdout:8/807: write d6/d2a/d68/fd0 [383539,50662] 0 2024-03-16T09:01:21.370 INFO:tasks.workunit.client.1.smithi019.stdout:1/634: creat d16/d1f/fb6 x:0 0 0 2024-03-16T09:01:21.373 INFO:tasks.workunit.client.1.smithi019.stdout:0/762: mkdir d7/d8/dec/dfc 0 2024-03-16T09:01:21.373 INFO:tasks.workunit.client.1.smithi019.stdout:6/816: dwrite de/d1c/f8d [0,4194304] 0 2024-03-16T09:01:21.373 INFO:tasks.workunit.client.1.smithi019.stdout:0/763: readlink d7/d8/dec/def/da1/da7/ld4 0 2024-03-16T09:01:21.373 INFO:tasks.workunit.client.1.smithi019.stdout:0/764: stat d7/d8/d32/d66/d6d/d8c/cb2 0 2024-03-16T09:01:21.374 INFO:tasks.workunit.client.1.smithi019.stdout:3/674: mknod d3/d1b/d40/d4a/cd3 0 2024-03-16T09:01:21.375 INFO:tasks.workunit.client.1.smithi019.stdout:3/675: write d3/d1b/d88/fc2 [164151,126147] 0 2024-03-16T09:01:21.375 INFO:tasks.workunit.client.1.smithi019.stdout:3/676: truncate d3/d9b/f9f 859040 0 2024-03-16T09:01:21.376 INFO:tasks.workunit.client.1.smithi019.stdout:2/771: dwrite d5/d6/d104/dba/dfc/d74/f95 [0,4194304] 0 2024-03-16T09:01:21.381 INFO:tasks.workunit.client.0.smithi007.stdout:3/850: dwrite d15/f2c [4194304,4194304] 0 2024-03-16T09:01:21.382 INFO:tasks.workunit.client.1.smithi019.stdout:8/808: dwrite d6/d2a/f88 [0,4194304] 0 2024-03-16T09:01:21.386 INFO:tasks.workunit.client.0.smithi007.stdout:3/851: mknod d15/d18/d1f/d3b/d87/c10a 0 2024-03-16T09:01:21.393 INFO:tasks.workunit.client.0.smithi007.stdout:3/852: dread d15/d18/d1f/f2e [0,4194304] 0 2024-03-16T09:01:21.393 INFO:tasks.workunit.client.0.smithi007.stdout:3/853: write d15/d18/d1f/d29/f31 [2446272,107257] 0 2024-03-16T09:01:21.393 INFO:tasks.workunit.client.0.smithi007.stdout:3/854: chown d15/d18/d1f/d29/d43/d72 7 1 2024-03-16T09:01:21.395 INFO:tasks.workunit.client.0.smithi007.stdout:6/777: dwrite d2/d13/d3f/d50/d72/fb4 [0,4194304] 0 2024-03-16T09:01:21.395 INFO:tasks.workunit.client.0.smithi007.stdout:6/778: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fdb [1002819,71124] 0 2024-03-16T09:01:21.396 INFO:tasks.workunit.client.0.smithi007.stdout:3/855: link d15/d18/d1f/d3b/f44 d15/d18/d1f/d77/f10b 0 2024-03-16T09:01:21.410 INFO:tasks.workunit.client.0.smithi007.stdout:3/856: rmdir d15/d18/d1f/d3b 39 2024-03-16T09:01:21.410 INFO:tasks.workunit.client.0.smithi007.stdout:3/857: chown d15/d18/d6a/ca9 189116 1 2024-03-16T09:01:21.411 INFO:tasks.workunit.client.0.smithi007.stdout:3/858: dread - d15/d18/d6a/f100 zero size 2024-03-16T09:01:21.412 INFO:tasks.workunit.client.0.smithi007.stdout:3/859: link d15/d18/d1f/d29/d43/c62 d15/d18/d1f/d29/d43/c10c 0 2024-03-16T09:01:21.413 INFO:tasks.workunit.client.0.smithi007.stdout:3/860: rmdir d15/d18/d1f/d29/d3e/dc0/da5 39 2024-03-16T09:01:21.414 INFO:tasks.workunit.client.0.smithi007.stdout:3/861: creat d15/d18/d1f/d3b/dae/f10d x:0 0 0 2024-03-16T09:01:21.415 INFO:tasks.workunit.client.0.smithi007.stdout:3/862: mkdir d15/d18/d1f/d3b/dae/d10e 0 2024-03-16T09:01:21.416 INFO:tasks.workunit.client.0.smithi007.stdout:3/863: fdatasync d15/d18/d1f/d29/f25 0 2024-03-16T09:01:21.421 INFO:tasks.workunit.client.0.smithi007.stdout:6/779: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/fa7 [0,4194304] 0 2024-03-16T09:01:21.421 INFO:tasks.workunit.client.0.smithi007.stdout:6/780: creat d2/d82/f100 x:0 0 0 2024-03-16T09:01:21.438 INFO:tasks.workunit.client.1.smithi019.stdout:6/817: unlink de/f3c 0 2024-03-16T09:01:21.439 INFO:tasks.workunit.client.1.smithi019.stdout:3/677: creat d3/d8b/dae/fd4 x:0 0 0 2024-03-16T09:01:21.439 INFO:tasks.workunit.client.1.smithi019.stdout:8/809: rmdir d6/d2a/d3c/dab 39 2024-03-16T09:01:21.439 INFO:tasks.workunit.client.1.smithi019.stdout:8/810: write d6/de/fb2 [306608,49419] 0 2024-03-16T09:01:21.440 INFO:tasks.workunit.client.1.smithi019.stdout:8/811: creat d6/d2a/d3c/dab/f11b x:0 0 0 2024-03-16T09:01:21.440 INFO:tasks.workunit.client.1.smithi019.stdout:8/812: fsync f2 0 2024-03-16T09:01:21.441 INFO:tasks.workunit.client.1.smithi019.stdout:6/818: getdents de/d26/d32/d97/d66/d72/dd0 0 2024-03-16T09:01:21.442 INFO:tasks.workunit.client.1.smithi019.stdout:3/678: rename d3/d8b/dae/l4b to d3/d1b/dc7/ld5 0 2024-03-16T09:01:21.443 INFO:tasks.workunit.client.1.smithi019.stdout:3/679: mkdir d3/d8b/dae/dd6 0 2024-03-16T09:01:21.444 INFO:tasks.workunit.client.1.smithi019.stdout:6/819: getdents de/d26/d32/d97/d66/d72/d7f 0 2024-03-16T09:01:21.445 INFO:tasks.workunit.client.0.smithi007.stdout:6/781: dwrite d2/d82/fa0 [0,4194304] 0 2024-03-16T09:01:21.445 INFO:tasks.workunit.client.1.smithi019.stdout:3/680: readlink d3/d1b/l1d 0 2024-03-16T09:01:21.445 INFO:tasks.workunit.client.1.smithi019.stdout:3/681: fsync d3/d8b/dae/d5a/f61 0 2024-03-16T09:01:21.447 INFO:tasks.workunit.client.1.smithi019.stdout:6/820: mknod de/d1c/c106 0 2024-03-16T09:01:21.448 INFO:tasks.workunit.client.0.smithi007.stdout:6/782: rename d2/d29/c88 to d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/c101 0 2024-03-16T09:01:21.450 INFO:tasks.workunit.client.1.smithi019.stdout:2/772: link d5/d6/d2b/c42 d5/d6/d2b/d2f/d35/c10b 0 2024-03-16T09:01:21.450 INFO:tasks.workunit.client.1.smithi019.stdout:3/682: creat d3/d8b/dae/d5a/d75/d7a/dcd/fd7 x:0 0 0 2024-03-16T09:01:21.451 INFO:tasks.workunit.client.1.smithi019.stdout:1/635: write d16/f9c [353313,14143] 0 2024-03-16T09:01:21.452 INFO:tasks.workunit.client.1.smithi019.stdout:2/773: creat d5/de/dab/f10c x:0 0 0 2024-03-16T09:01:21.452 INFO:tasks.workunit.client.1.smithi019.stdout:3/683: symlink d3/d1b/d88/ld8 0 2024-03-16T09:01:21.453 INFO:tasks.workunit.client.1.smithi019.stdout:1/636: mknod d16/d84/cb7 0 2024-03-16T09:01:21.455 INFO:tasks.workunit.client.1.smithi019.stdout:4/678: sync 2024-03-16T09:01:21.455 INFO:tasks.workunit.client.1.smithi019.stdout:5/743: sync 2024-03-16T09:01:21.455 INFO:tasks.workunit.client.1.smithi019.stdout:7/650: sync 2024-03-16T09:01:21.455 INFO:tasks.workunit.client.1.smithi019.stdout:7/651: fsync d9/d25/d28/f9c 0 2024-03-16T09:01:21.455 INFO:tasks.workunit.client.1.smithi019.stdout:9/543: sync 2024-03-16T09:01:21.455 INFO:tasks.workunit.client.1.smithi019.stdout:2/774: mknod d5/d6/d104/dba/dfc/d78/c10d 0 2024-03-16T09:01:21.457 INFO:tasks.workunit.client.0.smithi007.stdout:1/695: sync 2024-03-16T09:01:21.458 INFO:tasks.workunit.client.1.smithi019.stdout:2/775: creat d5/d6/d104/df7/d6f/f10e x:0 0 0 2024-03-16T09:01:21.462 INFO:tasks.workunit.client.0.smithi007.stdout:1/696: unlink d9/d6b/d7e/db0/cb1 0 2024-03-16T09:01:21.462 INFO:tasks.workunit.client.0.smithi007.stdout:1/697: chown d9/d6b/d7e/lb8 2 1 2024-03-16T09:01:21.462 INFO:tasks.workunit.client.0.smithi007.stdout:1/698: chown d9/dd/d1b/d28/d29/d7b/f88 1 1 2024-03-16T09:01:21.463 INFO:tasks.workunit.client.1.smithi019.stdout:1/637: dread d16/d1f/d43/f6d [0,4194304] 0 2024-03-16T09:01:21.464 INFO:tasks.workunit.client.1.smithi019.stdout:4/679: write d4/d6/d15/f67 [635744,1033] 0 2024-03-16T09:01:21.465 INFO:tasks.workunit.client.1.smithi019.stdout:9/544: dread d1/d98/dae/f35 [0,4194304] 0 2024-03-16T09:01:21.467 INFO:tasks.workunit.client.0.smithi007.stdout:1/699: link d9/dd/d1b/d2d/d30/c3c d9/dd/d5e/cdc 0 2024-03-16T09:01:21.468 INFO:tasks.workunit.client.0.smithi007.stdout:1/700: chown d9/dd/d1b/d2d/c2f 26 1 2024-03-16T09:01:21.468 INFO:tasks.workunit.client.0.smithi007.stdout:1/701: chown d9/f12 103492129 1 2024-03-16T09:01:21.468 INFO:tasks.workunit.client.0.smithi007.stdout:1/702: creat d9/dd/d5e/d89/fdd x:0 0 0 2024-03-16T09:01:21.468 INFO:tasks.workunit.client.0.smithi007.stdout:1/703: write d9/fca [683783,42314] 0 2024-03-16T09:01:21.469 INFO:tasks.workunit.client.1.smithi019.stdout:9/545: mkdir d1/d7/d86/dba 0 2024-03-16T09:01:21.470 INFO:tasks.workunit.client.0.smithi007.stdout:1/704: mkdir d9/d6b/d7e/db0/dde 0 2024-03-16T09:01:21.471 INFO:tasks.workunit.client.1.smithi019.stdout:9/546: mkdir d1/d7/dbb 0 2024-03-16T09:01:21.471 INFO:tasks.workunit.client.0.smithi007.stdout:1/705: symlink d9/dd/d1b/d28/d29/ldf 0 2024-03-16T09:01:21.473 INFO:tasks.workunit.client.0.smithi007.stdout:1/706: mkdir d9/dd/de0 0 2024-03-16T09:01:21.473 INFO:tasks.workunit.client.1.smithi019.stdout:7/652: dread d9/d25/f54 [0,4194304] 0 2024-03-16T09:01:21.474 INFO:tasks.workunit.client.1.smithi019.stdout:7/653: dread d9/da/d13/d18/d30/f97 [0,4194304] 0 2024-03-16T09:01:21.474 INFO:tasks.workunit.client.1.smithi019.stdout:7/654: dread - d9/da/d13/d18/d30/d75/f9d zero size 2024-03-16T09:01:21.474 INFO:tasks.workunit.client.1.smithi019.stdout:7/655: readlink d9/da/d13/l8e 0 2024-03-16T09:01:21.486 INFO:tasks.workunit.client.0.smithi007.stdout:1/707: dread d9/dd/d1b/d2d/f44 [0,4194304] 0 2024-03-16T09:01:21.487 INFO:tasks.workunit.client.1.smithi019.stdout:9/547: write d1/d7/de/d24/f6d [694839,60522] 0 2024-03-16T09:01:21.487 INFO:tasks.workunit.client.1.smithi019.stdout:9/548: chown d1/f6a 144509575 1 2024-03-16T09:01:21.487 INFO:tasks.workunit.client.0.smithi007.stdout:1/708: rename d9/dd/d1b/d28/f3e to d9/dd/d1b/d28/d29/d7b/dba/fe1 0 2024-03-16T09:01:21.487 INFO:tasks.workunit.client.0.smithi007.stdout:1/709: readlink d9/dd/l68 0 2024-03-16T09:01:21.489 INFO:tasks.workunit.client.1.smithi019.stdout:9/549: link d1/d7/de/d24/d44/d60/d74/f91 d1/d7/de/d24/d44/d60/d74/fbc 0 2024-03-16T09:01:21.490 INFO:tasks.workunit.client.1.smithi019.stdout:9/550: mknod d1/d7/de/da3/cbd 0 2024-03-16T09:01:21.491 INFO:tasks.workunit.client.1.smithi019.stdout:9/551: mknod d1/cbe 0 2024-03-16T09:01:21.492 INFO:tasks.workunit.client.1.smithi019.stdout:9/552: creat d1/d7/de/d24/d44/d60/fbf x:0 0 0 2024-03-16T09:01:21.492 INFO:tasks.workunit.client.1.smithi019.stdout:9/553: fsync d1/d7/de/f10 0 2024-03-16T09:01:21.512 INFO:tasks.workunit.client.1.smithi019.stdout:8/813: dwrite d6/d2a/d45/d57/d72/d79/db7/fe7 [0,4194304] 0 2024-03-16T09:01:21.513 INFO:tasks.workunit.client.1.smithi019.stdout:8/814: creat d6/d2a/d3c/dab/d30/d4a/f11c x:0 0 0 2024-03-16T09:01:21.515 INFO:tasks.workunit.client.1.smithi019.stdout:8/815: mkdir d6/d2a/d45/d57/d72/d79/da5/d11d 0 2024-03-16T09:01:21.517 INFO:tasks.workunit.client.1.smithi019.stdout:2/776: rename d5/d6/d104/dba/dfc/l77 to d5/d5a/l10f 0 2024-03-16T09:01:21.519 INFO:tasks.workunit.client.1.smithi019.stdout:2/777: link d5/d6/c6c d5/d6/d104/df7/c110 0 2024-03-16T09:01:21.519 INFO:tasks.workunit.client.1.smithi019.stdout:1/638: rename d16/d1f/d43/d55/la6 to d16/d18/lb8 0 2024-03-16T09:01:21.521 INFO:tasks.workunit.client.1.smithi019.stdout:9/554: rename d1/d7/d86/dba to d1/d7/dc0 0 2024-03-16T09:01:21.524 INFO:tasks.workunit.client.1.smithi019.stdout:8/816: dread d6/d2a/d3c/f44 [0,4194304] 0 2024-03-16T09:01:21.525 INFO:tasks.workunit.client.1.smithi019.stdout:8/817: getdents d6/d2a/d3c/dab/d30/d4a/d67/dee/d10d 0 2024-03-16T09:01:21.525 INFO:tasks.workunit.client.1.smithi019.stdout:8/818: readlink d6/d2a/d45/d19/d74/l91 0 2024-03-16T09:01:21.525 INFO:tasks.workunit.client.1.smithi019.stdout:2/778: rename d5/d6/d2b/dfb to d5/d111 0 2024-03-16T09:01:21.526 INFO:tasks.workunit.client.1.smithi019.stdout:8/819: getdents d6/d2a/d45/d57 0 2024-03-16T09:01:21.527 INFO:tasks.workunit.client.1.smithi019.stdout:8/820: chown d6/d2a/d55 3602 1 2024-03-16T09:01:21.527 INFO:tasks.workunit.client.1.smithi019.stdout:9/555: rename d1/d7/de/da3/fb0 to d1/d92/fc1 0 2024-03-16T09:01:21.527 INFO:tasks.workunit.client.0.smithi007.stdout:6/783: dwrite d2/d13/d62/f68 [4194304,4194304] 0 2024-03-16T09:01:21.528 INFO:tasks.workunit.client.0.smithi007.stdout:6/784: write d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c/ff0 [375227,121541] 0 2024-03-16T09:01:21.528 INFO:tasks.workunit.client.1.smithi019.stdout:5/744: dwrite d6/dd/f5c [0,4194304] 0 2024-03-16T09:01:21.528 INFO:tasks.workunit.client.1.smithi019.stdout:3/684: dwrite d3/d8b/dae/fd4 [0,4194304] 0 2024-03-16T09:01:21.530 INFO:tasks.workunit.client.0.smithi007.stdout:6/785: creat d2/d13/d3f/d50/d78/f102 x:0 0 0 2024-03-16T09:01:21.530 INFO:tasks.workunit.client.0.smithi007.stdout:6/786: readlink d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/l23 0 2024-03-16T09:01:21.533 INFO:tasks.workunit.client.0.smithi007.stdout:6/787: mkdir d2/d10/d3d/d6d/dbf/d103 0 2024-03-16T09:01:21.534 INFO:tasks.workunit.client.1.smithi019.stdout:2/779: mkdir d5/d6/d2b/d112 0 2024-03-16T09:01:21.534 INFO:tasks.workunit.client.1.smithi019.stdout:2/780: write d5/de/f4a [4958711,44986] 0 2024-03-16T09:01:21.535 INFO:tasks.workunit.client.0.smithi007.stdout:6/788: symlink d2/d13/d4b/d94/l104 0 2024-03-16T09:01:21.537 INFO:tasks.workunit.client.0.smithi007.stdout:6/789: mkdir d2/d13/d105 0 2024-03-16T09:01:21.538 INFO:tasks.workunit.client.1.smithi019.stdout:8/821: write d6/d2a/d3c/f46 [1383239,2592] 0 2024-03-16T09:01:21.538 INFO:tasks.workunit.client.1.smithi019.stdout:2/781: write d5/f61 [423684,84609] 0 2024-03-16T09:01:21.538 INFO:tasks.workunit.client.1.smithi019.stdout:7/656: dwrite d9/d25/d28/d48/f7b [4194304,4194304] 0 2024-03-16T09:01:21.538 INFO:tasks.workunit.client.1.smithi019.stdout:7/657: readlink d9/da/l52 0 2024-03-16T09:01:21.538 INFO:tasks.workunit.client.1.smithi019.stdout:7/658: read d9/da/d13/d18/d1a/fa8 [85617,21211] 0 2024-03-16T09:01:21.539 INFO:tasks.workunit.client.1.smithi019.stdout:8/822: mkdir d6/d2a/d68/d11e 0 2024-03-16T09:01:21.539 INFO:tasks.workunit.client.1.smithi019.stdout:7/659: mkdir d9/db/d12/dcf 0 2024-03-16T09:01:21.540 INFO:tasks.workunit.client.1.smithi019.stdout:2/782: unlink d5/d6/d2b/d89/c7b 0 2024-03-16T09:01:21.542 INFO:tasks.workunit.client.1.smithi019.stdout:7/660: link f8 d9/da/d13/d18/d30/d3e/fd0 0 2024-03-16T09:01:21.542 INFO:tasks.workunit.client.1.smithi019.stdout:7/661: fdatasync d9/da/d13/d18/d30/d75/da4/fc2 0 2024-03-16T09:01:21.545 INFO:tasks.workunit.client.0.smithi007.stdout:1/710: dwrite d9/d6b/fa0 [0,4194304] 0 2024-03-16T09:01:21.545 INFO:tasks.workunit.client.0.smithi007.stdout:1/711: write d9/dd/d1b/d58/fa7 [625345,69248] 0 2024-03-16T09:01:21.546 INFO:tasks.workunit.client.1.smithi019.stdout:7/662: mknod d9/d25/d93/cd1 0 2024-03-16T09:01:21.547 INFO:tasks.workunit.client.1.smithi019.stdout:1/639: dwrite d16/d1f/f27 [0,4194304] 0 2024-03-16T09:01:21.547 INFO:tasks.workunit.client.1.smithi019.stdout:3/685: write f2 [2376772,22888] 0 2024-03-16T09:01:21.547 INFO:tasks.workunit.client.1.smithi019.stdout:7/663: symlink d9/d25/d93/ld2 0 2024-03-16T09:01:21.554 INFO:tasks.workunit.client.0.smithi007.stdout:1/712: rename d9/c23 to d9/dd/d5e/ce2 0 2024-03-16T09:01:21.555 INFO:tasks.workunit.client.1.smithi019.stdout:9/556: dwrite d1/d7/d21/f7b [0,4194304] 0 2024-03-16T09:01:21.555 INFO:tasks.workunit.client.1.smithi019.stdout:9/557: readlink d1/d7/de/l96 0 2024-03-16T09:01:21.556 INFO:tasks.workunit.client.1.smithi019.stdout:1/640: creat d16/d31/fb9 x:0 0 0 2024-03-16T09:01:21.556 INFO:tasks.workunit.client.1.smithi019.stdout:1/641: write d16/d1f/d4f/f57 [2103144,87955] 0 2024-03-16T09:01:21.556 INFO:tasks.workunit.client.1.smithi019.stdout:1/642: creat d16/d1f/d3c/d79/fba x:0 0 0 2024-03-16T09:01:21.556 INFO:tasks.workunit.client.0.smithi007.stdout:1/713: symlink d9/dd/d1b/d2d/le3 0 2024-03-16T09:01:21.559 INFO:tasks.workunit.client.0.smithi007.stdout:1/714: mkdir d9/dd/d1b/d28/d4c/de4 0 2024-03-16T09:01:21.559 INFO:tasks.workunit.client.0.smithi007.stdout:1/715: fdatasync f5 0 2024-03-16T09:01:21.560 INFO:tasks.workunit.client.1.smithi019.stdout:3/686: stat d3/d8b/dae/d5a/c5d 0 2024-03-16T09:01:21.563 INFO:tasks.workunit.client.1.smithi019.stdout:9/558: mkdir d1/d7/de/d24/d44/d60/d8b/dc2 0 2024-03-16T09:01:21.563 INFO:tasks.workunit.client.1.smithi019.stdout:9/559: fdatasync d1/d7/f56 0 2024-03-16T09:01:21.565 INFO:tasks.workunit.client.1.smithi019.stdout:9/560: dread d1/d7/f3e [0,4194304] 0 2024-03-16T09:01:21.565 INFO:tasks.workunit.client.1.smithi019.stdout:9/561: stat d1/d7/de/l69 0 2024-03-16T09:01:21.565 INFO:tasks.workunit.client.1.smithi019.stdout:2/783: dwrite d5/d6/d2b/d2f/d35/d65/fdc [0,4194304] 0 2024-03-16T09:01:21.566 INFO:tasks.workunit.client.1.smithi019.stdout:1/643: creat d16/d1f/d3c/d79/db4/fbb x:0 0 0 2024-03-16T09:01:21.567 INFO:tasks.workunit.client.1.smithi019.stdout:3/687: symlink d3/d8b/dae/d3f/ld9 0 2024-03-16T09:01:21.567 INFO:tasks.workunit.client.1.smithi019.stdout:3/688: truncate d3/d8b/dae/f50 4367282 0 2024-03-16T09:01:21.567 INFO:tasks.workunit.client.1.smithi019.stdout:3/689: chown d3/d8b/dae/d5a/d75 1396865417 1 2024-03-16T09:01:21.570 INFO:tasks.workunit.client.1.smithi019.stdout:9/562: rmdir d1/d7/de/da3 39 2024-03-16T09:01:21.570 INFO:tasks.workunit.client.1.smithi019.stdout:9/563: write d1/d7/f68 [4590977,105395] 0 2024-03-16T09:01:21.571 INFO:tasks.workunit.client.1.smithi019.stdout:1/644: truncate d16/d1f/d3c/d79/f91 297865 0 2024-03-16T09:01:21.573 INFO:tasks.workunit.client.1.smithi019.stdout:3/690: unlink d3/d8b/dae/d5a/d75/f92 0 2024-03-16T09:01:21.575 INFO:tasks.workunit.client.1.smithi019.stdout:9/564: mkdir d1/d98/dae/dc3 0 2024-03-16T09:01:21.576 INFO:tasks.workunit.client.1.smithi019.stdout:1/645: mknod d16/d1f/d43/cbc 0 2024-03-16T09:01:21.577 INFO:tasks.workunit.client.1.smithi019.stdout:3/691: mknod d3/d8b/dae/d6a/cda 0 2024-03-16T09:01:21.577 INFO:tasks.workunit.client.1.smithi019.stdout:9/565: mknod d1/d7/de/d24/d44/d60/d74/cc4 0 2024-03-16T09:01:21.578 INFO:tasks.workunit.client.0.smithi007.stdout:1/716: dread d9/f39 [0,4194304] 0 2024-03-16T09:01:21.578 INFO:tasks.workunit.client.1.smithi019.stdout:3/692: dread d3/d9b/f9f [0,4194304] 0 2024-03-16T09:01:21.578 INFO:tasks.workunit.client.1.smithi019.stdout:1/646: mkdir d16/d31/d76/d92/d9f/daf/db2/d78/dbd 0 2024-03-16T09:01:21.579 INFO:tasks.workunit.client.1.smithi019.stdout:9/566: creat d1/d98/fc5 x:0 0 0 2024-03-16T09:01:21.580 INFO:tasks.workunit.client.1.smithi019.stdout:1/647: truncate d16/d1f/d4f/f50 3255196 0 2024-03-16T09:01:21.581 INFO:tasks.workunit.client.1.smithi019.stdout:3/693: rmdir d3/d89 39 2024-03-16T09:01:21.581 INFO:tasks.workunit.client.1.smithi019.stdout:3/694: chown d3/d8b/da8/cb3 724 1 2024-03-16T09:01:21.582 INFO:tasks.workunit.client.1.smithi019.stdout:1/648: rename d16/d1f/d4f/f62 to d16/d31/d76/fbe 0 2024-03-16T09:01:21.583 INFO:tasks.workunit.client.0.smithi007.stdout:1/717: write d9/dd/d1b/d2d/f44 [548794,65218] 0 2024-03-16T09:01:21.583 INFO:tasks.workunit.client.1.smithi019.stdout:1/649: rmdir d16/d31/d76/d92/d9f/daf/db2/d78 39 2024-03-16T09:01:21.583 INFO:tasks.workunit.client.1.smithi019.stdout:1/650: stat d16/d1f/d43/f47 0 2024-03-16T09:01:21.583 INFO:tasks.workunit.client.1.smithi019.stdout:1/651: fsync d16/d1f/d43/f70 0 2024-03-16T09:01:21.583 INFO:tasks.workunit.client.1.smithi019.stdout:1/652: truncate f9 1392445 0 2024-03-16T09:01:21.586 INFO:tasks.workunit.client.0.smithi007.stdout:7/851: sync 2024-03-16T09:01:21.586 INFO:tasks.workunit.client.0.smithi007.stdout:4/738: sync 2024-03-16T09:01:21.587 INFO:tasks.workunit.client.0.smithi007.stdout:0/967: sync 2024-03-16T09:01:21.587 INFO:tasks.workunit.client.0.smithi007.stdout:0/968: write d3/d9/f93 [1431664,117900] 0 2024-03-16T09:01:21.588 INFO:tasks.workunit.client.0.smithi007.stdout:7/852: truncate d0/d19/d1f/f4b 2395432 0 2024-03-16T09:01:21.590 INFO:tasks.workunit.client.0.smithi007.stdout:4/739: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/ffd x:0 0 0 2024-03-16T09:01:21.590 INFO:tasks.workunit.client.0.smithi007.stdout:4/740: fsync d4/f4d 0 2024-03-16T09:01:21.590 INFO:tasks.workunit.client.0.smithi007.stdout:4/741: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/ffe x:0 0 0 2024-03-16T09:01:21.590 INFO:tasks.workunit.client.0.smithi007.stdout:4/742: truncate d4/d14/d64/d68/fc4 592956 0 2024-03-16T09:01:21.591 INFO:tasks.workunit.client.1.smithi019.stdout:9/567: dread d1/d7/de/d24/d44/d60/f64 [0,4194304] 0 2024-03-16T09:01:21.592 INFO:tasks.workunit.client.0.smithi007.stdout:0/969: truncate d3/d9/da/d48/d4b/f51 593680 0 2024-03-16T09:01:21.597 INFO:tasks.workunit.client.0.smithi007.stdout:0/970: truncate d3/d9/db4/d1c/d72/f82 163484 0 2024-03-16T09:01:21.597 INFO:tasks.workunit.client.1.smithi019.stdout:9/568: creat d1/d7/dbb/fc6 x:0 0 0 2024-03-16T09:01:21.597 INFO:tasks.workunit.client.1.smithi019.stdout:9/569: chown d1/d7/de/d24/d5b/c71 1595278 1 2024-03-16T09:01:21.597 INFO:tasks.workunit.client.1.smithi019.stdout:9/570: fsync d1/d92/fc1 0 2024-03-16T09:01:21.598 INFO:tasks.workunit.client.1.smithi019.stdout:9/571: chown d1/d7/de/d24/c29 246127 1 2024-03-16T09:01:21.598 INFO:tasks.workunit.client.0.smithi007.stdout:0/971: write d3/d9/da/d48/d9e/f117 [348739,99312] 0 2024-03-16T09:01:21.600 INFO:tasks.workunit.client.1.smithi019.stdout:9/572: symlink d1/d7/de/d24/lc7 0 2024-03-16T09:01:21.600 INFO:tasks.workunit.client.1.smithi019.stdout:9/573: stat d1/d7/de/d24/d44/d60/d74/fad 0 2024-03-16T09:01:21.601 INFO:tasks.workunit.client.0.smithi007.stdout:4/743: write d4/d14/d1c/f29 [958564,69739] 0 2024-03-16T09:01:21.603 INFO:tasks.workunit.client.0.smithi007.stdout:4/744: truncate d4/fa3 1366200 0 2024-03-16T09:01:21.604 INFO:tasks.workunit.client.0.smithi007.stdout:4/745: mkdir d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b/dff 0 2024-03-16T09:01:21.606 INFO:tasks.workunit.client.0.smithi007.stdout:4/746: unlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/fad 0 2024-03-16T09:01:21.610 INFO:tasks.workunit.client.0.smithi007.stdout:0/972: write d3/d9/db4/d2c/f5e [2424201,42265] 0 2024-03-16T09:01:21.610 INFO:tasks.workunit.client.0.smithi007.stdout:0/973: creat d3/d9/da/d48/d25/d43/d1a/f13c x:0 0 0 2024-03-16T09:01:21.611 INFO:tasks.workunit.client.1.smithi019.stdout:1/653: dwrite d16/f37 [0,4194304] 0 2024-03-16T09:01:21.612 INFO:tasks.workunit.client.1.smithi019.stdout:1/654: truncate d16/f5d 621813 0 2024-03-16T09:01:21.613 INFO:tasks.workunit.client.1.smithi019.stdout:1/655: creat d16/d84/fbf x:0 0 0 2024-03-16T09:01:21.614 INFO:tasks.workunit.client.0.smithi007.stdout:0/974: rename d3/d9/da/d18/d27/lc9 to d3/d6c/dbd/dd7/d7c/dec/l13d 0 2024-03-16T09:01:21.614 INFO:tasks.workunit.client.0.smithi007.stdout:0/975: readlink d3/d9/da/d92/d9f/de8/d123/l131 0 2024-03-16T09:01:21.615 INFO:tasks.workunit.client.1.smithi019.stdout:1/656: mknod d16/d31/d71/d89/cc0 0 2024-03-16T09:01:21.615 INFO:tasks.workunit.client.1.smithi019.stdout:1/657: write d16/d18/f1b [1706767,115180] 0 2024-03-16T09:01:21.615 INFO:tasks.workunit.client.0.smithi007.stdout:6/790: dwrite d2/f20 [0,4194304] 0 2024-03-16T09:01:21.616 INFO:tasks.workunit.client.0.smithi007.stdout:0/976: symlink d3/d9/da/d48/d25/d43/d1a/l13e 0 2024-03-16T09:01:21.624 INFO:tasks.workunit.client.0.smithi007.stdout:6/791: rmdir d2/d13/d105 0 2024-03-16T09:01:21.624 INFO:tasks.workunit.client.0.smithi007.stdout:6/792: readlink d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/lf5 0 2024-03-16T09:01:21.626 INFO:tasks.workunit.client.0.smithi007.stdout:6/793: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/f5c 266249 0 2024-03-16T09:01:21.627 INFO:tasks.workunit.client.0.smithi007.stdout:6/794: creat d2/d13/d3f/d50/d78/dbe/dcb/f106 x:0 0 0 2024-03-16T09:01:21.627 INFO:tasks.workunit.client.0.smithi007.stdout:6/795: dread - d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/fe4 zero size 2024-03-16T09:01:21.629 INFO:tasks.workunit.client.0.smithi007.stdout:6/796: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/f107 x:0 0 0 2024-03-16T09:01:21.631 INFO:tasks.workunit.client.0.smithi007.stdout:6/797: unlink d2/d13/d3f/d50/d78/d81/lb2 0 2024-03-16T09:01:21.631 INFO:tasks.workunit.client.0.smithi007.stdout:6/798: chown d2/d13/d3f/d50/d78/dbe/dcb/d1e/fd0 26217 1 2024-03-16T09:01:21.631 INFO:tasks.workunit.client.0.smithi007.stdout:6/799: chown d2/d13/d3f/d50/d78/dbe/dcb/d1e/l61 37667444 1 2024-03-16T09:01:21.631 INFO:tasks.workunit.client.0.smithi007.stdout:6/800: rename d2/d13/d3f/d50/d78 to d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/d108 22 2024-03-16T09:01:21.631 INFO:tasks.workunit.client.0.smithi007.stdout:6/801: fsync d2/fa 0 2024-03-16T09:01:21.632 INFO:tasks.workunit.client.0.smithi007.stdout:6/802: link d2/d10/leb d2/d13/d3f/d50/d78/dbe/dcb/l109 0 2024-03-16T09:01:21.642 INFO:tasks.workunit.client.1.smithi019.stdout:2/784: dwrite d5/d2e/f50 [0,4194304] 0 2024-03-16T09:01:21.642 INFO:tasks.workunit.client.1.smithi019.stdout:2/785: readlink d5/d5a/l10f 0 2024-03-16T09:01:21.644 INFO:tasks.workunit.client.1.smithi019.stdout:2/786: rename d5/de/l97 to d5/d6/d2b/d2f/d35/d4b/dca/l113 0 2024-03-16T09:01:21.645 INFO:tasks.workunit.client.1.smithi019.stdout:2/787: unlink d5/d6/d104/lfe 0 2024-03-16T09:01:21.646 INFO:tasks.workunit.client.1.smithi019.stdout:2/788: creat d5/d6/d104/dba/dfc/f114 x:0 0 0 2024-03-16T09:01:21.647 INFO:tasks.workunit.client.1.smithi019.stdout:2/789: write d5/d6/d2b/f18 [2795491,73454] 0 2024-03-16T09:01:21.650 INFO:tasks.workunit.client.0.smithi007.stdout:6/803: dread d2/f46 [0,4194304] 0 2024-03-16T09:01:21.662 INFO:tasks.workunit.client.1.smithi019.stdout:8/823: dwrite d6/d2a/d45/d57/fbb [0,4194304] 0 2024-03-16T09:01:21.663 INFO:tasks.workunit.client.1.smithi019.stdout:7/664: dwrite d9/d25/d93/fbc [0,4194304] 0 2024-03-16T09:01:21.665 INFO:tasks.workunit.client.1.smithi019.stdout:8/824: rename d6/d2a/d45/d19/l71 to d6/d2a/d45/d57/d72/d79/da5/d11d/l11f 0 2024-03-16T09:01:21.665 INFO:tasks.workunit.client.1.smithi019.stdout:8/825: write d6/de/f6a [524583,113278] 0 2024-03-16T09:01:21.668 INFO:tasks.workunit.client.1.smithi019.stdout:8/826: creat d6/d2a/d45/d2c/de3/f120 x:0 0 0 2024-03-16T09:01:21.669 INFO:tasks.workunit.client.1.smithi019.stdout:8/827: dread - d6/ff2 zero size 2024-03-16T09:01:21.669 INFO:tasks.workunit.client.1.smithi019.stdout:3/695: dwrite d3/d8b/dae/f50 [0,4194304] 0 2024-03-16T09:01:21.669 INFO:tasks.workunit.client.1.smithi019.stdout:3/696: read d3/d1b/d40/d4a/f98 [2601791,105278] 0 2024-03-16T09:01:21.669 INFO:tasks.workunit.client.1.smithi019.stdout:9/574: dwrite d1/d7/de/d24/d44/d60/f97 [0,4194304] 0 2024-03-16T09:01:21.669 INFO:tasks.workunit.client.1.smithi019.stdout:9/575: dread - d1/d7/de/d24/d44/d60/d74/f80 zero size 2024-03-16T09:01:21.669 INFO:tasks.workunit.client.1.smithi019.stdout:7/665: rename d9/db/d12/f6d to d9/da/d13/d18/d30/d3e/fd3 0 2024-03-16T09:01:21.669 INFO:tasks.workunit.client.1.smithi019.stdout:7/666: write d9/d25/d28/f9c [693985,8765] 0 2024-03-16T09:01:21.670 INFO:tasks.workunit.client.1.smithi019.stdout:0/765: sync 2024-03-16T09:01:21.670 INFO:tasks.workunit.client.0.smithi007.stdout:1/718: dwrite d9/dd/d5e/f65 [4194304,4194304] 0 2024-03-16T09:01:21.670 INFO:tasks.workunit.client.0.smithi007.stdout:1/719: write d9/dd/d1b/fd6 [580929,25684] 0 2024-03-16T09:01:21.670 INFO:tasks.workunit.client.0.smithi007.stdout:0/977: dwrite d3/d9/da/d18/d23/f45 [0,4194304] 0 2024-03-16T09:01:21.670 INFO:tasks.workunit.client.0.smithi007.stdout:6/804: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/ff4 [0,4194304] 0 2024-03-16T09:01:21.671 INFO:tasks.workunit.client.1.smithi019.stdout:3/697: creat d3/d1b/fdb x:0 0 0 2024-03-16T09:01:21.671 INFO:tasks.workunit.client.1.smithi019.stdout:3/698: fdatasync d3/d9b/fb6 0 2024-03-16T09:01:21.671 INFO:tasks.workunit.client.1.smithi019.stdout:8/828: link d6/d2a/d45/d57/d72/d79/db7/fe1 d6/d2a/d3c/dab/d30/d4a/dc7/f121 0 2024-03-16T09:01:21.673 INFO:tasks.workunit.client.1.smithi019.stdout:7/667: symlink d9/db/d12/d5b/dc3/ld4 0 2024-03-16T09:01:21.675 INFO:tasks.workunit.client.1.smithi019.stdout:3/699: rmdir d3/d8b/dae/d5a/d75/d7a 39 2024-03-16T09:01:21.675 INFO:tasks.workunit.client.1.smithi019.stdout:3/700: stat d3/d8b/dae/d6a/f7b 0 2024-03-16T09:01:21.676 INFO:tasks.workunit.client.1.smithi019.stdout:3/701: chown d3/d8b/dae/d3f/l44 0 1 2024-03-16T09:01:21.676 INFO:tasks.workunit.client.1.smithi019.stdout:0/766: write d7/d8/dec/def/d92/da6/fbb [880705,106245] 0 2024-03-16T09:01:21.678 INFO:tasks.workunit.client.0.smithi007.stdout:0/978: rename d3/d9/da/d48/d25/d43/d1a/l10f to d3/d9/da/d18/d23/d36/l13f 0 2024-03-16T09:01:21.679 INFO:tasks.workunit.client.1.smithi019.stdout:9/576: rename d1/d7/l46 to d1/d7/de/d24/d44/d76/d9c/lc8 0 2024-03-16T09:01:21.682 INFO:tasks.workunit.client.0.smithi007.stdout:1/720: read d9/dd/d1b/d58/f7d [3147447,6209] 0 2024-03-16T09:01:21.682 INFO:tasks.workunit.client.0.smithi007.stdout:1/721: write d9/dd/d5e/f9f [567818,69642] 0 2024-03-16T09:01:21.683 INFO:tasks.workunit.client.0.smithi007.stdout:6/805: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/d8f/d10a 0 2024-03-16T09:01:21.683 INFO:tasks.workunit.client.1.smithi019.stdout:0/767: link d7/d8/dec/def/l1f d7/d8/dec/dfc/lfd 0 2024-03-16T09:01:21.684 INFO:tasks.workunit.client.1.smithi019.stdout:8/829: dread d6/de/f76 [0,4194304] 0 2024-03-16T09:01:21.684 INFO:tasks.workunit.client.1.smithi019.stdout:8/830: write d6/d2a/d55/f112 [578082,110790] 0 2024-03-16T09:01:21.684 INFO:tasks.workunit.client.1.smithi019.stdout:9/577: mkdir d1/d7/d33/dc9 0 2024-03-16T09:01:21.684 INFO:tasks.workunit.client.0.smithi007.stdout:6/806: read d2/d29/fab [131651,49776] 0 2024-03-16T09:01:21.684 INFO:tasks.workunit.client.0.smithi007.stdout:6/807: readlink d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/lc8 0 2024-03-16T09:01:21.685 INFO:tasks.workunit.client.0.smithi007.stdout:6/808: readlink d2/d13/d3f/d50/d78/dbe/dcb/l32 0 2024-03-16T09:01:21.685 INFO:tasks.workunit.client.0.smithi007.stdout:1/722: dread d9/dd/d1b/d2d/d30/d45/fbd [0,4194304] 0 2024-03-16T09:01:21.685 INFO:tasks.workunit.client.1.smithi019.stdout:7/668: dread d9/da/f24 [0,4194304] 0 2024-03-16T09:01:21.685 INFO:tasks.workunit.client.0.smithi007.stdout:0/979: rename d3/d9/db4/d1c/f12d to d3/d9/da/d92/d9f/de8/f140 0 2024-03-16T09:01:21.687 INFO:tasks.workunit.client.1.smithi019.stdout:0/768: mknod d7/d1d/cfe 0 2024-03-16T09:01:21.687 INFO:tasks.workunit.client.1.smithi019.stdout:0/769: write d7/fe4 [245020,112379] 0 2024-03-16T09:01:21.687 INFO:tasks.workunit.client.1.smithi019.stdout:0/770: write d7/d8/f88 [1489993,84221] 0 2024-03-16T09:01:21.687 INFO:tasks.workunit.client.1.smithi019.stdout:9/578: mkdir d1/d7/d33/dc9/dca 0 2024-03-16T09:01:21.687 INFO:tasks.workunit.client.1.smithi019.stdout:9/579: write d1/d7/de/f2a [3555035,129856] 0 2024-03-16T09:01:21.687 INFO:tasks.workunit.client.1.smithi019.stdout:9/580: fdatasync d1/d7/de/d24/d44/d60/f97 0 2024-03-16T09:01:21.690 INFO:tasks.workunit.client.1.smithi019.stdout:7/669: rename d9/da/d13/f84 to d9/da/fd5 0 2024-03-16T09:01:21.690 INFO:tasks.workunit.client.1.smithi019.stdout:7/670: creat d9/da/d13/d18/d30/d3e/da5/fd6 x:0 0 0 2024-03-16T09:01:21.693 INFO:tasks.workunit.client.1.smithi019.stdout:3/702: dread d3/d1b/d40/d4a/f98 [0,4194304] 0 2024-03-16T09:01:21.694 INFO:tasks.workunit.client.0.smithi007.stdout:1/723: dread d9/dd/d1b/d28/f32 [0,4194304] 0 2024-03-16T09:01:21.694 INFO:tasks.workunit.client.1.smithi019.stdout:9/581: unlink d1/d7/de/f10 0 2024-03-16T09:01:21.695 INFO:tasks.workunit.client.1.smithi019.stdout:0/771: rename d7/f79 to d7/d8/dec/def/da1/fff 0 2024-03-16T09:01:21.696 INFO:tasks.workunit.client.1.smithi019.stdout:1/658: getdents d16/d1f/d4f 0 2024-03-16T09:01:21.699 INFO:tasks.workunit.client.1.smithi019.stdout:8/831: write d6/d2a/d45/d19/f1a [1574825,120566] 0 2024-03-16T09:01:21.699 INFO:tasks.workunit.client.1.smithi019.stdout:8/832: chown d6/d2a/d68/c11a 525941637 1 2024-03-16T09:01:21.700 INFO:tasks.workunit.client.1.smithi019.stdout:9/582: unlink d1/d98/dae/fa7 0 2024-03-16T09:01:21.701 INFO:tasks.workunit.client.0.smithi007.stdout:1/724: link d9/l98 d9/dd/d1b/d28/d29/d7b/le5 0 2024-03-16T09:01:21.702 INFO:tasks.workunit.client.1.smithi019.stdout:0/772: creat d7/d8/d32/df5/f100 x:0 0 0 2024-03-16T09:01:21.702 INFO:tasks.workunit.client.1.smithi019.stdout:0/773: write d7/d8/d32/d66/d6d/d8c/db8/d82/f6f [1150129,81263] 0 2024-03-16T09:01:21.703 INFO:tasks.workunit.client.1.smithi019.stdout:1/659: getdents d16/d31/d76/d92/d9f/daf 0 2024-03-16T09:01:21.703 INFO:tasks.workunit.client.1.smithi019.stdout:1/660: chown d16/d1f/d3c/d79/fba 703 1 2024-03-16T09:01:21.703 INFO:tasks.workunit.client.1.smithi019.stdout:2/790: unlink d5/d6/d2b/d2f/c59 0 2024-03-16T09:01:21.704 INFO:tasks.workunit.client.1.smithi019.stdout:3/703: mknod d3/d8b/dae/d5a/cdc 0 2024-03-16T09:01:21.705 INFO:tasks.workunit.client.1.smithi019.stdout:8/833: mknod d6/d2a/d45/d57/d72/d79/c122 0 2024-03-16T09:01:21.705 INFO:tasks.workunit.client.1.smithi019.stdout:8/834: readlink d6/d2a/d68/l106 0 2024-03-16T09:01:21.705 INFO:tasks.workunit.client.1.smithi019.stdout:8/835: readlink d6/d2a/d45/d57/d72/d79/d7b/lc2 0 2024-03-16T09:01:21.705 INFO:tasks.workunit.client.1.smithi019.stdout:8/836: chown d6/d2a/d45/d2c/f5c 10540881 1 2024-03-16T09:01:21.705 INFO:tasks.workunit.client.1.smithi019.stdout:8/837: write d6/d2a/d45/d57/d72/d79/db7/fe1 [731336,74175] 0 2024-03-16T09:01:21.706 INFO:tasks.workunit.client.1.smithi019.stdout:9/583: unlink d1/d7/de/d24/d5b/l5e 0 2024-03-16T09:01:21.706 INFO:tasks.workunit.client.1.smithi019.stdout:9/584: stat d1/d98/dae/f35 0 2024-03-16T09:01:21.707 INFO:tasks.workunit.client.1.smithi019.stdout:0/774: rmdir d7/d8/d32/d66/d6d/d8c 39 2024-03-16T09:01:21.708 INFO:tasks.workunit.client.0.smithi007.stdout:1/725: rename d9/d6b/fa0 to d9/dd/d1b/d28/d29/d7b/fe6 0 2024-03-16T09:01:21.708 INFO:tasks.workunit.client.0.smithi007.stdout:1/726: fdatasync d9/f22 0 2024-03-16T09:01:21.708 INFO:tasks.workunit.client.0.smithi007.stdout:1/727: write d9/f22 [3803854,93576] 0 2024-03-16T09:01:21.708 INFO:tasks.workunit.client.1.smithi019.stdout:2/791: symlink d5/d6/d104/dba/dfc/d78/l115 0 2024-03-16T09:01:21.708 INFO:tasks.workunit.client.1.smithi019.stdout:2/792: readlink d5/d6/d2b/d2f/d35/l9e 0 2024-03-16T09:01:21.710 INFO:tasks.workunit.client.1.smithi019.stdout:1/661: rmdir d16/d31/d76/d92 39 2024-03-16T09:01:21.710 INFO:tasks.workunit.client.1.smithi019.stdout:8/838: link d6/d2a/d68/lfc d6/d2a/d3c/l123 0 2024-03-16T09:01:21.711 INFO:tasks.workunit.client.1.smithi019.stdout:3/704: symlink d3/d9b/ldd 0 2024-03-16T09:01:21.712 INFO:tasks.workunit.client.1.smithi019.stdout:1/662: getdents d16/d31/d76/d92/d9f 0 2024-03-16T09:01:21.713 INFO:tasks.workunit.client.1.smithi019.stdout:8/839: mknod d6/d2a/d45/d57/d72/d79/dc1/c124 0 2024-03-16T09:01:21.713 INFO:tasks.workunit.client.1.smithi019.stdout:8/840: write d6/d2a/d3c/dab/d30/d4a/fef [1198427,80730] 0 2024-03-16T09:01:21.715 INFO:tasks.workunit.client.1.smithi019.stdout:1/663: rename d16/d31/fb9 to d16/da1/fc1 0 2024-03-16T09:01:21.732 INFO:tasks.workunit.client.1.smithi019.stdout:1/664: dread d16/d18/f1a [0,4194304] 0 2024-03-16T09:01:21.732 INFO:tasks.workunit.client.1.smithi019.stdout:1/665: stat d16/d1f/d43/l52 0 2024-03-16T09:01:21.733 INFO:tasks.workunit.client.0.smithi007.stdout:1/728: dread d9/dd/d1b/d2d/d30/d45/fc8 [0,4194304] 0 2024-03-16T09:01:21.733 INFO:tasks.workunit.client.1.smithi019.stdout:2/793: write d5/d6/d104/df7/d6f/da6/f8b [3882176,2535] 0 2024-03-16T09:01:21.734 INFO:tasks.workunit.client.0.smithi007.stdout:1/729: getdents d9/dd/d1b/d58 0 2024-03-16T09:01:21.735 INFO:tasks.workunit.client.0.smithi007.stdout:1/730: write d9/d6b/d7e/dac/dbb/fbe [1039542,40914] 0 2024-03-16T09:01:21.735 INFO:tasks.workunit.client.1.smithi019.stdout:8/841: write d6/d2a/d45/d19/d74/f7e [3747277,17604] 0 2024-03-16T09:01:21.735 INFO:tasks.workunit.client.1.smithi019.stdout:8/842: chown d6/d2a/d45/d19/l48 424589744 1 2024-03-16T09:01:21.737 INFO:tasks.workunit.client.1.smithi019.stdout:2/794: mkdir d5/d6/d116 0 2024-03-16T09:01:21.738 INFO:tasks.workunit.client.1.smithi019.stdout:0/775: dwrite d7/d8/d32/fa5 [4194304,4194304] 0 2024-03-16T09:01:21.738 INFO:tasks.workunit.client.1.smithi019.stdout:0/776: write d7/d1d/f4b [749470,12595] 0 2024-03-16T09:01:21.739 INFO:tasks.workunit.client.1.smithi019.stdout:0/777: truncate d7/d8/f23 292228 0 2024-03-16T09:01:21.740 INFO:tasks.workunit.client.1.smithi019.stdout:2/795: rename d5/d2e/ff1 to d5/d5a/f117 0 2024-03-16T09:01:21.740 INFO:tasks.workunit.client.1.smithi019.stdout:2/796: chown d5/d6/d2b/d2f/d35/d65/fb4 444136246 1 2024-03-16T09:01:21.741 INFO:tasks.workunit.client.1.smithi019.stdout:2/797: truncate d5/d6/d104/dba/dfc/d74/fa3 590506 0 2024-03-16T09:01:21.741 INFO:tasks.workunit.client.0.smithi007.stdout:3/864: sync 2024-03-16T09:01:21.741 INFO:tasks.workunit.client.0.smithi007.stdout:9/836: sync 2024-03-16T09:01:21.741 INFO:tasks.workunit.client.0.smithi007.stdout:5/815: sync 2024-03-16T09:01:21.741 INFO:tasks.workunit.client.0.smithi007.stdout:2/971: sync 2024-03-16T09:01:21.741 INFO:tasks.workunit.client.0.smithi007.stdout:3/865: stat d15/d18 0 2024-03-16T09:01:21.741 INFO:tasks.workunit.client.0.smithi007.stdout:9/837: stat d24/d4a/fcc 0 2024-03-16T09:01:21.741 INFO:tasks.workunit.client.0.smithi007.stdout:3/866: chown d15/l40 653159067 1 2024-03-16T09:01:21.742 INFO:tasks.workunit.client.1.smithi019.stdout:0/778: unlink d7/d1d/d2e/c31 0 2024-03-16T09:01:21.742 INFO:tasks.workunit.client.1.smithi019.stdout:0/779: write d7/d8/d32/d66/d6d/d8c/db5/ffa [57299,28695] 0 2024-03-16T09:01:21.742 INFO:tasks.workunit.client.1.smithi019.stdout:2/798: creat d5/d5a/d80/f118 x:0 0 0 2024-03-16T09:01:21.746 INFO:tasks.workunit.client.0.smithi007.stdout:9/838: rmdir d24/d4a/d96/db9/df2 0 2024-03-16T09:01:21.746 INFO:tasks.workunit.client.0.smithi007.stdout:5/816: symlink d1/d7/d1c/d19/df7/d110/l129 0 2024-03-16T09:01:21.746 INFO:tasks.workunit.client.0.smithi007.stdout:5/817: readlink d1/d7/d1c/d19/d34/l5d 0 2024-03-16T09:01:21.746 INFO:tasks.workunit.client.0.smithi007.stdout:5/818: write d1/d7/d1c/d88/da2/f45 [1419168,64566] 0 2024-03-16T09:01:21.747 INFO:tasks.workunit.client.1.smithi019.stdout:2/799: creat d5/d6/f119 x:0 0 0 2024-03-16T09:01:21.749 INFO:tasks.workunit.client.0.smithi007.stdout:2/972: mkdir dc/df/daa/d12c/d144 0 2024-03-16T09:01:21.749 INFO:tasks.workunit.client.0.smithi007.stdout:3/867: creat d15/d18/f10f x:0 0 0 2024-03-16T09:01:21.749 INFO:tasks.workunit.client.0.smithi007.stdout:3/868: chown d15/d18/d1f/d29/d7c/f109 179 1 2024-03-16T09:01:21.751 INFO:tasks.workunit.client.1.smithi019.stdout:2/800: unlink d5/d6/d2b/f98 0 2024-03-16T09:01:21.752 INFO:tasks.workunit.client.1.smithi019.stdout:2/801: write d5/d6/d104/dba/dfc/f114 [397122,108735] 0 2024-03-16T09:01:21.752 INFO:tasks.workunit.client.1.smithi019.stdout:2/802: readlink d5/lb7 0 2024-03-16T09:01:21.752 INFO:tasks.workunit.client.1.smithi019.stdout:2/803: chown d5/de/da2/faa 103533 1 2024-03-16T09:01:21.754 INFO:tasks.workunit.client.0.smithi007.stdout:9/839: rename d24/d4a/d96/db9/f77 to d24/d4c/d86/daa/d7f/ff7 0 2024-03-16T09:01:21.755 INFO:tasks.workunit.client.0.smithi007.stdout:5/819: creat d1/d7/d1c/d88/d11e/f12a x:0 0 0 2024-03-16T09:01:21.755 INFO:tasks.workunit.client.1.smithi019.stdout:2/804: getdents d5/d6/d2b/d2f/d35/d4b/dca 0 2024-03-16T09:01:21.757 INFO:tasks.workunit.client.1.smithi019.stdout:0/780: write d7/d8/d32/d4a/d61/dbf/fe6 [745255,125226] 0 2024-03-16T09:01:21.757 INFO:tasks.workunit.client.1.smithi019.stdout:2/805: readlink d5/d6/d2b/l2a 0 2024-03-16T09:01:21.757 INFO:tasks.workunit.client.1.smithi019.stdout:0/781: dread - d7/d8/d32/d66/d6d/d8c/db5/fd7 zero size 2024-03-16T09:01:21.759 INFO:tasks.workunit.client.1.smithi019.stdout:0/782: dread d7/d1d/f67 [0,4194304] 0 2024-03-16T09:01:21.759 INFO:tasks.workunit.client.1.smithi019.stdout:0/783: creat d7/d1d/f101 x:0 0 0 2024-03-16T09:01:21.761 INFO:tasks.workunit.client.0.smithi007.stdout:5/820: rmdir d1/d7/d1c/d19/d34/d3e/d62/def 0 2024-03-16T09:01:21.762 INFO:tasks.workunit.client.0.smithi007.stdout:3/869: mkdir d15/d18/d6a/db1/d110 0 2024-03-16T09:01:21.763 INFO:tasks.workunit.client.1.smithi019.stdout:2/806: rmdir d5/d6/d2b/d2f/d35 39 2024-03-16T09:01:21.763 INFO:tasks.workunit.client.1.smithi019.stdout:2/807: truncate d5/d2e/d4f/ffa 155892 0 2024-03-16T09:01:21.763 INFO:tasks.workunit.client.1.smithi019.stdout:2/808: symlink d5/de/dab/l11a 0 2024-03-16T09:01:21.786 INFO:tasks.workunit.client.1.smithi019.stdout:7/671: dwrite d9/da/d13/d18/d30/d3e/fd3 [0,4194304] 0 2024-03-16T09:01:21.786 INFO:tasks.workunit.client.1.smithi019.stdout:7/672: readlink d9/db/d12/d5b/dc3/ld4 0 2024-03-16T09:01:21.786 INFO:tasks.workunit.client.1.smithi019.stdout:7/673: read - d9/d25/d28/f8b zero size 2024-03-16T09:01:21.793 INFO:tasks.workunit.client.1.smithi019.stdout:7/674: mkdir d9/da/d13/d18/d30/d9e/dd7 0 2024-03-16T09:01:21.794 INFO:tasks.workunit.client.1.smithi019.stdout:7/675: truncate f8 1983502 0 2024-03-16T09:01:21.794 INFO:tasks.workunit.client.1.smithi019.stdout:7/676: mkdir d9/d25/d28/d48/d8d/dba/dad/dd8 0 2024-03-16T09:01:21.795 INFO:tasks.workunit.client.1.smithi019.stdout:7/677: chown d9/da/d13/d18/d1a/fa8 5 1 2024-03-16T09:01:21.796 INFO:tasks.workunit.client.1.smithi019.stdout:7/678: creat d9/da/d13/d18/d1a/fd9 x:0 0 0 2024-03-16T09:01:21.797 INFO:tasks.workunit.client.1.smithi019.stdout:7/679: rename d9/da/d13/d18/d1a/l61 to d9/db/d12/lda 0 2024-03-16T09:01:21.841 INFO:tasks.workunit.client.1.smithi019.stdout:3/705: dwrite d3/d11/f69 [0,4194304] 0 2024-03-16T09:01:21.841 INFO:tasks.workunit.client.1.smithi019.stdout:3/706: truncate d3/d8b/dae/d5a/fbb 470781 0 2024-03-16T09:01:21.844 INFO:tasks.workunit.client.0.smithi007.stdout:6/809: dwrite d2/d13/fcc [0,4194304] 0 2024-03-16T09:01:21.844 INFO:tasks.workunit.client.0.smithi007.stdout:6/810: read - d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c/fc6 zero size 2024-03-16T09:01:21.847 INFO:tasks.workunit.client.0.smithi007.stdout:0/980: dwrite d3/d6c/dbd/dd7/d7b/ff3 [4194304,4194304] 0 2024-03-16T09:01:21.847 INFO:tasks.workunit.client.1.smithi019.stdout:1/666: dwrite d16/da1/fb0 [0,4194304] 0 2024-03-16T09:01:21.851 INFO:tasks.workunit.client.0.smithi007.stdout:0/981: rmdir d3/d9/da/d48/d25/d11f 39 2024-03-16T09:01:21.861 INFO:tasks.workunit.client.0.smithi007.stdout:0/982: write d3/d9/db4/d60/fe5 [2631941,36656] 0 2024-03-16T09:01:21.862 INFO:tasks.workunit.client.0.smithi007.stdout:0/983: stat d3/d9/da/d48/d25/d43/d1a 0 2024-03-16T09:01:21.862 INFO:tasks.workunit.client.0.smithi007.stdout:0/984: chown d3/d9/da/d48/d25/d87/l3b 29457 1 2024-03-16T09:01:21.864 INFO:tasks.workunit.client.1.smithi019.stdout:6/821: sync 2024-03-16T09:01:21.864 INFO:tasks.workunit.client.1.smithi019.stdout:8/843: dwrite d6/d2a/d3c/dab/d30/d4a/d67/d95/de0/f117 [0,4194304] 0 2024-03-16T09:01:21.864 INFO:tasks.workunit.client.1.smithi019.stdout:8/844: chown d6/d2a/d45/d19/d74/f92 2636981 1 2024-03-16T09:01:21.864 INFO:tasks.workunit.client.1.smithi019.stdout:8/845: chown d6/d2a/d45/f42 64052 1 2024-03-16T09:01:21.865 INFO:tasks.workunit.client.1.smithi019.stdout:9/585: dwrite d1/d98/dae/f58 [0,4194304] 0 2024-03-16T09:01:21.865 INFO:tasks.workunit.client.0.smithi007.stdout:0/985: rename d3/d9/db4/d2c/d5b/f96 to d3/d9/da/d48/f141 0 2024-03-16T09:01:21.865 INFO:tasks.workunit.client.0.smithi007.stdout:0/986: fsync d3/d9/db4/d60/f129 0 2024-03-16T09:01:21.867 INFO:tasks.workunit.client.1.smithi019.stdout:2/809: dwrite d5/d2e/f50 [4194304,4194304] 0 2024-03-16T09:01:21.867 INFO:tasks.workunit.client.0.smithi007.stdout:1/731: dwrite d9/dd/d1b/d58/f7d [4194304,4194304] 0 2024-03-16T09:01:21.868 INFO:tasks.workunit.client.0.smithi007.stdout:0/987: mknod d3/d9/da/d48/d25/d43/d1a/d58/c142 0 2024-03-16T09:01:21.868 INFO:tasks.workunit.client.0.smithi007.stdout:0/988: chown d3/d1f/c136 263397387 1 2024-03-16T09:01:21.869 INFO:tasks.workunit.client.0.smithi007.stdout:3/870: dwrite d15/d18/d1f/d29/d43/fe6 [0,4194304] 0 2024-03-16T09:01:21.869 INFO:tasks.workunit.client.1.smithi019.stdout:8/846: mknod d6/d2a/d45/d103/c125 0 2024-03-16T09:01:21.869 INFO:tasks.workunit.client.1.smithi019.stdout:7/680: dwrite d9/da/d13/f94 [0,4194304] 0 2024-03-16T09:01:21.869 INFO:tasks.workunit.client.1.smithi019.stdout:0/784: fsync d7/d8/f88 0 2024-03-16T09:01:21.870 INFO:tasks.workunit.client.0.smithi007.stdout:2/973: dwrite dc/df/d24/d28/d4c/d5e/f5b [0,4194304] 0 2024-03-16T09:01:21.870 INFO:tasks.workunit.client.0.smithi007.stdout:5/821: dwrite d1/d7/d1c/d19/d34/d4d/d99/fd7 [0,4194304] 0 2024-03-16T09:01:21.870 INFO:tasks.workunit.client.0.smithi007.stdout:1/732: creat d9/dd/d1b/d28/d4c/de4/fe7 x:0 0 0 2024-03-16T09:01:21.870 INFO:tasks.workunit.client.1.smithi019.stdout:2/810: rename d5/d6/d2b/d2f/fdb to d5/d111/f11b 0 2024-03-16T09:01:21.871 INFO:tasks.workunit.client.1.smithi019.stdout:8/847: symlink d6/d2a/d3c/dab/d30/l126 0 2024-03-16T09:01:21.873 INFO:tasks.workunit.client.0.smithi007.stdout:0/989: dread d3/d9/da/d48/fbe [0,4194304] 0 2024-03-16T09:01:21.873 INFO:tasks.workunit.client.1.smithi019.stdout:8/848: write d6/d2a/d45/d19/fb8 [70314,13991] 0 2024-03-16T09:01:21.873 INFO:tasks.workunit.client.1.smithi019.stdout:8/849: write d6/d2a/d3c/dab/d98/fca [1202636,71485] 0 2024-03-16T09:01:21.873 INFO:tasks.workunit.client.1.smithi019.stdout:9/586: write d1/d7/de/d24/d44/d60/f64 [2250010,55390] 0 2024-03-16T09:01:21.875 INFO:tasks.workunit.client.1.smithi019.stdout:0/785: creat d7/d1d/d2e/f102 x:0 0 0 2024-03-16T09:01:21.875 INFO:tasks.workunit.client.1.smithi019.stdout:0/786: truncate d7/d8/d32/d66/d6d/d8c/db8/d82/faa 590305 0 2024-03-16T09:01:21.875 INFO:tasks.workunit.client.1.smithi019.stdout:0/787: stat d7/d8 0 2024-03-16T09:01:21.875 INFO:tasks.workunit.client.0.smithi007.stdout:5/822: link d1/d7/d1c/d88/fd6 d1/d7/d1c/d19/d34/d1d/dee/f12b 0 2024-03-16T09:01:21.877 INFO:tasks.workunit.client.1.smithi019.stdout:9/587: creat d1/d7/de/da3/fcb x:0 0 0 2024-03-16T09:01:21.878 INFO:tasks.workunit.client.0.smithi007.stdout:2/974: mknod dc/d27/da3/d143/c145 0 2024-03-16T09:01:21.879 INFO:tasks.workunit.client.0.smithi007.stdout:1/733: truncate d9/dd/d1b/d2d/f44 3253917 0 2024-03-16T09:01:21.879 INFO:tasks.workunit.client.1.smithi019.stdout:0/788: rmdir d7/d8/dec/def/d92/dc6 39 2024-03-16T09:01:21.881 INFO:tasks.workunit.client.0.smithi007.stdout:1/734: write d9/dd/d5e/d89/fd0 [578498,84570] 0 2024-03-16T09:01:21.881 INFO:tasks.workunit.client.0.smithi007.stdout:0/990: mknod d3/c143 0 2024-03-16T09:01:21.881 INFO:tasks.workunit.client.1.smithi019.stdout:9/588: mkdir d1/d7/de/d24/d44/d76/d9c/dcc 0 2024-03-16T09:01:21.883 INFO:tasks.workunit.client.1.smithi019.stdout:0/789: rmdir d7/d8 39 2024-03-16T09:01:21.884 INFO:tasks.workunit.client.1.smithi019.stdout:7/681: dread d9/db/f87 [0,4194304] 0 2024-03-16T09:01:21.885 INFO:tasks.workunit.client.0.smithi007.stdout:0/991: mknod d3/d9/da/d48/d25/d87/dbb/c144 0 2024-03-16T09:01:21.886 INFO:tasks.workunit.client.1.smithi019.stdout:9/589: mknod d1/d7/de/d24/d44/d60/d8b/dc2/ccd 0 2024-03-16T09:01:21.886 INFO:tasks.workunit.client.1.smithi019.stdout:9/590: chown d1/d7/la9 341 1 2024-03-16T09:01:21.890 INFO:tasks.workunit.client.1.smithi019.stdout:0/790: rename d7/d8/d32/d4a/c7a to d7/d8/d32/d66/d6d/d8c/c103 0 2024-03-16T09:01:21.892 INFO:tasks.workunit.client.0.smithi007.stdout:1/735: write d9/dd/d1b/d28/d29/d7b/f82 [1736496,22646] 0 2024-03-16T09:01:21.893 INFO:tasks.workunit.client.1.smithi019.stdout:9/591: mkdir d1/d7/de/d24/d5b/dce 0 2024-03-16T09:01:21.895 INFO:tasks.workunit.client.1.smithi019.stdout:0/791: mknod d7/d8/d32/d66/c104 0 2024-03-16T09:01:21.895 INFO:tasks.workunit.client.1.smithi019.stdout:9/592: dread d1/d7/f56 [0,4194304] 0 2024-03-16T09:01:21.897 INFO:tasks.workunit.client.1.smithi019.stdout:7/682: symlink d9/ldb 0 2024-03-16T09:01:21.899 INFO:tasks.workunit.client.1.smithi019.stdout:9/593: mkdir d1/d7/de/d24/d44/d76/d9c/dcf 0 2024-03-16T09:01:21.899 INFO:tasks.workunit.client.1.smithi019.stdout:9/594: fsync d1/d7/de/d24/d44/d76/f5f 0 2024-03-16T09:01:21.901 INFO:tasks.workunit.client.1.smithi019.stdout:9/595: mknod d1/d7/d33/dc9/dca/cd0 0 2024-03-16T09:01:21.901 INFO:tasks.workunit.client.1.smithi019.stdout:9/596: write d1/d7/d21/da2/f9e [415485,73554] 0 2024-03-16T09:01:21.901 INFO:tasks.workunit.client.1.smithi019.stdout:9/597: creat d1/d98/fd1 x:0 0 0 2024-03-16T09:01:21.902 INFO:tasks.workunit.client.1.smithi019.stdout:9/598: truncate d1/d7/fb2 1000512 0 2024-03-16T09:01:21.902 INFO:tasks.workunit.client.1.smithi019.stdout:9/599: write d1/d7/d21/da2/f9e [860421,114770] 0 2024-03-16T09:01:21.903 INFO:tasks.workunit.client.1.smithi019.stdout:0/792: rename d7/d1d/d63/d89/d8a to d7/d8/d32/d4a/d61/d68/dd0/d105 0 2024-03-16T09:01:21.904 INFO:tasks.workunit.client.0.smithi007.stdout:0/992: write d3/d9/da/d18/f6a [715812,85314] 0 2024-03-16T09:01:21.905 INFO:tasks.workunit.client.1.smithi019.stdout:0/793: unlink d7/d8/dec/def/d92/da6/fd5 0 2024-03-16T09:01:21.906 INFO:tasks.workunit.client.0.smithi007.stdout:0/993: creat d3/d6c/dbd/dd7/d7c/dec/f145 x:0 0 0 2024-03-16T09:01:21.907 INFO:tasks.workunit.client.1.smithi019.stdout:0/794: creat d7/d8/dec/def/da1/ddf/f106 x:0 0 0 2024-03-16T09:01:21.907 INFO:tasks.workunit.client.1.smithi019.stdout:0/795: truncate d7/d8/dec/def/da1/da7/fae 813955 0 2024-03-16T09:01:21.910 INFO:tasks.workunit.client.1.smithi019.stdout:0/796: mkdir d7/d8/dec/def/d6c/d87/d107 0 2024-03-16T09:01:21.911 INFO:tasks.workunit.client.0.smithi007.stdout:0/994: link d3/d9/da/d18/d23/f81 d3/d9/da/d48/d25/d87/f146 0 2024-03-16T09:01:21.911 INFO:tasks.workunit.client.0.smithi007.stdout:0/995: chown d3/d9/da/d18/d27/fbc 0 1 2024-03-16T09:01:21.915 INFO:tasks.workunit.client.1.smithi019.stdout:9/600: dread d1/d7/de/d24/d5b/f6e [0,4194304] 0 2024-03-16T09:01:21.916 INFO:tasks.workunit.client.1.smithi019.stdout:9/601: mknod d1/d7/d33/cd2 0 2024-03-16T09:01:21.918 INFO:tasks.workunit.client.1.smithi019.stdout:9/602: link d1/d7/de/d24/d44/d60/f97 d1/d7/d67/d77/fd3 0 2024-03-16T09:01:21.921 INFO:tasks.workunit.client.1.smithi019.stdout:3/707: truncate d3/d11/f69 1980719 0 2024-03-16T09:01:21.922 INFO:tasks.workunit.client.1.smithi019.stdout:3/708: unlink d3/d1b/d88/ld8 0 2024-03-16T09:01:21.922 INFO:tasks.workunit.client.1.smithi019.stdout:3/709: fsync d3/d8b/dae/f72 0 2024-03-16T09:01:21.923 INFO:tasks.workunit.client.1.smithi019.stdout:3/710: creat d3/d8b/dae/d5a/d75/fde x:0 0 0 2024-03-16T09:01:21.923 INFO:tasks.workunit.client.0.smithi007.stdout:9/840: getdents d24/d4c/d86/daa/d7f 0 2024-03-16T09:01:21.924 INFO:tasks.workunit.client.1.smithi019.stdout:3/711: creat d3/d8b/dae/d5a/d75/d7a/fdf x:0 0 0 2024-03-16T09:01:21.924 INFO:tasks.workunit.client.1.smithi019.stdout:3/712: readlink d3/d1b/l1d 0 2024-03-16T09:01:21.925 INFO:tasks.workunit.client.1.smithi019.stdout:3/713: fsync d3/d8b/dae/d6a/fb9 0 2024-03-16T09:01:21.925 INFO:tasks.workunit.client.0.smithi007.stdout:9/841: mknod d24/d4c/d86/daa/d95/cf8 0 2024-03-16T09:01:21.925 INFO:tasks.workunit.client.1.smithi019.stdout:3/714: dread d3/d8b/dae/d5a/fbb [0,4194304] 0 2024-03-16T09:01:21.926 INFO:tasks.workunit.client.0.smithi007.stdout:9/842: creat d24/d4a/d96/db9/db1/dbb/ff9 x:0 0 0 2024-03-16T09:01:21.927 INFO:tasks.workunit.client.0.smithi007.stdout:9/843: rmdir d24/d4c/d86/daa/d7f/d80/d87 39 2024-03-16T09:01:21.930 INFO:tasks.workunit.client.0.smithi007.stdout:9/844: dread d24/d4a/d96/db9/fa3 [0,4194304] 0 2024-03-16T09:01:21.931 INFO:tasks.workunit.client.0.smithi007.stdout:9/845: dread d24/d4c/d86/daa/d7f/d8f/db5/fc9 [0,4194304] 0 2024-03-16T09:01:21.933 INFO:tasks.workunit.client.0.smithi007.stdout:9/846: link d24/d4c/ce0 d24/d4a/d55/dbf/de9/dec/cfa 0 2024-03-16T09:01:21.933 INFO:tasks.workunit.client.0.smithi007.stdout:9/847: read d24/d4c/f63 [502958,80725] 0 2024-03-16T09:01:21.933 INFO:tasks.workunit.client.1.smithi019.stdout:9/603: write d1/d7/de/d24/f42 [858306,49013] 0 2024-03-16T09:01:21.933 INFO:tasks.workunit.client.1.smithi019.stdout:9/604: chown d1/d7/de/d24/d5b/f6c 49302306 1 2024-03-16T09:01:21.934 INFO:tasks.workunit.client.0.smithi007.stdout:9/848: mknod d24/d4c/d86/daa/d7f/d8f/dd0/cfb 0 2024-03-16T09:01:21.934 INFO:tasks.workunit.client.0.smithi007.stdout:9/849: symlink d24/d4a/d55/dbf/de9/lfc 0 2024-03-16T09:01:21.935 INFO:tasks.workunit.client.1.smithi019.stdout:9/605: link d1/d7/d33/d59/f85 d1/d7/de/fd4 0 2024-03-16T09:01:21.935 INFO:tasks.workunit.client.1.smithi019.stdout:9/606: chown d1/d7/d67/c6f 13082021 1 2024-03-16T09:01:21.935 INFO:tasks.workunit.client.1.smithi019.stdout:9/607: mknod d1/d7/d21/da2/cd5 0 2024-03-16T09:01:21.935 INFO:tasks.workunit.client.1.smithi019.stdout:9/608: dread - d1/d7/de/da3/fcb zero size 2024-03-16T09:01:21.936 INFO:tasks.workunit.client.1.smithi019.stdout:9/609: read - d1/d98/fc5 zero size 2024-03-16T09:01:21.936 INFO:tasks.workunit.client.0.smithi007.stdout:9/850: dread d24/d4a/da2/db7/fc3 [0,4194304] 0 2024-03-16T09:01:21.936 INFO:tasks.workunit.client.1.smithi019.stdout:9/610: unlink d1/d7/d33/d59/f73 0 2024-03-16T09:01:21.937 INFO:tasks.workunit.client.0.smithi007.stdout:9/851: link d24/d4c/d86/daa/d7f/d80/d87/d89/ce7 d24/d4a/d55/dbf/de9/dec/cfd 0 2024-03-16T09:01:21.937 INFO:tasks.workunit.client.0.smithi007.stdout:9/852: stat d24/d4c/d86/daa/d7f/d80/d87/dda 0 2024-03-16T09:01:21.938 INFO:tasks.workunit.client.0.smithi007.stdout:9/853: dread - d24/d8e/fa7 zero size 2024-03-16T09:01:21.945 INFO:tasks.workunit.client.1.smithi019.stdout:9/611: dread d1/d92/fc1 [0,4194304] 0 2024-03-16T09:01:21.945 INFO:tasks.workunit.client.1.smithi019.stdout:9/612: chown d1/d98/dae/dc3 88 1 2024-03-16T09:01:21.945 INFO:tasks.workunit.client.1.smithi019.stdout:9/613: chown d1/d7/f81 348 1 2024-03-16T09:01:21.952 INFO:tasks.workunit.client.0.smithi007.stdout:9/854: dread d24/d4a/d55/f5b [0,4194304] 0 2024-03-16T09:01:21.952 INFO:tasks.workunit.client.0.smithi007.stdout:9/855: creat d24/d4c/d86/daa/d7f/d8f/ffe x:0 0 0 2024-03-16T09:01:21.953 INFO:tasks.workunit.client.1.smithi019.stdout:9/614: dread d1/d7/d67/d77/fd3 [0,4194304] 0 2024-03-16T09:01:21.954 INFO:tasks.workunit.client.0.smithi007.stdout:9/856: mknod d24/d4a/cff 0 2024-03-16T09:01:21.954 INFO:tasks.workunit.client.1.smithi019.stdout:9/615: dread d1/d7/de/fd4 [0,4194304] 0 2024-03-16T09:01:21.955 INFO:tasks.workunit.client.0.smithi007.stdout:9/857: rename d24/d8e/ff1 to d24/d4a/d96/db9/dca/ddc/de8/f100 0 2024-03-16T09:01:21.955 INFO:tasks.workunit.client.0.smithi007.stdout:9/858: stat d24/d4a/da2/dd5/le6 0 2024-03-16T09:01:21.956 INFO:tasks.workunit.client.0.smithi007.stdout:9/859: creat d24/d4a/da2/db7/f101 x:0 0 0 2024-03-16T09:01:21.956 INFO:tasks.workunit.client.0.smithi007.stdout:9/860: write d24/f2d [3439411,86389] 0 2024-03-16T09:01:21.957 INFO:tasks.workunit.client.0.smithi007.stdout:9/861: truncate d24/d4a/d96/db9/db1/fce 2912624 0 2024-03-16T09:01:21.960 INFO:tasks.workunit.client.1.smithi019.stdout:4/680: sync 2024-03-16T09:01:21.960 INFO:tasks.workunit.client.1.smithi019.stdout:5/745: sync 2024-03-16T09:01:21.960 INFO:tasks.workunit.client.1.smithi019.stdout:5/746: read - d6/dd/d47/f86 zero size 2024-03-16T09:01:21.960 INFO:tasks.workunit.client.1.smithi019.stdout:4/681: read d4/d6/d15/d1f/fb8 [3110876,91685] 0 2024-03-16T09:01:21.965 INFO:tasks.workunit.client.0.smithi007.stdout:9/862: dread d24/d4c/d86/daa/d7f/d80/d87/d89/fbc [0,4194304] 0 2024-03-16T09:01:21.970 INFO:tasks.workunit.client.0.smithi007.stdout:6/811: dwrite d2/d13/d24/fec [4194304,4194304] 0 2024-03-16T09:01:21.971 INFO:tasks.workunit.client.1.smithi019.stdout:1/667: dwrite d16/d31/d71/f8d [0,4194304] 0 2024-03-16T09:01:21.971 INFO:tasks.workunit.client.1.smithi019.stdout:1/668: chown d16/d31/d76/d92/d9f/daf/db2/d78 367 1 2024-03-16T09:01:21.971 INFO:tasks.workunit.client.1.smithi019.stdout:2/811: dwrite d5/de/da2/fc1 [0,4194304] 0 2024-03-16T09:01:21.971 INFO:tasks.workunit.client.1.smithi019.stdout:2/812: fdatasync d5/ff 0 2024-03-16T09:01:21.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/871: dwrite d15/f2c [8388608,4194304] 0 2024-03-16T09:01:21.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/872: write d15/d18/d1f/d29/f5b [1004308,76193] 0 2024-03-16T09:01:21.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/873: write d15/d18/f10f [172949,92690] 0 2024-03-16T09:01:21.971 INFO:tasks.workunit.client.0.smithi007.stdout:3/874: stat d15/d18/d1f/d29/d43 0 2024-03-16T09:01:21.972 INFO:tasks.workunit.client.1.smithi019.stdout:8/850: dwrite d6/d2a/d45/d57/d72/d79/db7/fe7 [0,4194304] 0 2024-03-16T09:01:21.972 INFO:tasks.workunit.client.1.smithi019.stdout:8/851: truncate d6/d2a/d68/f86 708181 0 2024-03-16T09:01:21.973 INFO:tasks.workunit.client.0.smithi007.stdout:0/996: dwrite d3/d9/da/d48/d4b/f52 [0,4194304] 0 2024-03-16T09:01:21.974 INFO:tasks.workunit.client.0.smithi007.stdout:1/736: dwrite d9/dd/d1b/d2d/d30/d45/f5c [0,4194304] 0 2024-03-16T09:01:21.974 INFO:tasks.workunit.client.0.smithi007.stdout:1/737: read f0 [5268061,102691] 0 2024-03-16T09:01:21.974 INFO:tasks.workunit.client.1.smithi019.stdout:1/669: creat d16/d31/d76/fc2 x:0 0 0 2024-03-16T09:01:21.974 INFO:tasks.workunit.client.1.smithi019.stdout:7/683: dwrite d9/da/d13/f17 [0,4194304] 0 2024-03-16T09:01:21.976 INFO:tasks.workunit.client.1.smithi019.stdout:1/670: creat d16/d31/d76/d92/d9f/fc3 x:0 0 0 2024-03-16T09:01:21.977 INFO:tasks.workunit.client.0.smithi007.stdout:6/812: rename d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/f63 to d2/d13/d3f/d50/d72/f10b 0 2024-03-16T09:01:21.977 INFO:tasks.workunit.client.0.smithi007.stdout:6/813: write d2/f90 [722712,28279] 0 2024-03-16T09:01:21.978 INFO:tasks.workunit.client.0.smithi007.stdout:2/975: dwrite dc/d27/f8f [0,4194304] 0 2024-03-16T09:01:21.978 INFO:tasks.workunit.client.0.smithi007.stdout:2/976: write dc/df/d24/d28/d4c/f64 [735917,34500] 0 2024-03-16T09:01:21.981 INFO:tasks.workunit.client.1.smithi019.stdout:2/813: mkdir d5/d6/d104/df7/d6f/da6/d83/d11c 0 2024-03-16T09:01:21.982 INFO:tasks.workunit.client.0.smithi007.stdout:3/875: creat d15/d18/d1f/dce/dd6/ddf/f111 x:0 0 0 2024-03-16T09:01:21.985 INFO:tasks.workunit.client.1.smithi019.stdout:1/671: symlink d16/d1f/d3c/d79/db4/lc4 0 2024-03-16T09:01:21.988 INFO:tasks.workunit.client.0.smithi007.stdout:3/876: write d15/d18/d1f/d29/f5b [500360,46658] 0 2024-03-16T09:01:21.989 INFO:tasks.workunit.client.0.smithi007.stdout:0/997: mkdir d3/d6c/dbd/dd7/d7c/dec/d147 0 2024-03-16T09:01:21.989 INFO:tasks.workunit.client.0.smithi007.stdout:0/998: write d3/d6c/dbd/dd7/d7b/f11e [283926,45093] 0 2024-03-16T09:01:21.990 INFO:tasks.workunit.client.0.smithi007.stdout:9/863: dwrite d24/d4c/d86/daa/d7f/d80/f8c [0,4194304] 0 2024-03-16T09:01:21.994 INFO:tasks.workunit.client.1.smithi019.stdout:1/672: creat d16/d1f/d3c/d79/d96/dac/fc5 x:0 0 0 2024-03-16T09:01:21.996 INFO:tasks.workunit.client.1.smithi019.stdout:1/673: mkdir d16/d1f/d3c/d79/d96/dc6 0 2024-03-16T09:01:22.001 INFO:tasks.workunit.client.1.smithi019.stdout:0/797: rename d7/d8/dec/def/d6c/d87 to d7/d8/dec/d108 0 2024-03-16T09:01:22.002 INFO:tasks.workunit.client.1.smithi019.stdout:2/814: symlink d5/d6/d104/df7/d6f/da6/d83/d11c/l11d 0 2024-03-16T09:01:22.003 INFO:tasks.workunit.client.1.smithi019.stdout:2/815: fdatasync d5/d6/f28 0 2024-03-16T09:01:22.003 INFO:tasks.workunit.client.0.smithi007.stdout:5/823: rmdir d1/d7/d1c 39 2024-03-16T09:01:22.004 INFO:tasks.workunit.client.1.smithi019.stdout:0/798: mkdir d7/d8/d32/d4a/d61/d68/dea/d109 0 2024-03-16T09:01:22.007 INFO:tasks.workunit.client.1.smithi019.stdout:2/816: getdents d5/d2e 0 2024-03-16T09:01:22.007 INFO:tasks.workunit.client.1.smithi019.stdout:2/817: chown d5/d6/d2b/d53 1533199822 1 2024-03-16T09:01:22.008 INFO:tasks.workunit.client.1.smithi019.stdout:3/715: rename d3/d8b/dae/d3f/fa4 to d3/d1b/fe0 0 2024-03-16T09:01:22.008 INFO:tasks.workunit.client.1.smithi019.stdout:2/818: truncate d5/d6/d2b/f3c 10790505 0 2024-03-16T09:01:22.010 INFO:tasks.workunit.client.1.smithi019.stdout:0/799: truncate d7/d1d/d2e/f3c 1053992 0 2024-03-16T09:01:22.010 INFO:tasks.workunit.client.1.smithi019.stdout:0/800: chown d7/d8/d32/l3f 703124 1 2024-03-16T09:01:22.012 INFO:tasks.workunit.client.1.smithi019.stdout:1/674: dwrite d16/d18/f1b [0,4194304] 0 2024-03-16T09:01:22.012 INFO:tasks.workunit.client.1.smithi019.stdout:9/616: rename d1/d7/de/da3/lac to d1/d7/dc0/ld6 0 2024-03-16T09:01:22.014 INFO:tasks.workunit.client.1.smithi019.stdout:3/716: creat d3/d8b/dca/fe1 x:0 0 0 2024-03-16T09:01:22.018 INFO:tasks.workunit.client.0.smithi007.stdout:2/977: truncate dc/df/d24/d28/d2e/dc5/fd5 3813831 0 2024-03-16T09:01:22.018 INFO:tasks.workunit.client.1.smithi019.stdout:1/675: rmdir d16/d1f/d3c/d79/d96/dc6 0 2024-03-16T09:01:22.020 INFO:tasks.workunit.client.0.smithi007.stdout:3/877: mknod d15/d18/d1f/d29/d43/d72/c112 0 2024-03-16T09:01:22.020 INFO:tasks.workunit.client.0.smithi007.stdout:3/878: fdatasync d15/d18/d1f/d29/d43/ff9 0 2024-03-16T09:01:22.020 INFO:tasks.workunit.client.0.smithi007.stdout:3/879: read f10 [3104828,7414] 0 2024-03-16T09:01:22.021 INFO:tasks.workunit.client.1.smithi019.stdout:5/747: rename d6/d1f/d78/cbd to d6/dd/d47/d70/db2/d89/cef 0 2024-03-16T09:01:22.022 INFO:tasks.workunit.client.1.smithi019.stdout:5/748: truncate d6/dd/d47/d70/db2/d66/fb6 991229 0 2024-03-16T09:01:22.022 INFO:tasks.workunit.client.0.smithi007.stdout:0/999: mkdir d3/d6c/dbd/dd7/d7c/dec/d147/d148 0 2024-03-16T09:01:22.022 INFO:tasks.workunit.client.1.smithi019.stdout:9/617: unlink d1/d7/de/la8 0 2024-03-16T09:01:22.024 INFO:tasks.workunit.client.1.smithi019.stdout:0/801: getdents d7 0 2024-03-16T09:01:22.024 INFO:tasks.workunit.client.1.smithi019.stdout:0/802: chown d7/d1d/d63 3 1 2024-03-16T09:01:22.024 INFO:tasks.workunit.client.1.smithi019.stdout:0/803: read d7/d1d/f4b [984792,6901] 0 2024-03-16T09:01:22.025 INFO:tasks.workunit.client.0.smithi007.stdout:1/738: dwrite d9/dd/d1b/d28/d29/d7b/f9b [0,4194304] 0 2024-03-16T09:01:22.025 INFO:tasks.workunit.client.0.smithi007.stdout:1/739: chown d9/dd/d1b/d58/f66 25131 1 2024-03-16T09:01:22.025 INFO:tasks.workunit.client.1.smithi019.stdout:1/676: rmdir d16/d31/d76/d92/d9f/daf/db2/d78/dbd 0 2024-03-16T09:01:22.026 INFO:tasks.workunit.client.0.smithi007.stdout:9/864: mkdir d24/d4a/d96/db9/dca/ddc/de8/d102 0 2024-03-16T09:01:22.026 INFO:tasks.workunit.client.0.smithi007.stdout:9/865: fsync d24/d4c/d86/daa/d7f/d8f/db5/fc7 0 2024-03-16T09:01:22.027 INFO:tasks.workunit.client.0.smithi007.stdout:5/824: rmdir d1/d7/d1c/d19/df7/d105 39 2024-03-16T09:01:22.027 INFO:tasks.workunit.client.0.smithi007.stdout:5/825: write d1/d7/d1c/d19/f26 [3032697,116604] 0 2024-03-16T09:01:22.028 INFO:tasks.workunit.client.1.smithi019.stdout:2/819: rmdir d5/d6/d2b/d2f/d35 39 2024-03-16T09:01:22.029 INFO:tasks.workunit.client.1.smithi019.stdout:2/820: fsync d5/d6/d2b/f5b 0 2024-03-16T09:01:22.030 INFO:tasks.workunit.client.1.smithi019.stdout:5/749: symlink d6/d19/d34/dd2/lf0 0 2024-03-16T09:01:22.030 INFO:tasks.workunit.client.1.smithi019.stdout:2/821: dread d5/d5a/f9d [0,4194304] 0 2024-03-16T09:01:22.031 INFO:tasks.workunit.client.1.smithi019.stdout:9/618: creat d1/d7/de/d24/d44/d76/d9c/dcc/fd7 x:0 0 0 2024-03-16T09:01:22.033 INFO:tasks.workunit.client.1.smithi019.stdout:9/619: dread d1/d98/dae/f35 [0,4194304] 0 2024-03-16T09:01:22.033 INFO:tasks.workunit.client.1.smithi019.stdout:4/682: rename d4/d6/d15/c95 to d4/d6/d15/d1f/cdd 0 2024-03-16T09:01:22.033 INFO:tasks.workunit.client.1.smithi019.stdout:4/683: fsync d4/d64/d99/f40 0 2024-03-16T09:01:22.034 INFO:tasks.workunit.client.1.smithi019.stdout:2/822: symlink d5/d6/d2b/d2f/d35/d4b/l11e 0 2024-03-16T09:01:22.035 INFO:tasks.workunit.client.1.smithi019.stdout:3/717: rename d3/d8b/dae/d3f/cc4 to d3/ce2 0 2024-03-16T09:01:22.036 INFO:tasks.workunit.client.0.smithi007.stdout:1/740: creat d9/d6b/fe8 x:0 0 0 2024-03-16T09:01:22.037 INFO:tasks.workunit.client.0.smithi007.stdout:6/814: dwrite d2/d13/d3f/d50/d78/dbe/dcb/f25 [0,4194304] 0 2024-03-16T09:01:22.039 INFO:tasks.workunit.client.1.smithi019.stdout:9/620: rename d1/d7/de/c1c to d1/d7/de/d24/d44/d60/d74/cd8 0 2024-03-16T09:01:22.040 INFO:tasks.workunit.client.1.smithi019.stdout:9/621: chown d1/d7/de/l96 316 1 2024-03-16T09:01:22.041 INFO:tasks.workunit.client.1.smithi019.stdout:3/718: creat d3/d8b/da8/fe3 x:0 0 0 2024-03-16T09:01:22.041 INFO:tasks.workunit.client.0.smithi007.stdout:5/826: symlink d1/d7/d1c/d19/df7/l12c 0 2024-03-16T09:01:22.042 INFO:tasks.workunit.client.1.smithi019.stdout:4/684: getdents d4/d64 0 2024-03-16T09:01:22.045 INFO:tasks.workunit.client.1.smithi019.stdout:3/719: unlink d3/d1b/d88/d91/dbe/cc5 0 2024-03-16T09:01:22.045 INFO:tasks.workunit.client.1.smithi019.stdout:3/720: dread - d3/d8b/da8/fe3 zero size 2024-03-16T09:01:22.047 INFO:tasks.workunit.client.0.smithi007.stdout:6/815: mknod d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/d8f/c10c 0 2024-03-16T09:01:22.047 INFO:tasks.workunit.client.1.smithi019.stdout:4/685: rename d4/d64/d9b/fa1 to d4/d73/fde 0 2024-03-16T09:01:22.049 INFO:tasks.workunit.client.1.smithi019.stdout:4/686: creat d4/d6c/d89/fdf x:0 0 0 2024-03-16T09:01:22.049 INFO:tasks.workunit.client.1.smithi019.stdout:4/687: read - d4/d6/d15/d9a/fb1 zero size 2024-03-16T09:01:22.050 INFO:tasks.workunit.client.0.smithi007.stdout:5/827: creat d1/d7/d1c/d19/d34/d4d/d99/f12d x:0 0 0 2024-03-16T09:01:22.051 INFO:tasks.workunit.client.1.smithi019.stdout:3/721: rename d3/d11/f69 to d3/d8b/dae/d5a/d75/d7a/fe4 0 2024-03-16T09:01:22.053 INFO:tasks.workunit.client.1.smithi019.stdout:4/688: mknod d4/d6/d15/da2/ce0 0 2024-03-16T09:01:22.053 INFO:tasks.workunit.client.0.smithi007.stdout:6/816: unlink d2/fd 0 2024-03-16T09:01:22.055 INFO:tasks.workunit.client.1.smithi019.stdout:3/722: creat d3/d8b/da8/dcc/fe5 x:0 0 0 2024-03-16T09:01:22.055 INFO:tasks.workunit.client.0.smithi007.stdout:5/828: creat d1/d7/d16/d1b/d3c/f12e x:0 0 0 2024-03-16T09:01:22.055 INFO:tasks.workunit.client.0.smithi007.stdout:6/817: mknod d2/c10d 0 2024-03-16T09:01:22.055 INFO:tasks.workunit.client.1.smithi019.stdout:4/689: symlink d4/d1b/d2a/d4f/le1 0 2024-03-16T09:01:22.056 INFO:tasks.workunit.client.1.smithi019.stdout:4/690: fdatasync d4/d6/d15/d1f/fb8 0 2024-03-16T09:01:22.056 INFO:tasks.workunit.client.0.smithi007.stdout:6/818: getdents d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/ddc 0 2024-03-16T09:01:22.057 INFO:tasks.workunit.client.1.smithi019.stdout:4/691: creat d4/d6/d15/fe2 x:0 0 0 2024-03-16T09:01:22.057 INFO:tasks.workunit.client.1.smithi019.stdout:4/692: stat d4/ce 0 2024-03-16T09:01:22.059 INFO:tasks.workunit.client.0.smithi007.stdout:6/819: mknod d2/d10/d3d/d6d/dbf/d103/c10e 0 2024-03-16T09:01:22.060 INFO:tasks.workunit.client.0.smithi007.stdout:6/820: mknod d2/d13/d4b/d94/dea/df7/c10f 0 2024-03-16T09:01:22.061 INFO:tasks.workunit.client.0.smithi007.stdout:6/821: mknod d2/d10/d3d/d6d/dbf/d103/c110 0 2024-03-16T09:01:22.066 INFO:tasks.workunit.client.0.smithi007.stdout:6/822: unlink d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/fc5 0 2024-03-16T09:01:22.066 INFO:tasks.workunit.client.0.smithi007.stdout:6/823: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/f111 x:0 0 0 2024-03-16T09:01:22.069 INFO:tasks.workunit.client.1.smithi019.stdout:8/852: dwrite d6/d2a/d45/d57/d72/d79/fd5 [0,4194304] 0 2024-03-16T09:01:22.069 INFO:tasks.workunit.client.1.smithi019.stdout:8/853: stat d6/d2a/d45/d19/d74/f8a 0 2024-03-16T09:01:22.069 INFO:tasks.workunit.client.1.smithi019.stdout:8/854: truncate d6/d2a/d45/f47 1432267 0 2024-03-16T09:01:22.069 INFO:tasks.workunit.client.1.smithi019.stdout:8/855: fdatasync d6/de/f6a 0 2024-03-16T09:01:22.070 INFO:tasks.workunit.client.1.smithi019.stdout:7/684: dwrite d9/db/fb1 [4194304,4194304] 0 2024-03-16T09:01:22.072 INFO:tasks.workunit.client.1.smithi019.stdout:8/856: creat d6/d2a/d45/d57/d72/d79/da5/d11d/f127 x:0 0 0 2024-03-16T09:01:22.075 INFO:tasks.workunit.client.1.smithi019.stdout:8/857: creat d6/d2a/d45/d57/d72/d79/f128 x:0 0 0 2024-03-16T09:01:22.076 INFO:tasks.workunit.client.0.smithi007.stdout:7/853: sync 2024-03-16T09:01:22.077 INFO:tasks.workunit.client.0.smithi007.stdout:4/747: sync 2024-03-16T09:01:22.077 INFO:tasks.workunit.client.0.smithi007.stdout:4/748: readlink d4/d14/d64/l6c 0 2024-03-16T09:01:22.078 INFO:tasks.workunit.client.1.smithi019.stdout:8/858: dread d6/de/fb2 [0,4194304] 0 2024-03-16T09:01:22.080 INFO:tasks.workunit.client.0.smithi007.stdout:2/978: rename dc/df/d24/d28/d2e/dc5 to dc/df/d24/d28/dea/d102/d146 0 2024-03-16T09:01:22.081 INFO:tasks.workunit.client.1.smithi019.stdout:8/859: truncate d6/d2a/d45/d57/f60 4021779 0 2024-03-16T09:01:22.082 INFO:tasks.workunit.client.1.smithi019.stdout:8/860: read d6/f94 [267718,53465] 0 2024-03-16T09:01:22.083 INFO:tasks.workunit.client.0.smithi007.stdout:2/979: mkdir dc/df/d24/d28/d97/da4/d147 0 2024-03-16T09:01:22.086 INFO:tasks.workunit.client.0.smithi007.stdout:9/866: rename d24/d4a/d9e/fd3 to d24/d4c/d91/f103 0 2024-03-16T09:01:22.087 INFO:tasks.workunit.client.1.smithi019.stdout:8/861: mknod d6/d2a/d45/d2c/de3/c129 0 2024-03-16T09:01:22.087 INFO:tasks.workunit.client.1.smithi019.stdout:8/862: fdatasync d6/d2a/d3f/f7d 0 2024-03-16T09:01:22.090 INFO:tasks.workunit.client.0.smithi007.stdout:2/980: creat dc/df/d24/d28/d4c/ded/f148 x:0 0 0 2024-03-16T09:01:22.090 INFO:tasks.workunit.client.0.smithi007.stdout:9/867: dread d24/d4c/d86/daa/d7f/d80/d87/d89/f8a [0,4194304] 0 2024-03-16T09:01:22.090 INFO:tasks.workunit.client.0.smithi007.stdout:9/868: chown d24/f3f 43305120 1 2024-03-16T09:01:22.091 INFO:tasks.workunit.client.1.smithi019.stdout:8/863: mkdir d6/de/d51/d12a 0 2024-03-16T09:01:22.092 INFO:tasks.workunit.client.0.smithi007.stdout:9/869: creat d24/d4c/d86/daa/d95/f104 x:0 0 0 2024-03-16T09:01:22.093 INFO:tasks.workunit.client.1.smithi019.stdout:1/677: dwrite d16/f2f [4194304,4194304] 0 2024-03-16T09:01:22.093 INFO:tasks.workunit.client.1.smithi019.stdout:9/622: dwrite d1/d7/f51 [0,4194304] 0 2024-03-16T09:01:22.093 INFO:tasks.workunit.client.1.smithi019.stdout:9/623: chown d1/cbe 1506374 1 2024-03-16T09:01:22.094 INFO:tasks.workunit.client.1.smithi019.stdout:9/624: write d1/d7/f68 [4401768,116147] 0 2024-03-16T09:01:22.095 INFO:tasks.workunit.client.1.smithi019.stdout:1/678: symlink d16/d1f/d3c/d79/d96/lc7 0 2024-03-16T09:01:22.096 INFO:tasks.workunit.client.1.smithi019.stdout:1/679: chown d16/d84 11 1 2024-03-16T09:01:22.096 INFO:tasks.workunit.client.1.smithi019.stdout:1/680: fsync d16/f5d 0 2024-03-16T09:01:22.096 INFO:tasks.workunit.client.1.smithi019.stdout:9/625: rmdir d1/d7/d33/dc9/dca 39 2024-03-16T09:01:22.098 INFO:tasks.workunit.client.1.smithi019.stdout:1/681: symlink d16/d31/d71/lc8 0 2024-03-16T09:01:22.098 INFO:tasks.workunit.client.1.smithi019.stdout:1/682: write d16/d31/d71/f75 [470268,123314] 0 2024-03-16T09:01:22.099 INFO:tasks.workunit.client.1.smithi019.stdout:1/683: readlink d16/d31/d76/d92/d9f/daf/db2/d8f/la8 0 2024-03-16T09:01:22.099 INFO:tasks.workunit.client.0.smithi007.stdout:2/981: dread dc/df/d24/d28/d4c/daf/fe1 [0,4194304] 0 2024-03-16T09:01:22.099 INFO:tasks.workunit.client.0.smithi007.stdout:1/741: dwrite d9/f85 [0,4194304] 0 2024-03-16T09:01:22.100 INFO:tasks.workunit.client.1.smithi019.stdout:9/626: mknod d1/d7/de/d24/d44/d76/cd9 0 2024-03-16T09:01:22.100 INFO:tasks.workunit.client.1.smithi019.stdout:9/627: fdatasync d1/d98/dae/f58 0 2024-03-16T09:01:22.101 INFO:tasks.workunit.client.1.smithi019.stdout:3/723: dwrite d3/d8b/dae/d5a/f8c [0,4194304] 0 2024-03-16T09:01:22.101 INFO:tasks.workunit.client.1.smithi019.stdout:1/684: mknod d16/d1f/d3c/d79/db4/cc9 0 2024-03-16T09:01:22.103 INFO:tasks.workunit.client.1.smithi019.stdout:8/864: write d6/d2a/d45/d19/f43 [3274325,2922] 0 2024-03-16T09:01:22.103 INFO:tasks.workunit.client.1.smithi019.stdout:8/865: fsync d6/fd4 0 2024-03-16T09:01:22.103 INFO:tasks.workunit.client.1.smithi019.stdout:9/628: rename d1/d7/la9 to d1/d7/de/d24/d5b/lda 0 2024-03-16T09:01:22.106 INFO:tasks.workunit.client.1.smithi019.stdout:2/823: unlink d5/d6/d104/df7/c81 0 2024-03-16T09:01:22.107 INFO:tasks.workunit.client.1.smithi019.stdout:3/724: link d3/d8b/da8/cb3 d3/d89/ce6 0 2024-03-16T09:01:22.107 INFO:tasks.workunit.client.1.smithi019.stdout:3/725: truncate d3/d8b/dae/d6a/f70 521217 0 2024-03-16T09:01:22.107 INFO:tasks.workunit.client.1.smithi019.stdout:3/726: chown d3/d1b/d40/d4a/db7/lc0 14064 1 2024-03-16T09:01:22.109 INFO:tasks.workunit.client.1.smithi019.stdout:9/629: mkdir d1/d7/d86/ddb 0 2024-03-16T09:01:22.109 INFO:tasks.workunit.client.1.smithi019.stdout:9/630: chown d1/d98/dae/f7a 0 1 2024-03-16T09:01:22.109 INFO:tasks.workunit.client.0.smithi007.stdout:5/829: dwrite d1/d7/d1c/d19/d34/d3e/d41/f71 [0,4194304] 0 2024-03-16T09:01:22.109 INFO:tasks.workunit.client.0.smithi007.stdout:5/830: read - d1/d7/d1c/d19/d34/d1d/f11d zero size 2024-03-16T09:01:22.111 INFO:tasks.workunit.client.1.smithi019.stdout:9/631: chown d1/d7/l3d 2037128 1 2024-03-16T09:01:22.111 INFO:tasks.workunit.client.1.smithi019.stdout:9/632: truncate d1/d7/f56 1043045 0 2024-03-16T09:01:22.111 INFO:tasks.workunit.client.1.smithi019.stdout:9/633: write d1/d7/d21/da2/fb8 [40160,99501] 0 2024-03-16T09:01:22.111 INFO:tasks.workunit.client.1.smithi019.stdout:9/634: chown d1/d7/d33/dc9/dca/cd0 14 1 2024-03-16T09:01:22.111 INFO:tasks.workunit.client.1.smithi019.stdout:9/635: creat d1/d7/de/d24/d44/d76/d9c/dcc/fdc x:0 0 0 2024-03-16T09:01:22.115 INFO:tasks.workunit.client.1.smithi019.stdout:7/685: dwrite d9/db/d12/f55 [0,4194304] 0 2024-03-16T09:01:22.115 INFO:tasks.workunit.client.1.smithi019.stdout:9/636: symlink d1/d7/d67/d77/ldd 0 2024-03-16T09:01:22.115 INFO:tasks.workunit.client.1.smithi019.stdout:9/637: getdents d1/d7/de/d24/d44/d76/d9c/dcf 0 2024-03-16T09:01:22.117 INFO:tasks.workunit.client.0.smithi007.stdout:6/824: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fdb [0,4194304] 0 2024-03-16T09:01:22.118 INFO:tasks.workunit.client.0.smithi007.stdout:7/854: dwrite d0/dc/d5a/fdc [0,4194304] 0 2024-03-16T09:01:22.118 INFO:tasks.workunit.client.0.smithi007.stdout:7/855: stat d0/df5/d10e 0 2024-03-16T09:01:22.118 INFO:tasks.workunit.client.0.smithi007.stdout:5/831: rename d1/d7/d1c/d19/d34/d1d/da8/fba to d1/d7/d1c/d19/df7/f12f 0 2024-03-16T09:01:22.121 INFO:tasks.workunit.client.1.smithi019.stdout:7/686: rmdir d9/da/d13/d18 39 2024-03-16T09:01:22.122 INFO:tasks.workunit.client.1.smithi019.stdout:7/687: dread d9/db/d12/d5b/f65 [0,4194304] 0 2024-03-16T09:01:22.124 INFO:tasks.workunit.client.1.smithi019.stdout:9/638: mknod d1/d7/dc0/cde 0 2024-03-16T09:01:22.124 INFO:tasks.workunit.client.1.smithi019.stdout:9/639: chown d1/d7/de/d24/d5b/lda 106 1 2024-03-16T09:01:22.124 INFO:tasks.workunit.client.0.smithi007.stdout:9/870: write d24/f34 [1732017,117886] 0 2024-03-16T09:01:22.126 INFO:tasks.workunit.client.0.smithi007.stdout:4/749: dwrite d4/d14/d64/d68/fc4 [0,4194304] 0 2024-03-16T09:01:22.126 INFO:tasks.workunit.client.0.smithi007.stdout:4/750: fsync d4/d14/d64/d68/d82/dac/dfc/d8f/da8/dbe/fef 0 2024-03-16T09:01:22.127 INFO:tasks.workunit.client.1.smithi019.stdout:9/640: creat d1/d98/dae/dc3/fdf x:0 0 0 2024-03-16T09:01:22.128 INFO:tasks.workunit.client.0.smithi007.stdout:5/832: link d1/d7/d1c/d19/d34/de/f49 d1/d7/d1c/d19/df7/f130 0 2024-03-16T09:01:22.129 INFO:tasks.workunit.client.0.smithi007.stdout:5/833: chown d1/d7/cb9 25743531 1 2024-03-16T09:01:22.129 INFO:tasks.workunit.client.0.smithi007.stdout:5/834: read - d1/d7/d1c/d19/df7/d110/f124 zero size 2024-03-16T09:01:22.135 INFO:tasks.workunit.client.1.smithi019.stdout:5/750: getdents d6/d19/d34/dd2 0 2024-03-16T09:01:22.136 INFO:tasks.workunit.client.1.smithi019.stdout:5/751: mkdir d6/dd/dde/df1 0 2024-03-16T09:01:22.138 INFO:tasks.workunit.client.0.smithi007.stdout:4/751: link d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/fde d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/f100 0 2024-03-16T09:01:22.138 INFO:tasks.workunit.client.0.smithi007.stdout:4/752: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/cd1 28 1 2024-03-16T09:01:22.143 INFO:tasks.workunit.client.0.smithi007.stdout:5/835: creat d1/d7/d1c/d19/d115/f131 x:0 0 0 2024-03-16T09:01:22.146 INFO:tasks.workunit.client.0.smithi007.stdout:9/871: rename d24/d4c/d86/daa/f88 to d24/d4c/f105 0 2024-03-16T09:01:22.147 INFO:tasks.workunit.client.0.smithi007.stdout:9/872: stat d24/d4c/d86/daa 0 2024-03-16T09:01:22.147 INFO:tasks.workunit.client.0.smithi007.stdout:9/873: chown d24/d4c/d86/c8b 1769 1 2024-03-16T09:01:22.147 INFO:tasks.workunit.client.0.smithi007.stdout:9/874: fdatasync d24/d4a/da2/db7/fc3 0 2024-03-16T09:01:22.147 INFO:tasks.workunit.client.0.smithi007.stdout:9/875: readlink d24/d4c/d86/daa/d7f/d80/l83 0 2024-03-16T09:01:22.147 INFO:tasks.workunit.client.0.smithi007.stdout:4/753: truncate d4/d14/d64/f88 331377 0 2024-03-16T09:01:22.150 INFO:tasks.workunit.client.0.smithi007.stdout:5/836: mkdir d1/d7/d1c/d19/d34/d5e/d6f/d132 0 2024-03-16T09:01:22.150 INFO:tasks.workunit.client.0.smithi007.stdout:5/837: readlink d1/d7/d1c/d88/da2/dae/lbb 0 2024-03-16T09:01:22.151 INFO:tasks.workunit.client.0.smithi007.stdout:5/838: stat d1/d7/d1c/d88/da2/d89/db3/f3d 0 2024-03-16T09:01:22.151 INFO:tasks.workunit.client.0.smithi007.stdout:5/839: chown d1/d7/d1c/d19/df7/d105/f123 640849 1 2024-03-16T09:01:22.156 INFO:tasks.workunit.client.0.smithi007.stdout:5/840: mkdir d1/d7/d1c/d88/da2/dae/df3/d133 0 2024-03-16T09:01:22.156 INFO:tasks.workunit.client.0.smithi007.stdout:5/841: read d1/d7/d1c/f15 [2190523,75813] 0 2024-03-16T09:01:22.157 INFO:tasks.workunit.client.1.smithi019.stdout:3/727: dwrite d3/d8b/da8/dcc/fe5 [0,4194304] 0 2024-03-16T09:01:22.157 INFO:tasks.workunit.client.0.smithi007.stdout:2/982: dwrite dc/df/d24/d28/d2e/f45 [0,4194304] 0 2024-03-16T09:01:22.157 INFO:tasks.workunit.client.0.smithi007.stdout:2/983: fdatasync dc/df/d24/d28/d2e/d35/f98 0 2024-03-16T09:01:22.160 INFO:tasks.workunit.client.0.smithi007.stdout:2/984: mkdir dc/df/d24/d28/d4c/d5e/d149 0 2024-03-16T09:01:22.160 INFO:tasks.workunit.client.1.smithi019.stdout:3/728: link d3/d1b/f9e d3/d8b/dae/d5a/d80/fe7 0 2024-03-16T09:01:22.163 INFO:tasks.workunit.client.0.smithi007.stdout:2/985: unlink dc/f3a 0 2024-03-16T09:01:22.163 INFO:tasks.workunit.client.0.smithi007.stdout:2/986: readlink dc/l26 0 2024-03-16T09:01:22.164 INFO:tasks.workunit.client.0.smithi007.stdout:2/987: rmdir dc/df/d24/d28/d97/da4/db8 39 2024-03-16T09:01:22.164 INFO:tasks.workunit.client.0.smithi007.stdout:2/988: truncate dc/df/d24/d28/d2e/f45 4735274 0 2024-03-16T09:01:22.166 INFO:tasks.workunit.client.0.smithi007.stdout:2/989: creat dc/df/d24/d28/d4c/dbc/f14a x:0 0 0 2024-03-16T09:01:22.176 INFO:tasks.workunit.client.0.smithi007.stdout:5/842: dread d1/d7/d1c/d19/d34/d4d/d99/fd7 [0,4194304] 0 2024-03-16T09:01:22.178 INFO:tasks.workunit.client.0.smithi007.stdout:5/843: rename d1/d7/d1c/d19/d34/d3e/cb5 to d1/d7/d1c/d88/da2/df4/c134 0 2024-03-16T09:01:22.178 INFO:tasks.workunit.client.0.smithi007.stdout:5/844: truncate d1/d7/d16/ffa 512261 0 2024-03-16T09:01:22.178 INFO:tasks.workunit.client.0.smithi007.stdout:5/845: readlink d1/d7/d1c/d19/d34/d3e/d41/d48/lc4 0 2024-03-16T09:01:22.178 INFO:tasks.workunit.client.0.smithi007.stdout:1/742: dwrite d9/dd/d1b/f49 [0,4194304] 0 2024-03-16T09:01:22.179 INFO:tasks.workunit.client.0.smithi007.stdout:5/846: mkdir d1/d7/d1c/d19/d34/d5e/d6f/d72/d135 0 2024-03-16T09:01:22.179 INFO:tasks.workunit.client.0.smithi007.stdout:7/856: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/db3/fee [0,4194304] 0 2024-03-16T09:01:22.180 INFO:tasks.workunit.client.1.smithi019.stdout:2/824: dwrite d5/d6/d104/dba/dfc/d74/f95 [0,4194304] 0 2024-03-16T09:01:22.180 INFO:tasks.workunit.client.0.smithi007.stdout:6/825: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/fc0 [0,4194304] 0 2024-03-16T09:01:22.183 INFO:tasks.workunit.client.0.smithi007.stdout:1/743: symlink d9/dd/d1b/d2d/d30/le9 0 2024-03-16T09:01:22.183 INFO:tasks.workunit.client.0.smithi007.stdout:9/876: dwrite d24/d4a/fcc [0,4194304] 0 2024-03-16T09:01:22.184 INFO:tasks.workunit.client.0.smithi007.stdout:4/754: dwrite d4/d14/d64/d68/f56 [0,4194304] 0 2024-03-16T09:01:22.185 INFO:tasks.workunit.client.1.smithi019.stdout:9/641: getdents d1/d7/d86 0 2024-03-16T09:01:22.185 INFO:tasks.workunit.client.1.smithi019.stdout:9/642: creat d1/d7/d67/d77/fe0 x:0 0 0 2024-03-16T09:01:22.186 INFO:tasks.workunit.client.1.smithi019.stdout:9/643: read d1/d7/f50 [147943,99607] 0 2024-03-16T09:01:22.186 INFO:tasks.workunit.client.1.smithi019.stdout:9/644: dread - d1/d7/de/d24/d44/d76/d9c/dcc/fd7 zero size 2024-03-16T09:01:22.186 INFO:tasks.workunit.client.1.smithi019.stdout:9/645: write d1/d7/de/d24/d44/d60/f9f [414752,9742] 0 2024-03-16T09:01:22.186 INFO:tasks.workunit.client.1.smithi019.stdout:9/646: write d1/d7/de/d24/d44/d76/d9c/dcc/fd7 [738252,98429] 0 2024-03-16T09:01:22.186 INFO:tasks.workunit.client.0.smithi007.stdout:2/990: dwrite dc/df/d24/ff0 [0,4194304] 0 2024-03-16T09:01:22.187 INFO:tasks.workunit.client.0.smithi007.stdout:7/857: creat d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f12c x:0 0 0 2024-03-16T09:01:22.189 INFO:tasks.workunit.client.1.smithi019.stdout:5/752: dwrite d6/d19/f7f [0,4194304] 0 2024-03-16T09:01:22.189 INFO:tasks.workunit.client.1.smithi019.stdout:7/688: dwrite d9/da/d13/d18/fac [0,4194304] 0 2024-03-16T09:01:22.192 INFO:tasks.workunit.client.1.smithi019.stdout:9/647: rmdir d1/d7/de/d24/d44/d60 39 2024-03-16T09:01:22.192 INFO:tasks.workunit.client.0.smithi007.stdout:1/744: mknod d9/dd/cea 0 2024-03-16T09:01:22.194 INFO:tasks.workunit.client.1.smithi019.stdout:2/825: symlink d5/d6/d2b/l11f 0 2024-03-16T09:01:22.194 INFO:tasks.workunit.client.1.smithi019.stdout:2/826: stat d5/d6/d104/dba/dfc 0 2024-03-16T09:01:22.194 INFO:tasks.workunit.client.1.smithi019.stdout:2/827: stat d5/d6/d2b/d2f/d35/l8e 0 2024-03-16T09:01:22.194 INFO:tasks.workunit.client.1.smithi019.stdout:2/828: fsync d5/d2e/d4f/fe1 0 2024-03-16T09:01:22.197 INFO:tasks.workunit.client.1.smithi019.stdout:5/753: mknod d6/dd/d37/cf2 0 2024-03-16T09:01:22.197 INFO:tasks.workunit.client.1.smithi019.stdout:5/754: truncate d6/dd/d47/d70/f97 572406 0 2024-03-16T09:01:22.197 INFO:tasks.workunit.client.1.smithi019.stdout:5/755: write d6/d1f/f5a [4696173,46247] 0 2024-03-16T09:01:22.197 INFO:tasks.workunit.client.0.smithi007.stdout:5/847: dwrite d1/d7/d1c/d19/d34/d3e/d41/f4c [4194304,4194304] 0 2024-03-16T09:01:22.197 INFO:tasks.workunit.client.0.smithi007.stdout:1/745: dread d9/fca [0,4194304] 0 2024-03-16T09:01:22.198 INFO:tasks.workunit.client.1.smithi019.stdout:2/829: mknod d5/d6/d104/dba/c120 0 2024-03-16T09:01:22.198 INFO:tasks.workunit.client.0.smithi007.stdout:7/858: write d0/d19/d1f/f116 [2311691,129093] 0 2024-03-16T09:01:22.199 INFO:tasks.workunit.client.1.smithi019.stdout:7/689: symlink d9/da/ldc 0 2024-03-16T09:01:22.199 INFO:tasks.workunit.client.1.smithi019.stdout:7/690: read - d9/db/d12/fc4 zero size 2024-03-16T09:01:22.200 INFO:tasks.workunit.client.1.smithi019.stdout:9/648: mknod d1/d7/de/d24/ce1 0 2024-03-16T09:01:22.200 INFO:tasks.workunit.client.0.smithi007.stdout:9/877: creat d24/d4a/d96/f106 x:0 0 0 2024-03-16T09:01:22.208 INFO:tasks.workunit.client.0.smithi007.stdout:4/755: rename d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/cae to d4/d14/d1c/c101 0 2024-03-16T09:01:22.209 INFO:tasks.workunit.client.0.smithi007.stdout:1/746: dread d9/dd/d1b/d28/f46 [0,4194304] 0 2024-03-16T09:01:22.209 INFO:tasks.workunit.client.1.smithi019.stdout:9/649: mkdir d1/d7/dc0/de2 0 2024-03-16T09:01:22.210 INFO:tasks.workunit.client.1.smithi019.stdout:5/756: rmdir d6/dd/d47/d70/db2/d89/dcc 39 2024-03-16T09:01:22.211 INFO:tasks.workunit.client.0.smithi007.stdout:2/991: creat dc/df/d24/d28/d2e/d6e/f14b x:0 0 0 2024-03-16T09:01:22.212 INFO:tasks.workunit.client.1.smithi019.stdout:3/729: fsync d3/d8b/dae/d5a/d80/fe7 0 2024-03-16T09:01:22.212 INFO:tasks.workunit.client.1.smithi019.stdout:3/730: chown d3/d11/l39 8639 1 2024-03-16T09:01:22.212 INFO:tasks.workunit.client.1.smithi019.stdout:3/731: read - d3/d9b/fb6 zero size 2024-03-16T09:01:22.212 INFO:tasks.workunit.client.1.smithi019.stdout:3/732: chown d3/d8b/dca/fe1 472441191 1 2024-03-16T09:01:22.212 INFO:tasks.workunit.client.1.smithi019.stdout:3/733: truncate d3/d1b/fdb 562383 0 2024-03-16T09:01:22.212 INFO:tasks.workunit.client.1.smithi019.stdout:3/734: write d3/d8b/fa3 [349100,22401] 0 2024-03-16T09:01:22.214 INFO:tasks.workunit.client.1.smithi019.stdout:9/650: mkdir d1/d7/de/d24/d44/d60/d74/de3 0 2024-03-16T09:01:22.214 INFO:tasks.workunit.client.1.smithi019.stdout:9/651: write d1/d7/de/d24/f99 [286544,108757] 0 2024-03-16T09:01:22.216 INFO:tasks.workunit.client.0.smithi007.stdout:4/756: dread d4/d14/d64/d68/d82/dac/dfc/d8f/da8/fe [0,4194304] 0 2024-03-16T09:01:22.217 INFO:tasks.workunit.client.1.smithi019.stdout:5/757: mkdir d6/dd/df3 0 2024-03-16T09:01:22.220 INFO:tasks.workunit.client.1.smithi019.stdout:5/758: dread d6/dd/d37/d76/fb7 [0,4194304] 0 2024-03-16T09:01:22.220 INFO:tasks.workunit.client.0.smithi007.stdout:6/826: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/fcf [0,4194304] 0 2024-03-16T09:01:22.220 INFO:tasks.workunit.client.0.smithi007.stdout:6/827: chown d2 14224465 1 2024-03-16T09:01:22.221 INFO:tasks.workunit.client.0.smithi007.stdout:6/828: write d2/d82/fa2 [29053,3311] 0 2024-03-16T09:01:22.221 INFO:tasks.workunit.client.0.smithi007.stdout:6/829: truncate d2/d10/f70 1515859 0 2024-03-16T09:01:22.222 INFO:tasks.workunit.client.1.smithi019.stdout:2/830: dread d5/d6/d104/df7/d6f/da6/d83/fbd [0,4194304] 0 2024-03-16T09:01:22.222 INFO:tasks.workunit.client.1.smithi019.stdout:2/831: chown d5/d5a/c92 650 1 2024-03-16T09:01:22.224 INFO:tasks.workunit.client.1.smithi019.stdout:7/691: dwrite d9/db/d12/d5b/fb4 [4194304,4194304] 0 2024-03-16T09:01:22.226 INFO:tasks.workunit.client.1.smithi019.stdout:2/832: getdents d5/d6/d104/dba/de8 0 2024-03-16T09:01:22.226 INFO:tasks.workunit.client.0.smithi007.stdout:2/992: write dc/d27/f4f [3854779,127537] 0 2024-03-16T09:01:22.226 INFO:tasks.workunit.client.0.smithi007.stdout:2/993: dread - dc/df/d121/fb6 zero size 2024-03-16T09:01:22.227 INFO:tasks.workunit.client.1.smithi019.stdout:6/822: sync 2024-03-16T09:01:22.228 INFO:tasks.workunit.client.1.smithi019.stdout:6/823: write de/f49 [4972705,27855] 0 2024-03-16T09:01:22.229 INFO:tasks.workunit.client.1.smithi019.stdout:2/833: truncate d5/d6/fb0 1549919 0 2024-03-16T09:01:22.229 INFO:tasks.workunit.client.1.smithi019.stdout:2/834: readlink d5/d2e/l8f 0 2024-03-16T09:01:22.231 INFO:tasks.workunit.client.0.smithi007.stdout:9/878: dwrite d24/d4c/d86/daa/d95/f6f [0,4194304] 0 2024-03-16T09:01:22.231 INFO:tasks.workunit.client.1.smithi019.stdout:7/692: rename d9/da/d13/d18/d1a to d9/d25/d28/d48/ddd 0 2024-03-16T09:01:22.233 INFO:tasks.workunit.client.0.smithi007.stdout:5/848: symlink d1/d7/d1c/d19/df7/d105/l136 0 2024-03-16T09:01:22.236 INFO:tasks.workunit.client.1.smithi019.stdout:7/693: link d9/d25/f54 d9/d25/d28/d48/d8d/dba/dad/dd8/fde 0 2024-03-16T09:01:22.239 INFO:tasks.workunit.client.0.smithi007.stdout:7/859: rename d0/d19/d1f/d69/f6b to d0/dc/d5a/dba/dd9/f12d 0 2024-03-16T09:01:22.241 INFO:tasks.workunit.client.1.smithi019.stdout:2/835: unlink d5/d6/d2b/l68 0 2024-03-16T09:01:22.241 INFO:tasks.workunit.client.1.smithi019.stdout:2/836: dread - d5/d5a/d80/f118 zero size 2024-03-16T09:01:22.242 INFO:tasks.workunit.client.0.smithi007.stdout:5/849: dread d1/d7/d1c/d19/d34/d5e/d6f/f51 [0,4194304] 0 2024-03-16T09:01:22.244 INFO:tasks.workunit.client.0.smithi007.stdout:1/747: creat d9/dd/d1b/d28/d4c/feb x:0 0 0 2024-03-16T09:01:22.244 INFO:tasks.workunit.client.0.smithi007.stdout:1/748: fsync d9/dd/d1b/d28/d29/fab 0 2024-03-16T09:01:22.244 INFO:tasks.workunit.client.0.smithi007.stdout:1/749: write d9/f35 [2788591,46877] 0 2024-03-16T09:01:22.246 INFO:tasks.workunit.client.1.smithi019.stdout:6/824: write de/d26/d32/d97/d66/d72/f82 [2870824,21252] 0 2024-03-16T09:01:22.247 INFO:tasks.workunit.client.0.smithi007.stdout:4/757: rmdir d4/d14/d64 39 2024-03-16T09:01:22.247 INFO:tasks.workunit.client.0.smithi007.stdout:4/758: write d4/d14/d64/d68/d82/dac/dfc/fb1 [849681,53740] 0 2024-03-16T09:01:22.250 INFO:tasks.workunit.client.1.smithi019.stdout:6/825: rmdir de/d1c/d9a/d9e/df0 0 2024-03-16T09:01:22.250 INFO:tasks.workunit.client.0.smithi007.stdout:2/994: symlink dc/df/d6c/l14c 0 2024-03-16T09:01:22.251 INFO:tasks.workunit.client.0.smithi007.stdout:2/995: chown dc/df/d24/d28/d2e/d35/f98 1936 1 2024-03-16T09:01:22.251 INFO:tasks.workunit.client.0.smithi007.stdout:1/750: dread d9/dd/d1b/f2c [0,4194304] 0 2024-03-16T09:01:22.251 INFO:tasks.workunit.client.0.smithi007.stdout:9/879: rmdir d24/d4c 39 2024-03-16T09:01:22.251 INFO:tasks.workunit.client.0.smithi007.stdout:9/880: write d24/d4a/da2/db7/fc3 [471313,74852] 0 2024-03-16T09:01:22.252 INFO:tasks.workunit.client.1.smithi019.stdout:7/694: dread d9/da/d13/fb8 [0,4194304] 0 2024-03-16T09:01:22.252 INFO:tasks.workunit.client.1.smithi019.stdout:7/695: chown d9/da/d13/d18/c79 43518 1 2024-03-16T09:01:22.252 INFO:tasks.workunit.client.1.smithi019.stdout:6/826: mkdir de/d26/d32/d97/d66/ddc/d107 0 2024-03-16T09:01:22.252 INFO:tasks.workunit.client.1.smithi019.stdout:7/696: symlink d9/d25/d28/d48/ddd/ldf 0 2024-03-16T09:01:22.252 INFO:tasks.workunit.client.1.smithi019.stdout:7/697: chown d9/d25/d28/d48/l95 713915 1 2024-03-16T09:01:22.253 INFO:tasks.workunit.client.1.smithi019.stdout:7/698: dread - d9/d25/d28/d48/f76 zero size 2024-03-16T09:01:22.253 INFO:tasks.workunit.client.1.smithi019.stdout:7/699: truncate d9/d25/d28/f88 5111768 0 2024-03-16T09:01:22.253 INFO:tasks.workunit.client.0.smithi007.stdout:5/850: rename d1/d7/d1c/d88/da2/d89/db3/f94 to d1/d7/d16/f137 0 2024-03-16T09:01:22.254 INFO:tasks.workunit.client.0.smithi007.stdout:5/851: readlink d1/d7/d1c/d88/da2/d89/db3/l65 0 2024-03-16T09:01:22.255 INFO:tasks.workunit.client.0.smithi007.stdout:4/759: mkdir d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d102 0 2024-03-16T09:01:22.258 INFO:tasks.workunit.client.0.smithi007.stdout:1/751: truncate d9/f19 1448073 0 2024-03-16T09:01:22.259 INFO:tasks.workunit.client.0.smithi007.stdout:5/852: rmdir d1/d7/d1c/d19/d34/d5e/d6f/da9 39 2024-03-16T09:01:22.259 INFO:tasks.workunit.client.0.smithi007.stdout:5/853: write d1/d7/d1c/d19/d34/d3e/d41/da6/f119 [258469,10704] 0 2024-03-16T09:01:22.259 INFO:tasks.workunit.client.0.smithi007.stdout:9/881: dread d24/d4c/d86/daa/d95/f6f [0,4194304] 0 2024-03-16T09:01:22.261 INFO:tasks.workunit.client.0.smithi007.stdout:2/996: link dc/df/f12e dc/df/d24/d28/d2e/f14d 0 2024-03-16T09:01:22.262 INFO:tasks.workunit.client.0.smithi007.stdout:1/752: symlink d9/d6b/d7e/d91/lec 0 2024-03-16T09:01:22.264 INFO:tasks.workunit.client.0.smithi007.stdout:5/854: rename d1/d7/d1c/d88/da2/d89/db3/l21 to d1/d7/d1c/d19/df7/d105/l138 0 2024-03-16T09:01:22.264 INFO:tasks.workunit.client.0.smithi007.stdout:5/855: fdatasync d1/d7/d1c/f122 0 2024-03-16T09:01:22.264 INFO:tasks.workunit.client.1.smithi019.stdout:6/827: dread de/d26/d32/d4e/f9b [0,4194304] 0 2024-03-16T09:01:22.264 INFO:tasks.workunit.client.1.smithi019.stdout:6/828: readlink de/d26/d32/d4e/d75/laf 0 2024-03-16T09:01:22.265 INFO:tasks.workunit.client.0.smithi007.stdout:2/997: mkdir dc/df/d24/d126/d14e 0 2024-03-16T09:01:22.265 INFO:tasks.workunit.client.0.smithi007.stdout:9/882: dread d24/d4a/d55/dbf/de9/ff3 [0,4194304] 0 2024-03-16T09:01:22.265 INFO:tasks.workunit.client.1.smithi019.stdout:6/829: symlink de/d26/d32/dc1/l108 0 2024-03-16T09:01:22.266 INFO:tasks.workunit.client.0.smithi007.stdout:5/856: mknod d1/d7/d1c/d19/d34/d4d/d99/c139 0 2024-03-16T09:01:22.269 INFO:tasks.workunit.client.0.smithi007.stdout:5/857: dread d1/d7/d1c/d19/d34/d3e/d41/da6/fc9 [0,4194304] 0 2024-03-16T09:01:22.269 INFO:tasks.workunit.client.0.smithi007.stdout:5/858: write d1/d7/d1c/d88/da2/d89/fb6 [749234,126431] 0 2024-03-16T09:01:22.278 INFO:tasks.workunit.client.0.smithi007.stdout:5/859: dread d1/d7/d1c/d19/d34/d4d/d99/fa3 [0,4194304] 0 2024-03-16T09:01:22.279 INFO:tasks.workunit.client.0.smithi007.stdout:5/860: write d1/d7/d1c/d88/da2/d89/db3/d2a/d79/fbf [1239199,77545] 0 2024-03-16T09:01:22.279 INFO:tasks.workunit.client.1.smithi019.stdout:9/652: dwrite d1/d7/de/d24/d44/d76/f5f [0,4194304] 0 2024-03-16T09:01:22.282 INFO:tasks.workunit.client.0.smithi007.stdout:5/861: creat d1/d7/d1c/d19/d34/de/f13a x:0 0 0 2024-03-16T09:01:22.282 INFO:tasks.workunit.client.0.smithi007.stdout:1/753: write d9/dd/f14 [210308,5856] 0 2024-03-16T09:01:22.284 INFO:tasks.workunit.client.1.smithi019.stdout:3/735: dwrite d3/d8b/dca/fe1 [0,4194304] 0 2024-03-16T09:01:22.284 INFO:tasks.workunit.client.1.smithi019.stdout:3/736: write d3/d8b/dae/f50 [5294574,6145] 0 2024-03-16T09:01:22.284 INFO:tasks.workunit.client.1.smithi019.stdout:3/737: chown d3/d1b/d40/d4a 348 1 2024-03-16T09:01:22.284 INFO:tasks.workunit.client.0.smithi007.stdout:9/883: write d24/d4a/fcc [1182050,66472] 0 2024-03-16T09:01:22.286 INFO:tasks.workunit.client.1.smithi019.stdout:5/759: dwrite d6/dd/f1d [0,4194304] 0 2024-03-16T09:01:22.286 INFO:tasks.workunit.client.0.smithi007.stdout:6/830: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/ff2 [0,4194304] 0 2024-03-16T09:01:22.286 INFO:tasks.workunit.client.0.smithi007.stdout:5/862: truncate d1/d7/d1c/d88/da2/dae/fde 6349625 0 2024-03-16T09:01:22.290 INFO:tasks.workunit.client.1.smithi019.stdout:6/830: dwrite de/d26/d32/d97/d66/d72/d7f/dd5/fe9 [0,4194304] 0 2024-03-16T09:01:22.290 INFO:tasks.workunit.client.1.smithi019.stdout:7/700: dwrite d9/d25/d28/d48/d8d/dba/fca [0,4194304] 0 2024-03-16T09:01:22.290 INFO:tasks.workunit.client.1.smithi019.stdout:7/701: write d9/db/d12/d5b/f65 [964115,5744] 0 2024-03-16T09:01:22.291 INFO:tasks.workunit.client.0.smithi007.stdout:1/754: creat d9/dd/d1b/d28/d29/d7b/dba/dda/fed x:0 0 0 2024-03-16T09:01:22.291 INFO:tasks.workunit.client.0.smithi007.stdout:1/755: getdents d9/dd/de0 0 2024-03-16T09:01:22.293 INFO:tasks.workunit.client.1.smithi019.stdout:6/831: unlink de/d26/d32/d97/d66/la7 0 2024-03-16T09:01:22.296 INFO:tasks.workunit.client.1.smithi019.stdout:7/702: rename d9/d25/d28/d48/f7b to d9/d25/d28/d48/d8d/dba/dad/dd8/fe0 0 2024-03-16T09:01:22.296 INFO:tasks.workunit.client.1.smithi019.stdout:7/703: truncate d9/d25/d28/d48/ddd/fd9 453822 0 2024-03-16T09:01:22.298 INFO:tasks.workunit.client.0.smithi007.stdout:2/998: dwrite dc/d27/f142 [0,4194304] 0 2024-03-16T09:01:22.299 INFO:tasks.workunit.client.0.smithi007.stdout:9/884: mkdir d24/d4c/d86/daa/d7f/d80/de3/d107 0 2024-03-16T09:01:22.301 INFO:tasks.workunit.client.1.smithi019.stdout:7/704: rename d9/db/d12/d5b/d33 to d9/da/d13/d18/d30/d75/de1 0 2024-03-16T09:01:22.303 INFO:tasks.workunit.client.1.smithi019.stdout:7/705: creat d9/fe2 x:0 0 0 2024-03-16T09:01:22.303 INFO:tasks.workunit.client.1.smithi019.stdout:7/706: fdatasync d9/da/d13/d9a/f77 0 2024-03-16T09:01:22.305 INFO:tasks.workunit.client.1.smithi019.stdout:9/653: dwrite d1/d98/dae/f57 [0,4194304] 0 2024-03-16T09:01:22.306 INFO:tasks.workunit.client.1.smithi019.stdout:7/707: mknod d9/da/d13/d18/d30/d75/de1/ce3 0 2024-03-16T09:01:22.307 INFO:tasks.workunit.client.1.smithi019.stdout:6/832: dread de/d26/d32/f4f [0,4194304] 0 2024-03-16T09:01:22.308 INFO:tasks.workunit.client.1.smithi019.stdout:6/833: stat de/d1c/fea 0 2024-03-16T09:01:22.309 INFO:tasks.workunit.client.1.smithi019.stdout:7/708: truncate d9/db/d12/f26 1116379 0 2024-03-16T09:01:22.311 INFO:tasks.workunit.client.1.smithi019.stdout:3/738: dwrite d3/d89/f38 [0,4194304] 0 2024-03-16T09:01:22.311 INFO:tasks.workunit.client.1.smithi019.stdout:3/739: fsync d3/d8b/dae/f59 0 2024-03-16T09:01:22.311 INFO:tasks.workunit.client.1.smithi019.stdout:3/740: chown d3/d1b/d40/d4a/cd3 102 1 2024-03-16T09:01:22.311 INFO:tasks.workunit.client.1.smithi019.stdout:3/741: fsync d3/d11/f3b 0 2024-03-16T09:01:22.311 INFO:tasks.workunit.client.1.smithi019.stdout:3/742: read - d3/d9b/fb6 zero size 2024-03-16T09:01:22.312 INFO:tasks.workunit.client.1.smithi019.stdout:6/834: rename de/d26/d32/d4e/f9b to de/d26/d32/d97/d66/d72/f109 0 2024-03-16T09:01:22.312 INFO:tasks.workunit.client.1.smithi019.stdout:6/835: chown de/d26/d32/d97/d51/lbe 5 1 2024-03-16T09:01:22.313 INFO:tasks.workunit.client.0.smithi007.stdout:5/863: mkdir d1/d7/d1c/d19/d34/d1d/da8/d111/d13b 0 2024-03-16T09:01:22.314 INFO:tasks.workunit.client.1.smithi019.stdout:9/654: getdents d1/d7/d67/d77 0 2024-03-16T09:01:22.314 INFO:tasks.workunit.client.1.smithi019.stdout:9/655: write d1/d98/dae/f3c [1484725,3542] 0 2024-03-16T09:01:22.314 INFO:tasks.workunit.client.1.smithi019.stdout:9/656: chown d1/d7/d67/d77/fe0 0 1 2024-03-16T09:01:22.314 INFO:tasks.workunit.client.1.smithi019.stdout:9/657: fsync d1/d7/de/d24/d44/d60/d74/f80 0 2024-03-16T09:01:22.315 INFO:tasks.workunit.client.0.smithi007.stdout:2/999: mknod dc/df/d24/d28/d97/c14f 0 2024-03-16T09:01:22.315 INFO:tasks.workunit.client.1.smithi019.stdout:3/743: mknod d3/d8b/da8/dcc/ce8 0 2024-03-16T09:01:22.315 INFO:tasks.workunit.client.0.smithi007.stdout:9/885: truncate d24/f3c 3220401 0 2024-03-16T09:01:22.318 INFO:tasks.workunit.client.0.smithi007.stdout:9/886: dread d24/d4c/f63 [0,4194304] 0 2024-03-16T09:01:22.318 INFO:tasks.workunit.client.1.smithi019.stdout:7/709: rename d9/da/l44 to d9/da/d13/d18/d30/d75/de1/le4 0 2024-03-16T09:01:22.318 INFO:tasks.workunit.client.1.smithi019.stdout:7/710: chown d9/db/fa7 1559168 1 2024-03-16T09:01:22.323 INFO:tasks.workunit.client.1.smithi019.stdout:9/658: creat d1/d7/d67/fe4 x:0 0 0 2024-03-16T09:01:22.325 INFO:tasks.workunit.client.0.smithi007.stdout:1/756: dwrite d9/dd/d1b/d28/d29/f80 [0,4194304] 0 2024-03-16T09:01:22.326 INFO:tasks.workunit.client.1.smithi019.stdout:5/760: dwrite d6/dd/d61/fd4 [0,4194304] 0 2024-03-16T09:01:22.326 INFO:tasks.workunit.client.1.smithi019.stdout:5/761: truncate d6/dd/f1d 4791407 0 2024-03-16T09:01:22.327 INFO:tasks.workunit.client.1.smithi019.stdout:7/711: rename d9/db/d12/l98 to d9/db/d12/dcf/le5 0 2024-03-16T09:01:22.328 INFO:tasks.workunit.client.1.smithi019.stdout:7/712: stat d9/da/d13/d18/d30/d75/c7d 0 2024-03-16T09:01:22.328 INFO:tasks.workunit.client.1.smithi019.stdout:7/713: chown d9/d25/d28/f57 5 1 2024-03-16T09:01:22.330 INFO:tasks.workunit.client.1.smithi019.stdout:6/836: write de/d26/d32/d4e/f59 [67647,37671] 0 2024-03-16T09:01:22.330 INFO:tasks.workunit.client.1.smithi019.stdout:0/804: sync 2024-03-16T09:01:22.331 INFO:tasks.workunit.client.0.smithi007.stdout:3/880: sync 2024-03-16T09:01:22.331 INFO:tasks.workunit.client.0.smithi007.stdout:3/881: write d15/f16 [2092590,93423] 0 2024-03-16T09:01:22.331 INFO:tasks.workunit.client.0.smithi007.stdout:3/882: chown d15/d18/d1f/ffd 5393392 1 2024-03-16T09:01:22.331 INFO:tasks.workunit.client.0.smithi007.stdout:3/883: fsync d15/d18/d1f/d29/d7c/f94 0 2024-03-16T09:01:22.331 INFO:tasks.workunit.client.1.smithi019.stdout:7/714: write d9/da/d13/d18/f5a [89306,76520] 0 2024-03-16T09:01:22.335 INFO:tasks.workunit.client.1.smithi019.stdout:6/837: link de/d26/le7 de/d26/d73/ded/l10a 0 2024-03-16T09:01:22.335 INFO:tasks.workunit.client.0.smithi007.stdout:3/884: stat d15/d18/d6a/fe9 0 2024-03-16T09:01:22.336 INFO:tasks.workunit.client.0.smithi007.stdout:6/831: read d2/d13/d3f/d50/d72/f10b [2049132,29018] 0 2024-03-16T09:01:22.336 INFO:tasks.workunit.client.0.smithi007.stdout:3/885: mkdir d15/d18/d1f/d3b/dae/d113 0 2024-03-16T09:01:22.337 INFO:tasks.workunit.client.1.smithi019.stdout:0/805: mkdir d7/d8/d32/df5/d10a 0 2024-03-16T09:01:22.338 INFO:tasks.workunit.client.0.smithi007.stdout:6/832: creat d2/d10/d3d/d6d/dbf/f112 x:0 0 0 2024-03-16T09:01:22.339 INFO:tasks.workunit.client.0.smithi007.stdout:3/886: symlink d15/l114 0 2024-03-16T09:01:22.341 INFO:tasks.workunit.client.0.smithi007.stdout:6/833: mkdir d2/d13/d3f/d86/d113 0 2024-03-16T09:01:22.341 INFO:tasks.workunit.client.0.smithi007.stdout:6/834: fdatasync d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/fe2 0 2024-03-16T09:01:22.343 INFO:tasks.workunit.client.0.smithi007.stdout:3/887: mknod d15/d18/d1f/d29/d43/c115 0 2024-03-16T09:01:22.344 INFO:tasks.workunit.client.0.smithi007.stdout:6/835: truncate d2/d13/d62/f7b 1042864 0 2024-03-16T09:01:22.344 INFO:tasks.workunit.client.0.smithi007.stdout:6/836: fsync d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/f98 0 2024-03-16T09:01:22.344 INFO:tasks.workunit.client.0.smithi007.stdout:6/837: dread - d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/fd9 zero size 2024-03-16T09:01:22.347 INFO:tasks.workunit.client.1.smithi019.stdout:0/806: unlink d7/d8/dec/def/da1/da7/ld4 0 2024-03-16T09:01:22.348 INFO:tasks.workunit.client.0.smithi007.stdout:6/838: link d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/lf8 d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/l114 0 2024-03-16T09:01:22.348 INFO:tasks.workunit.client.0.smithi007.stdout:6/839: chown d2/c10d 5911 1 2024-03-16T09:01:22.350 INFO:tasks.workunit.client.0.smithi007.stdout:3/888: rename d15/d18/d6a/c74 to d15/d18/d1f/d29/d3e/d85/def/c116 0 2024-03-16T09:01:22.350 INFO:tasks.workunit.client.0.smithi007.stdout:3/889: readlink d15/l40 0 2024-03-16T09:01:22.350 INFO:tasks.workunit.client.0.smithi007.stdout:9/887: dwrite d24/f31 [4194304,4194304] 0 2024-03-16T09:01:22.352 INFO:tasks.workunit.client.1.smithi019.stdout:5/762: read d6/dd/f99 [2616231,93034] 0 2024-03-16T09:01:22.352 INFO:tasks.workunit.client.1.smithi019.stdout:5/763: truncate d6/dd/d47/f86 1043837 0 2024-03-16T09:01:22.353 INFO:tasks.workunit.client.0.smithi007.stdout:9/888: rename d24/f39 to d24/d4c/d86/daa/d7f/d80/d87/d89/f108 0 2024-03-16T09:01:22.353 INFO:tasks.workunit.client.0.smithi007.stdout:9/889: dread - d24/d4c/d86/daa/d7f/d8f/ffe zero size 2024-03-16T09:01:22.356 INFO:tasks.workunit.client.0.smithi007.stdout:9/890: rename d24/d4c/d86/daa/d7f/d8f/fb8 to d24/d4a/da2/dd5/f109 0 2024-03-16T09:01:22.356 INFO:tasks.workunit.client.0.smithi007.stdout:9/891: write d24/d4a/da2/db7/fc3 [1282124,46941] 0 2024-03-16T09:01:22.367 INFO:tasks.workunit.client.0.smithi007.stdout:5/864: dwrite d1/d7/d1c/d19/df7/d105/f123 [0,4194304] 0 2024-03-16T09:01:22.367 INFO:tasks.workunit.client.0.smithi007.stdout:5/865: chown d1/d7/d1c/d19/d34/d5e/d6f/f6c 36597297 1 2024-03-16T09:01:22.370 INFO:tasks.workunit.client.1.smithi019.stdout:0/807: write d7/d8/dec/def/f34 [761467,29777] 0 2024-03-16T09:01:22.370 INFO:tasks.workunit.client.1.smithi019.stdout:0/808: write d7/d8/d32/d66/d6d/d8c/db8/d82/fe3 [698907,28546] 0 2024-03-16T09:01:22.371 INFO:tasks.workunit.client.1.smithi019.stdout:0/809: symlink d7/d8/dec/def/d92/dc6/l10b 0 2024-03-16T09:01:22.381 INFO:tasks.workunit.client.1.smithi019.stdout:5/764: dread d6/dd/d37/d54/f5f [0,4194304] 0 2024-03-16T09:01:22.381 INFO:tasks.workunit.client.1.smithi019.stdout:5/765: write d6/dd/d47/d70/fed [118932,32512] 0 2024-03-16T09:01:22.382 INFO:tasks.workunit.client.0.smithi007.stdout:9/892: dread d24/d4c/d86/daa/d7f/d80/d87/d89/fae [0,4194304] 0 2024-03-16T09:01:22.382 INFO:tasks.workunit.client.1.smithi019.stdout:5/766: mkdir d6/d19/df4 0 2024-03-16T09:01:22.382 INFO:tasks.workunit.client.0.smithi007.stdout:9/893: creat d24/d4c/d91/f10a x:0 0 0 2024-03-16T09:01:22.383 INFO:tasks.workunit.client.1.smithi019.stdout:3/744: dwrite d3/f51 [0,4194304] 0 2024-03-16T09:01:22.384 INFO:tasks.workunit.client.1.smithi019.stdout:5/767: rename d6/d19/f7f to d6/dd/d47/d70/db2/d89/ff5 0 2024-03-16T09:01:22.385 INFO:tasks.workunit.client.1.smithi019.stdout:3/745: mkdir d3/d1b/d40/de9 0 2024-03-16T09:01:22.386 INFO:tasks.workunit.client.1.smithi019.stdout:3/746: creat d3/d1b/d40/dc3/fea x:0 0 0 2024-03-16T09:01:22.386 INFO:tasks.workunit.client.1.smithi019.stdout:5/768: getdents d6/d19/d34 0 2024-03-16T09:01:22.391 INFO:tasks.workunit.client.0.smithi007.stdout:9/894: write d24/f33 [1377384,80634] 0 2024-03-16T09:01:22.393 INFO:tasks.workunit.client.1.smithi019.stdout:6/838: dwrite de/d1c/f5d [0,4194304] 0 2024-03-16T09:01:22.394 INFO:tasks.workunit.client.0.smithi007.stdout:6/840: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/f64 [0,4194304] 0 2024-03-16T09:01:22.395 INFO:tasks.workunit.client.0.smithi007.stdout:5/866: dwrite d1/d7/d1c/f122 [0,4194304] 0 2024-03-16T09:01:22.395 INFO:tasks.workunit.client.1.smithi019.stdout:7/715: dwrite d9/db/d12/d5b/f38 [0,4194304] 0 2024-03-16T09:01:22.395 INFO:tasks.workunit.client.1.smithi019.stdout:9/659: dwrite d1/d7/de/d24/f6d [0,4194304] 0 2024-03-16T09:01:22.396 INFO:tasks.workunit.client.1.smithi019.stdout:9/660: read d1/d7/de/d24/d44/d76/d9c/dcc/fd7 [216688,106491] 0 2024-03-16T09:01:22.396 INFO:tasks.workunit.client.1.smithi019.stdout:9/661: write d1/d7/d67/d77/faa [71554,32390] 0 2024-03-16T09:01:22.396 INFO:tasks.workunit.client.1.smithi019.stdout:9/662: fdatasync d1/d7/d67/fe4 0 2024-03-16T09:01:22.397 INFO:tasks.workunit.client.0.smithi007.stdout:9/895: mkdir d24/d4c/d10b 0 2024-03-16T09:01:22.397 INFO:tasks.workunit.client.0.smithi007.stdout:9/896: truncate d24/d4c/d86/daa/d95/f68 434871 0 2024-03-16T09:01:22.397 INFO:tasks.workunit.client.0.smithi007.stdout:9/897: readlink d24/d4c/d86/daa/d95/l72 0 2024-03-16T09:01:22.398 INFO:tasks.workunit.client.0.smithi007.stdout:6/841: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c/f115 x:0 0 0 2024-03-16T09:01:22.398 INFO:tasks.workunit.client.0.smithi007.stdout:6/842: dread - d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/f92 zero size 2024-03-16T09:01:22.402 INFO:tasks.workunit.client.0.smithi007.stdout:5/867: creat d1/d7/d1c/d19/df7/d110/f13c x:0 0 0 2024-03-16T09:01:22.402 INFO:tasks.workunit.client.0.smithi007.stdout:5/868: write d1/d7/d1c/d19/df7/d110/f124 [245241,74100] 0 2024-03-16T09:01:22.408 INFO:tasks.workunit.client.1.smithi019.stdout:3/747: dwrite d3/d8b/dae/d5a/d75/d7a/f87 [0,4194304] 0 2024-03-16T09:01:22.409 INFO:tasks.workunit.client.1.smithi019.stdout:3/748: read d3/d1b/fe0 [2007184,15597] 0 2024-03-16T09:01:22.410 INFO:tasks.workunit.client.0.smithi007.stdout:9/898: mknod d24/d4a/d96/db9/dca/ddc/c10c 0 2024-03-16T09:01:22.410 INFO:tasks.workunit.client.0.smithi007.stdout:9/899: fsync d24/d4c/d86/daa/d7f/ff7 0 2024-03-16T09:01:22.410 INFO:tasks.workunit.client.1.smithi019.stdout:7/716: creat d9/da/d13/d18/d30/d9e/dd7/fe6 x:0 0 0 2024-03-16T09:01:22.410 INFO:tasks.workunit.client.1.smithi019.stdout:9/663: rmdir d1/d7/de/d24/d44/d60/d8b/dc2 39 2024-03-16T09:01:22.411 INFO:tasks.workunit.client.0.smithi007.stdout:6/843: write d2/d13/d3f/d50/d72/fa4 [1291328,80762] 0 2024-03-16T09:01:22.411 INFO:tasks.workunit.client.1.smithi019.stdout:6/839: read f7 [3538870,92394] 0 2024-03-16T09:01:22.414 INFO:tasks.workunit.client.1.smithi019.stdout:3/749: symlink d3/d8b/dae/d3f/leb 0 2024-03-16T09:01:22.414 INFO:tasks.workunit.client.1.smithi019.stdout:3/750: chown d3/d8b/dae/d5a/d80/f9c 34 1 2024-03-16T09:01:22.414 INFO:tasks.workunit.client.1.smithi019.stdout:3/751: creat d3/d8b/da8/dcc/fec x:0 0 0 2024-03-16T09:01:22.414 INFO:tasks.workunit.client.1.smithi019.stdout:3/752: fsync d3/d11/f49 0 2024-03-16T09:01:22.415 INFO:tasks.workunit.client.1.smithi019.stdout:7/717: creat d9/d25/d28/d48/fe7 x:0 0 0 2024-03-16T09:01:22.415 INFO:tasks.workunit.client.1.smithi019.stdout:9/664: mknod d1/d7/de/d24/d44/d60/d8b/ce5 0 2024-03-16T09:01:22.417 INFO:tasks.workunit.client.0.smithi007.stdout:6/844: mknod d2/d13/d3f/d50/d78/dbe/c116 0 2024-03-16T09:01:22.419 INFO:tasks.workunit.client.0.smithi007.stdout:9/900: getdents d24/d8e 0 2024-03-16T09:01:22.419 INFO:tasks.workunit.client.0.smithi007.stdout:9/901: write d24/d4c/d91/f103 [402577,128879] 0 2024-03-16T09:01:22.422 INFO:tasks.workunit.client.0.smithi007.stdout:9/902: symlink d24/d4c/d10b/l10d 0 2024-03-16T09:01:22.423 INFO:tasks.workunit.client.1.smithi019.stdout:6/840: dread de/d26/d32/d97/dae/db1/fcf [4194304,4194304] 0 2024-03-16T09:01:22.423 INFO:tasks.workunit.client.0.smithi007.stdout:9/903: creat d24/f10e x:0 0 0 2024-03-16T09:01:22.424 INFO:tasks.workunit.client.1.smithi019.stdout:3/753: write d3/f51 [2685981,87591] 0 2024-03-16T09:01:22.426 INFO:tasks.workunit.client.0.smithi007.stdout:9/904: getdents d24/d4c/d86/daa/d7f/d8f 0 2024-03-16T09:01:22.427 INFO:tasks.workunit.client.1.smithi019.stdout:3/754: rename d3/d1b/d40/d4a/l60 to d3/d1b/d40/dc3/led 0 2024-03-16T09:01:22.429 INFO:tasks.workunit.client.1.smithi019.stdout:3/755: symlink d3/d8b/lee 0 2024-03-16T09:01:22.429 INFO:tasks.workunit.client.1.smithi019.stdout:7/718: dread d9/da/d13/d9a/f77 [0,4194304] 0 2024-03-16T09:01:22.429 INFO:tasks.workunit.client.1.smithi019.stdout:7/719: write d9/db/d12/f73 [899921,47669] 0 2024-03-16T09:01:22.429 INFO:tasks.workunit.client.1.smithi019.stdout:7/720: dread - d9/d25/d28/d48/f4b zero size 2024-03-16T09:01:22.429 INFO:tasks.workunit.client.1.smithi019.stdout:7/721: readlink d9/da/d13/d18/d30/d3e/l68 0 2024-03-16T09:01:22.429 INFO:tasks.workunit.client.1.smithi019.stdout:7/722: fdatasync d9/da/d13/d9a/faf 0 2024-03-16T09:01:22.430 INFO:tasks.workunit.client.0.smithi007.stdout:9/905: rename d24/d4a/d55/dbf/de9/ceb to d24/d4c/d86/daa/d7f/c10f 0 2024-03-16T09:01:22.430 INFO:tasks.workunit.client.1.smithi019.stdout:3/756: mknod d3/d8b/dae/d5a/d75/d7a/cef 0 2024-03-16T09:01:22.430 INFO:tasks.workunit.client.1.smithi019.stdout:9/665: write d1/d7/de/d24/d44/d76/f9a [1282668,111723] 0 2024-03-16T09:01:22.431 INFO:tasks.workunit.client.1.smithi019.stdout:5/769: dwrite d6/dd/f44 [0,4194304] 0 2024-03-16T09:01:22.431 INFO:tasks.workunit.client.1.smithi019.stdout:5/770: fsync d6/dd/d37/d54/fb1 0 2024-03-16T09:01:22.431 INFO:tasks.workunit.client.1.smithi019.stdout:7/723: symlink d9/da/d13/d18/d30/d75/da4/le8 0 2024-03-16T09:01:22.431 INFO:tasks.workunit.client.1.smithi019.stdout:7/724: write f8 [1510826,105756] 0 2024-03-16T09:01:22.431 INFO:tasks.workunit.client.1.smithi019.stdout:7/725: creat d9/da/d13/d18/d30/d3e/da5/fe9 x:0 0 0 2024-03-16T09:01:22.433 INFO:tasks.workunit.client.0.smithi007.stdout:5/869: dwrite d1/d7/d16/d1b/f60 [4194304,4194304] 0 2024-03-16T09:01:22.434 INFO:tasks.workunit.client.0.smithi007.stdout:9/906: write d24/d4c/d86/daa/d95/f6f [3890304,126692] 0 2024-03-16T09:01:22.434 INFO:tasks.workunit.client.0.smithi007.stdout:9/907: write d24/d4a/d55/dbf/de9/ff3 [685279,104363] 0 2024-03-16T09:01:22.435 INFO:tasks.workunit.client.1.smithi019.stdout:5/771: symlink d6/d1f/d77/lf6 0 2024-03-16T09:01:22.435 INFO:tasks.workunit.client.0.smithi007.stdout:5/870: mkdir d1/d7/d1c/d88/da2/dae/d13d 0 2024-03-16T09:01:22.438 INFO:tasks.workunit.client.1.smithi019.stdout:7/726: unlink d9/db/d12/d5b/fb4 0 2024-03-16T09:01:22.439 INFO:tasks.workunit.client.1.smithi019.stdout:6/841: write de/d1c/d9a/d9e/fb5 [2136365,37103] 0 2024-03-16T09:01:22.439 INFO:tasks.workunit.client.0.smithi007.stdout:5/871: rename d1/d7/d1c/d19/d34/d4d/f77 to d1/d7/d1c/d19/d34/d1d/dee/f13e 0 2024-03-16T09:01:22.439 INFO:tasks.workunit.client.0.smithi007.stdout:5/872: chown d1/d7/d1c/d19/d34/d4d/d99/fd7 8527 1 2024-03-16T09:01:22.439 INFO:tasks.workunit.client.0.smithi007.stdout:5/873: stat d1/d7/d1c/df0 0 2024-03-16T09:01:22.439 INFO:tasks.workunit.client.0.smithi007.stdout:5/874: chown d1/d7/d1c/d19/dcc/d102/d10e 2110 1 2024-03-16T09:01:22.441 INFO:tasks.workunit.client.1.smithi019.stdout:9/666: write d1/d7/f81 [3872363,121444] 0 2024-03-16T09:01:22.443 INFO:tasks.workunit.client.1.smithi019.stdout:7/727: unlink d9/d25/d28/d48/f4b 0 2024-03-16T09:01:22.443 INFO:tasks.workunit.client.1.smithi019.stdout:7/728: readlink d9/d25/d28/d48/ddd/lc6 0 2024-03-16T09:01:22.443 INFO:tasks.workunit.client.0.smithi007.stdout:5/875: symlink d1/d7/d1c/d88/d11e/l13f 0 2024-03-16T09:01:22.444 INFO:tasks.workunit.client.0.smithi007.stdout:5/876: dread - d1/d7/d1c/d19/df7/f12f zero size 2024-03-16T09:01:22.445 INFO:tasks.workunit.client.1.smithi019.stdout:6/842: mknod de/d26/d32/d97/d51/d70/c10b 0 2024-03-16T09:01:22.446 INFO:tasks.workunit.client.0.smithi007.stdout:5/877: symlink d1/d7/d1c/d19/d34/d3e/d41/da6/l140 0 2024-03-16T09:01:22.446 INFO:tasks.workunit.client.0.smithi007.stdout:5/878: fdatasync d1/d7/d1c/d88/da2/d89/db3/f31 0 2024-03-16T09:01:22.447 INFO:tasks.workunit.client.0.smithi007.stdout:5/879: fdatasync d1/d7/d1c/d19/d34/d5e/f8e 0 2024-03-16T09:01:22.447 INFO:tasks.workunit.client.0.smithi007.stdout:5/880: write d1/d7/d1c/d19/d34/d1d/da8/fc7 [1083825,45113] 0 2024-03-16T09:01:22.447 INFO:tasks.workunit.client.0.smithi007.stdout:5/881: chown d1/d7/d1c/d19/dcc/d102/d10e 9 1 2024-03-16T09:01:22.447 INFO:tasks.workunit.client.0.smithi007.stdout:5/882: stat d1/d7/l42 0 2024-03-16T09:01:22.447 INFO:tasks.workunit.client.0.smithi007.stdout:5/883: write d1/d7/d1c/d19/df7/d105/f127 [614230,6033] 0 2024-03-16T09:01:22.447 INFO:tasks.workunit.client.1.smithi019.stdout:9/667: getdents d1/d7/de/d24/d44/d60 0 2024-03-16T09:01:22.449 INFO:tasks.workunit.client.1.smithi019.stdout:6/843: mkdir de/d10c 0 2024-03-16T09:01:22.455 INFO:tasks.workunit.client.1.smithi019.stdout:4/693: sync 2024-03-16T09:01:22.455 INFO:tasks.workunit.client.1.smithi019.stdout:8/866: sync 2024-03-16T09:01:22.455 INFO:tasks.workunit.client.1.smithi019.stdout:1/685: sync 2024-03-16T09:01:22.455 INFO:tasks.workunit.client.1.smithi019.stdout:8/867: fsync d6/d2a/d55/fe9 0 2024-03-16T09:01:22.455 INFO:tasks.workunit.client.1.smithi019.stdout:2/837: sync 2024-03-16T09:01:22.455 INFO:tasks.workunit.client.1.smithi019.stdout:2/838: creat d5/d6/d2b/d2f/d35/d65/f121 x:0 0 0 2024-03-16T09:01:22.455 INFO:tasks.workunit.client.1.smithi019.stdout:9/668: dread d1/d7/de/d24/d5b/f6e [0,4194304] 0 2024-03-16T09:01:22.455 INFO:tasks.workunit.client.1.smithi019.stdout:2/839: write d5/fac [876804,75530] 0 2024-03-16T09:01:22.456 INFO:tasks.workunit.client.1.smithi019.stdout:2/840: chown d5/d6/d2b/d89/fea 0 1 2024-03-16T09:01:22.456 INFO:tasks.workunit.client.1.smithi019.stdout:4/694: symlink d4/d6c/le3 0 2024-03-16T09:01:22.456 INFO:tasks.workunit.client.1.smithi019.stdout:8/868: mknod d6/d2a/d45/d57/d72/d79/da5/c12b 0 2024-03-16T09:01:22.456 INFO:tasks.workunit.client.1.smithi019.stdout:8/869: chown d6/d2a/d45/d57/d72/d79/fd5 1860973 1 2024-03-16T09:01:22.457 INFO:tasks.workunit.client.1.smithi019.stdout:7/729: write d9/da/d13/d18/d30/d3e/fd3 [2561647,82072] 0 2024-03-16T09:01:22.457 INFO:tasks.workunit.client.1.smithi019.stdout:7/730: truncate d9/da/d13/d18/d30/d9e/fc1 308720 0 2024-03-16T09:01:22.458 INFO:tasks.workunit.client.1.smithi019.stdout:1/686: unlink d16/d1f/d3c/d79/db4/fbb 0 2024-03-16T09:01:22.458 INFO:tasks.workunit.client.1.smithi019.stdout:4/695: mknod d4/d9e/ce4 0 2024-03-16T09:01:22.458 INFO:tasks.workunit.client.1.smithi019.stdout:1/687: truncate f12 5151050 0 2024-03-16T09:01:22.461 INFO:tasks.workunit.client.1.smithi019.stdout:4/696: rmdir d4/d1b/d49/d58/d80 39 2024-03-16T09:01:22.465 INFO:tasks.workunit.client.0.smithi007.stdout:5/884: write d1/d7/d1c/d19/d34/d4d/d99/fd7 [7251627,100672] 0 2024-03-16T09:01:22.466 INFO:tasks.workunit.client.1.smithi019.stdout:9/669: dread d1/d7/de/d24/d5b/f6e [0,4194304] 0 2024-03-16T09:01:22.467 INFO:tasks.workunit.client.1.smithi019.stdout:9/670: write d1/d7/de/d24/d44/d76/fa5 [939632,79588] 0 2024-03-16T09:01:22.467 INFO:tasks.workunit.client.1.smithi019.stdout:9/671: write d1/d98/dae/f22 [8041410,85139] 0 2024-03-16T09:01:22.468 INFO:tasks.workunit.client.0.smithi007.stdout:5/885: mknod d1/d7/d1c/d19/d34/d4d/de7/c141 0 2024-03-16T09:01:22.469 INFO:tasks.workunit.client.0.smithi007.stdout:5/886: write d1/d7/d1c/d88/da2/d89/db3/d2a/d81/f11a [653422,20281] 0 2024-03-16T09:01:22.469 INFO:tasks.workunit.client.0.smithi007.stdout:5/887: chown d1/d7/d1c/d19/d34/c13 118594 1 2024-03-16T09:01:22.471 INFO:tasks.workunit.client.1.smithi019.stdout:9/672: symlink d1/d7/dbb/le6 0 2024-03-16T09:01:22.472 INFO:tasks.workunit.client.1.smithi019.stdout:7/731: dread d9/db/fb1 [4194304,4194304] 0 2024-03-16T09:01:22.472 INFO:tasks.workunit.client.1.smithi019.stdout:9/673: truncate d1/d7/d21/da2/f9e 1378104 0 2024-03-16T09:01:22.473 INFO:tasks.workunit.client.0.smithi007.stdout:5/888: write d1/d7/d1c/d88/da2/f74 [113731,82555] 0 2024-03-16T09:01:22.473 INFO:tasks.workunit.client.1.smithi019.stdout:6/844: dread de/d26/d32/d4e/d75/fa2 [0,4194304] 0 2024-03-16T09:01:22.473 INFO:tasks.workunit.client.1.smithi019.stdout:6/845: fdatasync de/d26/d32/d4e/d57/fd4 0 2024-03-16T09:01:22.473 INFO:tasks.workunit.client.1.smithi019.stdout:6/846: fdatasync de/d1c/f8d 0 2024-03-16T09:01:22.474 INFO:tasks.workunit.client.1.smithi019.stdout:7/732: dread d9/d25/d28/d48/ddd/f91 [0,4194304] 0 2024-03-16T09:01:22.474 INFO:tasks.workunit.client.1.smithi019.stdout:7/733: creat d9/db/d12/fea x:0 0 0 2024-03-16T09:01:22.474 INFO:tasks.workunit.client.0.smithi007.stdout:5/889: symlink d1/d7/d1c/d19/d34/d1d/da8/d111/d13b/l142 0 2024-03-16T09:01:22.477 INFO:tasks.workunit.client.0.smithi007.stdout:5/890: link d1/d7/d1c/d19/d34/d4d/de7/c141 d1/d7/d16/d1b/c143 0 2024-03-16T09:01:22.477 INFO:tasks.workunit.client.0.smithi007.stdout:5/891: dread - d1/d7/d1c/d19/d34/d3e/d62/fdf zero size 2024-03-16T09:01:22.478 INFO:tasks.workunit.client.0.smithi007.stdout:5/892: creat d1/d7/d1c/d19/dcc/f144 x:0 0 0 2024-03-16T09:01:22.479 INFO:tasks.workunit.client.1.smithi019.stdout:4/697: dread d4/d64/d99/f6b [4194304,4194304] 0 2024-03-16T09:01:22.479 INFO:tasks.workunit.client.1.smithi019.stdout:4/698: write d4/d1b/d2a/d4f/fbc [323323,29290] 0 2024-03-16T09:01:22.481 INFO:tasks.workunit.client.0.smithi007.stdout:5/893: creat d1/d7/d1c/d19/d34/d3e/d41/da6/f145 x:0 0 0 2024-03-16T09:01:22.481 INFO:tasks.workunit.client.1.smithi019.stdout:4/699: mkdir d4/d1b/d2a/d3d/de5 0 2024-03-16T09:01:22.483 INFO:tasks.workunit.client.1.smithi019.stdout:9/674: dread d1/d7/de/d24/f6d [0,4194304] 0 2024-03-16T09:01:22.484 INFO:tasks.workunit.client.1.smithi019.stdout:9/675: rename d1/d7/d21/l4b to d1/d7/d67/le7 0 2024-03-16T09:01:22.484 INFO:tasks.workunit.client.1.smithi019.stdout:9/676: stat d1/d7/d67/d77/cb1 0 2024-03-16T09:01:22.485 INFO:tasks.workunit.client.1.smithi019.stdout:9/677: link d1/d7/dc0/ld6 d1/d7/de/d24/d5b/le8 0 2024-03-16T09:01:22.486 INFO:tasks.workunit.client.1.smithi019.stdout:9/678: chown d1/d7/f3e 31466420 1 2024-03-16T09:01:22.491 INFO:tasks.workunit.client.1.smithi019.stdout:7/734: write d9/db/f2c [1966322,78847] 0 2024-03-16T09:01:22.500 INFO:tasks.workunit.client.1.smithi019.stdout:7/735: write d9/da/d13/d18/d30/d75/f8c [2845948,117531] 0 2024-03-16T09:01:22.501 INFO:tasks.workunit.client.1.smithi019.stdout:7/736: mknod d9/ceb 0 2024-03-16T09:01:22.510 INFO:tasks.workunit.client.1.smithi019.stdout:6/847: write de/d26/d32/fd1 [1201983,49019] 0 2024-03-16T09:01:22.510 INFO:tasks.workunit.client.1.smithi019.stdout:6/848: fsync de/d26/d32/dc1/fca 0 2024-03-16T09:01:22.510 INFO:tasks.workunit.client.1.smithi019.stdout:5/772: dwrite d6/dd/d37/d76/fcf [0,4194304] 0 2024-03-16T09:01:22.510 INFO:tasks.workunit.client.1.smithi019.stdout:6/849: fsync de/d26/d32/d97/d66/d72/d7f/f81 0 2024-03-16T09:01:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:9/908: dwrite f1c [0,4194304] 0 2024-03-16T09:01:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:6/845: dwrite d2/d13/d62/f7b [0,4194304] 0 2024-03-16T09:01:22.511 INFO:tasks.workunit.client.1.smithi019.stdout:4/700: dread d4/d1b/d2a/d3d/f46 [0,4194304] 0 2024-03-16T09:01:22.512 INFO:tasks.workunit.client.1.smithi019.stdout:6/850: truncate de/fe6 3271841 0 2024-03-16T09:01:22.512 INFO:tasks.workunit.client.0.smithi007.stdout:9/909: mkdir d24/d4a/da2/de1/d110 0 2024-03-16T09:01:22.513 INFO:tasks.workunit.client.1.smithi019.stdout:5/773: link d6/dd/d47/d7a/lc1 d6/dd/d47/d70/lf7 0 2024-03-16T09:01:22.513 INFO:tasks.workunit.client.1.smithi019.stdout:5/774: creat d6/dd/d47/d70/ff8 x:0 0 0 2024-03-16T09:01:22.513 INFO:tasks.workunit.client.1.smithi019.stdout:1/688: dwrite d16/d1f/fb6 [0,4194304] 0 2024-03-16T09:01:22.513 INFO:tasks.workunit.client.1.smithi019.stdout:2/841: dwrite d5/d6/d2b/f82 [0,4194304] 0 2024-03-16T09:01:22.516 INFO:tasks.workunit.client.1.smithi019.stdout:1/689: dread d16/d1f/d4f/f60 [0,4194304] 0 2024-03-16T09:01:22.516 INFO:tasks.workunit.client.1.smithi019.stdout:8/870: dwrite d6/d2a/d3c/dab/f11b [0,4194304] 0 2024-03-16T09:01:22.516 INFO:tasks.workunit.client.1.smithi019.stdout:5/775: rename d6/d19/d34/dc2/fc4 to d6/dd/ff9 0 2024-03-16T09:01:22.518 INFO:tasks.workunit.client.1.smithi019.stdout:6/851: getdents de/d26/d32/d4e/d57 0 2024-03-16T09:01:22.518 INFO:tasks.workunit.client.1.smithi019.stdout:6/852: write de/d1c/f56 [1142824,59878] 0 2024-03-16T09:01:22.518 INFO:tasks.workunit.client.1.smithi019.stdout:9/679: dwrite d1/d7/d21/da2/f9e [0,4194304] 0 2024-03-16T09:01:22.518 INFO:tasks.workunit.client.0.smithi007.stdout:5/894: rename d1/d7/d1c/d88/da2/dae to d1/d7/d1c/d19/d34/d5e/d146 0 2024-03-16T09:01:22.519 INFO:tasks.workunit.client.0.smithi007.stdout:5/895: write d1/d7/d1c/d19/df7/f130 [548382,26099] 0 2024-03-16T09:01:22.522 INFO:tasks.workunit.client.0.smithi007.stdout:6/846: dread d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/ff2 [0,4194304] 0 2024-03-16T09:01:22.523 INFO:tasks.workunit.client.1.smithi019.stdout:2/842: mknod d5/d2e/d4f/c122 0 2024-03-16T09:01:22.526 INFO:tasks.workunit.client.0.smithi007.stdout:5/896: getdents d1/d7/d1c/d19/d34/d3e/d41/da6/de8 0 2024-03-16T09:01:22.527 INFO:tasks.workunit.client.1.smithi019.stdout:1/690: write d16/d31/f45 [844232,17267] 0 2024-03-16T09:01:22.528 INFO:tasks.workunit.client.1.smithi019.stdout:5/776: rename d6/dd/f16 to d6/dd/d47/d70/db2/ffa 0 2024-03-16T09:01:22.528 INFO:tasks.workunit.client.1.smithi019.stdout:5/777: chown d6/dd/d47/d70/db2/d89/dcc/fe7 8 1 2024-03-16T09:01:22.528 INFO:tasks.workunit.client.1.smithi019.stdout:5/778: fsync d6/d19/d34/fdf 0 2024-03-16T09:01:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:6/847: mknod d2/d10/d3d/d6d/c117 0 2024-03-16T09:01:22.532 INFO:tasks.workunit.client.1.smithi019.stdout:8/871: chown f5 4301127 1 2024-03-16T09:01:22.532 INFO:tasks.workunit.client.1.smithi019.stdout:8/872: dread - d6/d2a/d3f/f119 zero size 2024-03-16T09:01:22.532 INFO:tasks.workunit.client.1.smithi019.stdout:1/691: dread f9 [0,4194304] 0 2024-03-16T09:01:22.532 INFO:tasks.workunit.client.1.smithi019.stdout:1/692: chown d16/d1f/f41 27053256 1 2024-03-16T09:01:22.533 INFO:tasks.workunit.client.0.smithi007.stdout:5/897: mkdir d1/d7/d1c/d19/d34/d3e/d147 0 2024-03-16T09:01:22.533 INFO:tasks.workunit.client.0.smithi007.stdout:5/898: readlink d1/d7/d16/d1b/l11b 0 2024-03-16T09:01:22.534 INFO:tasks.workunit.client.0.smithi007.stdout:9/910: write d24/fc4 [3874043,39212] 0 2024-03-16T09:01:22.535 INFO:tasks.workunit.client.0.smithi007.stdout:9/911: stat d24/f4f 0 2024-03-16T09:01:22.535 INFO:tasks.workunit.client.0.smithi007.stdout:9/912: write d24/d4c/d86/daa/d95/f6f [308957,97614] 0 2024-03-16T09:01:22.537 INFO:tasks.workunit.client.0.smithi007.stdout:6/848: truncate d2/d13/d3f/d50/f53 2493450 0 2024-03-16T09:01:22.538 INFO:tasks.workunit.client.1.smithi019.stdout:6/853: symlink de/d26/d32/d97/d66/d72/db7/l10d 0 2024-03-16T09:01:22.538 INFO:tasks.workunit.client.1.smithi019.stdout:6/854: creat de/d1c/f10e x:0 0 0 2024-03-16T09:01:22.540 INFO:tasks.workunit.client.0.smithi007.stdout:6/849: link d2/d13/d4b/d94/dea/df7/lfd d2/d13/d3f/d50/d78/d81/l118 0 2024-03-16T09:01:22.540 INFO:tasks.workunit.client.0.smithi007.stdout:6/850: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/f98 [4408524,6406] 0 2024-03-16T09:01:22.540 INFO:tasks.workunit.client.0.smithi007.stdout:6/851: fsync d2/f90 0 2024-03-16T09:01:22.541 INFO:tasks.workunit.client.0.smithi007.stdout:5/899: dread d1/d7/d1c/d19/df7/d105/f127 [0,4194304] 0 2024-03-16T09:01:22.543 INFO:tasks.workunit.client.1.smithi019.stdout:9/680: truncate d1/d7/de/d24/d44/d60/d74/fad 146143 0 2024-03-16T09:01:22.543 INFO:tasks.workunit.client.1.smithi019.stdout:9/681: write d1/d7/fd [555585,60814] 0 2024-03-16T09:01:22.543 INFO:tasks.workunit.client.1.smithi019.stdout:9/682: write d1/d98/dae/f35 [1027755,94361] 0 2024-03-16T09:01:22.544 INFO:tasks.workunit.client.1.smithi019.stdout:5/779: mkdir d6/dd/d47/d70/dfb 0 2024-03-16T09:01:22.545 INFO:tasks.workunit.client.1.smithi019.stdout:8/873: write d6/d2a/d3c/dab/d30/d4a/d67/d95/de0/f117 [866862,23029] 0 2024-03-16T09:01:22.546 INFO:tasks.workunit.client.0.smithi007.stdout:5/900: write d1/d7/d1c/d19/d34/d3e/f43 [6218381,35615] 0 2024-03-16T09:01:22.547 INFO:tasks.workunit.client.1.smithi019.stdout:1/693: symlink d16/d1f/d3c/d79/d96/dac/lca 0 2024-03-16T09:01:22.549 INFO:tasks.workunit.client.1.smithi019.stdout:6/855: rmdir de/d26/d32/d97/d51/d70 39 2024-03-16T09:01:22.552 INFO:tasks.workunit.client.1.smithi019.stdout:4/701: dwrite d4/d6/f70 [0,4194304] 0 2024-03-16T09:01:22.553 INFO:tasks.workunit.client.1.smithi019.stdout:6/856: stat de/d26/d32/d4e/d75/lb2 0 2024-03-16T09:01:22.553 INFO:tasks.workunit.client.1.smithi019.stdout:6/857: chown de/d1c/d9a 1771332 1 2024-03-16T09:01:22.554 INFO:tasks.workunit.client.1.smithi019.stdout:7/737: dwrite d9/f3f [0,4194304] 0 2024-03-16T09:01:22.555 INFO:tasks.workunit.client.1.smithi019.stdout:2/843: dwrite d5/d6/d2b/d2f/d35/d65/f109 [0,4194304] 0 2024-03-16T09:01:22.555 INFO:tasks.workunit.client.1.smithi019.stdout:2/844: write d5/d5a/f9d [447559,91122] 0 2024-03-16T09:01:22.556 INFO:tasks.workunit.client.1.smithi019.stdout:8/874: rename d6/d2a/d55/f112 to d6/d2a/d45/d5d/f12c 0 2024-03-16T09:01:22.562 INFO:tasks.workunit.client.1.smithi019.stdout:4/702: symlink d4/d6/d15/le6 0 2024-03-16T09:01:22.564 INFO:tasks.workunit.client.1.smithi019.stdout:7/738: mkdir d9/db/d12/d5b/dc3/dec 0 2024-03-16T09:01:22.565 INFO:tasks.workunit.client.0.smithi007.stdout:5/901: symlink d1/l148 0 2024-03-16T09:01:22.565 INFO:tasks.workunit.client.0.smithi007.stdout:5/902: readlink d1/d7/d1c/d88/d11e/l13f 0 2024-03-16T09:01:22.566 INFO:tasks.workunit.client.0.smithi007.stdout:9/913: dwrite d24/d4c/d86/daa/d7f/d80/d87/d89/fb3 [4194304,4194304] 0 2024-03-16T09:01:22.568 INFO:tasks.workunit.client.1.smithi019.stdout:7/739: rename d9/d25/d93/fbc to d9/d25/d28/d48/fed 0 2024-03-16T09:01:22.569 INFO:tasks.workunit.client.1.smithi019.stdout:8/875: creat d6/d2a/d3c/dab/d30/d4a/d67/d95/f12d x:0 0 0 2024-03-16T09:01:22.569 INFO:tasks.workunit.client.1.smithi019.stdout:8/876: rename d6 to d6/de/d51/d12a/d12e 22 2024-03-16T09:01:22.570 INFO:tasks.workunit.client.0.smithi007.stdout:5/903: creat d1/d7/d1c/d88/da2/f149 x:0 0 0 2024-03-16T09:01:22.570 INFO:tasks.workunit.client.0.smithi007.stdout:5/904: read - d1/d7/d1c/d19/d34/d1d/f11d zero size 2024-03-16T09:01:22.570 INFO:tasks.workunit.client.0.smithi007.stdout:5/905: chown d1/d7/d1c/d19/d34/d5e/l5c 125238364 1 2024-03-16T09:01:22.571 INFO:tasks.workunit.client.1.smithi019.stdout:2/845: mknod d5/d6/d104/df7/dda/c123 0 2024-03-16T09:01:22.572 INFO:tasks.workunit.client.1.smithi019.stdout:8/877: unlink d6/d2a/d3c/dab/faa 0 2024-03-16T09:01:22.573 INFO:tasks.workunit.client.1.smithi019.stdout:7/740: mknod d9/cee 0 2024-03-16T09:01:22.573 INFO:tasks.workunit.client.1.smithi019.stdout:7/741: chown d9/da/d13/d18/d30/d3e/da5/fd6 7156816 1 2024-03-16T09:01:22.573 INFO:tasks.workunit.client.1.smithi019.stdout:7/742: dread - d9/da/d13/d18/d30/d75/da4/fc2 zero size 2024-03-16T09:01:22.573 INFO:tasks.workunit.client.1.smithi019.stdout:7/743: stat d9/da/fd5 0 2024-03-16T09:01:22.574 INFO:tasks.workunit.client.0.smithi007.stdout:5/906: mknod d1/d7/d1c/d19/d34/d5e/d146/c14a 0 2024-03-16T09:01:22.574 INFO:tasks.workunit.client.1.smithi019.stdout:4/703: dread d4/d6/d15/f2e [0,4194304] 0 2024-03-16T09:01:22.574 INFO:tasks.workunit.client.1.smithi019.stdout:4/704: truncate d4/d64/d99/f3b 1114658 0 2024-03-16T09:01:22.574 INFO:tasks.workunit.client.1.smithi019.stdout:2/846: symlink d5/d6/d104/df7/d6f/da6/d83/d11c/l124 0 2024-03-16T09:01:22.575 INFO:tasks.workunit.client.1.smithi019.stdout:8/878: mkdir d6/d2a/d45/d5d/d12f 0 2024-03-16T09:01:22.575 INFO:tasks.workunit.client.1.smithi019.stdout:8/879: readlink d6/d2a/d68/ld8 0 2024-03-16T09:01:22.576 INFO:tasks.workunit.client.0.smithi007.stdout:5/907: creat d1/d7/d1c/d19/d34/d3e/d147/f14b x:0 0 0 2024-03-16T09:01:22.577 INFO:tasks.workunit.client.0.smithi007.stdout:5/908: chown d1/d7/d1c/d88/da2/d89/db3/d2a/d79/l128 497 1 2024-03-16T09:01:22.578 INFO:tasks.workunit.client.1.smithi019.stdout:2/847: link d5/d6/d104/df7/d6f/d87/fe7 d5/d111/f125 0 2024-03-16T09:01:22.579 INFO:tasks.workunit.client.0.smithi007.stdout:5/909: creat d1/d7/d1c/d19/f14c x:0 0 0 2024-03-16T09:01:22.580 INFO:tasks.workunit.client.1.smithi019.stdout:2/848: link d5/d6/d104/df7/d6f/da6/c85 d5/d6/d2b/d2f/d35/d65/c126 0 2024-03-16T09:01:22.581 INFO:tasks.workunit.client.0.smithi007.stdout:5/910: getdents d1/d7/d1c/d19/d34/d5e/d6f 0 2024-03-16T09:01:22.582 INFO:tasks.workunit.client.0.smithi007.stdout:5/911: symlink d1/d7/d1c/d19/d34/d5e/d6f/l14d 0 2024-03-16T09:01:22.582 INFO:tasks.workunit.client.1.smithi019.stdout:2/849: rmdir d5/d2e 39 2024-03-16T09:01:22.583 INFO:tasks.workunit.client.1.smithi019.stdout:2/850: creat d5/d6/d104/df7/f127 x:0 0 0 2024-03-16T09:01:22.584 INFO:tasks.workunit.client.1.smithi019.stdout:2/851: mkdir d5/d111/d128 0 2024-03-16T09:01:22.593 INFO:tasks.workunit.client.1.smithi019.stdout:7/744: write d9/da/d13/d18/d30/d3e/f62 [708873,123599] 0 2024-03-16T09:01:22.600 INFO:tasks.workunit.client.1.smithi019.stdout:6/858: dwrite de/d26/d32/f4f [0,4194304] 0 2024-03-16T09:01:22.600 INFO:tasks.workunit.client.1.smithi019.stdout:5/780: dwrite d6/dd/d37/d76/f85 [0,4194304] 0 2024-03-16T09:01:22.602 INFO:tasks.workunit.client.1.smithi019.stdout:8/880: dread d6/ff1 [0,4194304] 0 2024-03-16T09:01:22.602 INFO:tasks.workunit.client.1.smithi019.stdout:8/881: chown d6/d2a/d45/d57/d72/d79/db7/c110 17 1 2024-03-16T09:01:22.603 INFO:tasks.workunit.client.1.smithi019.stdout:6/859: mkdir de/d26/d32/d4e/d10f 0 2024-03-16T09:01:22.603 INFO:tasks.workunit.client.1.smithi019.stdout:6/860: chown de/d26/d32/d97/fc5 3157 1 2024-03-16T09:01:22.603 INFO:tasks.workunit.client.1.smithi019.stdout:6/861: stat de/d26/d32/d4e/d75/l77 0 2024-03-16T09:01:22.604 INFO:tasks.workunit.client.1.smithi019.stdout:6/862: read - de/d26/d32/d97/d66/f74 zero size 2024-03-16T09:01:22.604 INFO:tasks.workunit.client.1.smithi019.stdout:8/882: getdents d6/d2a 0 2024-03-16T09:01:22.605 INFO:tasks.workunit.client.1.smithi019.stdout:6/863: link de/d26/d32/d97/d66/ddc/cfe de/c110 0 2024-03-16T09:01:22.605 INFO:tasks.workunit.client.1.smithi019.stdout:6/864: write de/d26/d32/d97/d66/ddc/ffc [836912,22791] 0 2024-03-16T09:01:22.613 INFO:tasks.workunit.client.1.smithi019.stdout:1/694: dwrite d16/d1f/d3c/f48 [0,4194304] 0 2024-03-16T09:01:22.615 INFO:tasks.workunit.client.0.smithi007.stdout:6/852: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/fc4 [0,4194304] 0 2024-03-16T09:01:22.619 INFO:tasks.workunit.client.1.smithi019.stdout:9/683: dwrite d1/d7/de/d24/d44/d60/f97 [0,4194304] 0 2024-03-16T09:01:22.619 INFO:tasks.workunit.client.1.smithi019.stdout:9/684: dread - d1/d98/fd1 zero size 2024-03-16T09:01:22.619 INFO:tasks.workunit.client.1.smithi019.stdout:9/685: creat d1/d7/d21/da2/fe9 x:0 0 0 2024-03-16T09:01:22.619 INFO:tasks.workunit.client.0.smithi007.stdout:6/853: rename d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c/fc6 to d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/f119 0 2024-03-16T09:01:22.619 INFO:tasks.workunit.client.0.smithi007.stdout:6/854: chown d2/d29/fab 465 1 2024-03-16T09:01:22.620 INFO:tasks.workunit.client.1.smithi019.stdout:1/695: rename d16/d31/d76/d92/d9f to d16/dcb 0 2024-03-16T09:01:22.623 INFO:tasks.workunit.client.1.smithi019.stdout:9/686: dread d1/d7/de/d24/f99 [0,4194304] 0 2024-03-16T09:01:22.624 INFO:tasks.workunit.client.1.smithi019.stdout:9/687: write d1/d7/fb2 [576663,102833] 0 2024-03-16T09:01:22.624 INFO:tasks.workunit.client.1.smithi019.stdout:9/688: stat d1/d98/dae/l17 0 2024-03-16T09:01:22.624 INFO:tasks.workunit.client.1.smithi019.stdout:1/696: mknod d16/d1f/d4f/d8a/ccc 0 2024-03-16T09:01:22.624 INFO:tasks.workunit.client.1.smithi019.stdout:1/697: truncate d16/d18/f1b 4386670 0 2024-03-16T09:01:22.625 INFO:tasks.workunit.client.1.smithi019.stdout:1/698: symlink d16/d31/d7b/lcd 0 2024-03-16T09:01:22.626 INFO:tasks.workunit.client.1.smithi019.stdout:9/689: symlink d1/d7/lea 0 2024-03-16T09:01:22.626 INFO:tasks.workunit.client.1.smithi019.stdout:9/690: truncate d1/d98/dae/f7a 883767 0 2024-03-16T09:01:22.627 INFO:tasks.workunit.client.1.smithi019.stdout:9/691: write d1/d7/de/d24/d44/d60/d74/fad [658755,22170] 0 2024-03-16T09:01:22.627 INFO:tasks.workunit.client.1.smithi019.stdout:1/699: mkdir d16/d31/d71/d89/dce 0 2024-03-16T09:01:22.627 INFO:tasks.workunit.client.1.smithi019.stdout:9/692: mknod d1/d98/dae/ceb 0 2024-03-16T09:01:22.628 INFO:tasks.workunit.client.1.smithi019.stdout:9/693: link d1/d7/de/l5c d1/d7/de/d24/d44/d60/d74/lec 0 2024-03-16T09:01:22.629 INFO:tasks.workunit.client.1.smithi019.stdout:9/694: mknod d1/d7/de/d24/d5b/dce/ced 0 2024-03-16T09:01:22.629 INFO:tasks.workunit.client.1.smithi019.stdout:9/695: chown d1/d7/d33/dc9/dca 2394 1 2024-03-16T09:01:22.630 INFO:tasks.workunit.client.1.smithi019.stdout:9/696: symlink d1/d7/de/d24/d5b/lee 0 2024-03-16T09:01:22.630 INFO:tasks.workunit.client.1.smithi019.stdout:9/697: fsync d1/d7/d21/f87 0 2024-03-16T09:01:22.631 INFO:tasks.workunit.client.1.smithi019.stdout:9/698: mkdir d1/d7/d33/d59/def 0 2024-03-16T09:01:22.631 INFO:tasks.workunit.client.1.smithi019.stdout:9/699: dread - d1/d7/de/d24/d44/d60/fbf zero size 2024-03-16T09:01:22.632 INFO:tasks.workunit.client.1.smithi019.stdout:9/700: mkdir d1/d98/dae/dc3/df0 0 2024-03-16T09:01:22.632 INFO:tasks.workunit.client.1.smithi019.stdout:4/705: dwrite d4/f26 [4194304,4194304] 0 2024-03-16T09:01:22.635 INFO:tasks.workunit.client.1.smithi019.stdout:9/701: symlink d1/d7/d86/lf1 0 2024-03-16T09:01:22.635 INFO:tasks.workunit.client.1.smithi019.stdout:4/706: rename d4/d6/d15/le6 to d4/d1b/d2a/d4f/le7 0 2024-03-16T09:01:22.636 INFO:tasks.workunit.client.1.smithi019.stdout:1/700: dread d16/d31/d71/f8d [0,4194304] 0 2024-03-16T09:01:22.637 INFO:tasks.workunit.client.1.smithi019.stdout:4/707: unlink d4/lb 0 2024-03-16T09:01:22.638 INFO:tasks.workunit.client.1.smithi019.stdout:4/708: rename d4/d1b/d2a/f4d to d4/d64/d9b/fe8 0 2024-03-16T09:01:22.640 INFO:tasks.workunit.client.1.smithi019.stdout:4/709: symlink d4/d1b/d49/d58/d80/d91/le9 0 2024-03-16T09:01:22.641 INFO:tasks.workunit.client.1.smithi019.stdout:4/710: symlink d4/d73/lea 0 2024-03-16T09:01:22.641 INFO:tasks.workunit.client.1.smithi019.stdout:4/711: mkdir d4/d6c/d89/deb 0 2024-03-16T09:01:22.648 INFO:tasks.workunit.client.1.smithi019.stdout:2/852: dwrite d5/d5a/d80/f108 [0,4194304] 0 2024-03-16T09:01:22.649 INFO:tasks.workunit.client.0.smithi007.stdout:5/912: dwrite d1/d7/d1c/d19/d34/d1d/dee/f12b [0,4194304] 0 2024-03-16T09:01:22.650 INFO:tasks.workunit.client.1.smithi019.stdout:2/853: mknod d5/d6/d2b/d2f/d35/d4b/dca/c129 0 2024-03-16T09:01:22.650 INFO:tasks.workunit.client.1.smithi019.stdout:2/854: fsync d5/de/f55 0 2024-03-16T09:01:22.651 INFO:tasks.workunit.client.1.smithi019.stdout:9/702: read d1/d7/de/f1a [2559896,111419] 0 2024-03-16T09:01:22.651 INFO:tasks.workunit.client.1.smithi019.stdout:9/703: fdatasync d1/d7/de/d24/f90 0 2024-03-16T09:01:22.651 INFO:tasks.workunit.client.1.smithi019.stdout:9/704: read d1/d7/de/d24/d44/d60/f72 [48634,125329] 0 2024-03-16T09:01:22.652 INFO:tasks.workunit.client.1.smithi019.stdout:9/705: write d1/f3 [4020309,112676] 0 2024-03-16T09:01:22.653 INFO:tasks.workunit.client.1.smithi019.stdout:9/706: read d1/d98/dae/f35 [138433,52782] 0 2024-03-16T09:01:22.654 INFO:tasks.workunit.client.1.smithi019.stdout:9/707: write d1/d7/f3e [344778,20990] 0 2024-03-16T09:01:22.654 INFO:tasks.workunit.client.1.smithi019.stdout:9/708: chown d1/d7/de/l69 118918 1 2024-03-16T09:01:22.655 INFO:tasks.workunit.client.1.smithi019.stdout:5/781: dwrite d6/d1f/fdd [0,4194304] 0 2024-03-16T09:01:22.655 INFO:tasks.workunit.client.1.smithi019.stdout:5/782: chown d6/d1f/f3f 0 1 2024-03-16T09:01:22.655 INFO:tasks.workunit.client.1.smithi019.stdout:0/810: sync 2024-03-16T09:01:22.656 INFO:tasks.workunit.client.1.smithi019.stdout:9/709: rename d1/d7/de/d24/d44/d60/d74/fbc to d1/d7/de/d24/ff2 0 2024-03-16T09:01:22.657 INFO:tasks.workunit.client.0.smithi007.stdout:6/855: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d1e/f85 [0,4194304] 0 2024-03-16T09:01:22.659 INFO:tasks.workunit.client.0.smithi007.stdout:6/856: symlink d2/d13/d3f/d50/l11a 0 2024-03-16T09:01:22.659 INFO:tasks.workunit.client.0.smithi007.stdout:6/857: readlink d2/d10/leb 0 2024-03-16T09:01:22.660 INFO:tasks.workunit.client.1.smithi019.stdout:8/883: dwrite d6/d2a/d3c/dab/d98/fca [0,4194304] 0 2024-03-16T09:01:22.661 INFO:tasks.workunit.client.1.smithi019.stdout:6/865: dwrite de/d26/d32/d97/d66/f90 [0,4194304] 0 2024-03-16T09:01:22.661 INFO:tasks.workunit.client.1.smithi019.stdout:1/701: dwrite f10 [0,4194304] 0 2024-03-16T09:01:22.662 INFO:tasks.workunit.client.0.smithi007.stdout:6/858: dread d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/f80 [0,4194304] 0 2024-03-16T09:01:22.662 INFO:tasks.workunit.client.0.smithi007.stdout:6/859: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/f11b x:0 0 0 2024-03-16T09:01:22.663 INFO:tasks.workunit.client.1.smithi019.stdout:8/884: link d6/d2a/d68/fd0 d6/d2a/d45/d57/d96/df5/f130 0 2024-03-16T09:01:22.676 INFO:tasks.workunit.client.0.smithi007.stdout:6/860: dread d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/f64 [0,4194304] 0 2024-03-16T09:01:22.677 INFO:tasks.workunit.client.0.smithi007.stdout:6/861: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/ddc 39 2024-03-16T09:01:22.677 INFO:tasks.workunit.client.0.smithi007.stdout:6/862: write d2/d13/d3f/d50/d78/dbe/dcb/f25 [5603935,11675] 0 2024-03-16T09:01:22.681 INFO:tasks.workunit.client.1.smithi019.stdout:2/855: dwrite d5/de/da2/faa [0,4194304] 0 2024-03-16T09:01:22.681 INFO:tasks.workunit.client.1.smithi019.stdout:4/712: dwrite d4/d6/d15/d1f/f44 [0,4194304] 0 2024-03-16T09:01:22.681 INFO:tasks.workunit.client.1.smithi019.stdout:8/885: dwrite d6/d2a/d45/d57/d96/df5/f130 [0,4194304] 0 2024-03-16T09:01:22.686 INFO:tasks.workunit.client.1.smithi019.stdout:8/886: link d6/d2a/d45/d57/d96/df5/l10e d6/d2a/d45/d5d/d12f/l131 0 2024-03-16T09:01:22.689 INFO:tasks.workunit.client.1.smithi019.stdout:8/887: rename d6/d2a/d45/d2c/f5c to d6/d2a/d45/d57/d72/d79/db7/f132 0 2024-03-16T09:01:22.690 INFO:tasks.workunit.client.1.smithi019.stdout:2/856: truncate d5/de/da2/faa 1461810 0 2024-03-16T09:01:22.692 INFO:tasks.workunit.client.1.smithi019.stdout:8/888: creat d6/d2a/d45/d57/d72/d79/dc1/f133 x:0 0 0 2024-03-16T09:01:22.692 INFO:tasks.workunit.client.1.smithi019.stdout:8/889: write d6/d2a/d45/d57/d72/d79/f128 [348667,2134] 0 2024-03-16T09:01:22.692 INFO:tasks.workunit.client.1.smithi019.stdout:2/857: creat d5/d6/d116/f12a x:0 0 0 2024-03-16T09:01:22.694 INFO:tasks.workunit.client.1.smithi019.stdout:8/890: rename d6/d2a/d45/fd7 to d6/d2a/d55/f134 0 2024-03-16T09:01:22.696 INFO:tasks.workunit.client.1.smithi019.stdout:2/858: truncate d5/d6/d104/dba/dfc/d74/fa3 833743 0 2024-03-16T09:01:22.696 INFO:tasks.workunit.client.1.smithi019.stdout:5/783: mknod d6/dd/d37/d54/cfc 0 2024-03-16T09:01:22.696 INFO:tasks.workunit.client.1.smithi019.stdout:2/859: chown d5/d6/d2b/d2f/fa4 106 1 2024-03-16T09:01:22.697 INFO:tasks.workunit.client.1.smithi019.stdout:0/811: mkdir d7/d8/dec/def/d48/d10c 0 2024-03-16T09:01:22.697 INFO:tasks.workunit.client.1.smithi019.stdout:3/757: sync 2024-03-16T09:01:22.697 INFO:tasks.workunit.client.1.smithi019.stdout:3/758: write d3/d8b/dae/d5a/d75/d7a/f87 [5206363,25538] 0 2024-03-16T09:01:22.697 INFO:tasks.workunit.client.1.smithi019.stdout:3/759: write d3/d1b/d88/d91/fc1 [159098,115562] 0 2024-03-16T09:01:22.698 INFO:tasks.workunit.client.1.smithi019.stdout:3/760: dread d3/d1b/fdb [0,4194304] 0 2024-03-16T09:01:22.698 INFO:tasks.workunit.client.1.smithi019.stdout:3/761: creat d3/d8b/da8/ff0 x:0 0 0 2024-03-16T09:01:22.698 INFO:tasks.workunit.client.1.smithi019.stdout:8/891: truncate d6/d2a/d45/d57/f100 1307442 0 2024-03-16T09:01:22.698 INFO:tasks.workunit.client.1.smithi019.stdout:1/702: mknod d16/dcb/daf/db2/d78/ccf 0 2024-03-16T09:01:22.700 INFO:tasks.workunit.client.1.smithi019.stdout:2/860: mknod d5/d6/d104/dba/dfc/d74/c12b 0 2024-03-16T09:01:22.700 INFO:tasks.workunit.client.1.smithi019.stdout:2/861: getdents d5/d6/d2b/d112 0 2024-03-16T09:01:22.700 INFO:tasks.workunit.client.1.smithi019.stdout:4/713: symlink d4/d6c/d79/lec 0 2024-03-16T09:01:22.702 INFO:tasks.workunit.client.1.smithi019.stdout:5/784: link d6/dd/d47/d70/db2/d89/dcc/le5 d6/dd/d47/d70/db2/d89/d9e/lfd 0 2024-03-16T09:01:22.703 INFO:tasks.workunit.client.1.smithi019.stdout:5/785: fsync d6/dd/d37/f58 0 2024-03-16T09:01:22.703 INFO:tasks.workunit.client.1.smithi019.stdout:2/862: symlink d5/d5a/l12c 0 2024-03-16T09:01:22.707 INFO:tasks.workunit.client.1.smithi019.stdout:6/866: truncate de/d26/d32/f4f 2501157 0 2024-03-16T09:01:22.707 INFO:tasks.workunit.client.1.smithi019.stdout:6/867: write de/d26/d32/d4e/d75/fa2 [3915869,49447] 0 2024-03-16T09:01:22.707 INFO:tasks.workunit.client.1.smithi019.stdout:6/868: readlink de/d26/d32/d97/l42 0 2024-03-16T09:01:22.707 INFO:tasks.workunit.client.1.smithi019.stdout:8/892: dread d6/d2a/d45/d57/d96/df5/f130 [0,4194304] 0 2024-03-16T09:01:22.708 INFO:tasks.workunit.client.1.smithi019.stdout:8/893: mknod d6/d2a/d3f/c135 0 2024-03-16T09:01:22.708 INFO:tasks.workunit.client.1.smithi019.stdout:8/894: stat d6/d2a/d3c/dab/d30/d4a/l5e 0 2024-03-16T09:01:22.709 INFO:tasks.workunit.client.1.smithi019.stdout:0/812: write d7/d1d/d63/d89/d9e/fc7 [1173609,120345] 0 2024-03-16T09:01:22.709 INFO:tasks.workunit.client.1.smithi019.stdout:1/703: mkdir d16/d1f/d4f/dd0 0 2024-03-16T09:01:22.709 INFO:tasks.workunit.client.1.smithi019.stdout:1/704: readlink d16/d1f/d4f/l97 0 2024-03-16T09:01:22.710 INFO:tasks.workunit.client.1.smithi019.stdout:4/714: rmdir d4/d1b/d49/d58/d80/d8e 39 2024-03-16T09:01:22.710 INFO:tasks.workunit.client.1.smithi019.stdout:4/715: truncate d4/d73/d8a/fd8 1761220 0 2024-03-16T09:01:22.711 INFO:tasks.workunit.client.1.smithi019.stdout:8/895: link d6/d2a/d3c/f46 d6/d2a/d3c/dab/d30/d4a/d67/f136 0 2024-03-16T09:01:22.711 INFO:tasks.workunit.client.1.smithi019.stdout:5/786: symlink d6/d1f/lfe 0 2024-03-16T09:01:22.715 INFO:tasks.workunit.client.1.smithi019.stdout:5/787: creat d6/dd/d47/fff x:0 0 0 2024-03-16T09:01:22.716 INFO:tasks.workunit.client.1.smithi019.stdout:5/788: mknod d6/d1f/d77/c100 0 2024-03-16T09:01:22.723 INFO:tasks.workunit.client.1.smithi019.stdout:1/705: write d16/f65 [3786740,35866] 0 2024-03-16T09:01:22.724 INFO:tasks.workunit.client.1.smithi019.stdout:3/762: dwrite d3/d1b/f6e [0,4194304] 0 2024-03-16T09:01:22.728 INFO:tasks.workunit.client.1.smithi019.stdout:3/763: creat d3/d1b/dc7/ff1 x:0 0 0 2024-03-16T09:01:22.729 INFO:tasks.workunit.client.1.smithi019.stdout:1/706: symlink d16/ld1 0 2024-03-16T09:01:22.730 INFO:tasks.workunit.client.1.smithi019.stdout:3/764: symlink d3/d1b/d40/de9/lf2 0 2024-03-16T09:01:22.730 INFO:tasks.workunit.client.1.smithi019.stdout:3/765: chown d3/d1b/dc7/ff1 231985896 1 2024-03-16T09:01:22.730 INFO:tasks.workunit.client.1.smithi019.stdout:3/766: truncate d3/d8b/da8/fe3 749237 0 2024-03-16T09:01:22.731 INFO:tasks.workunit.client.1.smithi019.stdout:3/767: write d3/d8b/dae/d5a/d75/d7a/dcd/fd7 [332730,98734] 0 2024-03-16T09:01:22.740 INFO:tasks.workunit.client.1.smithi019.stdout:4/716: dwrite d4/d64/d99/fca [0,4194304] 0 2024-03-16T09:01:22.740 INFO:tasks.workunit.client.1.smithi019.stdout:4/717: chown d4/d1b/d2a/c3a 1 1 2024-03-16T09:01:22.740 INFO:tasks.workunit.client.1.smithi019.stdout:4/718: dread - d4/d1b/d2a/d4f/f8b zero size 2024-03-16T09:01:22.741 INFO:tasks.workunit.client.1.smithi019.stdout:5/789: dwrite d6/dd/d47/d70/db2/d4c/f62 [0,4194304] 0 2024-03-16T09:01:22.741 INFO:tasks.workunit.client.1.smithi019.stdout:1/707: write d16/d18/f23 [1440042,26320] 0 2024-03-16T09:01:22.743 INFO:tasks.workunit.client.1.smithi019.stdout:5/790: creat d6/d19/f101 x:0 0 0 2024-03-16T09:01:22.745 INFO:tasks.workunit.client.1.smithi019.stdout:5/791: symlink d6/dd/dde/df1/l102 0 2024-03-16T09:01:22.745 INFO:tasks.workunit.client.1.smithi019.stdout:3/768: write d3/d89/f47 [5159097,111380] 0 2024-03-16T09:01:22.745 INFO:tasks.workunit.client.1.smithi019.stdout:3/769: write d3/d8b/da8/ff0 [836106,97622] 0 2024-03-16T09:01:22.745 INFO:tasks.workunit.client.1.smithi019.stdout:3/770: chown d3/d8b/dae/d5a/d75/d7a/cef 0 1 2024-03-16T09:01:22.748 INFO:tasks.workunit.client.1.smithi019.stdout:3/771: link d3/d89/cb2 d3/d11/cf3 0 2024-03-16T09:01:22.749 INFO:tasks.workunit.client.1.smithi019.stdout:3/772: symlink d3/d1b/d40/lf4 0 2024-03-16T09:01:22.749 INFO:tasks.workunit.client.1.smithi019.stdout:3/773: creat d3/d1b/d40/ff5 x:0 0 0 2024-03-16T09:01:22.751 INFO:tasks.workunit.client.1.smithi019.stdout:3/774: symlink d3/d8b/dae/d5a/d75/lf6 0 2024-03-16T09:01:22.752 INFO:tasks.workunit.client.1.smithi019.stdout:3/775: symlink d3/d8b/dae/d5a/d80/lf7 0 2024-03-16T09:01:22.753 INFO:tasks.workunit.client.1.smithi019.stdout:4/719: write d4/d1b/f57 [1882092,34942] 0 2024-03-16T09:01:22.753 INFO:tasks.workunit.client.1.smithi019.stdout:4/720: creat d4/d6/d15/da2/fed x:0 0 0 2024-03-16T09:01:22.754 INFO:tasks.workunit.client.1.smithi019.stdout:3/776: symlink d3/d1b/d88/d91/dbe/lf8 0 2024-03-16T09:01:22.754 INFO:tasks.workunit.client.1.smithi019.stdout:3/777: fdatasync d3/ff 0 2024-03-16T09:01:22.754 INFO:tasks.workunit.client.1.smithi019.stdout:5/792: dread d6/dd/f1d [0,4194304] 0 2024-03-16T09:01:22.755 INFO:tasks.workunit.client.1.smithi019.stdout:5/793: mknod d6/dd/d37/d76/c103 0 2024-03-16T09:01:22.756 INFO:tasks.workunit.client.1.smithi019.stdout:5/794: rmdir d6/dd/d47/d70/db2 39 2024-03-16T09:01:22.757 INFO:tasks.workunit.client.1.smithi019.stdout:5/795: write d6/d1f/d78/fe6 [56837,119507] 0 2024-03-16T09:01:22.757 INFO:tasks.workunit.client.1.smithi019.stdout:5/796: symlink d6/dd/d47/d70/db2/d66/l104 0 2024-03-16T09:01:22.758 INFO:tasks.workunit.client.1.smithi019.stdout:5/797: write d6/d1f/f55 [3248514,108819] 0 2024-03-16T09:01:22.759 INFO:tasks.workunit.client.1.smithi019.stdout:5/798: mkdir d6/dd/d47/d7a/d105 0 2024-03-16T09:01:22.759 INFO:tasks.workunit.client.1.smithi019.stdout:5/799: fsync d6/dd/d37/d76/f85 0 2024-03-16T09:01:22.759 INFO:tasks.workunit.client.1.smithi019.stdout:5/800: chown d6/dd/d61/lc7 0 1 2024-03-16T09:01:22.759 INFO:tasks.workunit.client.1.smithi019.stdout:5/801: creat d6/dd/d47/d70/f106 x:0 0 0 2024-03-16T09:01:22.759 INFO:tasks.workunit.client.1.smithi019.stdout:5/802: read d6/dd/f67 [3261135,108033] 0 2024-03-16T09:01:22.780 INFO:tasks.workunit.client.1.smithi019.stdout:1/708: dwrite d16/d1f/d3c/d79/f91 [0,4194304] 0 2024-03-16T09:01:22.781 INFO:tasks.workunit.client.1.smithi019.stdout:4/721: dwrite d4/d1b/d2a/f74 [0,4194304] 0 2024-03-16T09:01:22.783 INFO:tasks.workunit.client.1.smithi019.stdout:3/778: dread d3/d8b/da8/ff0 [0,4194304] 0 2024-03-16T09:01:22.784 INFO:tasks.workunit.client.1.smithi019.stdout:1/709: rename d16/dcb/daf/db2/l93 to d16/d1f/d3c/d79/d96/dac/ld2 0 2024-03-16T09:01:22.784 INFO:tasks.workunit.client.1.smithi019.stdout:1/710: creat d16/d1f/d3c/d79/fd3 x:0 0 0 2024-03-16T09:01:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:1/757: sync 2024-03-16T09:01:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:4/760: sync 2024-03-16T09:01:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:1/758: chown d9/ld2 2 1 2024-03-16T09:01:22.787 INFO:tasks.workunit.client.0.smithi007.stdout:7/860: sync 2024-03-16T09:01:22.788 INFO:tasks.workunit.client.1.smithi019.stdout:4/722: mkdir d4/d1b/d49/d58/d80/d8e/dee 0 2024-03-16T09:01:22.788 INFO:tasks.workunit.client.0.smithi007.stdout:7/861: truncate d0/d19/f48 2852023 0 2024-03-16T09:01:22.790 INFO:tasks.workunit.client.0.smithi007.stdout:7/862: write d0/dc/d5a/f89 [330124,35007] 0 2024-03-16T09:01:22.790 INFO:tasks.workunit.client.1.smithi019.stdout:5/803: rmdir d6/d19 39 2024-03-16T09:01:22.791 INFO:tasks.workunit.client.0.smithi007.stdout:4/761: mkdir d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d102/d103 0 2024-03-16T09:01:22.792 INFO:tasks.workunit.client.1.smithi019.stdout:4/723: fsync d4/d64/d99/f7b 0 2024-03-16T09:01:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:1/759: rename d9/dd/d1b/d28/d29/d7b/dba/dda/fed to d9/dd/d1b/dd8/fee 0 2024-03-16T09:01:22.792 INFO:tasks.workunit.client.0.smithi007.stdout:1/760: truncate d9/fca 1593089 0 2024-03-16T09:01:22.794 INFO:tasks.workunit.client.1.smithi019.stdout:3/779: getdents d3/d11 0 2024-03-16T09:01:22.794 INFO:tasks.workunit.client.1.smithi019.stdout:3/780: stat d3/d9b/f9f 0 2024-03-16T09:01:22.795 INFO:tasks.workunit.client.1.smithi019.stdout:5/804: mkdir d6/dd/d37/d76/d107 0 2024-03-16T09:01:22.795 INFO:tasks.workunit.client.0.smithi007.stdout:4/762: rename d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dd7/fb8 to d4/d14/d1c/f104 0 2024-03-16T09:01:22.798 INFO:tasks.workunit.client.0.smithi007.stdout:4/763: link d4/dcb/feb d4/d14/d64/d68/d82/dac/dfc/f105 0 2024-03-16T09:01:22.799 INFO:tasks.workunit.client.1.smithi019.stdout:3/781: mkdir d3/d89/df9 0 2024-03-16T09:01:22.800 INFO:tasks.workunit.client.0.smithi007.stdout:4/764: truncate d4/d14/d64/d68/f63 220784 0 2024-03-16T09:01:22.801 INFO:tasks.workunit.client.1.smithi019.stdout:5/805: creat d6/dd/d47/d70/db2/d66/f108 x:0 0 0 2024-03-16T09:01:22.801 INFO:tasks.workunit.client.1.smithi019.stdout:5/806: fdatasync d6/dd/d47/d70/db2/d89/f96 0 2024-03-16T09:01:22.802 INFO:tasks.workunit.client.0.smithi007.stdout:4/765: creat d4/f106 x:0 0 0 2024-03-16T09:01:22.804 INFO:tasks.workunit.client.1.smithi019.stdout:1/711: dwrite f12 [0,4194304] 0 2024-03-16T09:01:22.816 INFO:tasks.workunit.client.0.smithi007.stdout:1/761: dwrite d9/f62 [0,4194304] 0 2024-03-16T09:01:22.816 INFO:tasks.workunit.client.0.smithi007.stdout:1/762: write d9/dd/d1b/d2d/d30/f8e [881559,88147] 0 2024-03-16T09:01:22.816 INFO:tasks.workunit.client.0.smithi007.stdout:1/763: fsync d9/dd/d1b/d28/d4c/f63 0 2024-03-16T09:01:22.816 INFO:tasks.workunit.client.0.smithi007.stdout:1/764: write d9/d6b/fe8 [713209,42780] 0 2024-03-16T09:01:22.816 INFO:tasks.workunit.client.0.smithi007.stdout:1/765: write d9/dd/d1b/dd8/fee [697374,107130] 0 2024-03-16T09:01:22.817 INFO:tasks.workunit.client.0.smithi007.stdout:1/766: dread - d9/d6b/fd1 zero size 2024-03-16T09:01:22.817 INFO:tasks.workunit.client.0.smithi007.stdout:1/767: fdatasync d9/dd/d1b/d28/d29/d7b/db4/fce 0 2024-03-16T09:01:22.817 INFO:tasks.workunit.client.0.smithi007.stdout:4/766: dread d4/dcb/feb [0,4194304] 0 2024-03-16T09:01:22.818 INFO:tasks.workunit.client.0.smithi007.stdout:1/768: mkdir d9/def 0 2024-03-16T09:01:22.818 INFO:tasks.workunit.client.0.smithi007.stdout:1/769: write d9/dd/d1b/d2d/d30/fc3 [727258,88833] 0 2024-03-16T09:01:22.819 INFO:tasks.workunit.client.0.smithi007.stdout:4/767: link d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/cea d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/c107 0 2024-03-16T09:01:22.819 INFO:tasks.workunit.client.0.smithi007.stdout:4/768: fsync d4/d14/d1c/d5b/f62 0 2024-03-16T09:01:22.820 INFO:tasks.workunit.client.1.smithi019.stdout:4/724: rmdir d4 39 2024-03-16T09:01:22.820 INFO:tasks.workunit.client.0.smithi007.stdout:1/770: creat d9/ff0 x:0 0 0 2024-03-16T09:01:22.820 INFO:tasks.workunit.client.0.smithi007.stdout:1/771: fdatasync d9/fca 0 2024-03-16T09:01:22.821 INFO:tasks.workunit.client.1.smithi019.stdout:3/782: dwrite d3/d11/f6f [0,4194304] 0 2024-03-16T09:01:22.822 INFO:tasks.workunit.client.0.smithi007.stdout:1/772: link d9/fbc d9/dd/d5e/d89/ff1 0 2024-03-16T09:01:22.824 INFO:tasks.workunit.client.0.smithi007.stdout:1/773: symlink d9/dd/d1b/dd8/lf2 0 2024-03-16T09:01:22.825 INFO:tasks.workunit.client.1.smithi019.stdout:5/807: dwrite d6/d1f/fa4 [0,4194304] 0 2024-03-16T09:01:22.825 INFO:tasks.workunit.client.1.smithi019.stdout:5/808: write d6/dd/f67 [2671618,15632] 0 2024-03-16T09:01:22.825 INFO:tasks.workunit.client.1.smithi019.stdout:5/809: chown d6/dd/d47/d70/db2/d4c/fc6 37 1 2024-03-16T09:01:22.825 INFO:tasks.workunit.client.1.smithi019.stdout:4/725: symlink d4/d9e/lef 0 2024-03-16T09:01:22.826 INFO:tasks.workunit.client.1.smithi019.stdout:4/726: write d4/d64/d99/fca [4533750,62513] 0 2024-03-16T09:01:22.826 INFO:tasks.workunit.client.1.smithi019.stdout:1/712: rmdir d16/d84 39 2024-03-16T09:01:22.827 INFO:tasks.workunit.client.0.smithi007.stdout:1/774: rename d9/dd/d5e/f79 to d9/dd/d1b/dd8/ff3 0 2024-03-16T09:01:22.828 INFO:tasks.workunit.client.1.smithi019.stdout:3/783: creat d3/d1b/d40/d4a/db7/ffa x:0 0 0 2024-03-16T09:01:22.833 INFO:tasks.workunit.client.1.smithi019.stdout:5/810: symlink d6/d19/d34/l109 0 2024-03-16T09:01:22.833 INFO:tasks.workunit.client.1.smithi019.stdout:5/811: fdatasync d6/dd/d47/d70/fed 0 2024-03-16T09:01:22.834 INFO:tasks.workunit.client.1.smithi019.stdout:4/727: getdents d4/d1b/d49/d58/d80 0 2024-03-16T09:01:22.835 INFO:tasks.workunit.client.1.smithi019.stdout:1/713: mkdir d16/dcb/dd4 0 2024-03-16T09:01:22.836 INFO:tasks.workunit.client.0.smithi007.stdout:4/769: dwrite d4/d99/fbc [0,4194304] 0 2024-03-16T09:01:22.836 INFO:tasks.workunit.client.0.smithi007.stdout:4/770: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/l52 12826 1 2024-03-16T09:01:22.840 INFO:tasks.workunit.client.1.smithi019.stdout:5/812: unlink d6/dd/d37/d76/fcf 0 2024-03-16T09:01:22.843 INFO:tasks.workunit.client.0.smithi007.stdout:1/775: dread d9/d6b/d7e/dac/dbb/fbe [0,4194304] 0 2024-03-16T09:01:22.843 INFO:tasks.workunit.client.1.smithi019.stdout:4/728: rename d4/d73/ld2 to d4/d1b/lf0 0 2024-03-16T09:01:22.844 INFO:tasks.workunit.client.0.smithi007.stdout:1/776: creat d9/dd/d1b/d28/ff4 x:0 0 0 2024-03-16T09:01:22.844 INFO:tasks.workunit.client.1.smithi019.stdout:1/714: mkdir d16/dcb/dd4/dd5 0 2024-03-16T09:01:22.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/771: dread d4/d99/fbc [0,4194304] 0 2024-03-16T09:01:22.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/772: chown d4/d14/d64/d68/fc4 240101 1 2024-03-16T09:01:22.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/773: creat d4/d14/d64/d9f/f108 x:0 0 0 2024-03-16T09:01:22.845 INFO:tasks.workunit.client.1.smithi019.stdout:5/813: mkdir d6/d1f/d77/d10a 0 2024-03-16T09:01:22.847 INFO:tasks.workunit.client.1.smithi019.stdout:4/729: mknod d4/d9e/cf1 0 2024-03-16T09:01:22.850 INFO:tasks.workunit.client.1.smithi019.stdout:5/814: symlink d6/dd/d47/d70/db2/d89/dcc/l10b 0 2024-03-16T09:01:22.852 INFO:tasks.workunit.client.1.smithi019.stdout:1/715: rmdir d16/d1f/d3c/d79/d96 39 2024-03-16T09:01:22.852 INFO:tasks.workunit.client.1.smithi019.stdout:1/716: fsync d16/d1f/f41 0 2024-03-16T09:01:22.852 INFO:tasks.workunit.client.1.smithi019.stdout:1/717: chown d16/da1/fb0 1834 1 2024-03-16T09:01:22.852 INFO:tasks.workunit.client.1.smithi019.stdout:3/784: dwrite d3/d1b/d40/dc3/fea [0,4194304] 0 2024-03-16T09:01:22.856 INFO:tasks.workunit.client.0.smithi007.stdout:1/777: dread d9/dd/d1b/d28/f60 [0,4194304] 0 2024-03-16T09:01:22.856 INFO:tasks.workunit.client.1.smithi019.stdout:1/718: unlink d16/d18/f39 0 2024-03-16T09:01:22.857 INFO:tasks.workunit.client.1.smithi019.stdout:7/745: sync 2024-03-16T09:01:22.857 INFO:tasks.workunit.client.1.smithi019.stdout:9/710: sync 2024-03-16T09:01:22.857 INFO:tasks.workunit.client.1.smithi019.stdout:7/746: readlink d9/da/d13/l8e 0 2024-03-16T09:01:22.857 INFO:tasks.workunit.client.1.smithi019.stdout:9/711: readlink d1/d7/de/l5c 0 2024-03-16T09:01:22.858 INFO:tasks.workunit.client.1.smithi019.stdout:3/785: creat d3/d8b/dae/d6a/ffb x:0 0 0 2024-03-16T09:01:22.858 INFO:tasks.workunit.client.0.smithi007.stdout:4/774: dread d4/d14/d64/d68/f63 [0,4194304] 0 2024-03-16T09:01:22.859 INFO:tasks.workunit.client.1.smithi019.stdout:1/719: creat d16/d31/d7b/fd6 x:0 0 0 2024-03-16T09:01:22.861 INFO:tasks.workunit.client.0.smithi007.stdout:4/775: creat d4/d14/d64/d68/dbf/f109 x:0 0 0 2024-03-16T09:01:22.862 INFO:tasks.workunit.client.1.smithi019.stdout:3/786: unlink d3/ce2 0 2024-03-16T09:01:22.863 INFO:tasks.workunit.client.0.smithi007.stdout:4/776: chown d4/l72 31910 1 2024-03-16T09:01:22.863 INFO:tasks.workunit.client.0.smithi007.stdout:4/777: stat d4/d14/d64/d68/d82/dac/dfc/dd3/ff1 0 2024-03-16T09:01:22.863 INFO:tasks.workunit.client.1.smithi019.stdout:1/720: creat d16/d31/d71/fd7 x:0 0 0 2024-03-16T09:01:22.863 INFO:tasks.workunit.client.1.smithi019.stdout:1/721: fdatasync d16/d1f/f34 0 2024-03-16T09:01:22.864 INFO:tasks.workunit.client.1.smithi019.stdout:3/787: write d3/d11/f14 [5606806,101887] 0 2024-03-16T09:01:22.865 INFO:tasks.workunit.client.0.smithi007.stdout:4/778: mknod d4/d14/d64/d68/d82/c10a 0 2024-03-16T09:01:22.867 INFO:tasks.workunit.client.1.smithi019.stdout:1/722: rmdir d16/d1f/d4f/dd0 0 2024-03-16T09:01:22.868 INFO:tasks.workunit.client.0.smithi007.stdout:4/779: creat d4/d14/d1c/d5b/f10b x:0 0 0 2024-03-16T09:01:22.869 INFO:tasks.workunit.client.1.smithi019.stdout:3/788: symlink d3/d8b/lfc 0 2024-03-16T09:01:22.870 INFO:tasks.workunit.client.1.smithi019.stdout:3/789: stat d3/d8b/dae/d5a/f5c 0 2024-03-16T09:01:22.870 INFO:tasks.workunit.client.1.smithi019.stdout:3/790: readlink d3/d8b/dae/d6a/l7f 0 2024-03-16T09:01:22.870 INFO:tasks.workunit.client.0.smithi007.stdout:4/780: mknod d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b/dff/c10c 0 2024-03-16T09:01:22.871 INFO:tasks.workunit.client.1.smithi019.stdout:1/723: rename d16/c1d to d16/d31/d71/d89/cd8 0 2024-03-16T09:01:22.871 INFO:tasks.workunit.client.1.smithi019.stdout:1/724: chown d16/d18/f23 233 1 2024-03-16T09:01:22.871 INFO:tasks.workunit.client.1.smithi019.stdout:1/725: fdatasync d16/d31/d71/fd7 0 2024-03-16T09:01:22.871 INFO:tasks.workunit.client.1.smithi019.stdout:3/791: stat d3/d8b/da8/cb3 0 2024-03-16T09:01:22.871 INFO:tasks.workunit.client.1.smithi019.stdout:3/792: fdatasync d3/d1b/d88/fa5 0 2024-03-16T09:01:22.873 INFO:tasks.workunit.client.0.smithi007.stdout:4/781: link d4/d14/d64/d68/d82/dac/dfc/d8f/da8/dbe/fef d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/f10d 0 2024-03-16T09:01:22.873 INFO:tasks.workunit.client.0.smithi007.stdout:4/782: fdatasync d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/ffe 0 2024-03-16T09:01:22.874 INFO:tasks.workunit.client.0.smithi007.stdout:4/783: unlink d4/dcb/cd6 0 2024-03-16T09:01:22.876 INFO:tasks.workunit.client.0.smithi007.stdout:4/784: rename d4/d14/d64/d68/f6f to d4/f10e 0 2024-03-16T09:01:22.877 INFO:tasks.workunit.client.0.smithi007.stdout:4/785: symlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/l10f 0 2024-03-16T09:01:22.877 INFO:tasks.workunit.client.0.smithi007.stdout:4/786: truncate d4/d14/d64/d68/d82/dac/dfc/d8f/da8/f70 508151 0 2024-03-16T09:01:22.877 INFO:tasks.workunit.client.1.smithi019.stdout:3/793: dread d3/f30 [4194304,4194304] 0 2024-03-16T09:01:22.877 INFO:tasks.workunit.client.1.smithi019.stdout:3/794: creat d3/d8b/dae/d6a/ffd x:0 0 0 2024-03-16T09:01:22.878 INFO:tasks.workunit.client.0.smithi007.stdout:4/787: symlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/l110 0 2024-03-16T09:01:22.879 INFO:tasks.workunit.client.0.smithi007.stdout:4/788: creat d4/d14/d64/d68/d82/dac/dfc/d8f/f111 x:0 0 0 2024-03-16T09:01:22.879 INFO:tasks.workunit.client.0.smithi007.stdout:4/789: readlink d4/d14/d64/d9f/lec 0 2024-03-16T09:01:22.879 INFO:tasks.workunit.client.0.smithi007.stdout:4/790: truncate d4/d14/d64/fb5 708407 0 2024-03-16T09:01:22.879 INFO:tasks.workunit.client.1.smithi019.stdout:3/795: creat d3/d8b/dae/d5a/d80/ffe x:0 0 0 2024-03-16T09:01:22.880 INFO:tasks.workunit.client.0.smithi007.stdout:3/890: sync 2024-03-16T09:01:22.880 INFO:tasks.workunit.client.0.smithi007.stdout:9/914: sync 2024-03-16T09:01:22.882 INFO:tasks.workunit.client.0.smithi007.stdout:3/891: mknod d15/d18/d1f/d3b/dae/d10e/c117 0 2024-03-16T09:01:22.883 INFO:tasks.workunit.client.0.smithi007.stdout:3/892: rename d15/d18/d1f/d3b/d87/fb8 to d15/d18/d1f/d29/dfa/f118 0 2024-03-16T09:01:22.884 INFO:tasks.workunit.client.0.smithi007.stdout:3/893: creat d15/d18/d6a/f119 x:0 0 0 2024-03-16T09:01:22.884 INFO:tasks.workunit.client.0.smithi007.stdout:3/894: stat d15/d18/d6a/db1/fc1 0 2024-03-16T09:01:22.884 INFO:tasks.workunit.client.0.smithi007.stdout:3/895: write d15/d18/d1f/d29/d43/f69 [711889,12870] 0 2024-03-16T09:01:22.886 INFO:tasks.workunit.client.0.smithi007.stdout:3/896: creat d15/d18/d1f/dce/dd6/f11a x:0 0 0 2024-03-16T09:01:22.886 INFO:tasks.workunit.client.0.smithi007.stdout:3/897: dread - d15/d18/d1f/d29/d7c/ff8 zero size 2024-03-16T09:01:22.886 INFO:tasks.workunit.client.0.smithi007.stdout:3/898: chown d15/d18/d1f/d3b/d87/c8b 37489 1 2024-03-16T09:01:22.887 INFO:tasks.workunit.client.1.smithi019.stdout:9/712: dwrite d1/d98/dae/f58 [4194304,4194304] 0 2024-03-16T09:01:22.887 INFO:tasks.workunit.client.1.smithi019.stdout:7/747: dwrite d9/db/d12/fc4 [0,4194304] 0 2024-03-16T09:01:22.888 INFO:tasks.workunit.client.1.smithi019.stdout:4/730: dwrite d4/d6/d15/d9a/fa4 [4194304,4194304] 0 2024-03-16T09:01:22.899 INFO:tasks.workunit.client.0.smithi007.stdout:4/791: dread d4/d14/d64/d68/f56 [0,4194304] 0 2024-03-16T09:01:22.900 INFO:tasks.workunit.client.1.smithi019.stdout:4/731: write d4/d1b/dc1/fce [6138393,30448] 0 2024-03-16T09:01:22.901 INFO:tasks.workunit.client.1.smithi019.stdout:4/732: truncate d4/d64/d99/fca 3722456 0 2024-03-16T09:01:22.903 INFO:tasks.workunit.client.1.smithi019.stdout:3/796: getdents d3/d8b 0 2024-03-16T09:01:22.903 INFO:tasks.workunit.client.1.smithi019.stdout:3/797: chown d3/l66 1 1 2024-03-16T09:01:22.903 INFO:tasks.workunit.client.1.smithi019.stdout:3/798: chown d3/d1b/dc7/ff1 1 1 2024-03-16T09:01:22.904 INFO:tasks.workunit.client.0.smithi007.stdout:3/899: dread d15/d18/d1f/d3b/d87/dee/fff [0,4194304] 0 2024-03-16T09:01:22.904 INFO:tasks.workunit.client.0.smithi007.stdout:9/915: dwrite d24/f27 [4194304,4194304] 0 2024-03-16T09:01:22.905 INFO:tasks.workunit.client.1.smithi019.stdout:3/799: rmdir d3/d8b/dae/d6a 39 2024-03-16T09:01:22.906 INFO:tasks.workunit.client.0.smithi007.stdout:3/900: creat d15/d18/d1f/d29/d3e/dc0/da5/f11b x:0 0 0 2024-03-16T09:01:22.906 INFO:tasks.workunit.client.0.smithi007.stdout:3/901: dread - d15/d18/d1f/d56/f101 zero size 2024-03-16T09:01:22.907 INFO:tasks.workunit.client.1.smithi019.stdout:3/800: symlink d3/d89/df9/lff 0 2024-03-16T09:01:22.907 INFO:tasks.workunit.client.1.smithi019.stdout:3/801: chown d3/c8 483269635 1 2024-03-16T09:01:22.907 INFO:tasks.workunit.client.1.smithi019.stdout:3/802: fsync d3/d8b/dae/d5a/d75/d7a/f87 0 2024-03-16T09:01:22.907 INFO:tasks.workunit.client.1.smithi019.stdout:3/803: dread - d3/d11/fac zero size 2024-03-16T09:01:22.908 INFO:tasks.workunit.client.1.smithi019.stdout:3/804: mkdir d3/d11/d100 0 2024-03-16T09:01:22.909 INFO:tasks.workunit.client.0.smithi007.stdout:9/916: mkdir d24/d4a/da2/dd5/d111 0 2024-03-16T09:01:22.910 INFO:tasks.workunit.client.1.smithi019.stdout:9/713: dwrite d1/d98/fc5 [0,4194304] 0 2024-03-16T09:01:22.910 INFO:tasks.workunit.client.1.smithi019.stdout:9/714: chown d1/d7/c7e 373655 1 2024-03-16T09:01:22.912 INFO:tasks.workunit.client.0.smithi007.stdout:3/902: creat d15/d18/d1f/d3b/dae/d10e/f11c x:0 0 0 2024-03-16T09:01:22.915 INFO:tasks.workunit.client.1.smithi019.stdout:4/733: dread d4/d6/d15/fb3 [0,4194304] 0 2024-03-16T09:01:22.916 INFO:tasks.workunit.client.1.smithi019.stdout:4/734: mkdir d4/d6c/d79/df2 0 2024-03-16T09:01:22.917 INFO:tasks.workunit.client.1.smithi019.stdout:4/735: fdatasync d4/d6/f39 0 2024-03-16T09:01:22.917 INFO:tasks.workunit.client.1.smithi019.stdout:4/736: chown d4/d64/c7f 15828 1 2024-03-16T09:01:22.936 INFO:tasks.workunit.client.0.smithi007.stdout:1/778: dwrite d9/dd/d1b/d2d/d30/d45/fbd [0,4194304] 0 2024-03-16T09:01:22.937 INFO:tasks.workunit.client.0.smithi007.stdout:1/779: dread - d9/dd/d1b/d28/ff4 zero size 2024-03-16T09:01:22.937 INFO:tasks.workunit.client.0.smithi007.stdout:1/780: write d9/d6b/d7e/f92 [61548,114753] 0 2024-03-16T09:01:22.937 INFO:tasks.workunit.client.0.smithi007.stdout:4/792: dwrite d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/fa9 [4194304,4194304] 0 2024-03-16T09:01:22.937 INFO:tasks.workunit.client.0.smithi007.stdout:3/903: dwrite d15/d18/f33 [0,4194304] 0 2024-03-16T09:01:22.937 INFO:tasks.workunit.client.0.smithi007.stdout:3/904: dread - d15/d18/d1f/d3b/ff3 zero size 2024-03-16T09:01:22.937 INFO:tasks.workunit.client.0.smithi007.stdout:3/905: fsync d15/d18/d1f/d29/d3e/d85/fcb 0 2024-03-16T09:01:22.938 INFO:tasks.workunit.client.1.smithi019.stdout:7/748: dwrite d9/da/f24 [0,4194304] 0 2024-03-16T09:01:22.938 INFO:tasks.workunit.client.1.smithi019.stdout:7/749: truncate d9/da/fae 945030 0 2024-03-16T09:01:22.938 INFO:tasks.workunit.client.1.smithi019.stdout:7/750: dread d9/d25/d28/d48/ddd/fd9 [0,4194304] 0 2024-03-16T09:01:22.940 INFO:tasks.workunit.client.1.smithi019.stdout:4/737: read d4/d64/d99/fca [276778,19339] 0 2024-03-16T09:01:22.942 INFO:tasks.workunit.client.0.smithi007.stdout:4/793: link d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/c7a d4/dcb/dcd/c112 0 2024-03-16T09:01:22.942 INFO:tasks.workunit.client.0.smithi007.stdout:4/794: write d4/d14/d64/d9f/f108 [608315,73784] 0 2024-03-16T09:01:22.943 INFO:tasks.workunit.client.1.smithi019.stdout:7/751: mknod d9/da/d13/d18/d30/d75/cef 0 2024-03-16T09:01:22.944 INFO:tasks.workunit.client.1.smithi019.stdout:7/752: creat d9/d25/d93/ff0 x:0 0 0 2024-03-16T09:01:22.947 INFO:tasks.workunit.client.0.smithi007.stdout:3/906: creat d15/d18/d1f/d3b/d87/f11d x:0 0 0 2024-03-16T09:01:22.947 INFO:tasks.workunit.client.1.smithi019.stdout:7/753: mknod d9/da/cf1 0 2024-03-16T09:01:22.950 INFO:tasks.workunit.client.0.smithi007.stdout:3/907: mkdir d15/d18/d1f/dce/dd6/ddf/d11e 0 2024-03-16T09:01:22.955 INFO:tasks.workunit.client.0.smithi007.stdout:3/908: creat d15/d18/d1f/d56/d98/f11f x:0 0 0 2024-03-16T09:01:22.957 INFO:tasks.workunit.client.0.smithi007.stdout:1/781: dwrite d9/dd/d1b/d28/f32 [4194304,4194304] 0 2024-03-16T09:01:22.961 INFO:tasks.workunit.client.1.smithi019.stdout:7/754: write d9/db/f47 [2405552,11608] 0 2024-03-16T09:01:22.961 INFO:tasks.workunit.client.1.smithi019.stdout:7/755: symlink d9/da/d13/d9a/dab/lf2 0 2024-03-16T09:01:22.961 INFO:tasks.workunit.client.1.smithi019.stdout:7/756: write d9/d25/d28/d48/d8d/dba/dad/dd8/fe0 [3236642,6099] 0 2024-03-16T09:01:22.964 INFO:tasks.workunit.client.0.smithi007.stdout:1/782: mknod d9/dd/d1b/d28/d4c/cf5 0 2024-03-16T09:01:22.964 INFO:tasks.workunit.client.0.smithi007.stdout:1/783: fsync d9/f85 0 2024-03-16T09:01:22.964 INFO:tasks.workunit.client.0.smithi007.stdout:1/784: chown d9/d6b/d7e/dac/dbb/fd4 439695 1 2024-03-16T09:01:22.964 INFO:tasks.workunit.client.0.smithi007.stdout:1/785: write d9/fbc [602161,10383] 0 2024-03-16T09:01:22.964 INFO:tasks.workunit.client.0.smithi007.stdout:1/786: readlink d9/dd/d1b/d58/lbf 0 2024-03-16T09:01:22.965 INFO:tasks.workunit.client.1.smithi019.stdout:9/715: dwrite d1/d98/dae/f7a [0,4194304] 0 2024-03-16T09:01:22.965 INFO:tasks.workunit.client.1.smithi019.stdout:9/716: fsync d1/d98/dae/f4e 0 2024-03-16T09:01:22.965 INFO:tasks.workunit.client.1.smithi019.stdout:9/717: write d1/d7/f81 [1829528,16128] 0 2024-03-16T09:01:22.965 INFO:tasks.workunit.client.0.smithi007.stdout:1/787: mknod d9/d6b/d7e/cf6 0 2024-03-16T09:01:22.966 INFO:tasks.workunit.client.1.smithi019.stdout:9/718: mkdir d1/d7/d33/df3 0 2024-03-16T09:01:22.966 INFO:tasks.workunit.client.1.smithi019.stdout:9/719: chown d1/d7/d67/d77/fe0 131269591 1 2024-03-16T09:01:22.967 INFO:tasks.workunit.client.0.smithi007.stdout:4/795: rmdir d4/d14/d64/d68/d82/dac/dfc/d8f 39 2024-03-16T09:01:22.968 INFO:tasks.workunit.client.0.smithi007.stdout:1/788: dread d9/dd/d1b/d2d/d30/f61 [0,4194304] 0 2024-03-16T09:01:22.968 INFO:tasks.workunit.client.1.smithi019.stdout:6/869: sync 2024-03-16T09:01:22.968 INFO:tasks.workunit.client.1.smithi019.stdout:2/863: sync 2024-03-16T09:01:22.968 INFO:tasks.workunit.client.1.smithi019.stdout:2/864: readlink d5/d5a/d80/l94 0 2024-03-16T09:01:22.969 INFO:tasks.workunit.client.1.smithi019.stdout:6/870: truncate de/d1c/f10e 983690 0 2024-03-16T09:01:22.969 INFO:tasks.workunit.client.0.smithi007.stdout:4/796: rmdir d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b/dff 39 2024-03-16T09:01:22.970 INFO:tasks.workunit.client.1.smithi019.stdout:2/865: rmdir d5/d6/d104/df7/d6f/da6/d83/d11c 39 2024-03-16T09:01:22.973 INFO:tasks.workunit.client.0.smithi007.stdout:3/909: read d15/d18/d6a/d78/fa8 [954232,20669] 0 2024-03-16T09:01:22.973 INFO:tasks.workunit.client.0.smithi007.stdout:3/910: write d15/d18/f23 [4080809,47777] 0 2024-03-16T09:01:22.974 INFO:tasks.workunit.client.1.smithi019.stdout:2/866: rename d5/d6/d2b/d2f/d35/d65/f121 to d5/d6/d104/dba/dfc/f12d 0 2024-03-16T09:01:22.975 INFO:tasks.workunit.client.1.smithi019.stdout:2/867: rename d5/d5a/c6a to d5/d6/d2b/d53/c12e 0 2024-03-16T09:01:22.978 INFO:tasks.workunit.client.0.smithi007.stdout:3/911: link d15/d18/c21 d15/c120 0 2024-03-16T09:01:22.979 INFO:tasks.workunit.client.1.smithi019.stdout:2/868: creat d5/d6/d104/df7/f12f x:0 0 0 2024-03-16T09:01:22.981 INFO:tasks.workunit.client.0.smithi007.stdout:3/912: getdents d15/d18/d1f/d3b 0 2024-03-16T09:01:22.982 INFO:tasks.workunit.client.1.smithi019.stdout:6/871: write de/f86 [3783358,91825] 0 2024-03-16T09:01:22.983 INFO:tasks.workunit.client.1.smithi019.stdout:2/869: creat d5/d5a/de6/f130 x:0 0 0 2024-03-16T09:01:22.983 INFO:tasks.workunit.client.0.smithi007.stdout:5/913: sync 2024-03-16T09:01:22.983 INFO:tasks.workunit.client.0.smithi007.stdout:6/863: sync 2024-03-16T09:01:22.983 INFO:tasks.workunit.client.0.smithi007.stdout:6/864: chown d2/d13/d4b/fa6 89 1 2024-03-16T09:01:22.984 INFO:tasks.workunit.client.1.smithi019.stdout:6/872: mknod de/d26/d32/d97/d66/d72/d7f/dcb/c111 0 2024-03-16T09:01:22.985 INFO:tasks.workunit.client.0.smithi007.stdout:3/913: mkdir d15/d18/d121 0 2024-03-16T09:01:22.988 INFO:tasks.workunit.client.0.smithi007.stdout:6/865: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d11c 0 2024-03-16T09:01:22.989 INFO:tasks.workunit.client.0.smithi007.stdout:5/914: symlink d1/d7/d1c/d19/d34/d5e/d6f/d132/l14e 0 2024-03-16T09:01:22.991 INFO:tasks.workunit.client.0.smithi007.stdout:6/866: truncate d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/f97 3468782 0 2024-03-16T09:01:22.991 INFO:tasks.workunit.client.0.smithi007.stdout:3/914: dread d15/d18/d1f/d3b/d87/dee/fff [0,4194304] 0 2024-03-16T09:01:22.992 INFO:tasks.workunit.client.0.smithi007.stdout:3/915: fsync d15/f16 0 2024-03-16T09:01:22.994 INFO:tasks.workunit.client.1.smithi019.stdout:9/720: dwrite d1/d7/f68 [0,4194304] 0 2024-03-16T09:01:22.994 INFO:tasks.workunit.client.1.smithi019.stdout:9/721: fdatasync d1/d7/de/d24/d44/d76/d9c/fb6 0 2024-03-16T09:01:22.994 INFO:tasks.workunit.client.1.smithi019.stdout:9/722: chown d1/d98/dae 3 1 2024-03-16T09:01:22.994 INFO:tasks.workunit.client.0.smithi007.stdout:5/915: creat d1/d7/d1c/d19/d34/d5e/d6f/d72/f14f x:0 0 0 2024-03-16T09:01:22.995 INFO:tasks.workunit.client.0.smithi007.stdout:3/916: link d15/d18/d6a/d78/l86 d15/d18/d1f/d3b/d87/l122 0 2024-03-16T09:01:22.995 INFO:tasks.workunit.client.1.smithi019.stdout:9/723: creat d1/d7/dc0/de2/ff4 x:0 0 0 2024-03-16T09:01:22.996 INFO:tasks.workunit.client.0.smithi007.stdout:5/916: getdents d1/d7/d1c/d88/da2/d89/db3 0 2024-03-16T09:01:22.996 INFO:tasks.workunit.client.0.smithi007.stdout:5/917: read d1/d7/d1c/d88/da2/d89/db3/d2a/d81/f11a [958358,37231] 0 2024-03-16T09:01:22.996 INFO:tasks.workunit.client.0.smithi007.stdout:5/918: truncate d1/d7/d1c/d88/d11e/f12a 984879 0 2024-03-16T09:01:22.996 INFO:tasks.workunit.client.0.smithi007.stdout:5/919: fsync d1/d7/d1c/d88/d11e/f12a 0 2024-03-16T09:01:22.996 INFO:tasks.workunit.client.0.smithi007.stdout:5/920: read - d1/d7/d1c/d19/d34/d5e/d146/df3/f10f zero size 2024-03-16T09:01:22.996 INFO:tasks.workunit.client.1.smithi019.stdout:9/724: creat d1/d7/de/d24/d44/d60/d74/de3/ff5 x:0 0 0 2024-03-16T09:01:22.997 INFO:tasks.workunit.client.1.smithi019.stdout:9/725: mkdir d1/d7/de/da3/df6 0 2024-03-16T09:01:22.997 INFO:tasks.workunit.client.0.smithi007.stdout:1/789: dwrite d9/f8b [0,4194304] 0 2024-03-16T09:01:22.998 INFO:tasks.workunit.client.0.smithi007.stdout:3/917: rename d15/d18/d1f/d29/d43/c62 to d15/d18/d1f/d3b/d87/dee/c123 0 2024-03-16T09:01:22.998 INFO:tasks.workunit.client.1.smithi019.stdout:9/726: creat d1/d7/de/d24/ff7 x:0 0 0 2024-03-16T09:01:22.999 INFO:tasks.workunit.client.0.smithi007.stdout:6/867: dread d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/ff4 [0,4194304] 0 2024-03-16T09:01:22.999 INFO:tasks.workunit.client.0.smithi007.stdout:4/797: dwrite d4/d14/d64/d9f/f42 [0,4194304] 0 2024-03-16T09:01:23.000 INFO:tasks.workunit.client.0.smithi007.stdout:5/921: rename d1/d7/d1c/d19/d34/d3e/d41/da6/l140 to d1/d7/d1c/d19/d34/d3e/d41/da6/de8/l150 0 2024-03-16T09:01:23.000 INFO:tasks.workunit.client.0.smithi007.stdout:5/922: truncate d1/d7/d1c/d19/d34/d1d/f11d 645290 0 2024-03-16T09:01:23.001 INFO:tasks.workunit.client.0.smithi007.stdout:3/918: creat d15/d18/d1f/d77/f124 x:0 0 0 2024-03-16T09:01:23.002 INFO:tasks.workunit.client.0.smithi007.stdout:6/868: truncate d2/d10/d3d/d6d/dbf/fd5 333672 0 2024-03-16T09:01:23.003 INFO:tasks.workunit.client.1.smithi019.stdout:2/870: dwrite d5/ff [0,4194304] 0 2024-03-16T09:01:23.004 INFO:tasks.workunit.client.1.smithi019.stdout:2/871: chown d5/d6/d2b/d53/f71 10877 1 2024-03-16T09:01:23.004 INFO:tasks.workunit.client.1.smithi019.stdout:2/872: write d5/de/f5d [551252,100374] 0 2024-03-16T09:01:23.004 INFO:tasks.workunit.client.0.smithi007.stdout:4/798: creat d4/d14/d64/d68/d82/f113 x:0 0 0 2024-03-16T09:01:23.005 INFO:tasks.workunit.client.0.smithi007.stdout:4/799: dread d4/d14/d64/d68/d82/dac/dfc/d8f/da8/f70 [0,4194304] 0 2024-03-16T09:01:23.005 INFO:tasks.workunit.client.0.smithi007.stdout:1/790: rename d9/dd/d1b/d28/c71 to d9/dd/d1b/d28/d29/d7b/dba/cf7 0 2024-03-16T09:01:23.007 INFO:tasks.workunit.client.0.smithi007.stdout:4/800: write d4/d14/f19 [4498354,30956] 0 2024-03-16T09:01:23.007 INFO:tasks.workunit.client.0.smithi007.stdout:4/801: dread - d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/fb4 zero size 2024-03-16T09:01:23.007 INFO:tasks.workunit.client.0.smithi007.stdout:4/802: fdatasync d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/fd9 0 2024-03-16T09:01:23.008 INFO:tasks.workunit.client.0.smithi007.stdout:5/923: mkdir d1/d7/d1c/d88/da2/df4/d151 0 2024-03-16T09:01:23.008 INFO:tasks.workunit.client.0.smithi007.stdout:5/924: write d1/d7/d1c/f15 [2191945,80713] 0 2024-03-16T09:01:23.010 INFO:tasks.workunit.client.1.smithi019.stdout:2/873: creat d5/d6/d2b/d2f/d35/d4b/f131 x:0 0 0 2024-03-16T09:01:23.012 INFO:tasks.workunit.client.1.smithi019.stdout:2/874: dread d5/d5a/f9d [0,4194304] 0 2024-03-16T09:01:23.012 INFO:tasks.workunit.client.0.smithi007.stdout:3/919: truncate d15/d18/d1f/d3b/d63/f66 2143938 0 2024-03-16T09:01:23.014 INFO:tasks.workunit.client.0.smithi007.stdout:6/869: creat d2/d13/d62/f11d x:0 0 0 2024-03-16T09:01:23.014 INFO:tasks.workunit.client.0.smithi007.stdout:6/870: creat d2/d13/d24/f11e x:0 0 0 2024-03-16T09:01:23.015 INFO:tasks.workunit.client.0.smithi007.stdout:1/791: mknod d9/d6b/d7e/d91/cf8 0 2024-03-16T09:01:23.016 INFO:tasks.workunit.client.0.smithi007.stdout:4/803: stat d4/dcb/dcd/c112 0 2024-03-16T09:01:23.016 INFO:tasks.workunit.client.0.smithi007.stdout:4/804: stat d4/d14/d64/d68/d82/dac/dfc/dd3 0 2024-03-16T09:01:23.017 INFO:tasks.workunit.client.1.smithi019.stdout:9/727: dwrite d1/d7/f68 [0,4194304] 0 2024-03-16T09:01:23.018 INFO:tasks.workunit.client.0.smithi007.stdout:3/920: truncate d15/d18/d6a/d78/f9e 1260076 0 2024-03-16T09:01:23.022 INFO:tasks.workunit.client.0.smithi007.stdout:5/925: dread d1/d7/d1c/d19/d34/d5e/d6f/d72/fab [0,4194304] 0 2024-03-16T09:01:23.022 INFO:tasks.workunit.client.0.smithi007.stdout:5/926: stat d1/d7/d1c/d88/da2/d89/db3/d2a/d81 0 2024-03-16T09:01:23.023 INFO:tasks.workunit.client.0.smithi007.stdout:5/927: dread - d1/d7/d1c/d19/d34/d3e/d41/da6/f145 zero size 2024-03-16T09:01:23.023 INFO:tasks.workunit.client.0.smithi007.stdout:6/871: link d2/d13/d3f/d50/d78/d81/lb3 d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/l11f 0 2024-03-16T09:01:23.024 INFO:tasks.workunit.client.0.smithi007.stdout:4/805: read d4/d14/d1c/d5b/f74 [3270050,10834] 0 2024-03-16T09:01:23.025 INFO:tasks.workunit.client.0.smithi007.stdout:1/792: symlink d9/dd/d1b/d28/d4c/de4/lf9 0 2024-03-16T09:01:23.025 INFO:tasks.workunit.client.0.smithi007.stdout:3/921: write d15/d18/d6a/f7e [1048179,60057] 0 2024-03-16T09:01:23.027 INFO:tasks.workunit.client.0.smithi007.stdout:3/922: dread f10 [4194304,4194304] 0 2024-03-16T09:01:23.027 INFO:tasks.workunit.client.0.smithi007.stdout:3/923: write d15/d18/d1f/d56/d98/f11f [280608,114529] 0 2024-03-16T09:01:23.027 INFO:tasks.workunit.client.0.smithi007.stdout:5/928: symlink d1/d7/d16/d1b/df1/l152 0 2024-03-16T09:01:23.028 INFO:tasks.workunit.client.0.smithi007.stdout:5/929: write d1/d7/d1c/d88/da2/d89/db3/f6e [1010117,123234] 0 2024-03-16T09:01:23.029 INFO:tasks.workunit.client.1.smithi019.stdout:9/728: read d1/d7/de/d24/ff2 [766782,11707] 0 2024-03-16T09:01:23.029 INFO:tasks.workunit.client.1.smithi019.stdout:9/729: stat d1/d7/de/d24/d5b/c71 0 2024-03-16T09:01:23.029 INFO:tasks.workunit.client.0.smithi007.stdout:6/872: getdents d2/d10 0 2024-03-16T09:01:23.029 INFO:tasks.workunit.client.0.smithi007.stdout:6/873: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/f120 x:0 0 0 2024-03-16T09:01:23.032 INFO:tasks.workunit.client.0.smithi007.stdout:4/806: creat d4/d14/d64/d68/d82/dac/f114 x:0 0 0 2024-03-16T09:01:23.034 INFO:tasks.workunit.client.0.smithi007.stdout:1/793: symlink d9/dd/d5e/d89/lfa 0 2024-03-16T09:01:23.037 INFO:tasks.workunit.client.0.smithi007.stdout:3/924: mknod d15/d18/d1f/d29/dfa/c125 0 2024-03-16T09:01:23.037 INFO:tasks.workunit.client.0.smithi007.stdout:3/925: chown d15/c17 3857 1 2024-03-16T09:01:23.037 INFO:tasks.workunit.client.0.smithi007.stdout:3/926: chown d15/d18/d1f/d29/d43/d72/l9b 11541 1 2024-03-16T09:01:23.037 INFO:tasks.workunit.client.0.smithi007.stdout:3/927: getdents d15/d18/d6a/db1/d110 0 2024-03-16T09:01:23.038 INFO:tasks.workunit.client.1.smithi019.stdout:9/730: dread d1/d98/dae/f58 [4194304,4194304] 0 2024-03-16T09:01:23.038 INFO:tasks.workunit.client.1.smithi019.stdout:9/731: write d1/d98/dae/dc3/fdf [1046963,31605] 0 2024-03-16T09:01:23.041 INFO:tasks.workunit.client.0.smithi007.stdout:4/807: rename d4/d14/d64/d68/d82/dac/dfc/dd3 to d4/dcb/d115 0 2024-03-16T09:01:23.042 INFO:tasks.workunit.client.1.smithi019.stdout:9/732: rename d1/d7/d33 to d1/d7/de/d24/d5b/dce/df8 0 2024-03-16T09:01:23.042 INFO:tasks.workunit.client.1.smithi019.stdout:9/733: symlink d1/d7/de/d24/d5b/dce/lf9 0 2024-03-16T09:01:23.044 INFO:tasks.workunit.client.1.smithi019.stdout:9/734: symlink d1/d7/lfa 0 2024-03-16T09:01:23.045 INFO:tasks.workunit.client.0.smithi007.stdout:4/808: mknod d4/d14/d64/d9f/c116 0 2024-03-16T09:01:23.045 INFO:tasks.workunit.client.1.smithi019.stdout:9/735: creat d1/d7/d21/ffb x:0 0 0 2024-03-16T09:01:23.045 INFO:tasks.workunit.client.1.smithi019.stdout:9/736: chown d1/d7/de/d24/d44/d76/d9c/dcc 142159 1 2024-03-16T09:01:23.046 INFO:tasks.workunit.client.0.smithi007.stdout:3/928: getdents d15/d18/d6a 0 2024-03-16T09:01:23.046 INFO:tasks.workunit.client.1.smithi019.stdout:9/737: symlink d1/d7/de/d24/d5b/dce/lfc 0 2024-03-16T09:01:23.049 INFO:tasks.workunit.client.0.smithi007.stdout:1/794: rename d9/dd/d1b/c1f to d9/dd/d1b/d2d/d30/cfb 0 2024-03-16T09:01:23.049 INFO:tasks.workunit.client.0.smithi007.stdout:1/795: chown d9/dd/d1b/d2d/d30/d45 464558704 1 2024-03-16T09:01:23.050 INFO:tasks.workunit.client.0.smithi007.stdout:4/809: mknod d4/d14/d64/d68/d82/dac/dfc/de8/c117 0 2024-03-16T09:01:23.051 INFO:tasks.workunit.client.0.smithi007.stdout:6/874: dwrite d2/d13/d3f/d50/d78/fcd [0,4194304] 0 2024-03-16T09:01:23.052 INFO:tasks.workunit.client.0.smithi007.stdout:5/930: dwrite d1/d7/d1c/f15 [0,4194304] 0 2024-03-16T09:01:23.052 INFO:tasks.workunit.client.0.smithi007.stdout:1/796: link d9/dd/d1b/d28/c6f d9/d6b/cfc 0 2024-03-16T09:01:23.053 INFO:tasks.workunit.client.0.smithi007.stdout:4/810: symlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/l118 0 2024-03-16T09:01:23.054 INFO:tasks.workunit.client.0.smithi007.stdout:6/875: dread d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/f5c [0,4194304] 0 2024-03-16T09:01:23.056 INFO:tasks.workunit.client.1.smithi019.stdout:9/738: write d1/d7/de/d24/d5b/f6c [3672539,24984] 0 2024-03-16T09:01:23.060 INFO:tasks.workunit.client.0.smithi007.stdout:3/929: dwrite d15/d18/d1f/d3b/dae/d10e/f11c [0,4194304] 0 2024-03-16T09:01:23.060 INFO:tasks.workunit.client.0.smithi007.stdout:3/930: write f5 [2180518,103556] 0 2024-03-16T09:01:23.065 INFO:tasks.workunit.client.1.smithi019.stdout:0/813: sync 2024-03-16T09:01:23.066 INFO:tasks.workunit.client.1.smithi019.stdout:8/896: sync 2024-03-16T09:01:23.066 INFO:tasks.workunit.client.1.smithi019.stdout:0/814: readlink d7/d1d/d63/d89/lbc 0 2024-03-16T09:01:23.066 INFO:tasks.workunit.client.1.smithi019.stdout:0/815: readlink d7/d8/d32/d4a/d61/lb1 0 2024-03-16T09:01:23.066 INFO:tasks.workunit.client.1.smithi019.stdout:0/816: stat d7/d8/d32/d66/d6d/d8c/db8/d82/c45 0 2024-03-16T09:01:23.068 INFO:tasks.workunit.client.1.smithi019.stdout:8/897: symlink d6/d2a/d45/d19/d74/l137 0 2024-03-16T09:01:23.072 INFO:tasks.workunit.client.0.smithi007.stdout:5/931: rename d1/d7/d1c/df0/lfd to d1/d7/d1c/d19/dcc/d102/d10e/l153 0 2024-03-16T09:01:23.072 INFO:tasks.workunit.client.0.smithi007.stdout:5/932: chown d1/d7/d1c/d88/da2/d89/db3/f31 27 1 2024-03-16T09:01:23.073 INFO:tasks.workunit.client.0.smithi007.stdout:4/811: rmdir d4/d99 39 2024-03-16T09:01:23.073 INFO:tasks.workunit.client.0.smithi007.stdout:4/812: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b 176 1 2024-03-16T09:01:23.073 INFO:tasks.workunit.client.0.smithi007.stdout:4/813: read - d4/d14/d64/d68/d82/dac/dfc/d8f/da8/fc6 zero size 2024-03-16T09:01:23.073 INFO:tasks.workunit.client.0.smithi007.stdout:4/814: chown d4/d14/d64/d68/lba 3974747 1 2024-03-16T09:01:23.074 INFO:tasks.workunit.client.0.smithi007.stdout:6/876: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/d8f/fb1 7078 0 2024-03-16T09:01:23.075 INFO:tasks.workunit.client.0.smithi007.stdout:6/877: write d2/d13/d62/f6b [1327110,11026] 0 2024-03-16T09:01:23.076 INFO:tasks.workunit.client.1.smithi019.stdout:0/817: read d7/d8/dec/def/d6c/f8e [956615,30401] 0 2024-03-16T09:01:23.076 INFO:tasks.workunit.client.1.smithi019.stdout:0/818: write d7/d8/d32/d66/d6d/d8c/db8/d82/f47 [2010079,74554] 0 2024-03-16T09:01:23.076 INFO:tasks.workunit.client.1.smithi019.stdout:8/898: read d6/ff1 [2027162,62399] 0 2024-03-16T09:01:23.078 INFO:tasks.workunit.client.1.smithi019.stdout:0/819: link l5 d7/d8/dec/def/d92/da6/l10d 0 2024-03-16T09:01:23.078 INFO:tasks.workunit.client.0.smithi007.stdout:3/931: symlink d15/d18/d1f/d3b/dae/d113/l126 0 2024-03-16T09:01:23.078 INFO:tasks.workunit.client.0.smithi007.stdout:3/932: fsync d15/f59 0 2024-03-16T09:01:23.079 INFO:tasks.workunit.client.1.smithi019.stdout:9/739: dwrite d1/d7/d67/d77/fe0 [0,4194304] 0 2024-03-16T09:01:23.080 INFO:tasks.workunit.client.1.smithi019.stdout:0/820: rmdir d7/d1d/d63/d72 0 2024-03-16T09:01:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:1/797: rename d9/dd/d1b/d2d/cd7 to d9/d6b/d7e/cfd 0 2024-03-16T09:01:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:4/815: mknod d4/d14/c119 0 2024-03-16T09:01:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:6/878: truncate d2/d13/d62/f7b 4177632 0 2024-03-16T09:01:23.092 INFO:tasks.workunit.client.0.smithi007.stdout:5/933: symlink d1/d7/d1c/d19/d34/d5e/d6f/da9/l154 0 2024-03-16T09:01:23.094 INFO:tasks.workunit.client.1.smithi019.stdout:9/740: write d1/d7/d67/d77/fd3 [1730942,92469] 0 2024-03-16T09:01:23.095 INFO:tasks.workunit.client.0.smithi007.stdout:4/816: getdents d4/d14/d64/d68/dbf 0 2024-03-16T09:01:23.096 INFO:tasks.workunit.client.0.smithi007.stdout:1/798: dread d9/dd/f14 [0,4194304] 0 2024-03-16T09:01:23.098 INFO:tasks.workunit.client.0.smithi007.stdout:7/863: sync 2024-03-16T09:01:23.098 INFO:tasks.workunit.client.0.smithi007.stdout:6/879: getdents d2/d13/d62 0 2024-03-16T09:01:23.099 INFO:tasks.workunit.client.1.smithi019.stdout:9/741: dread d1/d7/f45 [0,4194304] 0 2024-03-16T09:01:23.100 INFO:tasks.workunit.client.1.smithi019.stdout:0/821: dwrite d7/d8/dec/def/d48/f65 [0,4194304] 0 2024-03-16T09:01:23.100 INFO:tasks.workunit.client.1.smithi019.stdout:0/822: stat d7/d8/d32/d66/d6d/d8c/db5/ffa 0 2024-03-16T09:01:23.100 INFO:tasks.workunit.client.0.smithi007.stdout:4/817: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/f11a x:0 0 0 2024-03-16T09:01:23.103 INFO:tasks.workunit.client.0.smithi007.stdout:1/799: link d9/dd/d1b/d28/f46 d9/d6b/d7e/ffe 0 2024-03-16T09:01:23.103 INFO:tasks.workunit.client.0.smithi007.stdout:3/933: dwrite d15/d18/d1f/d29/d43/f69 [0,4194304] 0 2024-03-16T09:01:23.103 INFO:tasks.workunit.client.1.smithi019.stdout:0/823: creat d7/d8/dec/def/d48/d10c/f10e x:0 0 0 2024-03-16T09:01:23.105 INFO:tasks.workunit.client.0.smithi007.stdout:7/864: mkdir d0/dc/d38/d4e/d12e 0 2024-03-16T09:01:23.106 INFO:tasks.workunit.client.1.smithi019.stdout:8/899: dwrite d6/d2a/d45/d19/d74/f7e [0,4194304] 0 2024-03-16T09:01:23.107 INFO:tasks.workunit.client.0.smithi007.stdout:9/917: sync 2024-03-16T09:01:23.107 INFO:tasks.workunit.client.0.smithi007.stdout:9/918: write d24/d4c/d86/daa/d95/f104 [1965,1720] 0 2024-03-16T09:01:23.108 INFO:tasks.workunit.client.1.smithi019.stdout:9/742: rename d1/d7/l66 to d1/d92/lfd 0 2024-03-16T09:01:23.109 INFO:tasks.workunit.client.1.smithi019.stdout:0/824: mkdir d7/d1d/d63/d6e/db0/d10f 0 2024-03-16T09:01:23.111 INFO:tasks.workunit.client.0.smithi007.stdout:7/865: symlink d0/d19/d1f/d5d/d9d/df7/l12f 0 2024-03-16T09:01:23.111 INFO:tasks.workunit.client.0.smithi007.stdout:7/866: dread - d0/d19/d1f/d5d/d9d/df7/d73/ded/f11d zero size 2024-03-16T09:01:23.111 INFO:tasks.workunit.client.0.smithi007.stdout:7/867: chown d0/d19/d1f/d5d/d9d/df7/d73/f91 162 1 2024-03-16T09:01:23.114 INFO:tasks.workunit.client.0.smithi007.stdout:5/934: dwrite d1/d7/d16/ffa [0,4194304] 0 2024-03-16T09:01:23.114 INFO:tasks.workunit.client.0.smithi007.stdout:5/935: stat d1/d7/l46 0 2024-03-16T09:01:23.114 INFO:tasks.workunit.client.0.smithi007.stdout:5/936: chown d1/d7/d1c/d19/d34/fa4 6119374 1 2024-03-16T09:01:23.114 INFO:tasks.workunit.client.1.smithi019.stdout:8/900: symlink d6/d2a/d45/d57/d72/d79/dc1/l138 0 2024-03-16T09:01:23.117 INFO:tasks.workunit.client.0.smithi007.stdout:7/868: mknod d0/d19/d1f/d5d/d9d/df7/da1/c130 0 2024-03-16T09:01:23.118 INFO:tasks.workunit.client.0.smithi007.stdout:6/880: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d1e/f111 [0,4194304] 0 2024-03-16T09:01:23.122 INFO:tasks.workunit.client.1.smithi019.stdout:9/743: write d1/d7/f51 [3252066,85797] 0 2024-03-16T09:01:23.124 INFO:tasks.workunit.client.1.smithi019.stdout:9/744: mkdir d1/d7/de/d24/d5b/dce/df8/dfe 0 2024-03-16T09:01:23.125 INFO:tasks.workunit.client.1.smithi019.stdout:9/745: rmdir d1/d7/de/da3/df6 0 2024-03-16T09:01:23.126 INFO:tasks.workunit.client.0.smithi007.stdout:7/869: getdents d0/d19/d1f/d69/de1 0 2024-03-16T09:01:23.126 INFO:tasks.workunit.client.0.smithi007.stdout:7/870: truncate d0/d19/f4d 352492 0 2024-03-16T09:01:23.127 INFO:tasks.workunit.client.1.smithi019.stdout:9/746: creat d1/d7/de/d24/d44/d60/d8b/dc2/fff x:0 0 0 2024-03-16T09:01:23.127 INFO:tasks.workunit.client.1.smithi019.stdout:9/747: fdatasync d1/d7/d21/da2/f9e 0 2024-03-16T09:01:23.127 INFO:tasks.workunit.client.1.smithi019.stdout:9/748: mknod d1/d7/d21/da2/c100 0 2024-03-16T09:01:23.129 INFO:tasks.workunit.client.1.smithi019.stdout:9/749: write d1/d7/de/d24/d44/d60/f9f [301337,112360] 0 2024-03-16T09:01:23.129 INFO:tasks.workunit.client.1.smithi019.stdout:9/750: chown d1/d7/de/d24/d5b/dce/df8/d59/l83 1908764 1 2024-03-16T09:01:23.131 INFO:tasks.workunit.client.1.smithi019.stdout:8/901: rename d6/d2a/d3c/dab/d30/d4a/d67/d95 to d6/d2a/d45/d57/d72/d9f/d139 0 2024-03-16T09:01:23.131 INFO:tasks.workunit.client.1.smithi019.stdout:8/902: chown d6/d2a/d3f/c135 668344 1 2024-03-16T09:01:23.131 INFO:tasks.workunit.client.1.smithi019.stdout:8/903: write d6/de/f1b [4943608,30710] 0 2024-03-16T09:01:23.131 INFO:tasks.workunit.client.1.smithi019.stdout:8/904: chown d6/d2a/d3c/dab/d30/d4a/dc7/f10c 78 1 2024-03-16T09:01:23.134 INFO:tasks.workunit.client.0.smithi007.stdout:6/881: unlink d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/c8a 0 2024-03-16T09:01:23.134 INFO:tasks.workunit.client.0.smithi007.stdout:6/882: chown d2/d13/d62 3183 1 2024-03-16T09:01:23.135 INFO:tasks.workunit.client.0.smithi007.stdout:7/871: symlink d0/d19/d1f/d5d/d9d/df7/da1/l131 0 2024-03-16T09:01:23.135 INFO:tasks.workunit.client.1.smithi019.stdout:8/905: getdents d6/d2a/d3c/dab/d30/d4a/d67/dee 0 2024-03-16T09:01:23.136 INFO:tasks.workunit.client.0.smithi007.stdout:4/818: dread d4/d14/d64/d9f/f4b [0,4194304] 0 2024-03-16T09:01:23.137 INFO:tasks.workunit.client.0.smithi007.stdout:6/883: mknod d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/c121 0 2024-03-16T09:01:23.137 INFO:tasks.workunit.client.0.smithi007.stdout:7/872: mkdir d0/d19/d1f/d5d/d9d/d132 0 2024-03-16T09:01:23.137 INFO:tasks.workunit.client.0.smithi007.stdout:7/873: read - d0/d19/d1f/d69/de1/de9/f102 zero size 2024-03-16T09:01:23.138 INFO:tasks.workunit.client.0.smithi007.stdout:4/819: mkdir d4/d99/d11b 0 2024-03-16T09:01:23.139 INFO:tasks.workunit.client.0.smithi007.stdout:6/884: rename d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/df6 to d2/d13/d62/d122 0 2024-03-16T09:01:23.141 INFO:tasks.workunit.client.0.smithi007.stdout:4/820: rename d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/fee to d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/de9/f11c 0 2024-03-16T09:01:23.142 INFO:tasks.workunit.client.0.smithi007.stdout:4/821: creat d4/d14/d64/f11d x:0 0 0 2024-03-16T09:01:23.143 INFO:tasks.workunit.client.0.smithi007.stdout:6/885: dread d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/fd4 [0,4194304] 0 2024-03-16T09:01:23.143 INFO:tasks.workunit.client.0.smithi007.stdout:6/886: write d2/d13/d4b/f7c [4373769,30180] 0 2024-03-16T09:01:23.143 INFO:tasks.workunit.client.0.smithi007.stdout:6/887: chown d2/d13/d3f/d50/d78/d81/fdf 68 1 2024-03-16T09:01:23.143 INFO:tasks.workunit.client.0.smithi007.stdout:6/888: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/fe2 467482 0 2024-03-16T09:01:23.145 INFO:tasks.workunit.client.0.smithi007.stdout:6/889: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/d123 0 2024-03-16T09:01:23.145 INFO:tasks.workunit.client.0.smithi007.stdout:6/890: fdatasync d2/f46 0 2024-03-16T09:01:23.145 INFO:tasks.workunit.client.0.smithi007.stdout:6/891: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/f120 [426343,69156] 0 2024-03-16T09:01:23.145 INFO:tasks.workunit.client.0.smithi007.stdout:6/892: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/f35 [4901648,27308] 0 2024-03-16T09:01:23.145 INFO:tasks.workunit.client.0.smithi007.stdout:6/893: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/d84/f124 x:0 0 0 2024-03-16T09:01:23.147 INFO:tasks.workunit.client.0.smithi007.stdout:6/894: creat d2/d13/d3f/d86/f125 x:0 0 0 2024-03-16T09:01:23.147 INFO:tasks.workunit.client.0.smithi007.stdout:6/895: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/d84/f9e [782011,113382] 0 2024-03-16T09:01:23.147 INFO:tasks.workunit.client.0.smithi007.stdout:6/896: write d2/d29/fab [872960,81910] 0 2024-03-16T09:01:23.149 INFO:tasks.workunit.client.0.smithi007.stdout:6/897: mknod d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/c126 0 2024-03-16T09:01:23.149 INFO:tasks.workunit.client.0.smithi007.stdout:6/898: chown d2/d13/d24/f6a 1966 1 2024-03-16T09:01:23.151 INFO:tasks.workunit.client.0.smithi007.stdout:6/899: mknod d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d11c/c127 0 2024-03-16T09:01:23.151 INFO:tasks.workunit.client.0.smithi007.stdout:6/900: creat d2/d13/d3f/d50/d78/d81/f128 x:0 0 0 2024-03-16T09:01:23.153 INFO:tasks.workunit.client.0.smithi007.stdout:6/901: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/d129 0 2024-03-16T09:01:23.155 INFO:tasks.workunit.client.0.smithi007.stdout:5/937: dread d1/d7/d1c/d19/d34/d4d/f101 [0,4194304] 0 2024-03-16T09:01:23.156 INFO:tasks.workunit.client.0.smithi007.stdout:5/938: mknod d1/d7/d16/d1b/c155 0 2024-03-16T09:01:23.156 INFO:tasks.workunit.client.0.smithi007.stdout:5/939: readlink d1/d7/d1c/laf 0 2024-03-16T09:01:23.158 INFO:tasks.workunit.client.0.smithi007.stdout:5/940: unlink d1/d7/d1c/d19/d34/d1d/l2e 0 2024-03-16T09:01:23.159 INFO:tasks.workunit.client.0.smithi007.stdout:7/874: dread d0/dc/d38/f7c [0,4194304] 0 2024-03-16T09:01:23.159 INFO:tasks.workunit.client.0.smithi007.stdout:5/941: mkdir d1/d7/d1c/d19/d156 0 2024-03-16T09:01:23.160 INFO:tasks.workunit.client.0.smithi007.stdout:7/875: mknod d0/d19/d1f/d69/c133 0 2024-03-16T09:01:23.161 INFO:tasks.workunit.client.0.smithi007.stdout:5/942: creat d1/d7/d1c/d19/d34/d5e/f157 x:0 0 0 2024-03-16T09:01:23.161 INFO:tasks.workunit.client.0.smithi007.stdout:5/943: chown d1/d7/d1c/d19/d34/d3e/d62/fdf 138732 1 2024-03-16T09:01:23.162 INFO:tasks.workunit.client.0.smithi007.stdout:5/944: write d1/d7/d1c/d19/d34/d1d/da8/fc7 [1276625,16404] 0 2024-03-16T09:01:23.162 INFO:tasks.workunit.client.0.smithi007.stdout:5/945: write d1/d7/d1c/d19/d34/d1d/dee/f13e [4930173,48616] 0 2024-03-16T09:01:23.162 INFO:tasks.workunit.client.0.smithi007.stdout:3/934: dwrite d15/d18/d1f/d29/d7c/f109 [0,4194304] 0 2024-03-16T09:01:23.162 INFO:tasks.workunit.client.0.smithi007.stdout:3/935: creat d15/d18/d1f/d56/d98/f127 x:0 0 0 2024-03-16T09:01:23.162 INFO:tasks.workunit.client.0.smithi007.stdout:3/936: stat d15/d18/d1f/d29/d43/c115 0 2024-03-16T09:01:23.162 INFO:tasks.workunit.client.0.smithi007.stdout:3/937: write d15/d18/d1f/f2e [1484946,41501] 0 2024-03-16T09:01:23.162 INFO:tasks.workunit.client.0.smithi007.stdout:1/800: dwrite d9/dd/d1b/d2d/d30/d45/fc8 [0,4194304] 0 2024-03-16T09:01:23.163 INFO:tasks.workunit.client.1.smithi019.stdout:0/825: dwrite d7/d8/dec/def/d48/f4e [4194304,4194304] 0 2024-03-16T09:01:23.163 INFO:tasks.workunit.client.0.smithi007.stdout:9/919: dwrite d24/f29 [0,4194304] 0 2024-03-16T09:01:23.163 INFO:tasks.workunit.client.0.smithi007.stdout:4/822: dwrite d4/d14/d1c/f32 [0,4194304] 0 2024-03-16T09:01:23.163 INFO:tasks.workunit.client.0.smithi007.stdout:4/823: chown d4/fa3 1 1 2024-03-16T09:01:23.165 INFO:tasks.workunit.client.1.smithi019.stdout:0/826: truncate d7/d1d/d2e/f39 221480 0 2024-03-16T09:01:23.166 INFO:tasks.workunit.client.1.smithi019.stdout:0/827: rmdir d7 39 2024-03-16T09:01:23.166 INFO:tasks.workunit.client.1.smithi019.stdout:0/828: write d7/d8/dec/def/da1/ddf/f106 [458112,6815] 0 2024-03-16T09:01:23.167 INFO:tasks.workunit.client.1.smithi019.stdout:0/829: truncate d7/d8/d32/d66/d6d/d8c/db8/d82/f6f 1214962 0 2024-03-16T09:01:23.169 INFO:tasks.workunit.client.0.smithi007.stdout:3/938: mknod d15/d18/d1f/dce/c128 0 2024-03-16T09:01:23.172 INFO:tasks.workunit.client.0.smithi007.stdout:1/801: unlink d9/dd/d1b/d28/d29/d7b/db4/fd9 0 2024-03-16T09:01:23.173 INFO:tasks.workunit.client.0.smithi007.stdout:9/920: mkdir d24/d4c/d86/daa/d7f/d80/de3/d107/d112 0 2024-03-16T09:01:23.173 INFO:tasks.workunit.client.0.smithi007.stdout:9/921: fsync d24/f27 0 2024-03-16T09:01:23.177 INFO:tasks.workunit.client.0.smithi007.stdout:5/946: dwrite d1/d7/d16/d1b/f8d [0,4194304] 0 2024-03-16T09:01:23.177 INFO:tasks.workunit.client.0.smithi007.stdout:5/947: dread - d1/d7/d1c/d19/df7/f12f zero size 2024-03-16T09:01:23.178 INFO:tasks.workunit.client.1.smithi019.stdout:5/815: sync 2024-03-16T09:01:23.178 INFO:tasks.workunit.client.1.smithi019.stdout:1/726: sync 2024-03-16T09:01:23.181 INFO:tasks.workunit.client.0.smithi007.stdout:4/824: unlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/la6 0 2024-03-16T09:01:23.181 INFO:tasks.workunit.client.0.smithi007.stdout:4/825: fdatasync d4/d14/d64/d68/d82/dac/dfc/d8f/da8/f1d 0 2024-03-16T09:01:23.181 INFO:tasks.workunit.client.1.smithi019.stdout:5/816: truncate d6/dd/d61/fa9 3299325 0 2024-03-16T09:01:23.182 INFO:tasks.workunit.client.1.smithi019.stdout:1/727: mkdir d16/d31/d7b/dd9 0 2024-03-16T09:01:23.183 INFO:tasks.workunit.client.1.smithi019.stdout:5/817: rmdir d6/dd/d47/dd0 39 2024-03-16T09:01:23.184 INFO:tasks.workunit.client.1.smithi019.stdout:1/728: link d16/d1f/d43/l52 d16/d31/d71/d89/dce/lda 0 2024-03-16T09:01:23.184 INFO:tasks.workunit.client.1.smithi019.stdout:1/729: chown d16/dcb/daf 263606353 1 2024-03-16T09:01:23.185 INFO:tasks.workunit.client.1.smithi019.stdout:5/818: symlink d6/d19/d34/l10c 0 2024-03-16T09:01:23.186 INFO:tasks.workunit.client.1.smithi019.stdout:1/730: mknod d16/d31/d71/d89/cdb 0 2024-03-16T09:01:23.187 INFO:tasks.workunit.client.1.smithi019.stdout:1/731: stat d16/d18/f6c 0 2024-03-16T09:01:23.187 INFO:tasks.workunit.client.1.smithi019.stdout:5/819: truncate d6/dd/d47/d70/db2/d89/f96 3195872 0 2024-03-16T09:01:23.189 INFO:tasks.workunit.client.0.smithi007.stdout:3/939: symlink d15/d18/d1f/d29/d43/df7/l129 0 2024-03-16T09:01:23.190 INFO:tasks.workunit.client.1.smithi019.stdout:5/820: rename d6/l65 to d6/dd/d47/d70/db2/d89/d9e/l10d 0 2024-03-16T09:01:23.191 INFO:tasks.workunit.client.0.smithi007.stdout:1/802: creat d9/dd/fff x:0 0 0 2024-03-16T09:01:23.191 INFO:tasks.workunit.client.0.smithi007.stdout:1/803: fsync d9/dd/d1b/f49 0 2024-03-16T09:01:23.191 INFO:tasks.workunit.client.0.smithi007.stdout:1/804: read d9/f19 [279218,60323] 0 2024-03-16T09:01:23.192 INFO:tasks.workunit.client.1.smithi019.stdout:5/821: symlink d6/dd/d61/l10e 0 2024-03-16T09:01:23.193 INFO:tasks.workunit.client.1.smithi019.stdout:5/822: rmdir d6/d19/d34/dc2 39 2024-03-16T09:01:23.193 INFO:tasks.workunit.client.0.smithi007.stdout:5/948: symlink d1/d7/d1c/d19/d156/l158 0 2024-03-16T09:01:23.193 INFO:tasks.workunit.client.0.smithi007.stdout:5/949: write d1/d7/d1c/d19/d115/f131 [569715,76932] 0 2024-03-16T09:01:23.193 INFO:tasks.workunit.client.0.smithi007.stdout:3/940: dread d15/d18/f10f [0,4194304] 0 2024-03-16T09:01:23.194 INFO:tasks.workunit.client.1.smithi019.stdout:5/823: mknod d6/dd/daf/c10f 0 2024-03-16T09:01:23.194 INFO:tasks.workunit.client.1.smithi019.stdout:5/824: write d6/dd/d37/f6b [394394,123862] 0 2024-03-16T09:01:23.196 INFO:tasks.workunit.client.1.smithi019.stdout:5/825: link d6/d1f/l3a d6/dd/d47/l110 0 2024-03-16T09:01:23.196 INFO:tasks.workunit.client.0.smithi007.stdout:4/826: unlink d4/d14/d1c/f3b 0 2024-03-16T09:01:23.197 INFO:tasks.workunit.client.1.smithi019.stdout:1/732: dread d16/d18/f1b [0,4194304] 0 2024-03-16T09:01:23.197 INFO:tasks.workunit.client.1.smithi019.stdout:1/733: getdents d16/dcb/daf/db2/d78/db1 0 2024-03-16T09:01:23.197 INFO:tasks.workunit.client.1.smithi019.stdout:1/734: fsync d16/d1f/d43/f6d 0 2024-03-16T09:01:23.198 INFO:tasks.workunit.client.0.smithi007.stdout:9/922: dwrite d24/f4f [0,4194304] 0 2024-03-16T09:01:23.199 INFO:tasks.workunit.client.1.smithi019.stdout:0/830: dwrite d7/d8/d32/d4a/d61/d68/f94 [0,4194304] 0 2024-03-16T09:01:23.199 INFO:tasks.workunit.client.1.smithi019.stdout:0/831: dread - d7/d8/d32/fcb zero size 2024-03-16T09:01:23.199 INFO:tasks.workunit.client.1.smithi019.stdout:1/735: unlink d16/d84/fb5 0 2024-03-16T09:01:23.204 INFO:tasks.workunit.client.0.smithi007.stdout:5/950: truncate d1/d7/d1c/d88/da2/d89/db3/d2a/d79/fbf 2937903 0 2024-03-16T09:01:23.205 INFO:tasks.workunit.client.0.smithi007.stdout:3/941: mknod d15/d18/d1f/d56/d98/c12a 0 2024-03-16T09:01:23.206 INFO:tasks.workunit.client.1.smithi019.stdout:0/832: symlink d7/d8/dec/def/d48/l110 0 2024-03-16T09:01:23.207 INFO:tasks.workunit.client.1.smithi019.stdout:1/736: stat d16/d1f/l85 0 2024-03-16T09:01:23.207 INFO:tasks.workunit.client.1.smithi019.stdout:1/737: fsync d16/d1f/d3c/f48 0 2024-03-16T09:01:23.207 INFO:tasks.workunit.client.1.smithi019.stdout:1/738: stat d16/d1f/d3c/d79/db4/lc4 0 2024-03-16T09:01:23.207 INFO:tasks.workunit.client.1.smithi019.stdout:1/739: readlink d16/d1f/d43/d55/l77 0 2024-03-16T09:01:23.207 INFO:tasks.workunit.client.1.smithi019.stdout:1/740: fdatasync f10 0 2024-03-16T09:01:23.209 INFO:tasks.workunit.client.0.smithi007.stdout:4/827: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/dbe/f11e x:0 0 0 2024-03-16T09:01:23.210 INFO:tasks.workunit.client.0.smithi007.stdout:9/923: rmdir d24/d4a/d55 39 2024-03-16T09:01:23.212 INFO:tasks.workunit.client.0.smithi007.stdout:5/951: creat d1/d7/d1c/d19/d34/d1d/da8/d111/f159 x:0 0 0 2024-03-16T09:01:23.216 INFO:tasks.workunit.client.0.smithi007.stdout:4/828: rename d4/d14/d64/d68/d82/dac/dfc/de8/c117 to d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/c11f 0 2024-03-16T09:01:23.218 INFO:tasks.workunit.client.1.smithi019.stdout:0/833: dread d7/d8/d32/fa5 [0,4194304] 0 2024-03-16T09:01:23.219 INFO:tasks.workunit.client.0.smithi007.stdout:5/952: mkdir d1/d7/d1c/d88/da2/d89/db3/d7d/d15a 0 2024-03-16T09:01:23.219 INFO:tasks.workunit.client.0.smithi007.stdout:5/953: write d1/d7/d1c/d19/d34/d4d/f101 [4333743,72758] 0 2024-03-16T09:01:23.219 INFO:tasks.workunit.client.0.smithi007.stdout:5/954: fsync d1/d7/d1c/d19/d34/f8c 0 2024-03-16T09:01:23.220 INFO:tasks.workunit.client.0.smithi007.stdout:5/955: dread d1/d7/d1c/d19/d34/d5e/d6f/d72/fd3 [0,4194304] 0 2024-03-16T09:01:23.220 INFO:tasks.workunit.client.0.smithi007.stdout:5/956: chown d1/d7/d1c/d88/da2/d89/db3/d7d 0 1 2024-03-16T09:01:23.222 INFO:tasks.workunit.client.0.smithi007.stdout:9/924: rename d24/d4c/d86/c8b to d24/d4a/d96/db9/dca/c113 0 2024-03-16T09:01:23.223 INFO:tasks.workunit.client.1.smithi019.stdout:5/826: dwrite d6/d1f/f55 [0,4194304] 0 2024-03-16T09:01:23.226 INFO:tasks.workunit.client.1.smithi019.stdout:1/741: write f10 [2251683,16727] 0 2024-03-16T09:01:23.226 INFO:tasks.workunit.client.0.smithi007.stdout:5/957: creat d1/d7/d1c/d19/d156/f15b x:0 0 0 2024-03-16T09:01:23.227 INFO:tasks.workunit.client.0.smithi007.stdout:1/805: dwrite d9/dd/d1b/d58/f66 [0,4194304] 0 2024-03-16T09:01:23.228 INFO:tasks.workunit.client.1.smithi019.stdout:1/742: unlink f10 0 2024-03-16T09:01:23.228 INFO:tasks.workunit.client.1.smithi019.stdout:1/743: write d16/d1f/d43/f4c [2152805,99134] 0 2024-03-16T09:01:23.228 INFO:tasks.workunit.client.0.smithi007.stdout:1/806: write d9/d6b/d7e/f92 [60169,53472] 0 2024-03-16T09:01:23.230 INFO:tasks.workunit.client.0.smithi007.stdout:4/829: rename d4/d14/f35 to d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/f120 0 2024-03-16T09:01:23.230 INFO:tasks.workunit.client.0.smithi007.stdout:4/830: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/fa2 3553 1 2024-03-16T09:01:23.230 INFO:tasks.workunit.client.0.smithi007.stdout:4/831: stat d4/d14/d64/d9f/f53 0 2024-03-16T09:01:23.231 INFO:tasks.workunit.client.0.smithi007.stdout:5/958: mkdir d1/d7/d1c/d88/da2/df4/d151/d15c 0 2024-03-16T09:01:23.235 INFO:tasks.workunit.client.0.smithi007.stdout:1/807: symlink d9/def/l100 0 2024-03-16T09:01:23.235 INFO:tasks.workunit.client.0.smithi007.stdout:1/808: readlink d9/dd/d1b/d28/l75 0 2024-03-16T09:01:23.240 INFO:tasks.workunit.client.0.smithi007.stdout:1/809: unlink d9/dd/d1b/ca1 0 2024-03-16T09:01:23.240 INFO:tasks.workunit.client.0.smithi007.stdout:1/810: creat d9/dd/d1b/d28/f101 x:0 0 0 2024-03-16T09:01:23.242 INFO:tasks.workunit.client.0.smithi007.stdout:3/942: dwrite d15/d18/d1f/f41 [0,4194304] 0 2024-03-16T09:01:23.243 INFO:tasks.workunit.client.0.smithi007.stdout:4/832: dread d4/f40 [0,4194304] 0 2024-03-16T09:01:23.243 INFO:tasks.workunit.client.0.smithi007.stdout:4/833: write d4/d14/fe6 [632391,38417] 0 2024-03-16T09:01:23.243 INFO:tasks.workunit.client.0.smithi007.stdout:4/834: read d4/dcb/fd8 [185891,85234] 0 2024-03-16T09:01:23.243 INFO:tasks.workunit.client.0.smithi007.stdout:1/811: mkdir d9/dd/d1b/d28/d29/d102 0 2024-03-16T09:01:23.247 INFO:tasks.workunit.client.1.smithi019.stdout:0/834: dwrite d7/d8/d32/d66/f91 [0,4194304] 0 2024-03-16T09:01:23.249 INFO:tasks.workunit.client.0.smithi007.stdout:3/943: symlink d15/d18/d121/l12b 0 2024-03-16T09:01:23.250 INFO:tasks.workunit.client.1.smithi019.stdout:0/835: creat d7/d8/dec/def/f111 x:0 0 0 2024-03-16T09:01:23.251 INFO:tasks.workunit.client.0.smithi007.stdout:3/944: read d15/d18/d1f/d56/f65 [976912,40170] 0 2024-03-16T09:01:23.252 INFO:tasks.workunit.client.0.smithi007.stdout:3/945: fsync d15/d18/f23 0 2024-03-16T09:01:23.252 INFO:tasks.workunit.client.0.smithi007.stdout:3/946: dread - d15/d18/d1f/d29/d3e/dda/fe5 zero size 2024-03-16T09:01:23.252 INFO:tasks.workunit.client.0.smithi007.stdout:4/835: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/f121 x:0 0 0 2024-03-16T09:01:23.252 INFO:tasks.workunit.client.0.smithi007.stdout:4/836: readlink d4/dcb/dcd/le0 0 2024-03-16T09:01:23.256 INFO:tasks.workunit.client.0.smithi007.stdout:3/947: truncate f10 713372 0 2024-03-16T09:01:23.261 INFO:tasks.workunit.client.1.smithi019.stdout:0/836: dread d7/d8/d32/d66/d6d/fc8 [4194304,4194304] 0 2024-03-16T09:01:23.269 INFO:tasks.workunit.client.0.smithi007.stdout:4/837: dread d4/d14/d1c/f32 [0,4194304] 0 2024-03-16T09:01:23.271 INFO:tasks.workunit.client.1.smithi019.stdout:1/744: dwrite d16/d1f/d3c/d79/fd3 [0,4194304] 0 2024-03-16T09:01:23.271 INFO:tasks.workunit.client.0.smithi007.stdout:5/959: dwrite d1/d7/d1c/d19/d34/d5e/f8e [0,4194304] 0 2024-03-16T09:01:23.272 INFO:tasks.workunit.client.0.smithi007.stdout:1/812: dwrite f0 [0,4194304] 0 2024-03-16T09:01:23.272 INFO:tasks.workunit.client.0.smithi007.stdout:4/838: rename d4/d14/d64/d68/d82/dac/dfc/de8/ff9 to d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/de9/f122 0 2024-03-16T09:01:23.273 INFO:tasks.workunit.client.0.smithi007.stdout:5/960: truncate d1/d7/d16/d1b/f8d 2443047 0 2024-03-16T09:01:23.273 INFO:tasks.workunit.client.0.smithi007.stdout:5/961: chown d1/d7/d1c/d19/d34/de/f49 7 1 2024-03-16T09:01:23.275 INFO:tasks.workunit.client.0.smithi007.stdout:3/948: truncate d15/d18/d1f/f41 3949731 0 2024-03-16T09:01:23.275 INFO:tasks.workunit.client.0.smithi007.stdout:4/839: read d4/d14/d1c/f6a [1384544,41689] 0 2024-03-16T09:01:23.276 INFO:tasks.workunit.client.1.smithi019.stdout:1/745: symlink d16/dcb/daf/db2/d8f/ldc 0 2024-03-16T09:01:23.276 INFO:tasks.workunit.client.1.smithi019.stdout:1/746: write d16/d31/d71/fd7 [579998,38838] 0 2024-03-16T09:01:23.276 INFO:tasks.workunit.client.1.smithi019.stdout:1/747: dread - d16/d18/f90 zero size 2024-03-16T09:01:23.276 INFO:tasks.workunit.client.1.smithi019.stdout:1/748: fdatasync d16/da1/fc1 0 2024-03-16T09:01:23.277 INFO:tasks.workunit.client.1.smithi019.stdout:0/837: write d7/d8/d32/d4a/f4d [576212,124570] 0 2024-03-16T09:01:23.277 INFO:tasks.workunit.client.1.smithi019.stdout:0/838: read - d7/d8/dec/def/f111 zero size 2024-03-16T09:01:23.280 INFO:tasks.workunit.client.1.smithi019.stdout:0/839: read d7/d8/d32/d66/d6d/d8c/db8/d82/f81 [57702,124033] 0 2024-03-16T09:01:23.280 INFO:tasks.workunit.client.1.smithi019.stdout:0/840: chown d7/d8/d32/d66/c7e 3 1 2024-03-16T09:01:23.288 INFO:tasks.workunit.client.0.smithi007.stdout:5/962: dwrite d1/d7/d1c/d19/d34/d1d/f11d [0,4194304] 0 2024-03-16T09:01:23.289 INFO:tasks.workunit.client.0.smithi007.stdout:3/949: rename d15/d18/d1f/d29/d43/d72/la2 to d15/d18/d1f/d3b/dae/l12c 0 2024-03-16T09:01:23.292 INFO:tasks.workunit.client.0.smithi007.stdout:4/840: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/f123 x:0 0 0 2024-03-16T09:01:23.294 INFO:tasks.workunit.client.0.smithi007.stdout:5/963: symlink d1/d7/d1c/d19/d34/d1d/da8/l15d 0 2024-03-16T09:01:23.294 INFO:tasks.workunit.client.0.smithi007.stdout:4/841: symlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/l124 0 2024-03-16T09:01:23.295 INFO:tasks.workunit.client.0.smithi007.stdout:5/964: mkdir d1/d7/d1c/d19/dcc/d15e 0 2024-03-16T09:01:23.295 INFO:tasks.workunit.client.0.smithi007.stdout:5/965: read - d1/d7/d1c/d19/d34/d3e/fe5 zero size 2024-03-16T09:01:23.296 INFO:tasks.workunit.client.0.smithi007.stdout:5/966: stat d1/d7/d1c/d19/d34/d1d 0 2024-03-16T09:01:23.296 INFO:tasks.workunit.client.0.smithi007.stdout:5/967: truncate d1/d7/d1c/d19/d34/d3e/d41/da6/f119 946220 0 2024-03-16T09:01:23.296 INFO:tasks.workunit.client.0.smithi007.stdout:3/950: rename d15/d18/d1f/d3b/d63/fb7 to d15/d18/d1f/f12d 0 2024-03-16T09:01:23.296 INFO:tasks.workunit.client.0.smithi007.stdout:3/951: write d15/d18/d1f/d29/d43/f7a [3498874,110380] 0 2024-03-16T09:01:23.297 INFO:tasks.workunit.client.0.smithi007.stdout:1/813: dwrite d9/dd/fff [0,4194304] 0 2024-03-16T09:01:23.298 INFO:tasks.workunit.client.0.smithi007.stdout:1/814: stat d9/dd/d1b/d2d/ca6 0 2024-03-16T09:01:23.298 INFO:tasks.workunit.client.0.smithi007.stdout:4/842: getdents d4/d14/d64/d68 0 2024-03-16T09:01:23.299 INFO:tasks.workunit.client.0.smithi007.stdout:5/968: creat d1/d7/d1c/d19/dcc/d102/f15f x:0 0 0 2024-03-16T09:01:23.302 INFO:tasks.workunit.client.0.smithi007.stdout:3/952: mknod d15/d18/d6a/c12e 0 2024-03-16T09:01:23.302 INFO:tasks.workunit.client.0.smithi007.stdout:3/953: dread - d15/d18/d1f/d3b/d87/ffe zero size 2024-03-16T09:01:23.302 INFO:tasks.workunit.client.1.smithi019.stdout:1/749: dwrite d16/d18/f24 [0,4194304] 0 2024-03-16T09:01:23.303 INFO:tasks.workunit.client.1.smithi019.stdout:0/841: dwrite d7/d8/f50 [0,4194304] 0 2024-03-16T09:01:23.304 INFO:tasks.workunit.client.1.smithi019.stdout:1/750: symlink d16/d1f/d3c/d79/db4/ldd 0 2024-03-16T09:01:23.304 INFO:tasks.workunit.client.0.smithi007.stdout:1/815: creat d9/dd/d1b/d28/d29/d7b/d90/f103 x:0 0 0 2024-03-16T09:01:23.304 INFO:tasks.workunit.client.0.smithi007.stdout:1/816: chown d9/dd/d1b/d2d/d30/d45/fbd 142 1 2024-03-16T09:01:23.305 INFO:tasks.workunit.client.1.smithi019.stdout:0/842: write d7/d1d/d2e/f39 [96904,98370] 0 2024-03-16T09:01:23.305 INFO:tasks.workunit.client.0.smithi007.stdout:4/843: mknod d4/d14/d64/d68/d82/dac/dfc/de8/c125 0 2024-03-16T09:01:23.305 INFO:tasks.workunit.client.1.smithi019.stdout:1/751: rmdir d16/dcb/daf/db2 39 2024-03-16T09:01:23.305 INFO:tasks.workunit.client.1.smithi019.stdout:1/752: fsync d16/d84/fbf 0 2024-03-16T09:01:23.306 INFO:tasks.workunit.client.1.smithi019.stdout:1/753: readlink d16/da1/lab 0 2024-03-16T09:01:23.306 INFO:tasks.workunit.client.1.smithi019.stdout:0/843: truncate d7/d8/dec/def/d92/da6/fbb 154387 0 2024-03-16T09:01:23.306 INFO:tasks.workunit.client.0.smithi007.stdout:3/954: truncate d15/d18/d1f/d3b/fca 164593 0 2024-03-16T09:01:23.308 INFO:tasks.workunit.client.1.smithi019.stdout:0/844: mkdir d7/d8/dec/d108/d107/d112 0 2024-03-16T09:01:23.311 INFO:tasks.workunit.client.0.smithi007.stdout:4/844: rename d4/d14/d64/d9f/ldf to d4/d14/d1c/d5b/l126 0 2024-03-16T09:01:23.314 INFO:tasks.workunit.client.0.smithi007.stdout:5/969: dread d1/d7/d1c/d19/d34/d5e/d6f/f6c [0,4194304] 0 2024-03-16T09:01:23.318 INFO:tasks.workunit.client.0.smithi007.stdout:1/817: dread d9/dd/fff [0,4194304] 0 2024-03-16T09:01:23.319 INFO:tasks.workunit.client.0.smithi007.stdout:1/818: write d9/dd/d1b/d28/d29/d7b/db4/fce [892124,34500] 0 2024-03-16T09:01:23.320 INFO:tasks.workunit.client.0.smithi007.stdout:1/819: creat d9/dd/d1b/d28/d29/f104 x:0 0 0 2024-03-16T09:01:23.321 INFO:tasks.workunit.client.0.smithi007.stdout:1/820: mkdir d9/d6b/d7e/dac/dbb/d105 0 2024-03-16T09:01:23.321 INFO:tasks.workunit.client.0.smithi007.stdout:1/821: write f5 [2060370,20140] 0 2024-03-16T09:01:23.321 INFO:tasks.workunit.client.0.smithi007.stdout:1/822: creat d9/dd/d1b/d28/d4c/de4/f106 x:0 0 0 2024-03-16T09:01:23.321 INFO:tasks.workunit.client.0.smithi007.stdout:1/823: dread - d9/dd/d1b/d28/ff4 zero size 2024-03-16T09:01:23.322 INFO:tasks.workunit.client.0.smithi007.stdout:1/824: write d9/d6b/fe8 [430576,94145] 0 2024-03-16T09:01:23.323 INFO:tasks.workunit.client.0.smithi007.stdout:1/825: symlink d9/l107 0 2024-03-16T09:01:23.324 INFO:tasks.workunit.client.0.smithi007.stdout:1/826: symlink d9/dd/d1b/d28/d29/d7b/db4/l108 0 2024-03-16T09:01:23.326 INFO:tasks.workunit.client.0.smithi007.stdout:1/827: rename d9/dd/d1b/d2d/f44 to d9/dd/d1b/d28/d4c/f109 0 2024-03-16T09:01:23.326 INFO:tasks.workunit.client.0.smithi007.stdout:1/828: dread - d9/dd/d1b/d28/f101 zero size 2024-03-16T09:01:23.327 INFO:tasks.workunit.client.0.smithi007.stdout:1/829: rename d9/dd/d1b/d2d/d30/d45/f5b to d9/dd/f10a 0 2024-03-16T09:01:23.328 INFO:tasks.workunit.client.0.smithi007.stdout:5/970: dread d1/d7/d1c/f80 [0,4194304] 0 2024-03-16T09:01:23.328 INFO:tasks.workunit.client.0.smithi007.stdout:5/971: fsync d1/d7/d1c/d19/d156/f15b 0 2024-03-16T09:01:23.329 INFO:tasks.workunit.client.1.smithi019.stdout:0/845: dwrite d7/d8/dec/def/da1/da7/fcf [0,4194304] 0 2024-03-16T09:01:23.329 INFO:tasks.workunit.client.1.smithi019.stdout:0/846: chown d7/d8/d32/d66/d6d/d8c/db8/cdd 32670503 1 2024-03-16T09:01:23.329 INFO:tasks.workunit.client.1.smithi019.stdout:1/754: dwrite d16/d1f/d43/f4c [0,4194304] 0 2024-03-16T09:01:23.330 INFO:tasks.workunit.client.0.smithi007.stdout:5/972: rename d1/d7/d1c/d19/d34/d5e/d146/lbb to d1/d7/d1c/d19/d34/d5e/d6f/d72/l160 0 2024-03-16T09:01:23.331 INFO:tasks.workunit.client.1.smithi019.stdout:3/805: sync 2024-03-16T09:01:23.332 INFO:tasks.workunit.client.0.smithi007.stdout:5/973: unlink d1/d7/d1c/d19/d34/d1d/dee/c126 0 2024-03-16T09:01:23.333 INFO:tasks.workunit.client.1.smithi019.stdout:0/847: mkdir d7/d8/d32/df5/d113 0 2024-03-16T09:01:23.333 INFO:tasks.workunit.client.1.smithi019.stdout:3/806: dread d3/d8b/fa3 [0,4194304] 0 2024-03-16T09:01:23.333 INFO:tasks.workunit.client.1.smithi019.stdout:3/807: creat d3/d1b/d40/f101 x:0 0 0 2024-03-16T09:01:23.334 INFO:tasks.workunit.client.1.smithi019.stdout:1/755: truncate d16/d31/f45 2919285 0 2024-03-16T09:01:23.335 INFO:tasks.workunit.client.0.smithi007.stdout:5/974: link d1/d7/d1c/f80 d1/d7/d1c/d19/d34/d5e/d6f/d132/f161 0 2024-03-16T09:01:23.335 INFO:tasks.workunit.client.0.smithi007.stdout:5/975: read - d1/ff6 zero size 2024-03-16T09:01:23.336 INFO:tasks.workunit.client.0.smithi007.stdout:3/955: truncate d15/d18/d1f/d3b/fca 1002803 0 2024-03-16T09:01:23.336 INFO:tasks.workunit.client.0.smithi007.stdout:3/956: write d15/d18/d1f/d3b/d87/ffe [141026,108] 0 2024-03-16T09:01:23.336 INFO:tasks.workunit.client.0.smithi007.stdout:3/957: write d15/d18/d6a/f7e [2222951,78547] 0 2024-03-16T09:01:23.337 INFO:tasks.workunit.client.1.smithi019.stdout:0/848: rename d7/d1d/f76 to d7/d8/dec/f114 0 2024-03-16T09:01:23.339 INFO:tasks.workunit.client.1.smithi019.stdout:0/849: write d7/d8/dec/def/d48/d10c/f10e [371389,104582] 0 2024-03-16T09:01:23.340 INFO:tasks.workunit.client.1.smithi019.stdout:3/808: symlink d3/d8b/dae/l102 0 2024-03-16T09:01:23.340 INFO:tasks.workunit.client.1.smithi019.stdout:3/809: truncate d3/f6c 2720160 0 2024-03-16T09:01:23.343 INFO:tasks.workunit.client.1.smithi019.stdout:1/756: symlink d16/d31/d76/lde 0 2024-03-16T09:01:23.344 INFO:tasks.workunit.client.1.smithi019.stdout:1/757: mkdir d16/d1f/d73/ddf 0 2024-03-16T09:01:23.345 INFO:tasks.workunit.client.1.smithi019.stdout:1/758: chown d16/d1f/d43/d55 67561096 1 2024-03-16T09:01:23.345 INFO:tasks.workunit.client.0.smithi007.stdout:1/830: dwrite d9/dd/d1b/d28/f36 [0,4194304] 0 2024-03-16T09:01:23.346 INFO:tasks.workunit.client.1.smithi019.stdout:1/759: mknod d16/dcb/daf/db2/d78/db1/ce0 0 2024-03-16T09:01:23.346 INFO:tasks.workunit.client.1.smithi019.stdout:1/760: write d16/f2b [1668319,40212] 0 2024-03-16T09:01:23.346 INFO:tasks.workunit.client.1.smithi019.stdout:1/761: chown d16/l3b 38592813 1 2024-03-16T09:01:23.347 INFO:tasks.workunit.client.1.smithi019.stdout:1/762: creat d16/d31/fe1 x:0 0 0 2024-03-16T09:01:23.348 INFO:tasks.workunit.client.1.smithi019.stdout:1/763: creat d16/d31/d71/d89/dce/fe2 x:0 0 0 2024-03-16T09:01:23.349 INFO:tasks.workunit.client.1.smithi019.stdout:1/764: fsync d16/d1f/d73/fa7 0 2024-03-16T09:01:23.376 INFO:tasks.workunit.client.0.smithi007.stdout:5/976: dwrite d1/d7/d1c/d19/d34/d3e/d41/f118 [0,4194304] 0 2024-03-16T09:01:23.378 INFO:tasks.workunit.client.0.smithi007.stdout:5/977: link d1/le0 d1/d7/d1c/d19/dcc/l162 0 2024-03-16T09:01:23.378 INFO:tasks.workunit.client.0.smithi007.stdout:5/978: getdents d1/d7/d1c/d88/da2/d89/db3/d7d/d15a 0 2024-03-16T09:01:23.389 INFO:tasks.workunit.client.1.smithi019.stdout:1/765: dwrite d16/dcb/fc3 [0,4194304] 0 2024-03-16T09:01:23.391 INFO:tasks.workunit.client.0.smithi007.stdout:3/958: dwrite d15/fe8 [0,4194304] 0 2024-03-16T09:01:23.391 INFO:tasks.workunit.client.0.smithi007.stdout:3/959: fsync d15/d18/d1f/d29/d43/ff9 0 2024-03-16T09:01:23.391 INFO:tasks.workunit.client.0.smithi007.stdout:3/960: chown d15/d18/d1f/d56/d98 10 1 2024-03-16T09:01:23.391 INFO:tasks.workunit.client.0.smithi007.stdout:4/845: dwrite d4/d14/d64/f88 [0,4194304] 0 2024-03-16T09:01:23.392 INFO:tasks.workunit.client.1.smithi019.stdout:0/850: dwrite d7/f2a [0,4194304] 0 2024-03-16T09:01:23.392 INFO:tasks.workunit.client.1.smithi019.stdout:0/851: truncate d7/d8/dec/def/d48/fe0 644040 0 2024-03-16T09:01:23.393 INFO:tasks.workunit.client.0.smithi007.stdout:1/831: dwrite d9/dd/f64 [0,4194304] 0 2024-03-16T09:01:23.393 INFO:tasks.workunit.client.0.smithi007.stdout:1/832: fdatasync d9/dd/d1b/d28/d29/fab 0 2024-03-16T09:01:23.393 INFO:tasks.workunit.client.0.smithi007.stdout:1/833: readlink d9/l9c 0 2024-03-16T09:01:23.394 INFO:tasks.workunit.client.1.smithi019.stdout:3/810: dwrite d3/d8b/dae/d5a/f8c [0,4194304] 0 2024-03-16T09:01:23.394 INFO:tasks.workunit.client.1.smithi019.stdout:3/811: stat d3/d8b/dae/dd6 0 2024-03-16T09:01:23.394 INFO:tasks.workunit.client.1.smithi019.stdout:0/852: mkdir d7/d8/d32/d66/d6d/d7c/d115 0 2024-03-16T09:01:23.394 INFO:tasks.workunit.client.1.smithi019.stdout:0/853: chown d7/d8/dec/def/l40 0 1 2024-03-16T09:01:23.398 INFO:tasks.workunit.client.0.smithi007.stdout:4/846: write d4/f22 [3710781,64216] 0 2024-03-16T09:01:23.401 INFO:tasks.workunit.client.1.smithi019.stdout:1/766: dread d16/d1f/fb6 [0,4194304] 0 2024-03-16T09:01:23.401 INFO:tasks.workunit.client.1.smithi019.stdout:1/767: read f14 [3154121,64566] 0 2024-03-16T09:01:23.401 INFO:tasks.workunit.client.0.smithi007.stdout:1/834: rename d9/dd/d1b/d2d/d30/le9 to d9/dd/d1b/d28/d29/d7b/l10b 0 2024-03-16T09:01:23.404 INFO:tasks.workunit.client.0.smithi007.stdout:1/835: creat d9/d6b/d7e/dac/f10c x:0 0 0 2024-03-16T09:01:23.405 INFO:tasks.workunit.client.1.smithi019.stdout:1/768: dread d16/f20 [0,4194304] 0 2024-03-16T09:01:23.405 INFO:tasks.workunit.client.1.smithi019.stdout:1/769: chown d16/d31/d71/d89/cc0 2235065 1 2024-03-16T09:01:23.405 INFO:tasks.workunit.client.1.smithi019.stdout:3/812: creat d3/d8b/dae/d6a/f103 x:0 0 0 2024-03-16T09:01:23.405 INFO:tasks.workunit.client.1.smithi019.stdout:3/813: dread - d3/d8b/dae/d3f/f7c zero size 2024-03-16T09:01:23.407 INFO:tasks.workunit.client.1.smithi019.stdout:3/814: creat d3/d8b/dae/d5a/d75/d7a/dd0/f104 x:0 0 0 2024-03-16T09:01:23.408 INFO:tasks.workunit.client.1.smithi019.stdout:3/815: truncate d3/d8b/dae/d5a/d80/f9c 3439367 0 2024-03-16T09:01:23.408 INFO:tasks.workunit.client.0.smithi007.stdout:9/925: sync 2024-03-16T09:01:23.409 INFO:tasks.workunit.client.0.smithi007.stdout:6/902: sync 2024-03-16T09:01:23.409 INFO:tasks.workunit.client.0.smithi007.stdout:7/876: sync 2024-03-16T09:01:23.409 INFO:tasks.workunit.client.0.smithi007.stdout:7/877: write d0/d19/d1f/d5d/d9d/df7/d3a/daf/f104 [956163,55861] 0 2024-03-16T09:01:23.409 INFO:tasks.workunit.client.1.smithi019.stdout:3/816: mkdir d3/d105 0 2024-03-16T09:01:23.412 INFO:tasks.workunit.client.1.smithi019.stdout:3/817: mknod d3/d8b/da8/dcc/c106 0 2024-03-16T09:01:23.412 INFO:tasks.workunit.client.1.smithi019.stdout:3/818: chown d3/d1b/d88/d91/dbe/lf8 763 1 2024-03-16T09:01:23.414 INFO:tasks.workunit.client.0.smithi007.stdout:6/903: rename d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/d84 to d2/d10/d3d/d12a 0 2024-03-16T09:01:23.414 INFO:tasks.workunit.client.0.smithi007.stdout:5/979: dwrite d1/d7/d1c/d19/d34/de/f10d [4194304,4194304] 0 2024-03-16T09:01:23.417 INFO:tasks.workunit.client.0.smithi007.stdout:6/904: mknod d2/d13/d3f/d86/d113/c12b 0 2024-03-16T09:01:23.418 INFO:tasks.workunit.client.0.smithi007.stdout:4/847: dread d4/d14/d1c/f29 [4194304,4194304] 0 2024-03-16T09:01:23.418 INFO:tasks.workunit.client.0.smithi007.stdout:4/848: chown d4/fa3 1057139679 1 2024-03-16T09:01:23.419 INFO:tasks.workunit.client.1.smithi019.stdout:3/819: dread d3/d1b/d40/f67 [0,4194304] 0 2024-03-16T09:01:23.421 INFO:tasks.workunit.client.1.smithi019.stdout:2/875: sync 2024-03-16T09:01:23.421 INFO:tasks.workunit.client.1.smithi019.stdout:5/827: sync 2024-03-16T09:01:23.421 INFO:tasks.workunit.client.1.smithi019.stdout:6/873: sync 2024-03-16T09:01:23.421 INFO:tasks.workunit.client.1.smithi019.stdout:8/906: sync 2024-03-16T09:01:23.421 INFO:tasks.workunit.client.1.smithi019.stdout:4/738: sync 2024-03-16T09:01:23.421 INFO:tasks.workunit.client.1.smithi019.stdout:8/907: chown d6/d2a/d3c/f44 15 1 2024-03-16T09:01:23.422 INFO:tasks.workunit.client.0.smithi007.stdout:5/980: dread d1/f2 [0,4194304] 0 2024-03-16T09:01:23.422 INFO:tasks.workunit.client.0.smithi007.stdout:6/905: mknod d2/d13/d62/c12c 0 2024-03-16T09:01:23.423 INFO:tasks.workunit.client.0.smithi007.stdout:4/849: mkdir d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/d127 0 2024-03-16T09:01:23.424 INFO:tasks.workunit.client.1.smithi019.stdout:7/757: sync 2024-03-16T09:01:23.424 INFO:tasks.workunit.client.1.smithi019.stdout:9/751: sync 2024-03-16T09:01:23.425 INFO:tasks.workunit.client.1.smithi019.stdout:4/739: rename d4/ce to d4/d1b/dc1/cf3 0 2024-03-16T09:01:23.425 INFO:tasks.workunit.client.1.smithi019.stdout:5/828: getdents d6/dd/d37/d76 0 2024-03-16T09:01:23.425 INFO:tasks.workunit.client.0.smithi007.stdout:5/981: unlink d1/d7/d16/d1b/c155 0 2024-03-16T09:01:23.425 INFO:tasks.workunit.client.0.smithi007.stdout:5/982: readlink d1/d7/d1c/d19/d34/d4d/lf9 0 2024-03-16T09:01:23.427 INFO:tasks.workunit.client.1.smithi019.stdout:7/758: link d9/da/c64 d9/d25/d28/d48/ddd/cf3 0 2024-03-16T09:01:23.427 INFO:tasks.workunit.client.1.smithi019.stdout:2/876: symlink d5/d6/d2b/d2f/d35/l132 0 2024-03-16T09:01:23.427 INFO:tasks.workunit.client.1.smithi019.stdout:9/752: read d1/d7/fd [2327671,70006] 0 2024-03-16T09:01:23.427 INFO:tasks.workunit.client.1.smithi019.stdout:5/829: creat d6/dd/dde/df1/f111 x:0 0 0 2024-03-16T09:01:23.429 INFO:tasks.workunit.client.1.smithi019.stdout:5/830: truncate d6/dd/f82 724677 0 2024-03-16T09:01:23.429 INFO:tasks.workunit.client.1.smithi019.stdout:0/854: dwrite d7/d8/d32/d66/d6d/d8c/db8/d82/f81 [0,4194304] 0 2024-03-16T09:01:23.429 INFO:tasks.workunit.client.1.smithi019.stdout:0/855: dread - d7/d8/dec/def/d48/fde zero size 2024-03-16T09:01:23.429 INFO:tasks.workunit.client.1.smithi019.stdout:5/831: mkdir d6/dd/d47/d7a/db9/d112 0 2024-03-16T09:01:23.429 INFO:tasks.workunit.client.1.smithi019.stdout:5/832: fsync d6/dd/d47/d70/db2/d4c/fc6 0 2024-03-16T09:01:23.430 INFO:tasks.workunit.client.0.smithi007.stdout:3/961: dwrite d15/d18/d1f/dce/dd6/ddf/f111 [0,4194304] 0 2024-03-16T09:01:23.430 INFO:tasks.workunit.client.1.smithi019.stdout:5/833: write d6/d19/f1b [418028,90927] 0 2024-03-16T09:01:23.430 INFO:tasks.workunit.client.1.smithi019.stdout:8/908: rename d6/de/l52 to d6/d2a/d3c/dab/d30/d4a/d67/l13a 0 2024-03-16T09:01:23.431 INFO:tasks.workunit.client.1.smithi019.stdout:0/856: rmdir d7/d8/dec/def/d92 39 2024-03-16T09:01:23.431 INFO:tasks.workunit.client.0.smithi007.stdout:4/850: symlink d4/d14/d64/d68/d82/dac/dfc/de8/l128 0 2024-03-16T09:01:23.433 INFO:tasks.workunit.client.1.smithi019.stdout:5/834: mknod d6/dd/d47/d70/c113 0 2024-03-16T09:01:23.434 INFO:tasks.workunit.client.0.smithi007.stdout:1/836: dwrite d9/dd/d1b/d28/d4c/fa4 [0,4194304] 0 2024-03-16T09:01:23.435 INFO:tasks.workunit.client.1.smithi019.stdout:0/857: mknod d7/d8/d32/d4a/d61/c116 0 2024-03-16T09:01:23.436 INFO:tasks.workunit.client.0.smithi007.stdout:4/851: dread d4/d14/d64/d68/d82/dac/dfc/fb1 [0,4194304] 0 2024-03-16T09:01:23.436 INFO:tasks.workunit.client.0.smithi007.stdout:5/983: rename d1/d7/d1c/d88/da2/d89/db3/d2a/c7f to d1/d7/d16/c163 0 2024-03-16T09:01:23.436 INFO:tasks.workunit.client.0.smithi007.stdout:5/984: chown d1/d7/d1c/d19/df7/d110/f124 3050181 1 2024-03-16T09:01:23.438 INFO:tasks.workunit.client.1.smithi019.stdout:0/858: mkdir d7/d8/dec/def/d92/d117 0 2024-03-16T09:01:23.438 INFO:tasks.workunit.client.1.smithi019.stdout:0/859: write d7/fe4 [1345499,115098] 0 2024-03-16T09:01:23.439 INFO:tasks.workunit.client.1.smithi019.stdout:0/860: chown d7/d8/d32/df5/d113 50072059 1 2024-03-16T09:01:23.439 INFO:tasks.workunit.client.1.smithi019.stdout:8/909: link d6/d2a/d45/lb4 d6/d2a/d55/l13b 0 2024-03-16T09:01:23.439 INFO:tasks.workunit.client.1.smithi019.stdout:9/753: rename d1/d98/dae/dc3 to d1/d7/de/d24/d44/d60/d74/d101 0 2024-03-16T09:01:23.440 INFO:tasks.workunit.client.0.smithi007.stdout:1/837: dread d9/f35 [0,4194304] 0 2024-03-16T09:01:23.441 INFO:tasks.workunit.client.0.smithi007.stdout:1/838: write d9/dd/d1b/f96 [270726,50088] 0 2024-03-16T09:01:23.441 INFO:tasks.workunit.client.0.smithi007.stdout:1/839: creat d9/dd/d5e/d89/f10d x:0 0 0 2024-03-16T09:01:23.441 INFO:tasks.workunit.client.1.smithi019.stdout:2/877: symlink d5/l133 0 2024-03-16T09:01:23.441 INFO:tasks.workunit.client.1.smithi019.stdout:9/754: creat d1/d7/f102 x:0 0 0 2024-03-16T09:01:23.441 INFO:tasks.workunit.client.1.smithi019.stdout:2/878: write d5/d2e/d4f/f57 [1988946,28617] 0 2024-03-16T09:01:23.443 INFO:tasks.workunit.client.1.smithi019.stdout:0/861: write d7/d8/d32/fa5 [3101205,35970] 0 2024-03-16T09:01:23.443 INFO:tasks.workunit.client.1.smithi019.stdout:9/755: link d1/d7/d86/f8c d1/d7/de/d24/d44/d60/d8b/f103 0 2024-03-16T09:01:23.444 INFO:tasks.workunit.client.1.smithi019.stdout:1/770: fdatasync d16/f2b 0 2024-03-16T09:01:23.445 INFO:tasks.workunit.client.1.smithi019.stdout:0/862: mkdir d7/d1d/d118 0 2024-03-16T09:01:23.445 INFO:tasks.workunit.client.1.smithi019.stdout:2/879: creat d5/d6/d104/f134 x:0 0 0 2024-03-16T09:01:23.446 INFO:tasks.workunit.client.0.smithi007.stdout:3/962: symlink d15/d18/d6a/db1/l12f 0 2024-03-16T09:01:23.446 INFO:tasks.workunit.client.1.smithi019.stdout:9/756: rename d1/d7/de/da3/cbd to d1/d98/c104 0 2024-03-16T09:01:23.446 INFO:tasks.workunit.client.1.smithi019.stdout:1/771: rmdir d16/dcb/daf/db2/d78 39 2024-03-16T09:01:23.448 INFO:tasks.workunit.client.1.smithi019.stdout:9/757: symlink d1/d7/de/d24/d5b/dce/df8/d59/def/l105 0 2024-03-16T09:01:23.448 INFO:tasks.workunit.client.1.smithi019.stdout:9/758: readlink d1/d7/de/da3/lb5 0 2024-03-16T09:01:23.448 INFO:tasks.workunit.client.1.smithi019.stdout:9/759: truncate d1/d7/de/d24/d44/d60/d74/f80 431494 0 2024-03-16T09:01:23.448 INFO:tasks.workunit.client.1.smithi019.stdout:9/760: readlink d1/d98/dae/l17 0 2024-03-16T09:01:23.448 INFO:tasks.workunit.client.1.smithi019.stdout:9/761: truncate d1/d7/de/d24/d44/d60/d8b/dc2/fff 84320 0 2024-03-16T09:01:23.451 INFO:tasks.workunit.client.0.smithi007.stdout:4/852: link d4/f40 d4/dcb/dcd/f129 0 2024-03-16T09:01:23.452 INFO:tasks.workunit.client.1.smithi019.stdout:8/910: dread d6/d2a/d3c/f44 [0,4194304] 0 2024-03-16T09:01:23.452 INFO:tasks.workunit.client.1.smithi019.stdout:8/911: chown d6/d2a/d45/d57/d72/d79/d7b/lc2 300 1 2024-03-16T09:01:23.454 INFO:tasks.workunit.client.1.smithi019.stdout:2/880: rename d5/d6/d2b/c37 to d5/de/c135 0 2024-03-16T09:01:23.454 INFO:tasks.workunit.client.1.smithi019.stdout:2/881: write d5/d6/d2b/d2f/f67 [1515329,26507] 0 2024-03-16T09:01:23.455 INFO:tasks.workunit.client.0.smithi007.stdout:5/985: symlink d1/d7/d1c/d19/d34/d3e/d62/l164 0 2024-03-16T09:01:23.461 INFO:tasks.workunit.client.1.smithi019.stdout:2/882: unlink d5/d2e/d4f/ffa 0 2024-03-16T09:01:23.464 INFO:tasks.workunit.client.0.smithi007.stdout:4/853: dread d4/d14/d64/d68/fc4 [0,4194304] 0 2024-03-16T09:01:23.464 INFO:tasks.workunit.client.0.smithi007.stdout:4/854: chown d4/d14/d64/d68/fc4 377 1 2024-03-16T09:01:23.464 INFO:tasks.workunit.client.0.smithi007.stdout:1/840: unlink d9/dd/d1b/d2d/d30/l34 0 2024-03-16T09:01:23.464 INFO:tasks.workunit.client.0.smithi007.stdout:1/841: readlink d9/dd/d1b/d2d/d30/d45/lcc 0 2024-03-16T09:01:23.464 INFO:tasks.workunit.client.0.smithi007.stdout:1/842: readlink d9/dd/d1b/d28/d29/lc9 0 2024-03-16T09:01:23.464 INFO:tasks.workunit.client.1.smithi019.stdout:9/762: dread d1/d7/d67/f82 [0,4194304] 0 2024-03-16T09:01:23.466 INFO:tasks.workunit.client.1.smithi019.stdout:2/883: creat d5/d6/d2b/d2f/d35/f136 x:0 0 0 2024-03-16T09:01:23.468 INFO:tasks.workunit.client.0.smithi007.stdout:1/843: creat d9/dd/d1b/d2d/f10e x:0 0 0 2024-03-16T09:01:23.470 INFO:tasks.workunit.client.0.smithi007.stdout:1/844: creat d9/dd/d1b/f10f x:0 0 0 2024-03-16T09:01:23.472 INFO:tasks.workunit.client.0.smithi007.stdout:4/855: write d4/d14/d64/d9f/f61 [4277905,69478] 0 2024-03-16T09:01:23.472 INFO:tasks.workunit.client.0.smithi007.stdout:5/986: rename d1/d7/d1c/d19/d156 to d1/d7/d1c/d19/d34/d1d/da8/d111/d165 0 2024-03-16T09:01:23.472 INFO:tasks.workunit.client.0.smithi007.stdout:5/987: write d1/d7/d1c/d88/da2/d89/db3/f3d [482563,34128] 0 2024-03-16T09:01:23.472 INFO:tasks.workunit.client.0.smithi007.stdout:1/845: read d9/dd/d1b/d28/d29/fab [326090,58055] 0 2024-03-16T09:01:23.474 INFO:tasks.workunit.client.0.smithi007.stdout:5/988: mkdir d1/d7/d1c/d19/d34/d5e/d6f/d72/d135/d166 0 2024-03-16T09:01:23.474 INFO:tasks.workunit.client.0.smithi007.stdout:5/989: chown d1/d7/d1c/d19/d34/d1d/f9e 6 1 2024-03-16T09:01:23.474 INFO:tasks.workunit.client.0.smithi007.stdout:5/990: chown d1/d7/d1c/d19/d34/d1d/da8/l15d 372623 1 2024-03-16T09:01:23.475 INFO:tasks.workunit.client.0.smithi007.stdout:5/991: creat d1/d7/d1c/d19/df7/d105/f167 x:0 0 0 2024-03-16T09:01:23.477 INFO:tasks.workunit.client.0.smithi007.stdout:4/856: dread d4/d14/d64/d68/fc4 [0,4194304] 0 2024-03-16T09:01:23.478 INFO:tasks.workunit.client.0.smithi007.stdout:4/857: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/d127/f12a x:0 0 0 2024-03-16T09:01:23.479 INFO:tasks.workunit.client.0.smithi007.stdout:4/858: mknod d4/d14/c12b 0 2024-03-16T09:01:23.479 INFO:tasks.workunit.client.0.smithi007.stdout:4/859: rmdir d4/d14/d64/d68/dbf 39 2024-03-16T09:01:23.480 INFO:tasks.workunit.client.0.smithi007.stdout:4/860: write d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/fa2 [658768,120965] 0 2024-03-16T09:01:23.480 INFO:tasks.workunit.client.0.smithi007.stdout:4/861: symlink d4/d14/d64/d68/d82/dac/dfc/de8/l12c 0 2024-03-16T09:01:23.481 INFO:tasks.workunit.client.0.smithi007.stdout:4/862: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d102/d103/f12d x:0 0 0 2024-03-16T09:01:23.482 INFO:tasks.workunit.client.0.smithi007.stdout:4/863: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d102/f12e x:0 0 0 2024-03-16T09:01:23.484 INFO:tasks.workunit.client.1.smithi019.stdout:4/740: dwrite d4/d1b/d49/d58/d80/d91/fd0 [0,4194304] 0 2024-03-16T09:01:23.486 INFO:tasks.workunit.client.1.smithi019.stdout:4/741: mkdir d4/d1b/d49/d58/d80/d91/df4 0 2024-03-16T09:01:23.486 INFO:tasks.workunit.client.1.smithi019.stdout:4/742: symlink d4/d1b/d2a/d4f/lf5 0 2024-03-16T09:01:23.491 INFO:tasks.workunit.client.0.smithi007.stdout:9/926: dwrite d24/d4c/d86/daa/d7f/d8f/db5/fc7 [4194304,4194304] 0 2024-03-16T09:01:23.492 INFO:tasks.workunit.client.1.smithi019.stdout:7/759: dwrite f4 [0,4194304] 0 2024-03-16T09:01:23.493 INFO:tasks.workunit.client.1.smithi019.stdout:7/760: truncate d9/da/fd5 1684901 0 2024-03-16T09:01:23.493 INFO:tasks.workunit.client.1.smithi019.stdout:7/761: write d9/d25/f54 [4456214,8563] 0 2024-03-16T09:01:23.493 INFO:tasks.workunit.client.1.smithi019.stdout:7/762: read - d9/d25/d28/d48/f76 zero size 2024-03-16T09:01:23.496 INFO:tasks.workunit.client.1.smithi019.stdout:9/763: dread d1/d7/f68 [0,4194304] 0 2024-03-16T09:01:23.496 INFO:tasks.workunit.client.1.smithi019.stdout:9/764: fsync d1/d7/de/d24/d44/d60/d74/fad 0 2024-03-16T09:01:23.497 INFO:tasks.workunit.client.1.smithi019.stdout:9/765: link d1/d7/l3d d1/d7/de/d24/d5b/dce/df8/df3/l106 0 2024-03-16T09:01:23.498 INFO:tasks.workunit.client.0.smithi007.stdout:9/927: read d24/f2d [2217860,67517] 0 2024-03-16T09:01:23.498 INFO:tasks.workunit.client.0.smithi007.stdout:9/928: fdatasync d24/d4c/d86/daa/d95/f6f 0 2024-03-16T09:01:23.501 INFO:tasks.workunit.client.0.smithi007.stdout:9/929: truncate d24/f29 1431035 0 2024-03-16T09:01:23.504 INFO:tasks.workunit.client.0.smithi007.stdout:4/864: dread d4/dcb/feb [0,4194304] 0 2024-03-16T09:01:23.505 INFO:tasks.workunit.client.0.smithi007.stdout:9/930: creat d24/d4a/da2/de1/d110/f114 x:0 0 0 2024-03-16T09:01:23.508 INFO:tasks.workunit.client.1.smithi019.stdout:9/766: dread d1/d7/f45 [0,4194304] 0 2024-03-16T09:01:23.508 INFO:tasks.workunit.client.1.smithi019.stdout:9/767: stat d1/d7/de/d24/d44/d60/f72 0 2024-03-16T09:01:23.509 INFO:tasks.workunit.client.0.smithi007.stdout:4/865: link d4/dcb/d115/ff1 d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d102/d103/f12f 0 2024-03-16T09:01:23.512 INFO:tasks.workunit.client.1.smithi019.stdout:6/874: dwrite de/d26/d32/d97/fb8 [0,4194304] 0 2024-03-16T09:01:23.513 INFO:tasks.workunit.client.0.smithi007.stdout:5/992: write d1/d7/d1c/d19/d34/d4d/d99/fd7 [1647975,115943] 0 2024-03-16T09:01:23.513 INFO:tasks.workunit.client.0.smithi007.stdout:9/931: creat d24/d4a/d96/db9/dca/ddc/f115 x:0 0 0 2024-03-16T09:01:23.516 INFO:tasks.workunit.client.0.smithi007.stdout:5/993: symlink d1/d7/d1c/d19/d34/d5e/d6f/da9/l168 0 2024-03-16T09:01:23.516 INFO:tasks.workunit.client.0.smithi007.stdout:9/932: mkdir d24/d4c/d86/daa/d7f/d8f/dd0/d116 0 2024-03-16T09:01:23.517 INFO:tasks.workunit.client.0.smithi007.stdout:9/933: truncate d24/d4a/d96/f106 321573 0 2024-03-16T09:01:23.517 INFO:tasks.workunit.client.1.smithi019.stdout:0/863: dwrite d7/d8/d32/d66/d6d/d8c/db8/f8f [0,4194304] 0 2024-03-16T09:01:23.520 INFO:tasks.workunit.client.1.smithi019.stdout:0/864: unlink d7/d8/d32/d66/d6d/d8c/db8/d82/fc2 0 2024-03-16T09:01:23.520 INFO:tasks.workunit.client.1.smithi019.stdout:0/865: dread d7/f46 [0,4194304] 0 2024-03-16T09:01:23.521 INFO:tasks.workunit.client.1.smithi019.stdout:0/866: read d7/d8/dec/def/d48/d10c/f10e [66609,33175] 0 2024-03-16T09:01:23.523 INFO:tasks.workunit.client.1.smithi019.stdout:0/867: unlink d7/d1d/f67 0 2024-03-16T09:01:23.524 INFO:tasks.workunit.client.1.smithi019.stdout:0/868: symlink d7/d1d/d118/l119 0 2024-03-16T09:01:23.524 INFO:tasks.workunit.client.1.smithi019.stdout:0/869: stat d7/d8/d32/d4a 0 2024-03-16T09:01:23.524 INFO:tasks.workunit.client.0.smithi007.stdout:5/994: dread d1/d7/d1c/d19/d34/d4d/f86 [0,4194304] 0 2024-03-16T09:01:23.525 INFO:tasks.workunit.client.1.smithi019.stdout:0/870: symlink d7/d8/d32/d4a/d61/l11a 0 2024-03-16T09:01:23.525 INFO:tasks.workunit.client.1.smithi019.stdout:0/871: write d7/d8/d32/d66/d6d/d7c/fcd [370287,29157] 0 2024-03-16T09:01:23.526 INFO:tasks.workunit.client.1.smithi019.stdout:0/872: write d7/d1d/d63/fee [1098875,76884] 0 2024-03-16T09:01:23.526 INFO:tasks.workunit.client.0.smithi007.stdout:5/995: creat d1/d7/d1c/d19/d34/f169 x:0 0 0 2024-03-16T09:01:23.526 INFO:tasks.workunit.client.0.smithi007.stdout:5/996: truncate d1/d7/d1c/d19/d34/d5e/d6f/d72/fd3 953987 0 2024-03-16T09:01:23.527 INFO:tasks.workunit.client.1.smithi019.stdout:0/873: creat d7/d1d/d63/d6e/db0/f11b x:0 0 0 2024-03-16T09:01:23.527 INFO:tasks.workunit.client.1.smithi019.stdout:9/768: dwrite d1/d7/f34 [0,4194304] 0 2024-03-16T09:01:23.528 INFO:tasks.workunit.client.1.smithi019.stdout:8/912: dwrite d6/de/fb2 [0,4194304] 0 2024-03-16T09:01:23.528 INFO:tasks.workunit.client.0.smithi007.stdout:5/997: dread d1/d7/d1c/d19/d34/de/f49 [0,4194304] 0 2024-03-16T09:01:23.528 INFO:tasks.workunit.client.0.smithi007.stdout:3/963: dwrite d15/d18/d1f/d29/d7c/fb0 [0,4194304] 0 2024-03-16T09:01:23.529 INFO:tasks.workunit.client.1.smithi019.stdout:9/769: unlink d1/d7/de/d24/d5b/dce/df8/l94 0 2024-03-16T09:01:23.529 INFO:tasks.workunit.client.1.smithi019.stdout:9/770: fdatasync d1/d7/f45 0 2024-03-16T09:01:23.530 INFO:tasks.workunit.client.1.smithi019.stdout:9/771: symlink d1/d7/de/d24/d44/d60/l107 0 2024-03-16T09:01:23.531 INFO:tasks.workunit.client.1.smithi019.stdout:9/772: write d1/d7/de/d24/d44/d76/d9c/dcc/fdc [69272,63046] 0 2024-03-16T09:01:23.532 INFO:tasks.workunit.client.1.smithi019.stdout:8/913: mknod d6/de/d51/df6/c13c 0 2024-03-16T09:01:23.532 INFO:tasks.workunit.client.1.smithi019.stdout:8/914: stat d6/l78 0 2024-03-16T09:01:23.533 INFO:tasks.workunit.client.1.smithi019.stdout:2/884: dwrite d5/d6/d2b/f18 [0,4194304] 0 2024-03-16T09:01:23.533 INFO:tasks.workunit.client.1.smithi019.stdout:2/885: write d5/d6/d2b/d2f/d35/f136 [28629,111244] 0 2024-03-16T09:01:23.533 INFO:tasks.workunit.client.1.smithi019.stdout:6/875: read de/d26/d32/d97/d66/d72/d7f/dd5/fe9 [2359750,103963] 0 2024-03-16T09:01:23.533 INFO:tasks.workunit.client.1.smithi019.stdout:9/773: creat d1/d7/de/d24/d5b/dce/df8/dfe/f108 x:0 0 0 2024-03-16T09:01:23.534 INFO:tasks.workunit.client.1.smithi019.stdout:6/876: chown c6 511161331 1 2024-03-16T09:01:23.534 INFO:tasks.workunit.client.1.smithi019.stdout:9/774: chown d1/d7/lea 312811 1 2024-03-16T09:01:23.534 INFO:tasks.workunit.client.1.smithi019.stdout:6/877: truncate de/d26/d32/d97/fc5 559509 0 2024-03-16T09:01:23.534 INFO:tasks.workunit.client.1.smithi019.stdout:7/763: dwrite d9/da/d13/d18/d30/d3e/da5/fe9 [0,4194304] 0 2024-03-16T09:01:23.534 INFO:tasks.workunit.client.1.smithi019.stdout:7/764: chown d9 752692 1 2024-03-16T09:01:23.534 INFO:tasks.workunit.client.1.smithi019.stdout:1/772: dwrite d16/d1f/d3c/d79/fba [0,4194304] 0 2024-03-16T09:01:23.535 INFO:tasks.workunit.client.0.smithi007.stdout:6/906: dwrite d2/d13/d3f/d50/d72/fa5 [0,4194304] 0 2024-03-16T09:01:23.536 INFO:tasks.workunit.client.1.smithi019.stdout:1/773: fsync d16/dcb/daf/db2/f8e 0 2024-03-16T09:01:23.536 INFO:tasks.workunit.client.1.smithi019.stdout:4/743: dwrite d4/d1b/d2a/d4f/da0/fc6 [0,4194304] 0 2024-03-16T09:01:23.536 INFO:tasks.workunit.client.1.smithi019.stdout:6/878: symlink de/d26/d32/d97/d2c/l112 0 2024-03-16T09:01:23.536 INFO:tasks.workunit.client.1.smithi019.stdout:6/879: write de/d26/f85 [2346532,72379] 0 2024-03-16T09:01:23.537 INFO:tasks.workunit.client.1.smithi019.stdout:6/880: write f7 [971287,59233] 0 2024-03-16T09:01:23.537 INFO:tasks.workunit.client.0.smithi007.stdout:5/998: mknod d1/d7/d16/d1b/df1/c16a 0 2024-03-16T09:01:23.538 INFO:tasks.workunit.client.1.smithi019.stdout:2/886: rename d5/d6/d104/dba/dfc/f12d to d5/d6/d104/dba/f137 0 2024-03-16T09:01:23.538 INFO:tasks.workunit.client.1.smithi019.stdout:2/887: fsync d5/d5a/f117 0 2024-03-16T09:01:23.544 INFO:tasks.workunit.client.1.smithi019.stdout:2/888: creat d5/d6/d2b/f138 x:0 0 0 2024-03-16T09:01:23.544 INFO:tasks.workunit.client.1.smithi019.stdout:2/889: write d5/d6/fb0 [1686997,121315] 0 2024-03-16T09:01:23.546 INFO:tasks.workunit.client.1.smithi019.stdout:1/774: dread f9 [0,4194304] 0 2024-03-16T09:01:23.550 INFO:tasks.workunit.client.0.smithi007.stdout:5/999: symlink d1/d7/d1c/d88/da2/d89/l16b 0 2024-03-16T09:01:23.551 INFO:tasks.workunit.client.0.smithi007.stdout:3/964: dwrite d15/d18/d1f/f2e [0,4194304] 0 2024-03-16T09:01:23.551 INFO:tasks.workunit.client.0.smithi007.stdout:3/965: creat d15/d18/d1f/d3b/dae/f130 x:0 0 0 2024-03-16T09:01:23.551 INFO:tasks.workunit.client.0.smithi007.stdout:3/966: dread - d15/d18/d1f/d56/f101 zero size 2024-03-16T09:01:23.552 INFO:tasks.workunit.client.1.smithi019.stdout:4/744: dread d4/d6/d15/f2e [0,4194304] 0 2024-03-16T09:01:23.552 INFO:tasks.workunit.client.1.smithi019.stdout:6/881: unlink de/d26/d32/d4e/d75/lb2 0 2024-03-16T09:01:23.559 INFO:tasks.workunit.client.1.smithi019.stdout:2/890: link d5/d2e/d4f/f57 d5/d6/d104/dba/f139 0 2024-03-16T09:01:23.559 INFO:tasks.workunit.client.1.smithi019.stdout:2/891: fdatasync d5/fc5 0 2024-03-16T09:01:23.562 INFO:tasks.workunit.client.1.smithi019.stdout:2/892: mkdir d5/d6/d2b/d2f/d35/d4b/d13a 0 2024-03-16T09:01:23.563 INFO:tasks.workunit.client.1.smithi019.stdout:7/765: mkdir d9/df4 0 2024-03-16T09:01:23.563 INFO:tasks.workunit.client.1.smithi019.stdout:7/766: chown d9/d25/d28/f8b 272 1 2024-03-16T09:01:23.563 INFO:tasks.workunit.client.1.smithi019.stdout:2/893: creat d5/f13b x:0 0 0 2024-03-16T09:01:23.564 INFO:tasks.workunit.client.0.smithi007.stdout:4/866: rmdir d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/d127 39 2024-03-16T09:01:23.564 INFO:tasks.workunit.client.1.smithi019.stdout:1/775: readlink d16/d1f/d43/l52 0 2024-03-16T09:01:23.565 INFO:tasks.workunit.client.1.smithi019.stdout:1/776: fdatasync d16/d31/d71/d89/dce/fe2 0 2024-03-16T09:01:23.565 INFO:tasks.workunit.client.1.smithi019.stdout:1/777: read - d16/dcb/daf/db2/f8e zero size 2024-03-16T09:01:23.565 INFO:tasks.workunit.client.1.smithi019.stdout:6/882: creat de/d26/d32/dce/f113 x:0 0 0 2024-03-16T09:01:23.566 INFO:tasks.workunit.client.1.smithi019.stdout:9/775: rename d1/d7/c61 to d1/d7/de/d24/d44/d76/d9c/dcf/c109 0 2024-03-16T09:01:23.568 INFO:tasks.workunit.client.1.smithi019.stdout:7/767: creat d9/da/d13/d9a/dab/ff5 x:0 0 0 2024-03-16T09:01:23.569 INFO:tasks.workunit.client.0.smithi007.stdout:9/934: fsync d24/d4a/d96/db9/dca/ddc/f115 0 2024-03-16T09:01:23.569 INFO:tasks.workunit.client.0.smithi007.stdout:9/935: chown d24/d4a/d55/dbf/de9/dec 606099797 1 2024-03-16T09:01:23.570 INFO:tasks.workunit.client.1.smithi019.stdout:6/883: mkdir de/d26/d32/d97/d66/d114 0 2024-03-16T09:01:23.572 INFO:tasks.workunit.client.1.smithi019.stdout:9/776: creat d1/d7/d86/f10a x:0 0 0 2024-03-16T09:01:23.573 INFO:tasks.workunit.client.1.smithi019.stdout:7/768: truncate d9/db/f2c 241700 0 2024-03-16T09:01:23.577 INFO:tasks.workunit.client.0.smithi007.stdout:9/936: dread d24/d4c/d86/daa/d95/f62 [0,4194304] 0 2024-03-16T09:01:23.580 INFO:tasks.workunit.client.0.smithi007.stdout:9/937: unlink d24/d4c/d86/daa/fe5 0 2024-03-16T09:01:23.581 INFO:tasks.workunit.client.0.smithi007.stdout:9/938: fdatasync d24/d4a/d96/db9/db1/fce 0 2024-03-16T09:01:23.581 INFO:tasks.workunit.client.1.smithi019.stdout:2/894: dread d5/d2e/f3b [0,4194304] 0 2024-03-16T09:01:23.581 INFO:tasks.workunit.client.1.smithi019.stdout:2/895: readlink d5/d5a/l12c 0 2024-03-16T09:01:23.582 INFO:tasks.workunit.client.1.smithi019.stdout:1/778: dread d16/da1/fb0 [0,4194304] 0 2024-03-16T09:01:23.583 INFO:tasks.workunit.client.1.smithi019.stdout:2/896: mknod d5/d6/d104/dba/dfc/d74/c13c 0 2024-03-16T09:01:23.584 INFO:tasks.workunit.client.1.smithi019.stdout:1/779: symlink d16/d31/d76/le3 0 2024-03-16T09:01:23.584 INFO:tasks.workunit.client.1.smithi019.stdout:2/897: getdents d5/d6/d104/df7/d6f 0 2024-03-16T09:01:23.585 INFO:tasks.workunit.client.1.smithi019.stdout:2/898: mknod d5/d6/d2b/d112/c13d 0 2024-03-16T09:01:23.586 INFO:tasks.workunit.client.1.smithi019.stdout:1/780: creat d16/d1f/d3c/fe4 x:0 0 0 2024-03-16T09:01:23.588 INFO:tasks.workunit.client.1.smithi019.stdout:1/781: creat d16/dcb/daf/db2/d78/fe5 x:0 0 0 2024-03-16T09:01:23.589 INFO:tasks.workunit.client.1.smithi019.stdout:1/782: link d16/d1f/d3c/f67 d16/dcb/dd4/fe6 0 2024-03-16T09:01:23.590 INFO:tasks.workunit.client.1.smithi019.stdout:1/783: fdatasync d16/d1f/d3c/f67 0 2024-03-16T09:01:23.595 INFO:tasks.workunit.client.1.smithi019.stdout:0/874: dwrite d7/f2a [0,4194304] 0 2024-03-16T09:01:23.595 INFO:tasks.workunit.client.1.smithi019.stdout:8/915: dwrite d6/d2a/d45/d57/d96/fad [0,4194304] 0 2024-03-16T09:01:23.595 INFO:tasks.workunit.client.1.smithi019.stdout:8/916: stat d6/f9e 0 2024-03-16T09:01:23.597 INFO:tasks.workunit.client.0.smithi007.stdout:4/867: dread d4/d14/f19 [0,4194304] 0 2024-03-16T09:01:23.601 INFO:tasks.workunit.client.0.smithi007.stdout:4/868: link d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/c94 d4/d14/c130 0 2024-03-16T09:01:23.601 INFO:tasks.workunit.client.0.smithi007.stdout:4/869: creat d4/d14/d64/f131 x:0 0 0 2024-03-16T09:01:23.602 INFO:tasks.workunit.client.0.smithi007.stdout:9/939: dread f7 [0,4194304] 0 2024-03-16T09:01:23.602 INFO:tasks.workunit.client.0.smithi007.stdout:6/907: dwrite d2/f90 [0,4194304] 0 2024-03-16T09:01:23.603 INFO:tasks.workunit.client.1.smithi019.stdout:1/784: read d16/d1f/d43/d55/f5e [3953626,18711] 0 2024-03-16T09:01:23.604 INFO:tasks.workunit.client.1.smithi019.stdout:1/785: creat d16/d1f/d4f/d8a/fe7 x:0 0 0 2024-03-16T09:01:23.606 INFO:tasks.workunit.client.1.smithi019.stdout:1/786: symlink d16/dcb/le8 0 2024-03-16T09:01:23.606 INFO:tasks.workunit.client.1.smithi019.stdout:0/875: dread d7/d8/d32/d66/f91 [0,4194304] 0 2024-03-16T09:01:23.606 INFO:tasks.workunit.client.1.smithi019.stdout:1/787: creat d16/d1f/d3c/fe9 x:0 0 0 2024-03-16T09:01:23.606 INFO:tasks.workunit.client.1.smithi019.stdout:0/876: fdatasync d7/d8/dec/def/f5a 0 2024-03-16T09:01:23.607 INFO:tasks.workunit.client.1.smithi019.stdout:0/877: stat d7/f19 0 2024-03-16T09:01:23.609 INFO:tasks.workunit.client.0.smithi007.stdout:9/940: link d24/d4a/fcc d24/d4c/d86/daa/d7f/d80/d87/dda/ddb/f117 0 2024-03-16T09:01:23.609 INFO:tasks.workunit.client.0.smithi007.stdout:9/941: readlink d24/d4c/d86/daa/d7f/d80/lc6 0 2024-03-16T09:01:23.609 INFO:tasks.workunit.client.0.smithi007.stdout:9/942: truncate d24/d4c/d86/daa/d7f/d80/d87/d89/fb3 9096007 0 2024-03-16T09:01:23.609 INFO:tasks.workunit.client.0.smithi007.stdout:9/943: dread - d24/d4a/f84 zero size 2024-03-16T09:01:23.609 INFO:tasks.workunit.client.0.smithi007.stdout:9/944: stat d24/d4c/d86/daa/d7f/d80/d87/d89/fae 0 2024-03-16T09:01:23.610 INFO:tasks.workunit.client.1.smithi019.stdout:1/788: creat d16/d1f/d3c/d79/d96/dac/fea x:0 0 0 2024-03-16T09:01:23.610 INFO:tasks.workunit.client.1.smithi019.stdout:1/789: readlink d16/d1f/d3c/d79/db4/ldd 0 2024-03-16T09:01:23.610 INFO:tasks.workunit.client.1.smithi019.stdout:1/790: creat d16/d1f/d3c/feb x:0 0 0 2024-03-16T09:01:23.611 INFO:tasks.workunit.client.0.smithi007.stdout:6/908: unlink d2/d82/f100 0 2024-03-16T09:01:23.612 INFO:tasks.workunit.client.1.smithi019.stdout:4/745: dwrite d4/d1b/d2a/d4f/da0/fc6 [0,4194304] 0 2024-03-16T09:01:23.613 INFO:tasks.workunit.client.0.smithi007.stdout:4/870: getdents d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93 0 2024-03-16T09:01:23.613 INFO:tasks.workunit.client.0.smithi007.stdout:4/871: dread - d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/ffe zero size 2024-03-16T09:01:23.616 INFO:tasks.workunit.client.1.smithi019.stdout:6/884: dwrite de/d26/d32/d4e/d75/fa2 [0,4194304] 0 2024-03-16T09:01:23.616 INFO:tasks.workunit.client.1.smithi019.stdout:4/746: creat d4/d1b/d2a/d3d/de5/ff6 x:0 0 0 2024-03-16T09:01:23.616 INFO:tasks.workunit.client.1.smithi019.stdout:4/747: chown d4/d6/f81 19 1 2024-03-16T09:01:23.618 INFO:tasks.workunit.client.0.smithi007.stdout:4/872: truncate d4/d14/d1c/d5b/f5e 3704680 0 2024-03-16T09:01:23.619 INFO:tasks.workunit.client.0.smithi007.stdout:6/909: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a 39 2024-03-16T09:01:23.621 INFO:tasks.workunit.client.1.smithi019.stdout:6/885: unlink f7 0 2024-03-16T09:01:23.621 INFO:tasks.workunit.client.1.smithi019.stdout:6/886: truncate de/d26/d32/d4e/d75/dcd/fe8 582906 0 2024-03-16T09:01:23.621 INFO:tasks.workunit.client.0.smithi007.stdout:4/873: symlink d4/d14/d64/d9f/d9a/l132 0 2024-03-16T09:01:23.622 INFO:tasks.workunit.client.1.smithi019.stdout:4/748: mkdir d4/d6c/d79/df7 0 2024-03-16T09:01:23.623 INFO:tasks.workunit.client.0.smithi007.stdout:6/910: truncate d2/f7 4592113 0 2024-03-16T09:01:23.623 INFO:tasks.workunit.client.0.smithi007.stdout:6/911: chown d2/d29/l5d 673 1 2024-03-16T09:01:23.624 INFO:tasks.workunit.client.1.smithi019.stdout:6/887: chown de/d26/d32/d4e/l71 264758 1 2024-03-16T09:01:23.624 INFO:tasks.workunit.client.1.smithi019.stdout:1/791: dread d16/d31/f45 [0,4194304] 0 2024-03-16T09:01:23.624 INFO:tasks.workunit.client.1.smithi019.stdout:1/792: truncate d16/f20 1796224 0 2024-03-16T09:01:23.624 INFO:tasks.workunit.client.0.smithi007.stdout:4/874: rmdir d4/d14/d64/d68/dbf 39 2024-03-16T09:01:23.625 INFO:tasks.workunit.client.1.smithi019.stdout:4/749: mkdir d4/d1b/d2a/d3d/df8 0 2024-03-16T09:01:23.625 INFO:tasks.workunit.client.1.smithi019.stdout:4/750: readlink d4/l6f 0 2024-03-16T09:01:23.625 INFO:tasks.workunit.client.1.smithi019.stdout:4/751: write f2 [3528204,116767] 0 2024-03-16T09:01:23.625 INFO:tasks.workunit.client.1.smithi019.stdout:9/777: dwrite d1/d98/dae/f22 [4194304,4194304] 0 2024-03-16T09:01:23.625 INFO:tasks.workunit.client.1.smithi019.stdout:2/899: dwrite d5/d6/d2b/f138 [0,4194304] 0 2024-03-16T09:01:23.625 INFO:tasks.workunit.client.1.smithi019.stdout:7/769: dwrite d9/d25/d28/d48/f76 [0,4194304] 0 2024-03-16T09:01:23.626 INFO:tasks.workunit.client.0.smithi007.stdout:6/912: mknod d2/d10/d3d/d6d/dbf/d103/c12d 0 2024-03-16T09:01:23.626 INFO:tasks.workunit.client.1.smithi019.stdout:8/917: dwrite d6/d2a/d45/d57/d72/d9f/d139/d9a/fa8 [0,4194304] 0 2024-03-16T09:01:23.626 INFO:tasks.workunit.client.1.smithi019.stdout:6/888: link de/d26/d32/d4e/d57/ffa de/d26/d100/f115 0 2024-03-16T09:01:23.627 INFO:tasks.workunit.client.0.smithi007.stdout:4/875: mknod d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/c133 0 2024-03-16T09:01:23.627 INFO:tasks.workunit.client.0.smithi007.stdout:4/876: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/ffd 2507 1 2024-03-16T09:01:23.628 INFO:tasks.workunit.client.1.smithi019.stdout:1/793: truncate d16/d1f/d3c/f48 1751203 0 2024-03-16T09:01:23.628 INFO:tasks.workunit.client.1.smithi019.stdout:1/794: readlink d16/d1f/d3c/d79/d96/dac/lca 0 2024-03-16T09:01:23.628 INFO:tasks.workunit.client.1.smithi019.stdout:1/795: creat d16/d1f/d73/fec x:0 0 0 2024-03-16T09:01:23.628 INFO:tasks.workunit.client.1.smithi019.stdout:1/796: creat d16/d1f/d3c/d79/d96/dac/fed x:0 0 0 2024-03-16T09:01:23.629 INFO:tasks.workunit.client.1.smithi019.stdout:4/752: mkdir d4/d6/df9 0 2024-03-16T09:01:23.629 INFO:tasks.workunit.client.1.smithi019.stdout:4/753: stat d4/l5 0 2024-03-16T09:01:23.631 INFO:tasks.workunit.client.0.smithi007.stdout:6/913: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d6f/f12e x:0 0 0 2024-03-16T09:01:23.631 INFO:tasks.workunit.client.1.smithi019.stdout:9/778: link d1/d7/f3e d1/d7/de/d24/d5b/dce/df8/df3/f10b 0 2024-03-16T09:01:23.634 INFO:tasks.workunit.client.0.smithi007.stdout:4/877: getdents d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84 0 2024-03-16T09:01:23.634 INFO:tasks.workunit.client.0.smithi007.stdout:4/878: readlink d4/d14/d64/d9f/lec 0 2024-03-16T09:01:23.634 INFO:tasks.workunit.client.0.smithi007.stdout:4/879: chown d4/d14/f95 189578 1 2024-03-16T09:01:23.635 INFO:tasks.workunit.client.1.smithi019.stdout:7/770: getdents d9/d25/d28/d48/d8d 0 2024-03-16T09:01:23.637 INFO:tasks.workunit.client.1.smithi019.stdout:0/878: dwrite d7/d1d/d2e/fc3 [0,4194304] 0 2024-03-16T09:01:23.638 INFO:tasks.workunit.client.1.smithi019.stdout:0/879: fdatasync d7/d1d/d63/f7b 0 2024-03-16T09:01:23.638 INFO:tasks.workunit.client.1.smithi019.stdout:0/880: chown d7/d8/d32/d66/c7e 385305 1 2024-03-16T09:01:23.638 INFO:tasks.workunit.client.0.smithi007.stdout:9/945: dwrite d24/d4c/d86/daa/d95/f6f [0,4194304] 0 2024-03-16T09:01:23.640 INFO:tasks.workunit.client.0.smithi007.stdout:4/880: truncate d4/d14/d64/d68/d82/dac/dfc/d8f/fc9 3843 0 2024-03-16T09:01:23.643 INFO:tasks.workunit.client.1.smithi019.stdout:8/918: symlink d6/l13d 0 2024-03-16T09:01:23.643 INFO:tasks.workunit.client.1.smithi019.stdout:6/889: creat de/d26/d32/d97/d2c/f116 x:0 0 0 2024-03-16T09:01:23.645 INFO:tasks.workunit.client.1.smithi019.stdout:1/797: unlink d16/d18/f1e 0 2024-03-16T09:01:23.646 INFO:tasks.workunit.client.1.smithi019.stdout:4/754: rename d4/d6/d15/l51 to d4/d1b/d2a/d4f/lfa 0 2024-03-16T09:01:23.647 INFO:tasks.workunit.client.1.smithi019.stdout:0/881: write d7/d8/d32/fa5 [4287478,83301] 0 2024-03-16T09:01:23.648 INFO:tasks.workunit.client.1.smithi019.stdout:6/890: mknod de/d26/d32/c117 0 2024-03-16T09:01:23.650 INFO:tasks.workunit.client.1.smithi019.stdout:1/798: unlink d16/d31/d76/fbe 0 2024-03-16T09:01:23.652 INFO:tasks.workunit.client.1.smithi019.stdout:4/755: creat d4/d73/d8a/dd6/ffb x:0 0 0 2024-03-16T09:01:23.652 INFO:tasks.workunit.client.1.smithi019.stdout:8/919: read d6/d2a/d45/d57/d96/fad [1190866,453] 0 2024-03-16T09:01:23.654 INFO:tasks.workunit.client.1.smithi019.stdout:0/882: mknod d7/d8/dec/def/c11c 0 2024-03-16T09:01:23.656 INFO:tasks.workunit.client.1.smithi019.stdout:6/891: rename de/d1c/f10e to de/d26/d32/d97/dae/db1/f118 0 2024-03-16T09:01:23.657 INFO:tasks.workunit.client.1.smithi019.stdout:1/799: mkdir d16/d1f/d3c/d79/db4/dee 0 2024-03-16T09:01:23.657 INFO:tasks.workunit.client.1.smithi019.stdout:8/920: dread d6/d2a/d45/d57/f65 [0,4194304] 0 2024-03-16T09:01:23.659 INFO:tasks.workunit.client.1.smithi019.stdout:0/883: dread d7/d8/dec/def/da1/da7/fae [0,4194304] 0 2024-03-16T09:01:23.659 INFO:tasks.workunit.client.1.smithi019.stdout:1/800: mknod d16/dcb/cef 0 2024-03-16T09:01:23.660 INFO:tasks.workunit.client.1.smithi019.stdout:8/921: dread d6/d2a/d45/d57/d72/d9f/fc4 [0,4194304] 0 2024-03-16T09:01:23.660 INFO:tasks.workunit.client.1.smithi019.stdout:8/922: fdatasync d6/d2a/d55/fe9 0 2024-03-16T09:01:23.660 INFO:tasks.workunit.client.1.smithi019.stdout:8/923: write d6/f9e [368912,742] 0 2024-03-16T09:01:23.663 INFO:tasks.workunit.client.1.smithi019.stdout:6/892: dread de/d26/d32/d4e/fa5 [0,4194304] 0 2024-03-16T09:01:23.663 INFO:tasks.workunit.client.1.smithi019.stdout:6/893: creat de/d26/f119 x:0 0 0 2024-03-16T09:01:23.665 INFO:tasks.workunit.client.1.smithi019.stdout:4/756: dread d4/d1b/d49/d58/d80/d91/fd0 [0,4194304] 0 2024-03-16T09:01:23.666 INFO:tasks.workunit.client.0.smithi007.stdout:6/914: dread d2/d13/d3f/d50/d78/dbe/dcb/d1e/f2f [0,4194304] 0 2024-03-16T09:01:23.667 INFO:tasks.workunit.client.1.smithi019.stdout:6/894: getdents de/d26/d32/d97/d66/ddc 0 2024-03-16T09:01:23.667 INFO:tasks.workunit.client.1.smithi019.stdout:6/895: dread - de/d26/d32/d97/d2c/f116 zero size 2024-03-16T09:01:23.667 INFO:tasks.workunit.client.1.smithi019.stdout:6/896: read de/d26/d32/d97/d66/d72/db7/feb [1163610,43957] 0 2024-03-16T09:01:23.668 INFO:tasks.workunit.client.1.smithi019.stdout:1/801: write d16/d1f/f74 [1985169,7119] 0 2024-03-16T09:01:23.669 INFO:tasks.workunit.client.0.smithi007.stdout:6/915: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/f12f x:0 0 0 2024-03-16T09:01:23.669 INFO:tasks.workunit.client.0.smithi007.stdout:6/916: chown d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/c121 6458344 1 2024-03-16T09:01:23.669 INFO:tasks.workunit.client.0.smithi007.stdout:6/917: dread - d2/d10/fe0 zero size 2024-03-16T09:01:23.669 INFO:tasks.workunit.client.0.smithi007.stdout:6/918: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fe1 977338 0 2024-03-16T09:01:23.669 INFO:tasks.workunit.client.0.smithi007.stdout:6/919: chown d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/fce 4070498 1 2024-03-16T09:01:23.669 INFO:tasks.workunit.client.0.smithi007.stdout:6/920: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/fe1 1796732 0 2024-03-16T09:01:23.670 INFO:tasks.workunit.client.1.smithi019.stdout:6/897: mkdir de/d26/d32/d97/d66/da4/d11a 0 2024-03-16T09:01:23.670 INFO:tasks.workunit.client.1.smithi019.stdout:1/802: mkdir d16/d31/d71/df0 0 2024-03-16T09:01:23.670 INFO:tasks.workunit.client.1.smithi019.stdout:1/803: stat d16/d1f/d43/l56 0 2024-03-16T09:01:23.671 INFO:tasks.workunit.client.1.smithi019.stdout:1/804: mkdir d16/d1f/d4f/df1 0 2024-03-16T09:01:23.671 INFO:tasks.workunit.client.1.smithi019.stdout:1/805: truncate f12 5263838 0 2024-03-16T09:01:23.674 INFO:tasks.workunit.client.1.smithi019.stdout:2/900: dwrite d5/d6/d2b/d2f/d35/d4b/f5c [0,4194304] 0 2024-03-16T09:01:23.677 INFO:tasks.workunit.client.1.smithi019.stdout:4/757: dread d4/d1b/d2a/f74 [0,4194304] 0 2024-03-16T09:01:23.677 INFO:tasks.workunit.client.1.smithi019.stdout:9/779: dwrite d1/d98/dae/f3c [0,4194304] 0 2024-03-16T09:01:23.677 INFO:tasks.workunit.client.1.smithi019.stdout:7/771: dwrite d9/da/d13/d18/f83 [0,4194304] 0 2024-03-16T09:01:23.677 INFO:tasks.workunit.client.1.smithi019.stdout:9/780: readlink d1/d7/de/d24/d5b/dce/df8/d59/def/l105 0 2024-03-16T09:01:23.677 INFO:tasks.workunit.client.1.smithi019.stdout:9/781: chown d1/d7/de/d24/d44/d60/d74/d101/df0 7 1 2024-03-16T09:01:23.678 INFO:tasks.workunit.client.1.smithi019.stdout:4/758: mknod d4/d1b/d2a/d3d/cfc 0 2024-03-16T09:01:23.678 INFO:tasks.workunit.client.1.smithi019.stdout:4/759: chown d4/d1b/f8f 229281 1 2024-03-16T09:01:23.679 INFO:tasks.workunit.client.0.smithi007.stdout:9/946: dwrite d24/d4c/d86/daa/d7f/d8f/dd0/fea [0,4194304] 0 2024-03-16T09:01:23.680 INFO:tasks.workunit.client.1.smithi019.stdout:6/898: fdatasync de/d26/d32/d97/d66/f94 0 2024-03-16T09:01:23.681 INFO:tasks.workunit.client.0.smithi007.stdout:9/947: mknod d24/d4c/d86/daa/d7f/d80/de3/d107/d112/c118 0 2024-03-16T09:01:23.681 INFO:tasks.workunit.client.0.smithi007.stdout:9/948: readlink d24/d4a/d96/db9/db1/led 0 2024-03-16T09:01:23.681 INFO:tasks.workunit.client.0.smithi007.stdout:9/949: fsync d24/d4a/d96/db9/db1/fce 0 2024-03-16T09:01:23.682 INFO:tasks.workunit.client.1.smithi019.stdout:7/772: unlink d9/d25/fcd 0 2024-03-16T09:01:23.683 INFO:tasks.workunit.client.1.smithi019.stdout:1/806: write d16/dcb/fc3 [423548,80196] 0 2024-03-16T09:01:23.684 INFO:tasks.workunit.client.1.smithi019.stdout:0/884: dread d7/d1d/d63/fee [0,4194304] 0 2024-03-16T09:01:23.686 INFO:tasks.workunit.client.1.smithi019.stdout:4/760: mkdir d4/d6c/d89/dfd 0 2024-03-16T09:01:23.686 INFO:tasks.workunit.client.1.smithi019.stdout:2/901: read d5/d6/d2b/d2f/f46 [3796499,33430] 0 2024-03-16T09:01:23.687 INFO:tasks.workunit.client.0.smithi007.stdout:9/950: unlink d24/d4c/d86/daa/d7f/d80/de3/d107/d112/c118 0 2024-03-16T09:01:23.687 INFO:tasks.workunit.client.1.smithi019.stdout:6/899: stat de/c12 0 2024-03-16T09:01:23.688 INFO:tasks.workunit.client.1.smithi019.stdout:2/902: creat d5/d6/d104/df7/d6f/d87/f13e x:0 0 0 2024-03-16T09:01:23.688 INFO:tasks.workunit.client.1.smithi019.stdout:2/903: creat d5/d6/f13f x:0 0 0 2024-03-16T09:01:23.688 INFO:tasks.workunit.client.1.smithi019.stdout:7/773: truncate d9/db/d12/f73 814316 0 2024-03-16T09:01:23.688 INFO:tasks.workunit.client.1.smithi019.stdout:7/774: stat d9/db/l89 0 2024-03-16T09:01:23.688 INFO:tasks.workunit.client.1.smithi019.stdout:7/775: fdatasync d9/da/fae 0 2024-03-16T09:01:23.689 INFO:tasks.workunit.client.0.smithi007.stdout:9/951: mkdir d24/d4a/d96/db9/dca/ddc/d119 0 2024-03-16T09:01:23.690 INFO:tasks.workunit.client.1.smithi019.stdout:1/807: creat d16/d31/d71/d89/ff2 x:0 0 0 2024-03-16T09:01:23.691 INFO:tasks.workunit.client.1.smithi019.stdout:0/885: mknod d7/d8/d32/df5/d113/c11d 0 2024-03-16T09:01:23.694 INFO:tasks.workunit.client.1.smithi019.stdout:4/761: link d4/d1b/d2a/d4f/f8b d4/d1b/d2a/d4f/ffe 0 2024-03-16T09:01:23.694 INFO:tasks.workunit.client.1.smithi019.stdout:4/762: write f2 [4361011,34057] 0 2024-03-16T09:01:23.694 INFO:tasks.workunit.client.0.smithi007.stdout:9/952: dread d24/d4c/d86/daa/d7f/d80/d87/d89/fc2 [0,4194304] 0 2024-03-16T09:01:23.695 INFO:tasks.workunit.client.1.smithi019.stdout:6/900: mknod de/d26/d32/d4e/d10f/c11b 0 2024-03-16T09:01:23.695 INFO:tasks.workunit.client.1.smithi019.stdout:6/901: chown de/d26/d32/d97/d51/d101 23046622 1 2024-03-16T09:01:23.695 INFO:tasks.workunit.client.1.smithi019.stdout:6/902: chown de/d26/d32/d4e/d57/lf5 6 1 2024-03-16T09:01:23.695 INFO:tasks.workunit.client.1.smithi019.stdout:6/903: readlink de/d1c/l102 0 2024-03-16T09:01:23.695 INFO:tasks.workunit.client.1.smithi019.stdout:6/904: chown de/d26/d32/d97/d51/lbe 239 1 2024-03-16T09:01:23.695 INFO:tasks.workunit.client.1.smithi019.stdout:6/905: chown de/d1c/d9a 0 1 2024-03-16T09:01:23.696 INFO:tasks.workunit.client.1.smithi019.stdout:3/820: sync 2024-03-16T09:01:23.696 INFO:tasks.workunit.client.1.smithi019.stdout:5/835: sync 2024-03-16T09:01:23.696 INFO:tasks.workunit.client.1.smithi019.stdout:2/904: rename d5/de/dab/cf9 to d5/d6/d104/df7/d10a/c140 0 2024-03-16T09:01:23.697 INFO:tasks.workunit.client.1.smithi019.stdout:5/836: mkdir d6/dd/d37/d76/d107/d114 0 2024-03-16T09:01:23.699 INFO:tasks.workunit.client.0.smithi007.stdout:9/953: rmdir d24/d4c/d86/daa/d7f/d8f/dd0/d116 0 2024-03-16T09:01:23.702 INFO:tasks.workunit.client.1.smithi019.stdout:3/821: dread d3/d8b/dae/f50 [4194304,4194304] 0 2024-03-16T09:01:23.702 INFO:tasks.workunit.client.1.smithi019.stdout:5/837: link d6/dd/d37/l50 d6/dd/d47/d7a/d105/l115 0 2024-03-16T09:01:23.702 INFO:tasks.workunit.client.0.smithi007.stdout:9/954: truncate d24/d4a/d55/dbf/de9/ff3 512465 0 2024-03-16T09:01:23.705 INFO:tasks.workunit.client.1.smithi019.stdout:5/838: creat d6/dd/d47/d7a/db9/d112/f116 x:0 0 0 2024-03-16T09:01:23.706 INFO:tasks.workunit.client.1.smithi019.stdout:6/906: dread de/f1a [0,4194304] 0 2024-03-16T09:01:23.706 INFO:tasks.workunit.client.1.smithi019.stdout:6/907: write de/d26/d32/f62 [366423,111949] 0 2024-03-16T09:01:23.706 INFO:tasks.workunit.client.1.smithi019.stdout:6/908: write de/d1c/fea [276793,13703] 0 2024-03-16T09:01:23.706 INFO:tasks.workunit.client.1.smithi019.stdout:6/909: creat de/d26/d32/d97/f11c x:0 0 0 2024-03-16T09:01:23.706 INFO:tasks.workunit.client.0.smithi007.stdout:9/955: creat d24/d4c/d86/daa/f11a x:0 0 0 2024-03-16T09:01:23.708 INFO:tasks.workunit.client.0.smithi007.stdout:9/956: mknod d24/d4c/d86/daa/d7f/d80/de3/c11b 0 2024-03-16T09:01:23.708 INFO:tasks.workunit.client.0.smithi007.stdout:9/957: fdatasync d24/f33 0 2024-03-16T09:01:23.710 INFO:tasks.workunit.client.0.smithi007.stdout:9/958: mkdir d24/d4c/d86/daa/d7f/d80/d87/d89/d11c 0 2024-03-16T09:01:23.710 INFO:tasks.workunit.client.0.smithi007.stdout:9/959: chown d24/d4c/d86/daa/d7f/d8f/db5/fc7 228 1 2024-03-16T09:01:23.711 INFO:tasks.workunit.client.1.smithi019.stdout:4/763: dread d4/d1b/f57 [0,4194304] 0 2024-03-16T09:01:23.711 INFO:tasks.workunit.client.1.smithi019.stdout:4/764: write d4/d1b/d2a/d3d/de5/ff6 [1021658,32172] 0 2024-03-16T09:01:23.713 INFO:tasks.workunit.client.0.smithi007.stdout:9/960: unlink d24/d4a/da2/dd5/le6 0 2024-03-16T09:01:23.718 INFO:tasks.workunit.client.1.smithi019.stdout:5/839: getdents d6/d1f/d78 0 2024-03-16T09:01:23.743 INFO:tasks.workunit.client.0.smithi007.stdout:4/881: dwrite d4/d14/f2d [0,4194304] 0 2024-03-16T09:01:23.744 INFO:tasks.workunit.client.1.smithi019.stdout:3/822: dwrite d3/d1b/d88/d91/fb0 [0,4194304] 0 2024-03-16T09:01:23.744 INFO:tasks.workunit.client.1.smithi019.stdout:7/776: dwrite d9/d25/d28/f88 [4194304,4194304] 0 2024-03-16T09:01:23.744 INFO:tasks.workunit.client.1.smithi019.stdout:2/905: dwrite d5/d6/d104/df7/d6f/da6/fa1 [0,4194304] 0 2024-03-16T09:01:23.744 INFO:tasks.workunit.client.1.smithi019.stdout:2/906: write d5/d6/d104/dba/dfc/d74/f106 [579025,115953] 0 2024-03-16T09:01:23.744 INFO:tasks.workunit.client.1.smithi019.stdout:2/907: chown d5/f23 22 1 2024-03-16T09:01:23.745 INFO:tasks.workunit.client.1.smithi019.stdout:2/908: stat d5/d6/d2b/l11f 0 2024-03-16T09:01:23.745 INFO:tasks.workunit.client.1.smithi019.stdout:1/808: dwrite fb [0,4194304] 0 2024-03-16T09:01:23.745 INFO:tasks.workunit.client.1.smithi019.stdout:1/809: stat d16/d1f/d3c/d79/d96/dac/cb3 0 2024-03-16T09:01:23.745 INFO:tasks.workunit.client.0.smithi007.stdout:9/961: dwrite d24/d4a/d96/db9/dca/ddc/f115 [0,4194304] 0 2024-03-16T09:01:23.746 INFO:tasks.workunit.client.0.smithi007.stdout:9/962: dread d24/d4c/d86/daa/d7f/d80/d87/d89/f8a [0,4194304] 0 2024-03-16T09:01:23.749 INFO:tasks.workunit.client.0.smithi007.stdout:4/882: write d4/d14/d1c/d5b/f62 [2300504,126000] 0 2024-03-16T09:01:23.750 INFO:tasks.workunit.client.0.smithi007.stdout:9/963: mknod d24/d4a/d96/db9/dca/ddc/de8/c11d 0 2024-03-16T09:01:23.750 INFO:tasks.workunit.client.0.smithi007.stdout:9/964: stat d24/fc4 0 2024-03-16T09:01:23.750 INFO:tasks.workunit.client.0.smithi007.stdout:9/965: read - d24/d4a/d55/faf zero size 2024-03-16T09:01:23.750 INFO:tasks.workunit.client.0.smithi007.stdout:9/966: fdatasync d24/d4a/d55/dbf/de9/ff0 0 2024-03-16T09:01:23.751 INFO:tasks.workunit.client.1.smithi019.stdout:3/823: symlink d3/d1b/d40/d4a/l107 0 2024-03-16T09:01:23.751 INFO:tasks.workunit.client.1.smithi019.stdout:3/824: chown d3/d8b/dae/d6a 230 1 2024-03-16T09:01:23.752 INFO:tasks.workunit.client.0.smithi007.stdout:4/883: getdents d4/d14/d64/d68/d82/dac/dfc 0 2024-03-16T09:01:23.754 INFO:tasks.workunit.client.1.smithi019.stdout:3/825: mknod d3/d11/c108 0 2024-03-16T09:01:23.754 INFO:tasks.workunit.client.1.smithi019.stdout:3/826: write d3/d8b/dae/d5a/d75/d7a/fdf [370969,15252] 0 2024-03-16T09:01:23.755 INFO:tasks.workunit.client.1.smithi019.stdout:7/777: dread d9/db/d12/fc4 [0,4194304] 0 2024-03-16T09:01:23.755 INFO:tasks.workunit.client.1.smithi019.stdout:7/778: creat d9/da/d13/d9a/dab/ff6 x:0 0 0 2024-03-16T09:01:23.755 INFO:tasks.workunit.client.1.smithi019.stdout:7/779: chown d9/da/c64 0 1 2024-03-16T09:01:23.756 INFO:tasks.workunit.client.0.smithi007.stdout:4/884: mkdir d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/d134 0 2024-03-16T09:01:23.756 INFO:tasks.workunit.client.0.smithi007.stdout:4/885: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/f135 x:0 0 0 2024-03-16T09:01:23.756 INFO:tasks.workunit.client.0.smithi007.stdout:4/886: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/de9/f11c 1544211 1 2024-03-16T09:01:23.756 INFO:tasks.workunit.client.0.smithi007.stdout:4/887: creat d4/dcb/dcd/f136 x:0 0 0 2024-03-16T09:01:23.756 INFO:tasks.workunit.client.1.smithi019.stdout:6/910: rename de/d26/d32/d4e to de/d26/d32/d97/dae/db1/d11d 0 2024-03-16T09:01:23.756 INFO:tasks.workunit.client.1.smithi019.stdout:6/911: read - de/d26/d73/f7e zero size 2024-03-16T09:01:23.759 INFO:tasks.workunit.client.1.smithi019.stdout:4/765: rename d4/lc0 to d4/d1b/lff 0 2024-03-16T09:01:23.761 INFO:tasks.workunit.client.1.smithi019.stdout:2/909: write d5/de/f4a [3061122,103585] 0 2024-03-16T09:01:23.761 INFO:tasks.workunit.client.1.smithi019.stdout:2/910: chown d5/d105 1447733 1 2024-03-16T09:01:23.762 INFO:tasks.workunit.client.1.smithi019.stdout:6/912: creat de/d1c/d9a/d9e/f11e x:0 0 0 2024-03-16T09:01:23.766 INFO:tasks.workunit.client.1.smithi019.stdout:5/840: rename d6/dd/d47/d70/ff8 to d6/dd/d47/d70/db2/d66/f117 0 2024-03-16T09:01:23.768 INFO:tasks.workunit.client.1.smithi019.stdout:1/810: rename d16/f2b to d16/d1f/ff3 0 2024-03-16T09:01:23.770 INFO:tasks.workunit.client.1.smithi019.stdout:5/841: mkdir d6/dd/d47/d56/d87/d118 0 2024-03-16T09:01:23.770 INFO:tasks.workunit.client.1.smithi019.stdout:5/842: creat d6/dd/d61/f119 x:0 0 0 2024-03-16T09:01:23.773 INFO:tasks.workunit.client.1.smithi019.stdout:6/913: rename de/d26/le7 to de/d26/d32/d97/d51/d101/l11f 0 2024-03-16T09:01:23.775 INFO:tasks.workunit.client.1.smithi019.stdout:1/811: mkdir d16/d18/df4 0 2024-03-16T09:01:23.775 INFO:tasks.workunit.client.1.smithi019.stdout:1/812: readlink d16/dcb/daf/db2/d8f/ldc 0 2024-03-16T09:01:23.778 INFO:tasks.workunit.client.1.smithi019.stdout:5/843: creat d6/d19/d34/dd2/f11a x:0 0 0 2024-03-16T09:01:23.778 INFO:tasks.workunit.client.1.smithi019.stdout:5/844: fsync d6/d19/d34/fe8 0 2024-03-16T09:01:23.778 INFO:tasks.workunit.client.1.smithi019.stdout:5/845: creat d6/d19/d34/dd2/f11b x:0 0 0 2024-03-16T09:01:23.779 INFO:tasks.workunit.client.1.smithi019.stdout:7/780: dread d9/da/f59 [4194304,4194304] 0 2024-03-16T09:01:23.779 INFO:tasks.workunit.client.1.smithi019.stdout:7/781: creat d9/d25/d28/d48/ff7 x:0 0 0 2024-03-16T09:01:23.781 INFO:tasks.workunit.client.1.smithi019.stdout:6/914: rename de/d26/d32/d97/d66/d72/d7f/cf3 to de/d26/d32/d97/dae/db1/d11d/d75/c120 0 2024-03-16T09:01:23.782 INFO:tasks.workunit.client.1.smithi019.stdout:6/915: truncate de/d26/f2a 5159374 0 2024-03-16T09:01:23.782 INFO:tasks.workunit.client.1.smithi019.stdout:6/916: write de/d26/d32/d97/dae/db1/d11d/fa5 [2043611,126128] 0 2024-03-16T09:01:23.782 INFO:tasks.workunit.client.1.smithi019.stdout:6/917: stat de/f45 0 2024-03-16T09:01:23.782 INFO:tasks.workunit.client.1.smithi019.stdout:6/918: chown de/d26/d32/d97/fb8 196053218 1 2024-03-16T09:01:23.782 INFO:tasks.workunit.client.1.smithi019.stdout:6/919: write de/d26/d32/fd1 [1813895,53178] 0 2024-03-16T09:01:23.782 INFO:tasks.workunit.client.1.smithi019.stdout:1/813: mknod d16/d1f/d3c/d79/db4/cf5 0 2024-03-16T09:01:23.782 INFO:tasks.workunit.client.1.smithi019.stdout:6/920: dread de/d26/f2a [4194304,4194304] 0 2024-03-16T09:01:23.784 INFO:tasks.workunit.client.1.smithi019.stdout:7/782: creat d9/db/d12/d5b/ff8 x:0 0 0 2024-03-16T09:01:23.788 INFO:tasks.workunit.client.1.smithi019.stdout:7/783: symlink d9/da/d13/d18/d30/d9e/lf9 0 2024-03-16T09:01:23.789 INFO:tasks.workunit.client.1.smithi019.stdout:7/784: creat d9/d25/d28/d48/ffa x:0 0 0 2024-03-16T09:01:23.789 INFO:tasks.workunit.client.1.smithi019.stdout:5/846: write d6/d1f/f55 [3483809,112622] 0 2024-03-16T09:01:23.791 INFO:tasks.workunit.client.1.smithi019.stdout:5/847: mkdir d6/d1f/d77/d10a/d11c 0 2024-03-16T09:01:23.792 INFO:tasks.workunit.client.0.smithi007.stdout:4/888: dwrite d4/d14/d1c/d5b/fa0 [0,4194304] 0 2024-03-16T09:01:23.794 INFO:tasks.workunit.client.1.smithi019.stdout:5/848: symlink d6/dd/d47/dd0/l11d 0 2024-03-16T09:01:23.795 INFO:tasks.workunit.client.1.smithi019.stdout:5/849: mknod d6/d1f/d77/c11e 0 2024-03-16T09:01:23.797 INFO:tasks.workunit.client.1.smithi019.stdout:5/850: rename d6/d1f/d77/d84/cdb to d6/dd/df3/c11f 0 2024-03-16T09:01:23.797 INFO:tasks.workunit.client.1.smithi019.stdout:5/851: chown d6/dd/d37/d54/l68 220571 1 2024-03-16T09:01:23.797 INFO:tasks.workunit.client.1.smithi019.stdout:5/852: dread - d6/dd/d37/fa6 zero size 2024-03-16T09:01:23.798 INFO:tasks.workunit.client.1.smithi019.stdout:1/814: write d16/d1f/d4f/f58 [2063784,26741] 0 2024-03-16T09:01:23.798 INFO:tasks.workunit.client.0.smithi007.stdout:9/967: dwrite d24/d4c/d86/daa/d7f/d80/d87/dda/ddb/f117 [0,4194304] 0 2024-03-16T09:01:23.799 INFO:tasks.workunit.client.1.smithi019.stdout:2/911: dwrite d5/d6/d104/df7/d6f/fb6 [0,4194304] 0 2024-03-16T09:01:23.799 INFO:tasks.workunit.client.0.smithi007.stdout:9/968: write d24/d4c/d86/daa/d7f/d80/d87/d89/fb3 [8453174,76315] 0 2024-03-16T09:01:23.800 INFO:tasks.workunit.client.1.smithi019.stdout:5/853: rename d6/d19/d34/dd2/f11a to d6/d1f/d77/d10a/d11c/f120 0 2024-03-16T09:01:23.800 INFO:tasks.workunit.client.1.smithi019.stdout:5/854: truncate d6/d19/f3c 323228 0 2024-03-16T09:01:23.800 INFO:tasks.workunit.client.1.smithi019.stdout:5/855: fsync d6/d1f/f5e 0 2024-03-16T09:01:23.802 INFO:tasks.workunit.client.1.smithi019.stdout:1/815: symlink d16/d18/df4/lf6 0 2024-03-16T09:01:23.803 INFO:tasks.workunit.client.1.smithi019.stdout:5/856: rmdir d6/dd/d37/d76/d107/d114 0 2024-03-16T09:01:23.804 INFO:tasks.workunit.client.1.smithi019.stdout:1/816: symlink d16/d1f/d4f/lf7 0 2024-03-16T09:01:23.804 INFO:tasks.workunit.client.1.smithi019.stdout:1/817: stat d16/d31/d76/le3 0 2024-03-16T09:01:23.804 INFO:tasks.workunit.client.1.smithi019.stdout:1/818: truncate d16/d1f/d3c/d79/d96/dac/fed 416669 0 2024-03-16T09:01:23.805 INFO:tasks.workunit.client.1.smithi019.stdout:1/819: symlink d16/d84/lf8 0 2024-03-16T09:01:23.807 INFO:tasks.workunit.client.1.smithi019.stdout:1/820: mkdir d16/dcb/daf/db2/d78/db1/df9 0 2024-03-16T09:01:23.807 INFO:tasks.workunit.client.1.smithi019.stdout:7/785: write d9/d25/f31 [4475661,107041] 0 2024-03-16T09:01:23.807 INFO:tasks.workunit.client.1.smithi019.stdout:4/766: dwrite d4/d6/d15/da2/fae [0,4194304] 0 2024-03-16T09:01:23.808 INFO:tasks.workunit.client.1.smithi019.stdout:1/821: creat d16/d31/d76/d92/ffa x:0 0 0 2024-03-16T09:01:23.813 INFO:tasks.workunit.client.1.smithi019.stdout:1/822: rmdir d16/dcb/dd4/dd5 0 2024-03-16T09:01:23.813 INFO:tasks.workunit.client.1.smithi019.stdout:1/823: truncate d16/d18/f6c 511653 0 2024-03-16T09:01:23.813 INFO:tasks.workunit.client.1.smithi019.stdout:1/824: chown d16/d31 92086 1 2024-03-16T09:01:23.814 INFO:tasks.workunit.client.1.smithi019.stdout:1/825: rmdir d16/dcb/daf/db2/d8f 39 2024-03-16T09:01:23.816 INFO:tasks.workunit.client.1.smithi019.stdout:1/826: symlink d16/dcb/daf/db2/lfb 0 2024-03-16T09:01:23.817 INFO:tasks.workunit.client.1.smithi019.stdout:2/912: dread d5/d6/d2b/f18 [0,4194304] 0 2024-03-16T09:01:23.817 INFO:tasks.workunit.client.1.smithi019.stdout:1/827: mknod d16/d1f/d73/ddf/cfc 0 2024-03-16T09:01:23.818 INFO:tasks.workunit.client.1.smithi019.stdout:2/913: mknod d5/d6/d116/c141 0 2024-03-16T09:01:23.819 INFO:tasks.workunit.client.1.smithi019.stdout:2/914: mkdir d5/d5a/d80/d142 0 2024-03-16T09:01:23.819 INFO:tasks.workunit.client.1.smithi019.stdout:2/915: chown d5/d6/d104/df7 20949577 1 2024-03-16T09:01:23.819 INFO:tasks.workunit.client.1.smithi019.stdout:1/828: unlink d16/f37 0 2024-03-16T09:01:23.819 INFO:tasks.workunit.client.1.smithi019.stdout:2/916: creat d5/d111/f143 x:0 0 0 2024-03-16T09:01:23.820 INFO:tasks.workunit.client.1.smithi019.stdout:2/917: creat d5/d6/d2b/d2f/d35/d65/f144 x:0 0 0 2024-03-16T09:01:23.820 INFO:tasks.workunit.client.0.smithi007.stdout:9/969: write d24/f4f [1306412,26647] 0 2024-03-16T09:01:23.820 INFO:tasks.workunit.client.0.smithi007.stdout:9/970: dread - d24/d8e/f99 zero size 2024-03-16T09:01:23.820 INFO:tasks.workunit.client.0.smithi007.stdout:9/971: readlink lc 0 2024-03-16T09:01:23.821 INFO:tasks.workunit.client.1.smithi019.stdout:1/829: symlink d16/d1f/d3c/lfd 0 2024-03-16T09:01:23.821 INFO:tasks.workunit.client.1.smithi019.stdout:1/830: write d16/d31/d7b/fd6 [207244,48102] 0 2024-03-16T09:01:23.822 INFO:tasks.workunit.client.1.smithi019.stdout:1/831: mknod d16/dcb/cfe 0 2024-03-16T09:01:23.823 INFO:tasks.workunit.client.1.smithi019.stdout:1/832: mknod d16/d1f/d73/cff 0 2024-03-16T09:01:23.823 INFO:tasks.workunit.client.1.smithi019.stdout:5/857: read d6/fb [2325935,114388] 0 2024-03-16T09:01:23.824 INFO:tasks.workunit.client.1.smithi019.stdout:5/858: dread - d6/dd/d47/d70/db2/d4c/f91 zero size 2024-03-16T09:01:23.824 INFO:tasks.workunit.client.1.smithi019.stdout:5/859: fdatasync d6/dd/d47/f86 0 2024-03-16T09:01:23.825 INFO:tasks.workunit.client.1.smithi019.stdout:5/860: mknod d6/d1f/d77/c121 0 2024-03-16T09:01:23.825 INFO:tasks.workunit.client.1.smithi019.stdout:5/861: chown d6/d1f/l39 115 1 2024-03-16T09:01:23.826 INFO:tasks.workunit.client.1.smithi019.stdout:5/862: symlink d6/dd/l122 0 2024-03-16T09:01:23.826 INFO:tasks.workunit.client.1.smithi019.stdout:5/863: stat d6/d19/d34/c5b 0 2024-03-16T09:01:23.827 INFO:tasks.workunit.client.1.smithi019.stdout:1/833: dread d16/f20 [0,4194304] 0 2024-03-16T09:01:23.827 INFO:tasks.workunit.client.1.smithi019.stdout:5/864: mknod d6/d19/c123 0 2024-03-16T09:01:23.828 INFO:tasks.workunit.client.1.smithi019.stdout:1/834: creat d16/d1f/d43/f100 x:0 0 0 2024-03-16T09:01:23.828 INFO:tasks.workunit.client.1.smithi019.stdout:5/865: write d6/d1f/f5a [4651320,13779] 0 2024-03-16T09:01:23.828 INFO:tasks.workunit.client.1.smithi019.stdout:5/866: read d6/d1f/f3f [35654,112054] 0 2024-03-16T09:01:23.829 INFO:tasks.workunit.client.0.smithi007.stdout:9/972: rmdir d24/d4c/d86 39 2024-03-16T09:01:23.830 INFO:tasks.workunit.client.0.smithi007.stdout:4/889: dwrite d4/d14/d64/d9f/f42 [4194304,4194304] 0 2024-03-16T09:01:23.830 INFO:tasks.workunit.client.1.smithi019.stdout:2/918: write d5/d6/d2b/d2f/d35/d65/fdc [656896,40968] 0 2024-03-16T09:01:23.830 INFO:tasks.workunit.client.1.smithi019.stdout:1/835: link d16/d1f/d3c/d79/db4/lc4 d16/d1f/d73/l101 0 2024-03-16T09:01:23.830 INFO:tasks.workunit.client.1.smithi019.stdout:6/921: dwrite de/d26/f104 [0,4194304] 0 2024-03-16T09:01:23.831 INFO:tasks.workunit.client.1.smithi019.stdout:4/767: dwrite d4/d64/d99/f40 [0,4194304] 0 2024-03-16T09:01:23.831 INFO:tasks.workunit.client.0.smithi007.stdout:4/890: mknod d4/d14/d64/d68/d82/dac/c137 0 2024-03-16T09:01:23.832 INFO:tasks.workunit.client.1.smithi019.stdout:1/836: unlink d16/d31/d71/f75 0 2024-03-16T09:01:23.833 INFO:tasks.workunit.client.1.smithi019.stdout:6/922: symlink de/d26/d32/dce/l121 0 2024-03-16T09:01:23.834 INFO:tasks.workunit.client.1.smithi019.stdout:5/867: link d6/dd/d47/d70/db2/d4c/l6a d6/dd/dde/l124 0 2024-03-16T09:01:23.836 INFO:tasks.workunit.client.0.smithi007.stdout:4/891: creat d4/d14/d64/d68/dbf/f138 x:0 0 0 2024-03-16T09:01:23.837 INFO:tasks.workunit.client.1.smithi019.stdout:4/768: creat d4/d1b/d49/d6e/f100 x:0 0 0 2024-03-16T09:01:23.837 INFO:tasks.workunit.client.1.smithi019.stdout:4/769: readlink d4/la6 0 2024-03-16T09:01:23.841 INFO:tasks.workunit.client.1.smithi019.stdout:1/837: mknod d16/d31/d71/d89/c102 0 2024-03-16T09:01:23.844 INFO:tasks.workunit.client.0.smithi007.stdout:4/892: dread d4/d14/d64/d68/d82/f8d [4194304,4194304] 0 2024-03-16T09:01:23.844 INFO:tasks.workunit.client.1.smithi019.stdout:4/770: rmdir d4/d1b/d2a/d3d/dd3 0 2024-03-16T09:01:23.844 INFO:tasks.workunit.client.1.smithi019.stdout:4/771: chown d4/l65 0 1 2024-03-16T09:01:23.845 INFO:tasks.workunit.client.0.smithi007.stdout:4/893: symlink d4/d14/d64/l139 0 2024-03-16T09:01:23.846 INFO:tasks.workunit.client.1.smithi019.stdout:1/838: creat d16/d84/f103 x:0 0 0 2024-03-16T09:01:23.846 INFO:tasks.workunit.client.0.smithi007.stdout:9/973: write d24/d4c/d86/daa/d7f/d8f/db5/fc7 [656561,41266] 0 2024-03-16T09:01:23.847 INFO:tasks.workunit.client.1.smithi019.stdout:4/772: creat d4/d1b/d2a/d3d/f101 x:0 0 0 2024-03-16T09:01:23.847 INFO:tasks.workunit.client.1.smithi019.stdout:4/773: fsync d4/d64/d99/fb9 0 2024-03-16T09:01:23.848 INFO:tasks.workunit.client.0.smithi007.stdout:4/894: mkdir d4/d14/d64/d68/d82/dac/dfc/d8f/d13a 0 2024-03-16T09:01:23.848 INFO:tasks.workunit.client.0.smithi007.stdout:4/895: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dd7 50411927 1 2024-03-16T09:01:23.849 INFO:tasks.workunit.client.1.smithi019.stdout:4/774: creat d4/d6c/d79/df2/f102 x:0 0 0 2024-03-16T09:01:23.849 INFO:tasks.workunit.client.0.smithi007.stdout:4/896: getdents d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dd7 0 2024-03-16T09:01:23.849 INFO:tasks.workunit.client.0.smithi007.stdout:4/897: stat d4/d99/ccc 0 2024-03-16T09:01:23.849 INFO:tasks.workunit.client.0.smithi007.stdout:4/898: dread - d4/d14/ff4 zero size 2024-03-16T09:01:23.851 INFO:tasks.workunit.client.0.smithi007.stdout:4/899: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b/dff/f13b x:0 0 0 2024-03-16T09:01:23.851 INFO:tasks.workunit.client.0.smithi007.stdout:4/900: stat d4/d14/d1c/d5b/fa0 0 2024-03-16T09:01:23.852 INFO:tasks.workunit.client.0.smithi007.stdout:4/901: symlink d4/d14/l13c 0 2024-03-16T09:01:23.852 INFO:tasks.workunit.client.0.smithi007.stdout:4/902: chown d4/d99 265467557 1 2024-03-16T09:01:23.853 INFO:tasks.workunit.client.0.smithi007.stdout:4/903: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/f13d x:0 0 0 2024-03-16T09:01:23.855 INFO:tasks.workunit.client.0.smithi007.stdout:9/974: write d24/d4c/d86/daa/d7f/d80/d87/d89/f9d [1303187,121364] 0 2024-03-16T09:01:23.855 INFO:tasks.workunit.client.0.smithi007.stdout:9/975: fdatasync d24/d4c/fac 0 2024-03-16T09:01:23.855 INFO:tasks.workunit.client.0.smithi007.stdout:9/976: chown d24/d4c/d86/daa/d95/c6a 235 1 2024-03-16T09:01:23.855 INFO:tasks.workunit.client.0.smithi007.stdout:9/977: truncate d24/d4c/d86/fd7 303462 0 2024-03-16T09:01:23.866 INFO:tasks.workunit.client.1.smithi019.stdout:4/775: write d4/d6/d15/d9a/fa4 [3163911,111499] 0 2024-03-16T09:01:23.866 INFO:tasks.workunit.client.1.smithi019.stdout:4/776: readlink d4/d1b/d2a/l36 0 2024-03-16T09:01:23.866 INFO:tasks.workunit.client.1.smithi019.stdout:4/777: chown d4/d1b/d49/d58/d80/d91/fd0 6303 1 2024-03-16T09:01:23.866 INFO:tasks.workunit.client.1.smithi019.stdout:4/778: write d4/d64/d99/fb9 [122132,52351] 0 2024-03-16T09:01:23.869 INFO:tasks.workunit.client.1.smithi019.stdout:4/779: dread d4/d64/d9b/fe8 [0,4194304] 0 2024-03-16T09:01:23.870 INFO:tasks.workunit.client.0.smithi007.stdout:4/904: dread d4/d14/d1c/f29 [4194304,4194304] 0 2024-03-16T09:01:23.872 INFO:tasks.workunit.client.1.smithi019.stdout:4/780: unlink d4/fa5 0 2024-03-16T09:01:23.875 INFO:tasks.workunit.client.1.smithi019.stdout:6/923: dwrite de/d26/d32/d97/d2c/f3a [0,4194304] 0 2024-03-16T09:01:23.875 INFO:tasks.workunit.client.1.smithi019.stdout:6/924: write de/d26/d32/d97/dae/db1/d11d/d57/fd4 [5790044,70448] 0 2024-03-16T09:01:23.875 INFO:tasks.workunit.client.1.smithi019.stdout:5/868: dwrite d6/d19/f3c [0,4194304] 0 2024-03-16T09:01:23.876 INFO:tasks.workunit.client.0.smithi007.stdout:9/978: dwrite d24/d4c/d86/fe4 [0,4194304] 0 2024-03-16T09:01:23.882 INFO:tasks.workunit.client.0.smithi007.stdout:7/878: sync 2024-03-16T09:01:23.882 INFO:tasks.workunit.client.0.smithi007.stdout:1/846: sync 2024-03-16T09:01:23.885 INFO:tasks.workunit.client.0.smithi007.stdout:7/879: rename d0/d6c to d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d134 0 2024-03-16T09:01:23.887 INFO:tasks.workunit.client.0.smithi007.stdout:7/880: symlink d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d134/dfd/l135 0 2024-03-16T09:01:23.887 INFO:tasks.workunit.client.1.smithi019.stdout:6/925: write de/d26/f2a [1593360,85051] 0 2024-03-16T09:01:23.888 INFO:tasks.workunit.client.0.smithi007.stdout:9/979: dread d24/fc4 [4194304,4194304] 0 2024-03-16T09:01:23.889 INFO:tasks.workunit.client.0.smithi007.stdout:7/881: mknod d0/d19/d1f/d5d/d9d/df8/d121/c136 0 2024-03-16T09:01:23.889 INFO:tasks.workunit.client.0.smithi007.stdout:7/882: chown d0/dc/d5a/dba/dd9/d93/f10a 25 1 2024-03-16T09:01:23.890 INFO:tasks.workunit.client.1.smithi019.stdout:4/781: read d4/f26 [8028858,66003] 0 2024-03-16T09:01:23.890 INFO:tasks.workunit.client.1.smithi019.stdout:2/919: dwrite d5/d6/d104/df7/d6f/d87/f13e [0,4194304] 0 2024-03-16T09:01:23.890 INFO:tasks.workunit.client.0.smithi007.stdout:7/883: mknod d0/d19/d1f/d5d/d9d/df7/d3a/daf/dec/c137 0 2024-03-16T09:01:23.890 INFO:tasks.workunit.client.0.smithi007.stdout:7/884: chown d0/d19/d1f/d69 6 1 2024-03-16T09:01:23.893 INFO:tasks.workunit.client.0.smithi007.stdout:9/980: dread d24/f26 [0,4194304] 0 2024-03-16T09:01:23.894 INFO:tasks.workunit.client.0.smithi007.stdout:9/981: readlink d24/d4a/d9e/lc0 0 2024-03-16T09:01:23.896 INFO:tasks.workunit.client.0.smithi007.stdout:4/905: dwrite d4/d14/fb2 [0,4194304] 0 2024-03-16T09:01:23.900 INFO:tasks.workunit.client.0.smithi007.stdout:7/885: read d0/d19/d1f/d5d/d9d/df7/d73/f91 [2879977,33566] 0 2024-03-16T09:01:23.900 INFO:tasks.workunit.client.0.smithi007.stdout:7/886: dread - d0/dc/d5a/dba/dd9/d93/fe7 zero size 2024-03-16T09:01:23.903 INFO:tasks.workunit.client.0.smithi007.stdout:7/887: creat d0/dc/d38/f138 x:0 0 0 2024-03-16T09:01:23.903 INFO:tasks.workunit.client.0.smithi007.stdout:7/888: chown d0/d19/d1f/d5d/d9d/df7/c122 4523711 1 2024-03-16T09:01:23.905 INFO:tasks.workunit.client.0.smithi007.stdout:4/906: dread d4/d14/f95 [0,4194304] 0 2024-03-16T09:01:23.906 INFO:tasks.workunit.client.1.smithi019.stdout:5/869: dwrite d6/dd/d37/d54/f5f [0,4194304] 0 2024-03-16T09:01:23.907 INFO:tasks.workunit.client.0.smithi007.stdout:4/907: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/f13e x:0 0 0 2024-03-16T09:01:23.907 INFO:tasks.workunit.client.0.smithi007.stdout:9/982: write d24/d4a/d96/db9/db1/fce [635668,102252] 0 2024-03-16T09:01:23.907 INFO:tasks.workunit.client.0.smithi007.stdout:9/983: write d24/d4c/d86/daa/d7f/d80/d87/d89/f9d [3433662,51574] 0 2024-03-16T09:01:23.915 INFO:tasks.workunit.client.0.smithi007.stdout:9/984: creat d24/d4a/d96/db9/f11e x:0 0 0 2024-03-16T09:01:23.916 INFO:tasks.workunit.client.0.smithi007.stdout:7/889: rename d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d9c to d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d139 0 2024-03-16T09:01:23.916 INFO:tasks.workunit.client.0.smithi007.stdout:7/890: fdatasync d0/d19/d1f/d69/f11e 0 2024-03-16T09:01:23.917 INFO:tasks.workunit.client.0.smithi007.stdout:9/985: dread d24/d4c/d91/fa5 [0,4194304] 0 2024-03-16T09:01:23.917 INFO:tasks.workunit.client.0.smithi007.stdout:4/908: rename d4/d14/d64/d9f/cca to d4/d14/d64/d68/c13f 0 2024-03-16T09:01:23.917 INFO:tasks.workunit.client.0.smithi007.stdout:4/909: creat d4/d14/d64/d68/dbf/f140 x:0 0 0 2024-03-16T09:01:23.918 INFO:tasks.workunit.client.0.smithi007.stdout:7/891: creat d0/d19/d1f/d5d/d9d/df7/d3a/daf/d117/f13a x:0 0 0 2024-03-16T09:01:23.919 INFO:tasks.workunit.client.0.smithi007.stdout:9/986: unlink d24/d4a/d55/c75 0 2024-03-16T09:01:23.919 INFO:tasks.workunit.client.0.smithi007.stdout:9/987: dread - d24/d4a/d55/faf zero size 2024-03-16T09:01:23.920 INFO:tasks.workunit.client.0.smithi007.stdout:4/910: mkdir d4/dcb/d115/d141 0 2024-03-16T09:01:23.921 INFO:tasks.workunit.client.0.smithi007.stdout:7/892: rename d0/d19/d1f/d69/l8c to d0/d19/d1f/d5d/d9d/df7/d3a/d11b/l13b 0 2024-03-16T09:01:23.922 INFO:tasks.workunit.client.0.smithi007.stdout:9/988: getdents d24/d4c/d10b 0 2024-03-16T09:01:23.923 INFO:tasks.workunit.client.0.smithi007.stdout:9/989: rename d24/d4c/d86/daa/d7f/d80/d87/d89/ce7 to d24/d4c/d86/daa/d7f/d80/d87/d89/d11c/c11f 0 2024-03-16T09:01:23.924 INFO:tasks.workunit.client.0.smithi007.stdout:9/990: mkdir d24/d4a/d55/dbf/d120 0 2024-03-16T09:01:23.924 INFO:tasks.workunit.client.0.smithi007.stdout:9/991: truncate d24/d4a/da2/db7/f101 752210 0 2024-03-16T09:01:23.928 INFO:tasks.workunit.client.0.smithi007.stdout:9/992: dread d24/d4c/d86/daa/d7f/ff7 [0,4194304] 0 2024-03-16T09:01:23.928 INFO:tasks.workunit.client.0.smithi007.stdout:9/993: write d24/d4c/f105 [4742041,8541] 0 2024-03-16T09:01:23.928 INFO:tasks.workunit.client.0.smithi007.stdout:4/911: dread d4/d14/d64/d9f/f4c [0,4194304] 0 2024-03-16T09:01:23.928 INFO:tasks.workunit.client.0.smithi007.stdout:4/912: chown d4/d14/d64/d68/cf8 8 1 2024-03-16T09:01:23.929 INFO:tasks.workunit.client.0.smithi007.stdout:4/913: link d4/d14/d1c/d5b/cab d4/d99/c142 0 2024-03-16T09:01:23.930 INFO:tasks.workunit.client.0.smithi007.stdout:4/914: mknod d4/d14/d64/d68/d82/dac/c143 0 2024-03-16T09:01:23.939 INFO:tasks.workunit.client.0.smithi007.stdout:1/847: dwrite d9/d6b/d7e/f92 [0,4194304] 0 2024-03-16T09:01:23.940 INFO:tasks.workunit.client.0.smithi007.stdout:1/848: write d9/dd/d1b/d28/d4c/f63 [4640416,490] 0 2024-03-16T09:01:23.940 INFO:tasks.workunit.client.1.smithi019.stdout:5/870: dwrite d6/d19/d34/fe8 [0,4194304] 0 2024-03-16T09:01:23.940 INFO:tasks.workunit.client.1.smithi019.stdout:5/871: readlink d6/d1f/d77/lf6 0 2024-03-16T09:01:23.940 INFO:tasks.workunit.client.1.smithi019.stdout:5/872: readlink l5 0 2024-03-16T09:01:23.941 INFO:tasks.workunit.client.1.smithi019.stdout:4/782: dwrite d4/d1b/d2a/d4f/da0/fc6 [0,4194304] 0 2024-03-16T09:01:23.941 INFO:tasks.workunit.client.1.smithi019.stdout:2/920: dwrite d5/d6/d2b/d53/f71 [0,4194304] 0 2024-03-16T09:01:23.941 INFO:tasks.workunit.client.1.smithi019.stdout:2/921: chown d5/de/dd4 595282239 1 2024-03-16T09:01:23.943 INFO:tasks.workunit.client.0.smithi007.stdout:1/849: creat d9/dd/de0/f110 x:0 0 0 2024-03-16T09:01:23.943 INFO:tasks.workunit.client.1.smithi019.stdout:5/873: rename d6/d19/d34/dd2 to d6/dd/dde/d125 0 2024-03-16T09:01:23.944 INFO:tasks.workunit.client.1.smithi019.stdout:2/922: link d5/d5a/c66 d5/d2e/d4f/c145 0 2024-03-16T09:01:23.946 INFO:tasks.workunit.client.0.smithi007.stdout:1/850: mknod d9/d6b/d7e/db0/c111 0 2024-03-16T09:01:23.946 INFO:tasks.workunit.client.0.smithi007.stdout:1/851: readlink d9/dd/d1b/d58/lbf 0 2024-03-16T09:01:23.950 INFO:tasks.workunit.client.1.smithi019.stdout:8/924: sync 2024-03-16T09:01:23.954 INFO:tasks.workunit.client.1.smithi019.stdout:4/783: dread d4/d1b/f5d [0,4194304] 0 2024-03-16T09:01:23.956 INFO:tasks.workunit.client.1.smithi019.stdout:6/926: dwrite de/d26/d73/fa9 [0,4194304] 0 2024-03-16T09:01:23.956 INFO:tasks.workunit.client.1.smithi019.stdout:6/927: stat de/d26/d32/d97/d2c/l83 0 2024-03-16T09:01:23.959 INFO:tasks.workunit.client.0.smithi007.stdout:7/893: dwrite d0/d19/d1f/f116 [0,4194304] 0 2024-03-16T09:01:23.959 INFO:tasks.workunit.client.0.smithi007.stdout:7/894: chown d0/d19/d1f/d5d/d9d/df7/d3a/f44 337005 1 2024-03-16T09:01:23.961 INFO:tasks.workunit.client.1.smithi019.stdout:6/928: creat de/d26/d32/f122 x:0 0 0 2024-03-16T09:01:23.961 INFO:tasks.workunit.client.1.smithi019.stdout:6/929: stat de/d26/d32/d97/d66/d72/d7f/f81 0 2024-03-16T09:01:23.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:23 smithi019 ceph-mon[36407]: pgmap v25: 65 pgs: 65 active+clean; 2.9 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 282 MiB/s rd, 347 MiB/s wr, 466 op/s 2024-03-16T09:01:23.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:23 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:01:23.963 INFO:tasks.workunit.client.1.smithi019.stdout:4/784: rename d4/d1b/d2a to d4/d73/d103 0 2024-03-16T09:01:23.965 INFO:tasks.workunit.client.1.smithi019.stdout:8/925: dread d6/d2a/d3c/dab/d30/d4a/fcf [0,4194304] 0 2024-03-16T09:01:23.965 INFO:tasks.workunit.client.1.smithi019.stdout:6/930: rename de/d26/d32/d97/d51/c8b to de/d10c/c123 0 2024-03-16T09:01:23.965 INFO:tasks.workunit.client.1.smithi019.stdout:2/923: dwrite d5/de/f32 [0,4194304] 0 2024-03-16T09:01:23.965 INFO:tasks.workunit.client.0.smithi007.stdout:4/915: dwrite d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b/f71 [0,4194304] 0 2024-03-16T09:01:23.966 INFO:tasks.workunit.client.0.smithi007.stdout:7/895: rename d0/d19/d1f/d69/f118 to d0/d19/d1f/d5d/d9d/f13c 0 2024-03-16T09:01:23.966 INFO:tasks.workunit.client.0.smithi007.stdout:7/896: write d0/d19/d1f/d5d/fc4 [503467,10133] 0 2024-03-16T09:01:23.966 INFO:tasks.workunit.client.0.smithi007.stdout:7/897: chown d0/d19/d1f/d5d/c8f 125906516 1 2024-03-16T09:01:23.966 INFO:tasks.workunit.client.1.smithi019.stdout:4/785: symlink d4/d1b/d49/d58/d80/l104 0 2024-03-16T09:01:23.966 INFO:tasks.workunit.client.1.smithi019.stdout:4/786: stat d4/d6/d15/c2f 0 2024-03-16T09:01:23.967 INFO:tasks.workunit.client.1.smithi019.stdout:8/926: creat d6/d2a/d45/d57/d96/df5/f13e x:0 0 0 2024-03-16T09:01:23.967 INFO:tasks.workunit.client.1.smithi019.stdout:8/927: stat d6/d2a/d45/d57/d72/d79/db7/fe1 0 2024-03-16T09:01:23.968 INFO:tasks.workunit.client.1.smithi019.stdout:6/931: truncate de/d1c/f28 6086964 0 2024-03-16T09:01:23.968 INFO:tasks.workunit.client.1.smithi019.stdout:6/932: fdatasync de/d26/d32/dc1/fca 0 2024-03-16T09:01:23.968 INFO:tasks.workunit.client.1.smithi019.stdout:6/933: creat de/d26/d32/d97/f124 x:0 0 0 2024-03-16T09:01:23.976 INFO:tasks.workunit.client.0.smithi007.stdout:4/916: chown d4/d14/d1c/c1f 1 1 2024-03-16T09:01:23.976 INFO:tasks.workunit.client.0.smithi007.stdout:4/917: creat d4/dcb/dcd/f144 x:0 0 0 2024-03-16T09:01:23.976 INFO:tasks.workunit.client.0.smithi007.stdout:4/918: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/dbe/f145 x:0 0 0 2024-03-16T09:01:23.977 INFO:tasks.workunit.client.0.smithi007.stdout:1/852: dwrite d9/f22 [0,4194304] 0 2024-03-16T09:01:23.978 INFO:tasks.workunit.client.1.smithi019.stdout:8/928: creat d6/d2a/d45/f13f x:0 0 0 2024-03-16T09:01:23.979 INFO:tasks.workunit.client.1.smithi019.stdout:8/929: read - d6/d2a/d45/f13f zero size 2024-03-16T09:01:23.979 INFO:tasks.workunit.client.0.smithi007.stdout:3/967: sync 2024-03-16T09:01:23.980 INFO:tasks.workunit.client.0.smithi007.stdout:1/853: dread d9/dd/d1b/f96 [0,4194304] 0 2024-03-16T09:01:23.981 INFO:tasks.workunit.client.0.smithi007.stdout:1/854: write d9/dd/d1b/d2d/f10e [205276,16562] 0 2024-03-16T09:01:23.981 INFO:tasks.workunit.client.0.smithi007.stdout:1/855: chown d9/d6b/d7e/dac/dbb/lcb 206711585 1 2024-03-16T09:01:23.984 INFO:tasks.workunit.client.1.smithi019.stdout:8/930: rename d6/de/f1b to d6/d2a/d3c/dab/d98/f140 0 2024-03-16T09:01:23.984 INFO:tasks.workunit.client.1.smithi019.stdout:8/931: chown d6/d2a/d68/fba 62872822 1 2024-03-16T09:01:23.986 INFO:tasks.workunit.client.0.smithi007.stdout:4/919: symlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/l146 0 2024-03-16T09:01:23.986 INFO:tasks.workunit.client.0.smithi007.stdout:4/920: write d4/d14/d1c/d5b/f8e [1079429,16942] 0 2024-03-16T09:01:23.990 INFO:tasks.workunit.client.0.smithi007.stdout:3/968: mkdir d15/d18/d1f/d29/d3e/dc0/da5/d131 0 2024-03-16T09:01:23.990 INFO:tasks.workunit.client.0.smithi007.stdout:3/969: truncate d15/d18/d1f/d29/d43/f7a 4683253 0 2024-03-16T09:01:23.990 INFO:tasks.workunit.client.0.smithi007.stdout:3/970: creat d15/d18/d1f/d29/d43/df7/f132 x:0 0 0 2024-03-16T09:01:23.990 INFO:tasks.workunit.client.0.smithi007.stdout:3/971: readlink d15/d18/d6a/db1/l12f 0 2024-03-16T09:01:23.990 INFO:tasks.workunit.client.0.smithi007.stdout:3/972: dread - d15/d18/d6a/f119 zero size 2024-03-16T09:01:23.990 INFO:tasks.workunit.client.0.smithi007.stdout:6/921: sync 2024-03-16T09:01:23.990 INFO:tasks.workunit.client.0.smithi007.stdout:6/922: truncate d2/d13/d3f/d50/d78/dbe/dcb/d1e/f111 4811525 0 2024-03-16T09:01:23.991 INFO:tasks.workunit.client.0.smithi007.stdout:7/898: rmdir d0/d19/d1f/d69/de1/de9 39 2024-03-16T09:01:23.991 INFO:tasks.workunit.client.0.smithi007.stdout:7/899: write d0/f3 [4663161,105199] 0 2024-03-16T09:01:23.991 INFO:tasks.workunit.client.1.smithi019.stdout:0/886: sync 2024-03-16T09:01:23.991 INFO:tasks.workunit.client.1.smithi019.stdout:3/827: sync 2024-03-16T09:01:23.991 INFO:tasks.workunit.client.1.smithi019.stdout:9/782: sync 2024-03-16T09:01:23.991 INFO:tasks.workunit.client.1.smithi019.stdout:0/887: write d7/f46 [342097,16436] 0 2024-03-16T09:01:23.992 INFO:tasks.workunit.client.1.smithi019.stdout:4/787: dwrite d4/d64/d9b/fb7 [0,4194304] 0 2024-03-16T09:01:23.992 INFO:tasks.workunit.client.1.smithi019.stdout:6/934: dwrite de/d26/d32/d97/d51/fba [0,4194304] 0 2024-03-16T09:01:23.992 INFO:tasks.workunit.client.1.smithi019.stdout:6/935: creat de/d26/d73/ded/f125 x:0 0 0 2024-03-16T09:01:23.992 INFO:tasks.workunit.client.1.smithi019.stdout:6/936: fdatasync de/d26/d32/d97/d51/d70/f88 0 2024-03-16T09:01:23.993 INFO:tasks.workunit.client.1.smithi019.stdout:9/783: link d1/d7/de/d24/d44/d76/l49 d1/d7/d67/d77/l10c 0 2024-03-16T09:01:23.994 INFO:tasks.workunit.client.0.smithi007.stdout:1/856: mkdir d9/dd/d1b/dd8/d112 0 2024-03-16T09:01:23.995 INFO:tasks.workunit.client.1.smithi019.stdout:3/828: rename d3/d1b/d40/d4a/db7/ffa to d3/d8b/dae/d5a/d75/d7a/f109 0 2024-03-16T09:01:23.995 INFO:tasks.workunit.client.0.smithi007.stdout:4/921: mkdir d4/d14/d147 0 2024-03-16T09:01:23.996 INFO:tasks.workunit.client.1.smithi019.stdout:8/932: dread d6/d2a/d3c/dab/d98/fca [0,4194304] 0 2024-03-16T09:01:23.996 INFO:tasks.workunit.client.1.smithi019.stdout:8/933: truncate d6/d2a/d45/d19/d74/f7e 4799587 0 2024-03-16T09:01:23.996 INFO:tasks.workunit.client.1.smithi019.stdout:4/788: rmdir d4/d1b 39 2024-03-16T09:01:23.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:23 smithi007 ceph-mon[29470]: pgmap v25: 65 pgs: 65 active+clean; 2.9 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 282 MiB/s rd, 347 MiB/s wr, 466 op/s 2024-03-16T09:01:23.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:23 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:01:23.998 INFO:tasks.workunit.client.1.smithi019.stdout:6/937: rmdir de/d1c/d9a 39 2024-03-16T09:01:23.998 INFO:tasks.workunit.client.1.smithi019.stdout:6/938: chown de/d26/d32/d97/d66/d72/dd0 1041 1 2024-03-16T09:01:23.999 INFO:tasks.workunit.client.0.smithi007.stdout:3/973: creat d15/d18/d1f/dce/f133 x:0 0 0 2024-03-16T09:01:24.000 INFO:tasks.workunit.client.1.smithi019.stdout:9/784: unlink d1/d7/de/d24/f42 0 2024-03-16T09:01:24.000 INFO:tasks.workunit.client.1.smithi019.stdout:9/785: chown d1/d7/d67/c6f 86134 1 2024-03-16T09:01:24.000 INFO:tasks.workunit.client.1.smithi019.stdout:4/789: mkdir d4/d105 0 2024-03-16T09:01:24.000 INFO:tasks.workunit.client.1.smithi019.stdout:4/790: chown d4/d6/d15/d1f/fbd 66273738 1 2024-03-16T09:01:24.001 INFO:tasks.workunit.client.1.smithi019.stdout:4/791: symlink d4/l106 0 2024-03-16T09:01:24.003 INFO:tasks.workunit.client.1.smithi019.stdout:4/792: rename d4/d73/d103/d4f/ffe to d4/d9e/f107 0 2024-03-16T09:01:24.004 INFO:tasks.workunit.client.0.smithi007.stdout:4/922: truncate d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/f100 3653806 0 2024-03-16T09:01:24.004 INFO:tasks.workunit.client.1.smithi019.stdout:4/793: creat d4/d6/df9/f108 x:0 0 0 2024-03-16T09:01:24.005 INFO:tasks.workunit.client.0.smithi007.stdout:3/974: link d15/d18/d1f/d29/d7c/f94 d15/d18/d1f/d29/d3e/f134 0 2024-03-16T09:01:24.006 INFO:tasks.workunit.client.1.smithi019.stdout:8/934: write d6/de/f6c [767947,99159] 0 2024-03-16T09:01:24.006 INFO:tasks.workunit.client.1.smithi019.stdout:3/829: dread d3/d11/f6f [0,4194304] 0 2024-03-16T09:01:24.007 INFO:tasks.workunit.client.1.smithi019.stdout:3/830: readlink d3/d1b/dc7/ld5 0 2024-03-16T09:01:24.007 INFO:tasks.workunit.client.0.smithi007.stdout:1/857: getdents d9/dd/d1b/d28/d29/d7b/d90 0 2024-03-16T09:01:24.008 INFO:tasks.workunit.client.1.smithi019.stdout:4/794: mkdir d4/d73/d103/d3d/de5/d109 0 2024-03-16T09:01:24.008 INFO:tasks.workunit.client.1.smithi019.stdout:0/888: write d7/f19 [193746,4033] 0 2024-03-16T09:01:24.009 INFO:tasks.workunit.client.1.smithi019.stdout:3/831: truncate d3/d11/f49 1891374 0 2024-03-16T09:01:24.009 INFO:tasks.workunit.client.1.smithi019.stdout:3/832: write d3/d8b/dae/f59 [4402573,127222] 0 2024-03-16T09:01:24.009 INFO:tasks.workunit.client.0.smithi007.stdout:3/975: unlink d15/l40 0 2024-03-16T09:01:24.009 INFO:tasks.workunit.client.0.smithi007.stdout:3/976: fsync d15/d18/d1f/d29/d3e/dc0/ff4 0 2024-03-16T09:01:24.009 INFO:tasks.workunit.client.0.smithi007.stdout:3/977: dread - d15/d18/d1f/d29/d3e/d85/fcb zero size 2024-03-16T09:01:24.010 INFO:tasks.workunit.client.0.smithi007.stdout:7/900: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/daf/dec/f106 [0,4194304] 0 2024-03-16T09:01:24.010 INFO:tasks.workunit.client.1.smithi019.stdout:9/786: read d1/d98/dae/f7a [2324973,53175] 0 2024-03-16T09:01:24.011 INFO:tasks.workunit.client.1.smithi019.stdout:4/795: creat d4/d9e/f10a x:0 0 0 2024-03-16T09:01:24.011 INFO:tasks.workunit.client.0.smithi007.stdout:4/923: dread d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/f120 [4194304,4194304] 0 2024-03-16T09:01:24.011 INFO:tasks.workunit.client.0.smithi007.stdout:4/924: write d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/d127/f12a [219657,66110] 0 2024-03-16T09:01:24.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/858: mknod d9/dd/d1b/d28/d29/d7b/db4/c113 0 2024-03-16T09:01:24.012 INFO:tasks.workunit.client.0.smithi007.stdout:1/859: readlink d9/l107 0 2024-03-16T09:01:24.014 INFO:tasks.workunit.client.1.smithi019.stdout:0/889: creat d7/d8/d32/d4a/d61/d68/dea/d109/f11e x:0 0 0 2024-03-16T09:01:24.016 INFO:tasks.workunit.client.0.smithi007.stdout:3/978: creat d15/d18/d6a/db1/d110/f135 x:0 0 0 2024-03-16T09:01:24.016 INFO:tasks.workunit.client.0.smithi007.stdout:3/979: chown d15/d18/d1f/d29/d3e/d85/fa6 2846 1 2024-03-16T09:01:24.016 INFO:tasks.workunit.client.0.smithi007.stdout:3/980: creat d15/d18/d1f/d3b/dae/d10e/f136 x:0 0 0 2024-03-16T09:01:24.016 INFO:tasks.workunit.client.1.smithi019.stdout:3/833: rename d3/d8b/dae/d3f/c45 to d3/d89/c10a 0 2024-03-16T09:01:24.018 INFO:tasks.workunit.client.1.smithi019.stdout:3/834: dread d3/d8b/dae/d5a/fbb [0,4194304] 0 2024-03-16T09:01:24.018 INFO:tasks.workunit.client.1.smithi019.stdout:3/835: creat d3/d8b/dae/d6a/f10b x:0 0 0 2024-03-16T09:01:24.018 INFO:tasks.workunit.client.1.smithi019.stdout:3/836: chown d3/d1b/d40/d4a/c5e 1 1 2024-03-16T09:01:24.018 INFO:tasks.workunit.client.1.smithi019.stdout:3/837: chown d3/d8b/dae/d5a/d75/d7a/fdf 3530 1 2024-03-16T09:01:24.021 INFO:tasks.workunit.client.1.smithi019.stdout:4/796: unlink d4/d6c/le3 0 2024-03-16T09:01:24.021 INFO:tasks.workunit.client.0.smithi007.stdout:4/925: symlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/d134/l148 0 2024-03-16T09:01:24.021 INFO:tasks.workunit.client.0.smithi007.stdout:4/926: readlink d4/d14/d64/d68/dbf/le2 0 2024-03-16T09:01:24.022 INFO:tasks.workunit.client.0.smithi007.stdout:1/860: creat d9/dd/d1b/d2d/d30/d45/f114 x:0 0 0 2024-03-16T09:01:24.024 INFO:tasks.workunit.client.1.smithi019.stdout:0/890: dread d7/d8/d32/d66/f91 [0,4194304] 0 2024-03-16T09:01:24.024 INFO:tasks.workunit.client.0.smithi007.stdout:1/861: dread d9/dd/d1b/d2d/d30/f6d [0,4194304] 0 2024-03-16T09:01:24.025 INFO:tasks.workunit.client.1.smithi019.stdout:9/787: rename d1/cbe to d1/d7/dc0/c10d 0 2024-03-16T09:01:24.027 INFO:tasks.workunit.client.1.smithi019.stdout:3/838: mknod d3/d11/d100/c10c 0 2024-03-16T09:01:24.027 INFO:tasks.workunit.client.0.smithi007.stdout:4/927: symlink d4/d14/d64/d68/d82/dac/dfc/d8f/d13a/l149 0 2024-03-16T09:01:24.028 INFO:tasks.workunit.client.1.smithi019.stdout:4/797: creat d4/da3/f10b x:0 0 0 2024-03-16T09:01:24.031 INFO:tasks.workunit.client.0.smithi007.stdout:3/981: getdents d15/d18/d1f/d29/d3e/dc0/da5 0 2024-03-16T09:01:24.031 INFO:tasks.workunit.client.0.smithi007.stdout:3/982: stat d15/d18/d1f/d29/d3e/d4c/f64 0 2024-03-16T09:01:24.032 INFO:tasks.workunit.client.1.smithi019.stdout:0/891: rename d7/d8/d32/d66/d6d/d8c/db8/d82/f81 to d7/d8/d32/d4a/d61/f11f 0 2024-03-16T09:01:24.033 INFO:tasks.workunit.client.0.smithi007.stdout:7/901: dwrite d0/dc/fca [0,4194304] 0 2024-03-16T09:01:24.033 INFO:tasks.workunit.client.1.smithi019.stdout:8/935: dwrite d6/d2a/d45/d57/d72/d79/db7/f132 [0,4194304] 0 2024-03-16T09:01:24.034 INFO:tasks.workunit.client.1.smithi019.stdout:6/939: dwrite de/d26/fff [0,4194304] 0 2024-03-16T09:01:24.035 INFO:tasks.workunit.client.1.smithi019.stdout:8/936: write d6/f9e [56990,55120] 0 2024-03-16T09:01:24.035 INFO:tasks.workunit.client.1.smithi019.stdout:3/839: mkdir d3/d1b/d88/d91/dbe/d10d 0 2024-03-16T09:01:24.035 INFO:tasks.workunit.client.0.smithi007.stdout:4/928: mkdir d4/dcb/dcd/d14a 0 2024-03-16T09:01:24.037 INFO:tasks.workunit.client.1.smithi019.stdout:0/892: mknod d7/d8/dec/def/da1/ddf/c120 0 2024-03-16T09:01:24.037 INFO:tasks.workunit.client.1.smithi019.stdout:0/893: readlink d7/d8/d32/d4a/l8d 0 2024-03-16T09:01:24.037 INFO:tasks.workunit.client.1.smithi019.stdout:0/894: write d7/d1d/d2e/f3c [1716941,39573] 0 2024-03-16T09:01:24.042 INFO:tasks.workunit.client.1.smithi019.stdout:6/940: creat de/d26/d32/d97/d51/d70/f126 x:0 0 0 2024-03-16T09:01:24.043 INFO:tasks.workunit.client.1.smithi019.stdout:3/840: symlink d3/d1b/d88/d91/dbe/d10d/l10e 0 2024-03-16T09:01:24.044 INFO:tasks.workunit.client.0.smithi007.stdout:7/902: rename d0/d19/cc8 to d0/d19/d1f/d69/c13d 0 2024-03-16T09:01:24.045 INFO:tasks.workunit.client.1.smithi019.stdout:0/895: rmdir d7/d8/d32/df5 39 2024-03-16T09:01:24.045 INFO:tasks.workunit.client.1.smithi019.stdout:0/896: chown d7/d8/dec/def/da1/ddf 140 1 2024-03-16T09:01:24.047 INFO:tasks.workunit.client.1.smithi019.stdout:0/897: mkdir d7/d1d/d63/d89/d9e/d121 0 2024-03-16T09:01:24.047 INFO:tasks.workunit.client.1.smithi019.stdout:0/898: read d7/d8/dec/def/d92/da6/fbb [76329,5783] 0 2024-03-16T09:01:24.052 INFO:tasks.workunit.client.1.smithi019.stdout:8/937: dread d6/d2a/d3c/dab/d30/f4f [0,4194304] 0 2024-03-16T09:01:24.054 INFO:tasks.workunit.client.1.smithi019.stdout:6/941: dread de/d26/fff [0,4194304] 0 2024-03-16T09:01:24.057 INFO:tasks.workunit.client.1.smithi019.stdout:6/942: mkdir de/d26/d32/dc1/d127 0 2024-03-16T09:01:24.057 INFO:tasks.workunit.client.1.smithi019.stdout:6/943: read de/d1c/fbc [7261022,62879] 0 2024-03-16T09:01:24.057 INFO:tasks.workunit.client.1.smithi019.stdout:9/788: dwrite d1/d7/d21/da2/fb8 [0,4194304] 0 2024-03-16T09:01:24.059 INFO:tasks.workunit.client.1.smithi019.stdout:8/938: mknod d6/d2a/d45/d57/d72/d79/db7/c141 0 2024-03-16T09:01:24.059 INFO:tasks.workunit.client.1.smithi019.stdout:8/939: symlink d6/d2a/d45/d19/d74/l142 0 2024-03-16T09:01:24.061 INFO:tasks.workunit.client.0.smithi007.stdout:3/983: dwrite d15/d18/d1f/f58 [0,4194304] 0 2024-03-16T09:01:24.061 INFO:tasks.workunit.client.0.smithi007.stdout:3/984: dread - d15/d18/d1f/d3b/d87/f11d zero size 2024-03-16T09:01:24.061 INFO:tasks.workunit.client.0.smithi007.stdout:3/985: dread - d15/d18/d1f/d56/fde zero size 2024-03-16T09:01:24.061 INFO:tasks.workunit.client.0.smithi007.stdout:3/986: readlink d15/d18/d6a/d78/l86 0 2024-03-16T09:01:24.061 INFO:tasks.workunit.client.1.smithi019.stdout:9/789: rename d1/d7/dbb/fc6 to d1/d7/de/d24/d44/d60/d74/d101/f10e 0 2024-03-16T09:01:24.063 INFO:tasks.workunit.client.1.smithi019.stdout:4/798: dwrite d4/d64/d9b/fe8 [0,4194304] 0 2024-03-16T09:01:24.063 INFO:tasks.workunit.client.1.smithi019.stdout:4/799: chown d4/d64/d9b/da9/lc3 1847 1 2024-03-16T09:01:24.063 INFO:tasks.workunit.client.1.smithi019.stdout:4/800: readlink d4/d73/d103/d3d/l86 0 2024-03-16T09:01:24.063 INFO:tasks.workunit.client.1.smithi019.stdout:8/940: link d6/d2a/d45/d5d/fa7 d6/d2a/d45/d2c/de3/f143 0 2024-03-16T09:01:24.066 INFO:tasks.workunit.client.0.smithi007.stdout:3/987: link d15/d18/d1f/f58 d15/d18/d1f/d56/d98/f137 0 2024-03-16T09:01:24.066 INFO:tasks.workunit.client.1.smithi019.stdout:4/801: mkdir d4/d73/d8a/dd6/d10c 0 2024-03-16T09:01:24.067 INFO:tasks.workunit.client.1.smithi019.stdout:1/839: sync 2024-03-16T09:01:24.067 INFO:tasks.workunit.client.1.smithi019.stdout:7/786: sync 2024-03-16T09:01:24.067 INFO:tasks.workunit.client.1.smithi019.stdout:7/787: creat d9/d25/d28/d48/ffb x:0 0 0 2024-03-16T09:01:24.068 INFO:tasks.workunit.client.1.smithi019.stdout:7/788: dread - d9/db/d12/fea zero size 2024-03-16T09:01:24.069 INFO:tasks.workunit.client.0.smithi007.stdout:3/988: mknod d15/d18/d1f/dce/dd6/c138 0 2024-03-16T09:01:24.069 INFO:tasks.workunit.client.1.smithi019.stdout:1/840: truncate d16/d18/f23 1167712 0 2024-03-16T09:01:24.070 INFO:tasks.workunit.client.1.smithi019.stdout:1/841: dread d16/d18/f6c [0,4194304] 0 2024-03-16T09:01:24.071 INFO:tasks.workunit.client.0.smithi007.stdout:3/989: creat d15/d18/d1f/d3b/d87/dee/f139 x:0 0 0 2024-03-16T09:01:24.071 INFO:tasks.workunit.client.0.smithi007.stdout:3/990: read d15/fa4 [812709,52172] 0 2024-03-16T09:01:24.072 INFO:tasks.workunit.client.1.smithi019.stdout:4/802: rename d4/d6/d15/f33 to d4/d64/f10d 0 2024-03-16T09:01:24.072 INFO:tasks.workunit.client.1.smithi019.stdout:4/803: write d4/d6/fcc [1205047,31225] 0 2024-03-16T09:01:24.072 INFO:tasks.workunit.client.1.smithi019.stdout:4/804: chown d4/d1b/f5d 233004 1 2024-03-16T09:01:24.072 INFO:tasks.workunit.client.1.smithi019.stdout:1/842: symlink d16/dcb/daf/l104 0 2024-03-16T09:01:24.073 INFO:tasks.workunit.client.1.smithi019.stdout:1/843: readlink d16/d1f/d4f/lf7 0 2024-03-16T09:01:24.073 INFO:tasks.workunit.client.0.smithi007.stdout:3/991: symlink d15/d18/d1f/d29/l13a 0 2024-03-16T09:01:24.075 INFO:tasks.workunit.client.1.smithi019.stdout:1/844: truncate d16/f65 2283570 0 2024-03-16T09:01:24.075 INFO:tasks.workunit.client.1.smithi019.stdout:1/845: stat d16/dcb/daf/db2/lfb 0 2024-03-16T09:01:24.075 INFO:tasks.workunit.client.1.smithi019.stdout:1/846: dread - d16/d84/fbf zero size 2024-03-16T09:01:24.075 INFO:tasks.workunit.client.1.smithi019.stdout:1/847: read - d16/d1f/d3c/fe4 zero size 2024-03-16T09:01:24.078 INFO:tasks.workunit.client.1.smithi019.stdout:7/789: rename d9/da to d9/d25/d28/d48/dfc 0 2024-03-16T09:01:24.080 INFO:tasks.workunit.client.1.smithi019.stdout:1/848: creat d16/d31/d71/df0/f105 x:0 0 0 2024-03-16T09:01:24.080 INFO:tasks.workunit.client.1.smithi019.stdout:1/849: write d16/d31/d76/d92/ffa [753899,71890] 0 2024-03-16T09:01:24.080 INFO:tasks.workunit.client.1.smithi019.stdout:1/850: truncate d16/dcb/daf/db2/f8e 698877 0 2024-03-16T09:01:24.080 INFO:tasks.workunit.client.1.smithi019.stdout:1/851: write d16/d1f/d43/f100 [354576,107994] 0 2024-03-16T09:01:24.080 INFO:tasks.workunit.client.1.smithi019.stdout:7/790: mknod d9/df4/cfd 0 2024-03-16T09:01:24.080 INFO:tasks.workunit.client.1.smithi019.stdout:7/791: read - d9/d25/d28/d48/d8d/dba/fc8 zero size 2024-03-16T09:01:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:3/992: rename d15/d18/d1f/d29/d3e/d85/def to d15/d18/d1f/d29/d3e/dc0/da5/d13b 0 2024-03-16T09:01:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:3/993: fdatasync d15/d18/d1f/d29/d7c/fba 0 2024-03-16T09:01:24.126 INFO:tasks.workunit.client.1.smithi019.stdout:7/792: write d9/db/f87 [1071652,22079] 0 2024-03-16T09:01:24.126 INFO:tasks.workunit.client.1.smithi019.stdout:7/793: dread - d9/d25/d28/d48/d8d/dba/fc9 zero size 2024-03-16T09:01:24.127 INFO:tasks.workunit.client.1.smithi019.stdout:3/841: dwrite d3/d8b/dae/fd4 [0,4194304] 0 2024-03-16T09:01:24.127 INFO:tasks.workunit.client.1.smithi019.stdout:3/842: chown d3/d8b/c97 0 1 2024-03-16T09:01:24.127 INFO:tasks.workunit.client.1.smithi019.stdout:7/794: rename d9/d25/d28/d48/d8d/dba/dad/dd8/fe0 to d9/d25/d28/d48/dfc/d13/d9a/ffe 0 2024-03-16T09:01:24.128 INFO:tasks.workunit.client.1.smithi019.stdout:3/843: mknod d3/d1b/d40/d4a/db7/c10f 0 2024-03-16T09:01:24.129 INFO:tasks.workunit.client.1.smithi019.stdout:0/899: dwrite d7/f19 [0,4194304] 0 2024-03-16T09:01:24.129 INFO:tasks.workunit.client.1.smithi019.stdout:0/900: chown d7/d1d/d63/d6e/l75 984 1 2024-03-16T09:01:24.129 INFO:tasks.workunit.client.1.smithi019.stdout:7/795: creat d9/fff x:0 0 0 2024-03-16T09:01:24.132 INFO:tasks.workunit.client.0.smithi007.stdout:7/903: dwrite d0/f11 [0,4194304] 0 2024-03-16T09:01:24.132 INFO:tasks.workunit.client.0.smithi007.stdout:7/904: write d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f8e [1043870,20812] 0 2024-03-16T09:01:24.133 INFO:tasks.workunit.client.1.smithi019.stdout:9/790: dwrite d1/d98/dae/f55 [0,4194304] 0 2024-03-16T09:01:24.133 INFO:tasks.workunit.client.1.smithi019.stdout:7/796: getdents d9/d25/d28/d48/dfc/d13/d18/d30/d3e/da5 0 2024-03-16T09:01:24.134 INFO:tasks.workunit.client.0.smithi007.stdout:7/905: symlink d0/d19/d1f/d69/l13e 0 2024-03-16T09:01:24.135 INFO:tasks.workunit.client.1.smithi019.stdout:6/944: dwrite de/d1c/fbc [0,4194304] 0 2024-03-16T09:01:24.135 INFO:tasks.workunit.client.1.smithi019.stdout:6/945: fsync de/d26/d32/d97/d66/ddc/ffc 0 2024-03-16T09:01:24.135 INFO:tasks.workunit.client.1.smithi019.stdout:8/941: dwrite d6/d2a/d45/d19/f43 [0,4194304] 0 2024-03-16T09:01:24.136 INFO:tasks.workunit.client.1.smithi019.stdout:8/942: fdatasync d6/d2a/d45/d19/f97 0 2024-03-16T09:01:24.136 INFO:tasks.workunit.client.0.smithi007.stdout:3/994: dwrite d15/d18/d1f/dce/dd6/ddf/f111 [0,4194304] 0 2024-03-16T09:01:24.136 INFO:tasks.workunit.client.0.smithi007.stdout:7/906: link d0/dc/lf d0/d19/d1f/d5d/d9d/df7/d3a/db3/l13f 0 2024-03-16T09:01:24.138 INFO:tasks.workunit.client.1.smithi019.stdout:4/805: dwrite d4/d73/d8a/dd6/ffb [0,4194304] 0 2024-03-16T09:01:24.139 INFO:tasks.workunit.client.1.smithi019.stdout:1/852: dwrite d16/d1f/d3c/d79/f91 [0,4194304] 0 2024-03-16T09:01:24.139 INFO:tasks.workunit.client.1.smithi019.stdout:8/943: mkdir d6/de/d51/d12a/d144 0 2024-03-16T09:01:24.140 INFO:tasks.workunit.client.1.smithi019.stdout:0/901: write d7/d1d/d2e/fc3 [1373442,85643] 0 2024-03-16T09:01:24.140 INFO:tasks.workunit.client.1.smithi019.stdout:0/902: creat d7/d1d/d63/d89/f122 x:0 0 0 2024-03-16T09:01:24.141 INFO:tasks.workunit.client.1.smithi019.stdout:0/903: write d7/d8/d32/d66/d6d/d8c/db8/d82/fe3 [766603,60632] 0 2024-03-16T09:01:24.148 INFO:tasks.workunit.client.1.smithi019.stdout:4/806: rename d4/l6f to d4/d6c/d79/df7/l10e 0 2024-03-16T09:01:24.148 INFO:tasks.workunit.client.1.smithi019.stdout:9/791: dread d1/d7/d21/f7b [0,4194304] 0 2024-03-16T09:01:24.149 INFO:tasks.workunit.client.1.smithi019.stdout:1/853: mknod d16/d1f/d4f/c106 0 2024-03-16T09:01:24.149 INFO:tasks.workunit.client.1.smithi019.stdout:1/854: fsync d16/dcb/daf/db2/f8e 0 2024-03-16T09:01:24.149 INFO:tasks.workunit.client.1.smithi019.stdout:1/855: chown d16/d84/f103 3244199 1 2024-03-16T09:01:24.149 INFO:tasks.workunit.client.1.smithi019.stdout:8/944: stat d6/d2a/d45/d2c/l89 0 2024-03-16T09:01:24.150 INFO:tasks.workunit.client.1.smithi019.stdout:0/904: mkdir d7/d8/d32/d4a/d61/d68/dea/d123 0 2024-03-16T09:01:24.150 INFO:tasks.workunit.client.1.smithi019.stdout:0/905: chown d7/d1d/d63/df1 1527001 1 2024-03-16T09:01:24.151 INFO:tasks.workunit.client.1.smithi019.stdout:7/797: dread d9/f74 [0,4194304] 0 2024-03-16T09:01:24.152 INFO:tasks.workunit.client.1.smithi019.stdout:4/807: mknod d4/d1b/d49/d58/c10f 0 2024-03-16T09:01:24.153 INFO:tasks.workunit.client.1.smithi019.stdout:8/945: dread d6/d2a/d45/d2c/de3/f143 [0,4194304] 0 2024-03-16T09:01:24.153 INFO:tasks.workunit.client.1.smithi019.stdout:9/792: mkdir d1/d7/de/d24/d44/d76/d10f 0 2024-03-16T09:01:24.154 INFO:tasks.workunit.client.1.smithi019.stdout:1/856: mknod d16/d31/d71/df0/c107 0 2024-03-16T09:01:24.156 INFO:tasks.workunit.client.1.smithi019.stdout:0/906: link d7/l24 d7/d8/dec/def/d6c/l124 0 2024-03-16T09:01:24.160 INFO:tasks.workunit.client.1.smithi019.stdout:9/793: unlink d1/d98/dae/f3c 0 2024-03-16T09:01:24.161 INFO:tasks.workunit.client.1.smithi019.stdout:8/946: symlink d6/d2a/d3c/dab/d30/d4a/l145 0 2024-03-16T09:01:24.161 INFO:tasks.workunit.client.1.smithi019.stdout:8/947: truncate f2 4991491 0 2024-03-16T09:01:24.162 INFO:tasks.workunit.client.1.smithi019.stdout:4/808: write d4/f35 [672589,128863] 0 2024-03-16T09:01:24.162 INFO:tasks.workunit.client.1.smithi019.stdout:4/809: creat d4/d1b/d49/d6e/f110 x:0 0 0 2024-03-16T09:01:24.163 INFO:tasks.workunit.client.1.smithi019.stdout:9/794: rename d1/d7/lea to d1/d7/de/d24/d44/d60/d8b/l110 0 2024-03-16T09:01:24.163 INFO:tasks.workunit.client.1.smithi019.stdout:9/795: fdatasync d1/d7/de/d24/d5b/f6e 0 2024-03-16T09:01:24.163 INFO:tasks.workunit.client.1.smithi019.stdout:9/796: chown d1/d98/dae/f4e 143 1 2024-03-16T09:01:24.163 INFO:tasks.workunit.client.1.smithi019.stdout:9/797: chown d1/d7/de/d24/d44/d60/f64 37 1 2024-03-16T09:01:24.164 INFO:tasks.workunit.client.1.smithi019.stdout:8/948: mkdir d6/d2a/d68/d11e/d146 0 2024-03-16T09:01:24.165 INFO:tasks.workunit.client.1.smithi019.stdout:9/798: link d1/d7/de/da3/lb5 d1/d7/d21/l111 0 2024-03-16T09:01:24.166 INFO:tasks.workunit.client.1.smithi019.stdout:9/799: mkdir d1/d7/de/d24/d44/d112 0 2024-03-16T09:01:24.168 INFO:tasks.workunit.client.1.smithi019.stdout:9/800: rename d1/d98/dae/f57 to d1/d7/de/d24/d44/d112/f113 0 2024-03-16T09:01:24.169 INFO:tasks.workunit.client.1.smithi019.stdout:9/801: symlink d1/d7/d21/da2/l114 0 2024-03-16T09:01:24.171 INFO:tasks.workunit.client.1.smithi019.stdout:9/802: rename d1/d7/de/d24/d44/d60/d8b/f103 to d1/d7/de/d24/d5b/dce/df8/dc9/f115 0 2024-03-16T09:01:24.171 INFO:tasks.workunit.client.1.smithi019.stdout:9/803: creat d1/f116 x:0 0 0 2024-03-16T09:01:24.171 INFO:tasks.workunit.client.1.smithi019.stdout:9/804: dread d1/d7/de/d24/d44/d60/d74/f80 [0,4194304] 0 2024-03-16T09:01:24.174 INFO:tasks.workunit.client.1.smithi019.stdout:9/805: dread d1/d7/d67/d77/faa [0,4194304] 0 2024-03-16T09:01:24.174 INFO:tasks.workunit.client.1.smithi019.stdout:9/806: write d1/d7/de/d24/d5b/dce/df8/dc9/f115 [1395531,102152] 0 2024-03-16T09:01:24.182 INFO:tasks.workunit.client.1.smithi019.stdout:9/807: dread d1/d7/de/d24/d44/d76/f5f [0,4194304] 0 2024-03-16T09:01:24.182 INFO:tasks.workunit.client.1.smithi019.stdout:9/808: write d1/d7/de/d24/d5b/f6e [208306,120111] 0 2024-03-16T09:01:24.187 INFO:tasks.workunit.client.1.smithi019.stdout:8/949: dread d6/f27 [0,4194304] 0 2024-03-16T09:01:24.190 INFO:tasks.workunit.client.1.smithi019.stdout:9/809: dread d1/d7/d67/d77/fd3 [0,4194304] 0 2024-03-16T09:01:24.190 INFO:tasks.workunit.client.1.smithi019.stdout:9/810: readlink d1/d7/de/d24/d44/d60/d8b/l110 0 2024-03-16T09:01:24.190 INFO:tasks.workunit.client.1.smithi019.stdout:8/950: dread d6/d2a/d3c/dab/d30/d4a/fef [0,4194304] 0 2024-03-16T09:01:24.191 INFO:tasks.workunit.client.1.smithi019.stdout:9/811: mkdir d1/d117 0 2024-03-16T09:01:24.191 INFO:tasks.workunit.client.1.smithi019.stdout:9/812: write d1/d7/de/d24/d44/d76/f9a [383776,2469] 0 2024-03-16T09:01:24.191 INFO:tasks.workunit.client.1.smithi019.stdout:9/813: chown d1/d98/fd1 135616 1 2024-03-16T09:01:24.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/814: dread - d1/d7/dc0/de2/ff4 zero size 2024-03-16T09:01:24.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/815: fdatasync d1/d7/de/d24/f39 0 2024-03-16T09:01:24.193 INFO:tasks.workunit.client.1.smithi019.stdout:8/951: mknod d6/d2a/d68/c147 0 2024-03-16T09:01:24.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/816: dread d1/d7/de/d24/d44/d60/d74/d101/fdf [0,4194304] 0 2024-03-16T09:01:24.200 INFO:tasks.workunit.client.1.smithi019.stdout:8/952: dread d6/d2a/d3c/dab/d30/d4a/d67/fdd [0,4194304] 0 2024-03-16T09:01:24.200 INFO:tasks.workunit.client.1.smithi019.stdout:8/953: dread - d6/d2a/d45/d57/d72/d9f/d139/f12d zero size 2024-03-16T09:01:24.200 INFO:tasks.workunit.client.1.smithi019.stdout:8/954: stat d6/d2a/d45/f32 0 2024-03-16T09:01:24.200 INFO:tasks.workunit.client.1.smithi019.stdout:8/955: write d6/d2a/d3c/dab/f11b [5033529,29120] 0 2024-03-16T09:01:24.201 INFO:tasks.workunit.client.1.smithi019.stdout:8/956: creat d6/d2a/d45/d57/d72/d79/da5/f148 x:0 0 0 2024-03-16T09:01:24.201 INFO:tasks.workunit.client.1.smithi019.stdout:5/874: sync 2024-03-16T09:01:24.202 INFO:tasks.workunit.client.0.smithi007.stdout:7/907: dwrite d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d139/f10f [0,4194304] 0 2024-03-16T09:01:24.202 INFO:tasks.workunit.client.0.smithi007.stdout:3/995: dwrite d15/d18/d1f/d77/fe0 [0,4194304] 0 2024-03-16T09:01:24.203 INFO:tasks.workunit.client.1.smithi019.stdout:7/798: dwrite d9/db/fb1 [0,4194304] 0 2024-03-16T09:01:24.203 INFO:tasks.workunit.client.1.smithi019.stdout:6/946: dwrite de/d26/d32/d97/dae/db1/d11d/d75/fa2 [0,4194304] 0 2024-03-16T09:01:24.203 INFO:tasks.workunit.client.1.smithi019.stdout:7/799: write d9/d25/d28/d48/d8d/dba/f7c [1171331,74571] 0 2024-03-16T09:01:24.204 INFO:tasks.workunit.client.1.smithi019.stdout:4/810: dwrite d4/d64/d99/f7b [0,4194304] 0 2024-03-16T09:01:24.208 INFO:tasks.workunit.client.0.smithi007.stdout:7/908: rename d0/dc/fd to d0/d19/d1f/d5d/d9d/df8/d121/f140 0 2024-03-16T09:01:24.208 INFO:tasks.workunit.client.0.smithi007.stdout:7/909: chown d0/d19/d1f/d5d/d9d/df7/d3a/db3/fee 1023 1 2024-03-16T09:01:24.209 INFO:tasks.workunit.client.1.smithi019.stdout:2/924: sync 2024-03-16T09:01:24.209 INFO:tasks.workunit.client.1.smithi019.stdout:6/947: mknod de/d26/d32/d97/d51/c128 0 2024-03-16T09:01:24.211 INFO:tasks.workunit.client.1.smithi019.stdout:8/957: getdents d6/d2a/d3c/dab/d30/d4a 0 2024-03-16T09:01:24.211 INFO:tasks.workunit.client.1.smithi019.stdout:8/958: chown d6/d2a/d45/d57/d72/lbd 345 1 2024-03-16T09:01:24.211 INFO:tasks.workunit.client.0.smithi007.stdout:3/996: stat d15/d18/d6a/d78/lcf 0 2024-03-16T09:01:24.213 INFO:tasks.workunit.client.1.smithi019.stdout:4/811: unlink d4/d6c/d79/lad 0 2024-03-16T09:01:24.213 INFO:tasks.workunit.client.1.smithi019.stdout:2/925: mkdir d5/d6/d104/dba/dfc/d146 0 2024-03-16T09:01:24.213 INFO:tasks.workunit.client.1.smithi019.stdout:2/926: read d5/d2e/d4f/f5e [2868576,67482] 0 2024-03-16T09:01:24.213 INFO:tasks.workunit.client.1.smithi019.stdout:2/927: chown d5/d6/fb0 1162966 1 2024-03-16T09:01:24.214 INFO:tasks.workunit.client.1.smithi019.stdout:7/800: rmdir d9/db/d12/d5b 39 2024-03-16T09:01:24.215 INFO:tasks.workunit.client.1.smithi019.stdout:8/959: getdents d6/d2a/d45/d103 0 2024-03-16T09:01:24.215 INFO:tasks.workunit.client.1.smithi019.stdout:8/960: chown d6/d2a/d45/d57/d96/l99 1587370 1 2024-03-16T09:01:24.218 INFO:tasks.workunit.client.0.smithi007.stdout:7/910: mknod d0/d19/d1f/d5d/d9d/df7/da1/c141 0 2024-03-16T09:01:24.219 INFO:tasks.workunit.client.1.smithi019.stdout:4/812: rename d4/d9e/lef to d4/d1b/d49/d6e/l111 0 2024-03-16T09:01:24.219 INFO:tasks.workunit.client.1.smithi019.stdout:4/813: stat d4/d1b/d49/d58/d80/d91/fd0 0 2024-03-16T09:01:24.220 INFO:tasks.workunit.client.0.smithi007.stdout:3/997: getdents d15/d18/d1f/dce/dd6 0 2024-03-16T09:01:24.220 INFO:tasks.workunit.client.0.smithi007.stdout:3/998: chown d15/d18/d1f/d29/d3e/dda/cf1 0 1 2024-03-16T09:01:24.220 INFO:tasks.workunit.client.0.smithi007.stdout:3/999: fsync d15/d18/f22 0 2024-03-16T09:01:24.221 INFO:tasks.workunit.client.1.smithi019.stdout:7/801: unlink d9/db/d12/dcf/le5 0 2024-03-16T09:01:24.221 INFO:tasks.workunit.client.1.smithi019.stdout:8/961: fsync d6/d2a/d3c/dab/f11b 0 2024-03-16T09:01:24.221 INFO:tasks.workunit.client.1.smithi019.stdout:8/962: chown d6/d2a/d45/d57/d72/d79/db7/c110 80 1 2024-03-16T09:01:24.225 INFO:tasks.workunit.client.1.smithi019.stdout:5/875: dwrite d6/dd/d47/d7a/db9/fbb [0,4194304] 0 2024-03-16T09:01:24.225 INFO:tasks.workunit.client.1.smithi019.stdout:5/876: write d6/dd/d47/d70/db2/d89/dcc/fe7 [298376,120477] 0 2024-03-16T09:01:24.225 INFO:tasks.workunit.client.1.smithi019.stdout:5/877: readlink d6/dd/d47/d7a/lbf 0 2024-03-16T09:01:24.225 INFO:tasks.workunit.client.1.smithi019.stdout:5/878: chown d6/dd/d47/d70 502086 1 2024-03-16T09:01:24.225 INFO:tasks.workunit.client.1.smithi019.stdout:5/879: dread - d6/d19/d34/fdf zero size 2024-03-16T09:01:24.225 INFO:tasks.workunit.client.1.smithi019.stdout:5/880: chown d6/dd/d47/d70/fed 1543902 1 2024-03-16T09:01:24.226 INFO:tasks.workunit.client.1.smithi019.stdout:2/928: rmdir d5/de/dab 39 2024-03-16T09:01:24.228 INFO:tasks.workunit.client.1.smithi019.stdout:9/817: getdents d1/d7/d21 0 2024-03-16T09:01:24.228 INFO:tasks.workunit.client.1.smithi019.stdout:2/929: symlink d5/d6/d2b/d2f/d35/d65/l147 0 2024-03-16T09:01:24.228 INFO:tasks.workunit.client.1.smithi019.stdout:2/930: dread - d5/d2e/d4f/fe1 zero size 2024-03-16T09:01:24.228 INFO:tasks.workunit.client.1.smithi019.stdout:2/931: write d5/d5a/d80/fb3 [1042650,69137] 0 2024-03-16T09:01:24.228 INFO:tasks.workunit.client.1.smithi019.stdout:2/932: chown d5/d6/d104/dba/dfc/c76 3814121 1 2024-03-16T09:01:24.229 INFO:tasks.workunit.client.1.smithi019.stdout:2/933: getdents d5/d6/d2b/d2f/d35/d4b/dca 0 2024-03-16T09:01:24.231 INFO:tasks.workunit.client.1.smithi019.stdout:5/881: rename d6/dd/d47/d70/f97 to d6/dd/d47/d7a/db9/d112/f126 0 2024-03-16T09:01:24.232 INFO:tasks.workunit.client.1.smithi019.stdout:4/814: dread d4/d6/d15/f67 [4194304,4194304] 0 2024-03-16T09:01:24.232 INFO:tasks.workunit.client.1.smithi019.stdout:4/815: chown d4/d6/d15/fb3 0 1 2024-03-16T09:01:24.233 INFO:tasks.workunit.client.1.smithi019.stdout:8/963: dread d6/fdb [0,4194304] 0 2024-03-16T09:01:24.234 INFO:tasks.workunit.client.1.smithi019.stdout:8/964: symlink d6/d2a/d45/d57/l149 0 2024-03-16T09:01:24.234 INFO:tasks.workunit.client.1.smithi019.stdout:8/965: write d6/d2a/d45/d57/d72/d9f/d139/f12d [503544,119566] 0 2024-03-16T09:01:24.234 INFO:tasks.workunit.client.1.smithi019.stdout:9/818: unlink d1/d7/de/d24/d44/d60/d74/cc4 0 2024-03-16T09:01:24.237 INFO:tasks.workunit.client.1.smithi019.stdout:8/966: symlink d6/d2a/d3c/dab/l14a 0 2024-03-16T09:01:24.237 INFO:tasks.workunit.client.1.smithi019.stdout:9/819: mkdir d1/d7/de/d24/d5b/d118 0 2024-03-16T09:01:24.238 INFO:tasks.workunit.client.1.smithi019.stdout:2/934: dread d5/ff [0,4194304] 0 2024-03-16T09:01:24.238 INFO:tasks.workunit.client.1.smithi019.stdout:2/935: readlink d5/d2e/l8f 0 2024-03-16T09:01:24.238 INFO:tasks.workunit.client.1.smithi019.stdout:4/816: mkdir d4/d64/d112 0 2024-03-16T09:01:24.238 INFO:tasks.workunit.client.1.smithi019.stdout:8/967: creat d6/d2a/d45/d57/f14b x:0 0 0 2024-03-16T09:01:24.239 INFO:tasks.workunit.client.1.smithi019.stdout:8/968: dread - d6/d2a/d45/d57/d72/d9f/d139/fe4 zero size 2024-03-16T09:01:24.239 INFO:tasks.workunit.client.1.smithi019.stdout:9/820: mkdir d1/d7/d86/d119 0 2024-03-16T09:01:24.240 INFO:tasks.workunit.client.1.smithi019.stdout:7/802: dwrite d9/d25/d28/d48/dfc/fae [0,4194304] 0 2024-03-16T09:01:24.240 INFO:tasks.workunit.client.0.smithi007.stdout:7/911: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/ff6 [0,4194304] 0 2024-03-16T09:01:24.243 INFO:tasks.workunit.client.1.smithi019.stdout:8/969: creat d6/d2a/d3c/dab/d30/d4a/d67/dee/d10d/f14c x:0 0 0 2024-03-16T09:01:24.243 INFO:tasks.workunit.client.1.smithi019.stdout:8/970: dread - d6/d2a/d45/d57/ff7 zero size 2024-03-16T09:01:24.243 INFO:tasks.workunit.client.1.smithi019.stdout:5/882: write d6/dd/d47/d7a/db9/fbb [3187132,124372] 0 2024-03-16T09:01:24.243 INFO:tasks.workunit.client.1.smithi019.stdout:8/971: creat d6/d2a/d45/d57/d72/d79/da5/f14d x:0 0 0 2024-03-16T09:01:24.243 INFO:tasks.workunit.client.1.smithi019.stdout:6/948: dwrite de/d26/d73/ded/fef [0,4194304] 0 2024-03-16T09:01:24.243 INFO:tasks.workunit.client.1.smithi019.stdout:6/949: write de/d1c/d9a/d9e/f11e [219334,75022] 0 2024-03-16T09:01:24.244 INFO:tasks.workunit.client.1.smithi019.stdout:6/950: write de/d26/fff [4565439,5746] 0 2024-03-16T09:01:24.244 INFO:tasks.workunit.client.0.smithi007.stdout:9/994: sync 2024-03-16T09:01:24.246 INFO:tasks.workunit.client.0.smithi007.stdout:7/912: unlink d0/d19/f26 0 2024-03-16T09:01:24.246 INFO:tasks.workunit.client.0.smithi007.stdout:7/913: truncate d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f12c 921224 0 2024-03-16T09:01:24.248 INFO:tasks.workunit.client.1.smithi019.stdout:5/883: mknod d6/d1f/d77/c127 0 2024-03-16T09:01:24.248 INFO:tasks.workunit.client.1.smithi019.stdout:2/936: dread d5/d6/d104/df7/d6f/d87/f13e [0,4194304] 0 2024-03-16T09:01:24.249 INFO:tasks.workunit.client.1.smithi019.stdout:6/951: truncate de/f15 1906785 0 2024-03-16T09:01:24.249 INFO:tasks.workunit.client.1.smithi019.stdout:6/952: readlink de/d26/d32/l34 0 2024-03-16T09:01:24.249 INFO:tasks.workunit.client.1.smithi019.stdout:8/972: symlink d6/d2a/d55/l14e 0 2024-03-16T09:01:24.251 INFO:tasks.workunit.client.1.smithi019.stdout:6/953: creat de/d26/d73/f129 x:0 0 0 2024-03-16T09:01:24.253 INFO:tasks.workunit.client.1.smithi019.stdout:6/954: link de/d1c/l6c de/d26/d32/d97/da1/d105/l12a 0 2024-03-16T09:01:24.254 INFO:tasks.workunit.client.1.smithi019.stdout:2/937: mkdir d5/d6/d104/dba/d148 0 2024-03-16T09:01:24.254 INFO:tasks.workunit.client.0.smithi007.stdout:7/914: creat d0/d19/d1f/d5d/d9d/df8/d121/f142 x:0 0 0 2024-03-16T09:01:24.256 INFO:tasks.workunit.client.1.smithi019.stdout:8/973: rename d6/d2a/d55/c73 to d6/d2a/d45/d2c/de3/c14f 0 2024-03-16T09:01:24.258 INFO:tasks.workunit.client.1.smithi019.stdout:4/817: dwrite d4/d6/d15/f67 [4194304,4194304] 0 2024-03-16T09:01:24.258 INFO:tasks.workunit.client.1.smithi019.stdout:2/938: symlink d5/d6/d104/df7/d6f/l149 0 2024-03-16T09:01:24.259 INFO:tasks.workunit.client.1.smithi019.stdout:6/955: rename de/c110 to de/d26/d32/d97/dae/db1/d11d/d75/de4/c12b 0 2024-03-16T09:01:24.259 INFO:tasks.workunit.client.1.smithi019.stdout:6/956: dread - de/d26/d73/f7e zero size 2024-03-16T09:01:24.259 INFO:tasks.workunit.client.0.smithi007.stdout:9/995: write d24/f54 [7710724,105969] 0 2024-03-16T09:01:24.259 INFO:tasks.workunit.client.1.smithi019.stdout:9/821: rmdir d1/d7/de/d24/d44/d60/d74 39 2024-03-16T09:01:24.260 INFO:tasks.workunit.client.0.smithi007.stdout:7/915: unlink d0/d19/d1f/d5d/d9d/df7/d7a/f6e 0 2024-03-16T09:01:24.260 INFO:tasks.workunit.client.0.smithi007.stdout:7/916: chown d0/d19/f2b 12 1 2024-03-16T09:01:24.262 INFO:tasks.workunit.client.1.smithi019.stdout:8/974: symlink d6/d2a/d45/d57/d72/d9f/d139/de0/l150 0 2024-03-16T09:01:24.262 INFO:tasks.workunit.client.1.smithi019.stdout:8/975: readlink d6/d2a/d45/d2c/l89 0 2024-03-16T09:01:24.262 INFO:tasks.workunit.client.1.smithi019.stdout:6/957: getdents de/d26/d32/d97/dae/db1 0 2024-03-16T09:01:24.263 INFO:tasks.workunit.client.0.smithi007.stdout:9/996: mknod d24/d4c/d86/daa/d7f/d80/d87/d89/c121 0 2024-03-16T09:01:24.264 INFO:tasks.workunit.client.1.smithi019.stdout:2/939: symlink d5/d6/d104/dba/d148/l14a 0 2024-03-16T09:01:24.264 INFO:tasks.workunit.client.1.smithi019.stdout:2/940: stat d5/d6/d104/df7/d6f/da6/fc2 0 2024-03-16T09:01:24.264 INFO:tasks.workunit.client.1.smithi019.stdout:2/941: write d5/d6/d104/df7/d6f/d87/ffd [559118,122914] 0 2024-03-16T09:01:24.265 INFO:tasks.workunit.client.1.smithi019.stdout:6/958: unlink de/d26/d32/d97/d66/d72/db7/feb 0 2024-03-16T09:01:24.265 INFO:tasks.workunit.client.1.smithi019.stdout:8/976: rmdir d6/de/d51/d12a 39 2024-03-16T09:01:24.265 INFO:tasks.workunit.client.1.smithi019.stdout:8/977: chown d6/d2a/d45/f26 2673 1 2024-03-16T09:01:24.267 INFO:tasks.workunit.client.1.smithi019.stdout:8/978: rename d6/d2a/d45/d19/d74/f7f to d6/d2a/d45/d57/d72/d79/db7/f151 0 2024-03-16T09:01:24.269 INFO:tasks.workunit.client.1.smithi019.stdout:4/818: write d4/d6/d15/f67 [6096517,101467] 0 2024-03-16T09:01:24.269 INFO:tasks.workunit.client.1.smithi019.stdout:4/819: truncate d4/d1b/d49/f7d 1047719 0 2024-03-16T09:01:24.269 INFO:tasks.workunit.client.0.smithi007.stdout:7/917: dread d0/f3 [0,4194304] 0 2024-03-16T09:01:24.273 INFO:tasks.workunit.client.1.smithi019.stdout:4/820: symlink d4/d73/d103/d3d/de5/l113 0 2024-03-16T09:01:24.275 INFO:tasks.workunit.client.1.smithi019.stdout:4/821: getdents d4/d64/d9b/da9 0 2024-03-16T09:01:24.277 INFO:tasks.workunit.client.1.smithi019.stdout:4/822: unlink d4/d6c/d79/df2/f102 0 2024-03-16T09:01:24.279 INFO:tasks.workunit.client.1.smithi019.stdout:6/959: write de/f49 [2166779,26433] 0 2024-03-16T09:01:24.279 INFO:tasks.workunit.client.1.smithi019.stdout:6/960: readlink de/d1c/d9a/d9e/lc3 0 2024-03-16T09:01:24.281 INFO:tasks.workunit.client.1.smithi019.stdout:6/961: mknod de/d26/d32/d97/d66/d72/d7f/dd5/c12c 0 2024-03-16T09:01:24.282 INFO:tasks.workunit.client.1.smithi019.stdout:6/962: mknod de/d10c/c12d 0 2024-03-16T09:01:24.286 INFO:tasks.workunit.client.1.smithi019.stdout:4/823: write d4/d6/d15/d9a/fa4 [6587939,45331] 0 2024-03-16T09:01:24.288 INFO:tasks.workunit.client.1.smithi019.stdout:4/824: symlink d4/d1b/d49/d58/d80/d91/df4/l114 0 2024-03-16T09:01:24.292 INFO:tasks.workunit.client.1.smithi019.stdout:6/963: write de/d26/f104 [2742315,49455] 0 2024-03-16T09:01:24.293 INFO:tasks.workunit.client.1.smithi019.stdout:6/964: getdents de/d26/d32/dc1 0 2024-03-16T09:01:24.295 INFO:tasks.workunit.client.1.smithi019.stdout:6/965: symlink de/d26/d32/d97/da1/d105/l12e 0 2024-03-16T09:01:24.295 INFO:tasks.workunit.client.1.smithi019.stdout:6/966: write de/d26/d32/d97/d66/d72/d7f/dd5/fe9 [2842405,10623] 0 2024-03-16T09:01:24.295 INFO:tasks.workunit.client.1.smithi019.stdout:6/967: creat de/d26/d32/dce/f12f x:0 0 0 2024-03-16T09:01:24.295 INFO:tasks.workunit.client.1.smithi019.stdout:6/968: chown de/d26/d32/d97/c23 282738591 1 2024-03-16T09:01:24.296 INFO:tasks.workunit.client.1.smithi019.stdout:6/969: mknod de/d26/d32/d97/d51/c130 0 2024-03-16T09:01:24.303 INFO:tasks.workunit.client.0.smithi007.stdout:1/862: sync 2024-03-16T09:01:24.303 INFO:tasks.workunit.client.0.smithi007.stdout:6/923: sync 2024-03-16T09:01:24.303 INFO:tasks.workunit.client.0.smithi007.stdout:4/929: sync 2024-03-16T09:01:24.304 INFO:tasks.workunit.client.0.smithi007.stdout:4/930: chown d4/d99/c142 17 1 2024-03-16T09:01:24.304 INFO:tasks.workunit.client.0.smithi007.stdout:4/931: write d4/d14/d64/d68/d82/dac/dfc/faa [105090,40409] 0 2024-03-16T09:01:24.304 INFO:tasks.workunit.client.0.smithi007.stdout:4/932: truncate d4/d14/d1c/f32 4499233 0 2024-03-16T09:01:24.304 INFO:tasks.workunit.client.0.smithi007.stdout:6/924: rmdir d2/d13/d3f 39 2024-03-16T09:01:24.306 INFO:tasks.workunit.client.0.smithi007.stdout:1/863: link d9/dd/d5e/c86 d9/dd/d1b/d28/d29/d7b/d90/c115 0 2024-03-16T09:01:24.306 INFO:tasks.workunit.client.0.smithi007.stdout:6/925: symlink d2/d13/d62/l130 0 2024-03-16T09:01:24.310 INFO:tasks.workunit.client.0.smithi007.stdout:4/933: dread d4/d14/d64/d68/d82/dac/dfc/fb1 [0,4194304] 0 2024-03-16T09:01:24.310 INFO:tasks.workunit.client.0.smithi007.stdout:4/934: write d4/d14/d1c/f6a [2714662,64210] 0 2024-03-16T09:01:24.310 INFO:tasks.workunit.client.0.smithi007.stdout:4/935: dread - d4/d14/d64/d68/d82/dac/dfc/d8f/da8/fc6 zero size 2024-03-16T09:01:24.310 INFO:tasks.workunit.client.0.smithi007.stdout:4/936: dread - d4/dcb/dcd/f144 zero size 2024-03-16T09:01:24.311 INFO:tasks.workunit.client.0.smithi007.stdout:4/937: unlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/l146 0 2024-03-16T09:01:24.313 INFO:tasks.workunit.client.0.smithi007.stdout:4/938: mknod d4/d14/d64/d68/d82/dac/dfc/d8f/d13a/c14b 0 2024-03-16T09:01:24.313 INFO:tasks.workunit.client.0.smithi007.stdout:4/939: write d4/dcb/dcd/fe7 [794379,68047] 0 2024-03-16T09:01:24.313 INFO:tasks.workunit.client.0.smithi007.stdout:4/940: write d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b/f71 [5036968,6190] 0 2024-03-16T09:01:24.319 INFO:tasks.workunit.client.1.smithi019.stdout:8/979: dwrite d6/d2a/d3c/dab/d30/d4a/d67/ff0 [0,4194304] 0 2024-03-16T09:01:24.319 INFO:tasks.workunit.client.1.smithi019.stdout:9/822: dwrite d1/d7/de/d24/d44/d60/d74/f80 [0,4194304] 0 2024-03-16T09:01:24.319 INFO:tasks.workunit.client.1.smithi019.stdout:9/823: read d1/d7/de/d24/f99 [145188,46135] 0 2024-03-16T09:01:24.320 INFO:tasks.workunit.client.0.smithi007.stdout:7/918: fsync d0/d19/d1f/d5d/d9d/df8/d121/f142 0 2024-03-16T09:01:24.322 INFO:tasks.workunit.client.1.smithi019.stdout:9/824: mkdir d1/d7/d86/d119/d11a 0 2024-03-16T09:01:24.322 INFO:tasks.workunit.client.1.smithi019.stdout:9/825: chown d1/d7/de/da3/cb9 14978 1 2024-03-16T09:01:24.323 INFO:tasks.workunit.client.0.smithi007.stdout:7/919: write d0/d19/d1f/d76/f11f [897044,28857] 0 2024-03-16T09:01:24.323 INFO:tasks.workunit.client.1.smithi019.stdout:9/826: getdents d1/d7/de/d24/d44/d60/d8b/dc2 0 2024-03-16T09:01:24.325 INFO:tasks.workunit.client.0.smithi007.stdout:9/997: dwrite d24/d4c/fac [0,4194304] 0 2024-03-16T09:01:24.326 INFO:tasks.workunit.client.1.smithi019.stdout:2/942: dwrite d5/d6/d104/dba/dfc/d78/f107 [0,4194304] 0 2024-03-16T09:01:24.327 INFO:tasks.workunit.client.0.smithi007.stdout:1/864: dwrite d9/dd/d1b/f2c [0,4194304] 0 2024-03-16T09:01:24.328 INFO:tasks.workunit.client.0.smithi007.stdout:7/920: unlink d0/d19/d1f/d5d/d9d/df8/c10b 0 2024-03-16T09:01:24.328 INFO:tasks.workunit.client.1.smithi019.stdout:4/825: dwrite d4/d64/d99/fca [0,4194304] 0 2024-03-16T09:01:24.330 INFO:tasks.workunit.client.1.smithi019.stdout:6/970: dwrite de/d26/d32/dce/f12f [0,4194304] 0 2024-03-16T09:01:24.331 INFO:tasks.workunit.client.1.smithi019.stdout:4/826: truncate d4/d6/d15/fb3 3946074 0 2024-03-16T09:01:24.331 INFO:tasks.workunit.client.1.smithi019.stdout:4/827: chown d4/l65 4 1 2024-03-16T09:01:24.332 INFO:tasks.workunit.client.1.smithi019.stdout:8/980: read d6/d2a/d3c/dab/d30/d4a/d67/ff0 [3046561,1443] 0 2024-03-16T09:01:24.338 INFO:tasks.workunit.client.0.smithi007.stdout:9/998: creat d24/d4c/d86/daa/d7f/d80/d87/dda/f122 x:0 0 0 2024-03-16T09:01:24.340 INFO:tasks.workunit.client.1.smithi019.stdout:2/943: write d5/de/da2/fc1 [2572072,72040] 0 2024-03-16T09:01:24.341 INFO:tasks.workunit.client.0.smithi007.stdout:1/865: mknod d9/def/c116 0 2024-03-16T09:01:24.341 INFO:tasks.workunit.client.0.smithi007.stdout:1/866: write d9/dd/d1b/d28/ff4 [206029,110776] 0 2024-03-16T09:01:24.341 INFO:tasks.workunit.client.0.smithi007.stdout:1/867: fdatasync d9/dd/d1b/fd6 0 2024-03-16T09:01:24.342 INFO:tasks.workunit.client.1.smithi019.stdout:8/981: creat d6/de/f152 x:0 0 0 2024-03-16T09:01:24.344 INFO:tasks.workunit.client.0.smithi007.stdout:9/999: read d24/f43 [1260129,75979] 0 2024-03-16T09:01:24.349 INFO:tasks.workunit.client.0.smithi007.stdout:1/868: link d9/dd/d1b/dd8/fee d9/dd/d1b/f117 0 2024-03-16T09:01:24.349 INFO:tasks.workunit.client.0.smithi007.stdout:1/869: creat d9/d6b/f118 x:0 0 0 2024-03-16T09:01:24.350 INFO:tasks.workunit.client.1.smithi019.stdout:8/982: dread d6/d2a/d3c/dab/d30/d4a/f4e [0,4194304] 0 2024-03-16T09:01:24.353 INFO:tasks.workunit.client.1.smithi019.stdout:8/983: unlink d6/d2a/d45/d5d/faf 0 2024-03-16T09:01:24.354 INFO:tasks.workunit.client.0.smithi007.stdout:1/870: rmdir d9/dd/d1b/d28/d4c/de4 39 2024-03-16T09:01:24.356 INFO:tasks.workunit.client.1.smithi019.stdout:8/984: creat d6/d2a/d45/d57/d72/d9f/f153 x:0 0 0 2024-03-16T09:01:24.356 INFO:tasks.workunit.client.1.smithi019.stdout:8/985: dread - d6/de/d51/f101 zero size 2024-03-16T09:01:24.356 INFO:tasks.workunit.client.1.smithi019.stdout:8/986: fsync d6/de/f6c 0 2024-03-16T09:01:24.357 INFO:tasks.workunit.client.0.smithi007.stdout:1/871: creat d9/d6b/d7e/dac/dbb/d105/f119 x:0 0 0 2024-03-16T09:01:24.357 INFO:tasks.workunit.client.0.smithi007.stdout:1/872: creat d9/dd/d1b/d28/d4c/f11a x:0 0 0 2024-03-16T09:01:24.358 INFO:tasks.workunit.client.1.smithi019.stdout:8/987: truncate d6/fd4 889060 0 2024-03-16T09:01:24.358 INFO:tasks.workunit.client.1.smithi019.stdout:8/988: fsync d6/d2a/d45/d19/fd3 0 2024-03-16T09:01:24.361 INFO:tasks.workunit.client.1.smithi019.stdout:8/989: mknod d6/d2a/d68/c154 0 2024-03-16T09:01:24.364 INFO:tasks.workunit.client.1.smithi019.stdout:6/971: symlink de/d26/d32/d97/d66/ddc/d107/l131 0 2024-03-16T09:01:24.364 INFO:tasks.workunit.client.1.smithi019.stdout:8/990: getdents d6/d2a/d45/d57/d72/d79/da5 0 2024-03-16T09:01:24.364 INFO:tasks.workunit.client.1.smithi019.stdout:9/827: dwrite d1/d92/fc1 [0,4194304] 0 2024-03-16T09:01:24.366 INFO:tasks.workunit.client.1.smithi019.stdout:4/828: rmdir d4/d1b/d49/d58/d80/d8e 39 2024-03-16T09:01:24.366 INFO:tasks.workunit.client.1.smithi019.stdout:4/829: write d4/d1b/f28 [1541760,71012] 0 2024-03-16T09:01:24.367 INFO:tasks.workunit.client.1.smithi019.stdout:4/830: truncate d4/d1b/d49/d58/d80/d91/fba 496526 0 2024-03-16T09:01:24.367 INFO:tasks.workunit.client.1.smithi019.stdout:8/991: truncate d6/d2a/f88 2567624 0 2024-03-16T09:01:24.367 INFO:tasks.workunit.client.1.smithi019.stdout:8/992: stat d6/de/f76 0 2024-03-16T09:01:24.370 INFO:tasks.workunit.client.1.smithi019.stdout:9/828: rename d1/d7/de/d24/d44/d60/lb4 to d1/d7/de/d24/d44/d76/d9c/l11b 0 2024-03-16T09:01:24.372 INFO:tasks.workunit.client.1.smithi019.stdout:8/993: stat d6/d2a/d3c/dab/c5f 0 2024-03-16T09:01:24.374 INFO:tasks.workunit.client.1.smithi019.stdout:4/831: rename d4/d64/d99/f6b to d4/d6/d15/d1f/f115 0 2024-03-16T09:01:24.374 INFO:tasks.workunit.client.1.smithi019.stdout:4/832: fsync d4/d73/d103/d3d/f101 0 2024-03-16T09:01:24.375 INFO:tasks.workunit.client.1.smithi019.stdout:4/833: symlink d4/d73/d103/d3d/de5/l116 0 2024-03-16T09:01:24.375 INFO:tasks.workunit.client.1.smithi019.stdout:4/834: write d4/d6/d15/fe2 [310231,42170] 0 2024-03-16T09:01:24.378 INFO:tasks.workunit.client.0.smithi007.stdout:1/873: rename d9/dd/d1b/d28/d29 to d9/d6b/d7e/db0/dde/d11b 0 2024-03-16T09:01:24.379 INFO:tasks.workunit.client.0.smithi007.stdout:1/874: creat d9/dd/d1b/d2d/d30/f11c x:0 0 0 2024-03-16T09:01:24.380 INFO:tasks.workunit.client.0.smithi007.stdout:1/875: chown d9/dd/d1b/d2d/d30/d45/lcc 1 1 2024-03-16T09:01:24.380 INFO:tasks.workunit.client.0.smithi007.stdout:1/876: stat d9/d6b/d7e/db0/dde/d11b/d7b/d90/c115 0 2024-03-16T09:01:24.382 INFO:tasks.workunit.client.0.smithi007.stdout:1/877: creat d9/d6b/d7e/d91/f11d x:0 0 0 2024-03-16T09:01:24.382 INFO:tasks.workunit.client.1.smithi019.stdout:3/844: sync 2024-03-16T09:01:24.384 INFO:tasks.workunit.client.1.smithi019.stdout:3/845: getdents d3/d1b/d88/d91/dbe 0 2024-03-16T09:01:24.384 INFO:tasks.workunit.client.1.smithi019.stdout:8/994: dread d6/de/f6c [0,4194304] 0 2024-03-16T09:01:24.387 INFO:tasks.workunit.client.1.smithi019.stdout:8/995: getdents d6/d2a/d45/d57/d72/d79/dc1 0 2024-03-16T09:01:24.391 INFO:tasks.workunit.client.0.smithi007.stdout:7/921: write d0/d19/f4d [212123,18088] 0 2024-03-16T09:01:24.398 INFO:tasks.workunit.client.1.smithi019.stdout:2/944: dwrite d5/de/dab/fe2 [0,4194304] 0 2024-03-16T09:01:24.398 INFO:tasks.workunit.client.1.smithi019.stdout:6/972: dwrite de/d26/d32/d97/d66/f90 [0,4194304] 0 2024-03-16T09:01:24.398 INFO:tasks.workunit.client.1.smithi019.stdout:6/973: write de/d26/d32/d97/d66/d72/dd0/fac [210173,1027] 0 2024-03-16T09:01:24.398 INFO:tasks.workunit.client.1.smithi019.stdout:6/974: readlink de/d26/d32/d97/d66/d72/db7/lec 0 2024-03-16T09:01:24.398 INFO:tasks.workunit.client.1.smithi019.stdout:6/975: creat de/d26/f132 x:0 0 0 2024-03-16T09:01:24.398 INFO:tasks.workunit.client.1.smithi019.stdout:6/976: fsync de/d26/d73/f7e 0 2024-03-16T09:01:24.398 INFO:tasks.workunit.client.1.smithi019.stdout:9/829: dwrite d1/d7/d21/fab [4194304,4194304] 0 2024-03-16T09:01:24.399 INFO:tasks.workunit.client.0.smithi007.stdout:7/922: rename d0/d19/d1f/d5d/d9d/df7/d3a/c65 to d0/d19/d1f/d69/de1/c143 0 2024-03-16T09:01:24.399 INFO:tasks.workunit.client.1.smithi019.stdout:4/835: write d4/d6/d15/fb3 [3755332,26717] 0 2024-03-16T09:01:24.399 INFO:tasks.workunit.client.1.smithi019.stdout:4/836: stat d4/d64/d9b/cb6 0 2024-03-16T09:01:24.400 INFO:tasks.workunit.client.1.smithi019.stdout:8/996: rmdir d6/d2a/d3c/dab 39 2024-03-16T09:01:24.402 INFO:tasks.workunit.client.1.smithi019.stdout:3/846: dwrite d3/d89/f47 [4194304,4194304] 0 2024-03-16T09:01:24.404 INFO:tasks.workunit.client.1.smithi019.stdout:2/945: rmdir d5/d6/d104/df7/d6f/da6/d83 39 2024-03-16T09:01:24.406 INFO:tasks.workunit.client.1.smithi019.stdout:8/997: truncate d6/d2a/d3c/dab/d98/f140 3015047 0 2024-03-16T09:01:24.406 INFO:tasks.workunit.client.0.smithi007.stdout:7/923: read d0/dc/d5a/fdc [1285545,91944] 0 2024-03-16T09:01:24.407 INFO:tasks.workunit.client.0.smithi007.stdout:7/924: stat d0/d19/d1f/f4b 0 2024-03-16T09:01:24.407 INFO:tasks.workunit.client.0.smithi007.stdout:7/925: fsync d0/d19/d1f/d76/f88 0 2024-03-16T09:01:24.407 INFO:tasks.workunit.client.0.smithi007.stdout:7/926: chown d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d139/d12b 439875 1 2024-03-16T09:01:24.408 INFO:tasks.workunit.client.0.smithi007.stdout:1/878: dwrite d9/dd/d1b/d2d/f6a [0,4194304] 0 2024-03-16T09:01:24.408 INFO:tasks.workunit.client.1.smithi019.stdout:6/977: truncate de/d1c/fbc 1104302 0 2024-03-16T09:01:24.408 INFO:tasks.workunit.client.1.smithi019.stdout:6/978: creat de/d26/d73/f133 x:0 0 0 2024-03-16T09:01:24.408 INFO:tasks.workunit.client.1.smithi019.stdout:6/979: chown de/d26/d32/d97/d51/d70/lc2 2 1 2024-03-16T09:01:24.409 INFO:tasks.workunit.client.1.smithi019.stdout:9/830: symlink d1/d7/d86/ddb/l11c 0 2024-03-16T09:01:24.409 INFO:tasks.workunit.client.1.smithi019.stdout:0/907: sync 2024-03-16T09:01:24.409 INFO:tasks.workunit.client.1.smithi019.stdout:0/908: fsync d7/d8/dec/def/da1/ddf/f106 0 2024-03-16T09:01:24.409 INFO:tasks.workunit.client.1.smithi019.stdout:0/909: creat d7/d8/d32/d4a/d61/d68/f125 x:0 0 0 2024-03-16T09:01:24.409 INFO:tasks.workunit.client.1.smithi019.stdout:0/910: truncate d7/f97 1520797 0 2024-03-16T09:01:24.409 INFO:tasks.workunit.client.1.smithi019.stdout:0/911: dread - d7/d8/d32/d4a/d61/d68/dea/d109/f11e zero size 2024-03-16T09:01:24.410 INFO:tasks.workunit.client.1.smithi019.stdout:2/946: symlink d5/d6/d104/df7/d6f/d87/l14b 0 2024-03-16T09:01:24.410 INFO:tasks.workunit.client.1.smithi019.stdout:4/837: rmdir d4/d6c/d79/df7 39 2024-03-16T09:01:24.410 INFO:tasks.workunit.client.1.smithi019.stdout:4/838: write d4/d6/d15/d1f/fb8 [1000308,53443] 0 2024-03-16T09:01:24.410 INFO:tasks.workunit.client.1.smithi019.stdout:3/847: mkdir d3/d1b/d40/d4a/db7/d110 0 2024-03-16T09:01:24.410 INFO:tasks.workunit.client.1.smithi019.stdout:3/848: chown d3/d8b/dae/lab 222 1 2024-03-16T09:01:24.411 INFO:tasks.workunit.client.1.smithi019.stdout:6/980: mknod de/d26/d32/d97/dae/db1/d11d/d75/c134 0 2024-03-16T09:01:24.412 INFO:tasks.workunit.client.1.smithi019.stdout:8/998: symlink d6/de/l155 0 2024-03-16T09:01:24.412 INFO:tasks.workunit.client.1.smithi019.stdout:8/999: chown d6/d2a/d45/d57/d72/d79/cda 8084306 1 2024-03-16T09:01:24.412 INFO:tasks.workunit.client.1.smithi019.stdout:9/831: mkdir d1/d117/d11d 0 2024-03-16T09:01:24.412 INFO:tasks.workunit.client.1.smithi019.stdout:9/832: fdatasync d1/d7/de/d24/d44/d60/d74/f91 0 2024-03-16T09:01:24.414 INFO:tasks.workunit.client.0.smithi007.stdout:7/927: symlink d0/dc/d5a/dba/dd9/d93/l144 0 2024-03-16T09:01:24.415 INFO:tasks.workunit.client.0.smithi007.stdout:7/928: stat d0/d19/d1f 0 2024-03-16T09:01:24.415 INFO:tasks.workunit.client.1.smithi019.stdout:1/857: sync 2024-03-16T09:01:24.415 INFO:tasks.workunit.client.1.smithi019.stdout:5/884: sync 2024-03-16T09:01:24.415 INFO:tasks.workunit.client.1.smithi019.stdout:5/885: readlink d6/d19/d34/l81 0 2024-03-16T09:01:24.415 INFO:tasks.workunit.client.1.smithi019.stdout:1/858: read d16/d1f/f2e [2548751,126420] 0 2024-03-16T09:01:24.415 INFO:tasks.workunit.client.1.smithi019.stdout:7/803: sync 2024-03-16T09:01:24.415 INFO:tasks.workunit.client.1.smithi019.stdout:7/804: chown d9/d25 7 1 2024-03-16T09:01:24.416 INFO:tasks.workunit.client.1.smithi019.stdout:3/849: write d3/d8b/dae/d5a/d75/d7a/fe4 [1582735,122039] 0 2024-03-16T09:01:24.416 INFO:tasks.workunit.client.1.smithi019.stdout:3/850: stat d3/d11/d100 0 2024-03-16T09:01:24.416 INFO:tasks.workunit.client.1.smithi019.stdout:4/839: symlink d4/d64/d99/l117 0 2024-03-16T09:01:24.418 INFO:tasks.workunit.client.1.smithi019.stdout:6/981: creat de/d26/d32/dc1/f135 x:0 0 0 2024-03-16T09:01:24.418 INFO:tasks.workunit.client.1.smithi019.stdout:6/982: fsync de/d1c/f56 0 2024-03-16T09:01:24.418 INFO:tasks.workunit.client.1.smithi019.stdout:6/983: fdatasync de/d26/d32/d97/d66/d72/f109 0 2024-03-16T09:01:24.418 INFO:tasks.workunit.client.1.smithi019.stdout:6/984: chown de/d26/d32/d97/f46 7313 1 2024-03-16T09:01:24.419 INFO:tasks.workunit.client.1.smithi019.stdout:6/985: fdatasync de/d26/d32/d97/d66/f90 0 2024-03-16T09:01:24.420 INFO:tasks.workunit.client.1.smithi019.stdout:1/859: write d16/d31/d76/d92/ffa [330325,61919] 0 2024-03-16T09:01:24.420 INFO:tasks.workunit.client.1.smithi019.stdout:1/860: chown d16/d31/d71/d89/c102 22960878 1 2024-03-16T09:01:24.420 INFO:tasks.workunit.client.1.smithi019.stdout:1/861: truncate d16/d31/d71/d89/ff2 426285 0 2024-03-16T09:01:24.420 INFO:tasks.workunit.client.0.smithi007.stdout:7/929: mkdir d0/d145 0 2024-03-16T09:01:24.420 INFO:tasks.workunit.client.0.smithi007.stdout:7/930: write d0/d19/d1f/d69/deb/ffa [362548,45197] 0 2024-03-16T09:01:24.420 INFO:tasks.workunit.client.0.smithi007.stdout:7/931: write d0/d19/d1f/d5d/d9d/df7/d7a/fd4 [768778,96057] 0 2024-03-16T09:01:24.420 INFO:tasks.workunit.client.0.smithi007.stdout:7/932: write d0/d19/d1f/d5d/d9d/df7/d3a/f96 [3975356,87723] 0 2024-03-16T09:01:24.420 INFO:tasks.workunit.client.0.smithi007.stdout:7/933: chown d0/dc/d38/d4e/fd5 940634 1 2024-03-16T09:01:24.421 INFO:tasks.workunit.client.1.smithi019.stdout:5/886: symlink d6/dd/d47/d70/db2/l128 0 2024-03-16T09:01:24.422 INFO:tasks.workunit.client.1.smithi019.stdout:5/887: fsync d6/dd/f67 0 2024-03-16T09:01:24.423 INFO:tasks.workunit.client.0.smithi007.stdout:7/934: mkdir d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d139/d12b/d146 0 2024-03-16T09:01:24.424 INFO:tasks.workunit.client.1.smithi019.stdout:0/912: getdents d7/d8/d32/d66/d6d/d8c/db5 0 2024-03-16T09:01:24.424 INFO:tasks.workunit.client.1.smithi019.stdout:0/913: read d7/d8/dec/def/f44 [3792782,2344] 0 2024-03-16T09:01:24.424 INFO:tasks.workunit.client.1.smithi019.stdout:0/914: fsync d7/d8/dec/def/da1/fff 0 2024-03-16T09:01:24.426 INFO:tasks.workunit.client.1.smithi019.stdout:4/840: mknod d4/d73/d103/d3d/c118 0 2024-03-16T09:01:24.426 INFO:tasks.workunit.client.1.smithi019.stdout:4/841: write d4/d6/d15/da2/fae [4685689,116517] 0 2024-03-16T09:01:24.427 INFO:tasks.workunit.client.1.smithi019.stdout:4/842: dread - d4/d1b/d49/d6e/f100 zero size 2024-03-16T09:01:24.427 INFO:tasks.workunit.client.0.smithi007.stdout:7/935: creat d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d134/f147 x:0 0 0 2024-03-16T09:01:24.427 INFO:tasks.workunit.client.1.smithi019.stdout:9/833: getdents d1/d7/de/d24/d5b 0 2024-03-16T09:01:24.429 INFO:tasks.workunit.client.0.smithi007.stdout:7/936: dread d0/d19/d1f/d69/fc6 [0,4194304] 0 2024-03-16T09:01:24.432 INFO:tasks.workunit.client.1.smithi019.stdout:3/851: rename d3/d1b/d88/d91 to d3/d8b/dae/d3f/d111 0 2024-03-16T09:01:24.433 INFO:tasks.workunit.client.0.smithi007.stdout:7/937: truncate d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d139/fe2 498304 0 2024-03-16T09:01:24.435 INFO:tasks.workunit.client.1.smithi019.stdout:3/852: rename d3/d8b/dae/d5a/cdc to d3/d8b/c112 0 2024-03-16T09:01:24.437 INFO:tasks.workunit.client.1.smithi019.stdout:6/986: rename de/d26/d32/d97/d66/d72/d7f/l8f to de/d26/d32/dce/l136 0 2024-03-16T09:01:24.437 INFO:tasks.workunit.client.1.smithi019.stdout:3/853: symlink d3/d8b/dae/d3f/d111/l113 0 2024-03-16T09:01:24.439 INFO:tasks.workunit.client.1.smithi019.stdout:0/915: rename d7/f46 to d7/d8/dec/def/d92/dc6/f126 0 2024-03-16T09:01:24.439 INFO:tasks.workunit.client.1.smithi019.stdout:0/916: fdatasync d7/d8/d32/d4a/f4d 0 2024-03-16T09:01:24.439 INFO:tasks.workunit.client.1.smithi019.stdout:0/917: stat d7/l51 0 2024-03-16T09:01:24.440 INFO:tasks.workunit.client.1.smithi019.stdout:0/918: read d7/d1d/d63/d6e/fc9 [27116,45387] 0 2024-03-16T09:01:24.440 INFO:tasks.workunit.client.1.smithi019.stdout:0/919: dread - d7/d8/d32/df5/f100 zero size 2024-03-16T09:01:24.440 INFO:tasks.workunit.client.1.smithi019.stdout:0/920: read - d7/d8/d32/d4a/d61/d68/dea/d109/f11e zero size 2024-03-16T09:01:24.440 INFO:tasks.workunit.client.1.smithi019.stdout:6/987: symlink de/d26/d32/d97/d66/d72/d7f/l137 0 2024-03-16T09:01:24.441 INFO:tasks.workunit.client.1.smithi019.stdout:3/854: mknod d3/d8b/dae/d5a/d75/d7a/dcd/c114 0 2024-03-16T09:01:24.441 INFO:tasks.workunit.client.1.smithi019.stdout:0/921: mkdir d7/d1d/d63/d89/d127 0 2024-03-16T09:01:24.442 INFO:tasks.workunit.client.1.smithi019.stdout:0/922: creat d7/f128 x:0 0 0 2024-03-16T09:01:24.443 INFO:tasks.workunit.client.1.smithi019.stdout:0/923: creat d7/d8/dec/def/d48/d10c/f129 x:0 0 0 2024-03-16T09:01:24.446 INFO:tasks.workunit.client.1.smithi019.stdout:0/924: rename d7/d1d/d63/d6e/f93 to d7/d8/dec/def/da1/da7/f12a 0 2024-03-16T09:01:24.446 INFO:tasks.workunit.client.1.smithi019.stdout:0/925: write d7/d8/d32/d4a/d61/f27 [268134,101150] 0 2024-03-16T09:01:24.446 INFO:tasks.workunit.client.0.smithi007.stdout:7/938: dread d0/d19/d1f/d5d/d9d/df7/f7f [0,4194304] 0 2024-03-16T09:01:24.447 INFO:tasks.workunit.client.0.smithi007.stdout:7/939: getdents d0/d19/d1f/d5d/d9d/d132 0 2024-03-16T09:01:24.447 INFO:tasks.workunit.client.1.smithi019.stdout:9/834: dread d1/d7/d67/f82 [0,4194304] 0 2024-03-16T09:01:24.447 INFO:tasks.workunit.client.1.smithi019.stdout:9/835: fdatasync d1/d7/de/d24/d44/d60/d74/d101/f10e 0 2024-03-16T09:01:24.447 INFO:tasks.workunit.client.0.smithi007.stdout:7/940: symlink d0/dc/d5a/dba/l148 0 2024-03-16T09:01:24.449 INFO:tasks.workunit.client.1.smithi019.stdout:0/926: dread d7/d1d/d63/fee [0,4194304] 0 2024-03-16T09:01:24.449 INFO:tasks.workunit.client.1.smithi019.stdout:0/927: stat d7/d8/dec/def/d48/d10c 0 2024-03-16T09:01:24.449 INFO:tasks.workunit.client.1.smithi019.stdout:0/928: truncate d7/d8/dec/def/f34 4290743 0 2024-03-16T09:01:24.449 INFO:tasks.workunit.client.1.smithi019.stdout:0/929: fsync d7/d1d/d2e/fc5 0 2024-03-16T09:01:24.450 INFO:tasks.workunit.client.1.smithi019.stdout:0/930: truncate d7/d8/d32/d66/d6d/d7c/fcd 60693 0 2024-03-16T09:01:24.454 INFO:tasks.workunit.client.1.smithi019.stdout:6/988: rmdir de/d26/d32/dc1 39 2024-03-16T09:01:24.455 INFO:tasks.workunit.client.1.smithi019.stdout:3/855: dread d3/d11/f1f [4194304,4194304] 0 2024-03-16T09:01:24.456 INFO:tasks.workunit.client.1.smithi019.stdout:3/856: symlink d3/d1b/dc7/l115 0 2024-03-16T09:01:24.463 INFO:tasks.workunit.client.1.smithi019.stdout:0/931: read d7/d8/d32/d66/d6d/d8c/db8/f8f [2630833,24578] 0 2024-03-16T09:01:24.463 INFO:tasks.workunit.client.1.smithi019.stdout:0/932: chown d7/d8/dec/def/f111 722271003 1 2024-03-16T09:01:24.463 INFO:tasks.workunit.client.1.smithi019.stdout:0/933: fdatasync d7/f6a 0 2024-03-16T09:01:24.464 INFO:tasks.workunit.client.1.smithi019.stdout:0/934: mkdir d7/d1d/d63/d6e/d12b 0 2024-03-16T09:01:24.464 INFO:tasks.workunit.client.1.smithi019.stdout:0/935: getdents d7/d8/d32/d66/d6d/d7c/d115 0 2024-03-16T09:01:24.465 INFO:tasks.workunit.client.1.smithi019.stdout:0/936: symlink d7/d8/dec/d108/d107/d112/l12c 0 2024-03-16T09:01:24.466 INFO:tasks.workunit.client.1.smithi019.stdout:0/937: creat d7/d8/d32/f12d x:0 0 0 2024-03-16T09:01:24.466 INFO:tasks.workunit.client.1.smithi019.stdout:0/938: stat d7/d8/d32/d66/d6d/d8c/db8/d82/ff0 0 2024-03-16T09:01:24.467 INFO:tasks.workunit.client.1.smithi019.stdout:0/939: truncate d7/d8/f23 262187 0 2024-03-16T09:01:24.467 INFO:tasks.workunit.client.1.smithi019.stdout:0/940: chown d7/d8/dec/def/d92/da6/cac 4933364 1 2024-03-16T09:01:24.467 INFO:tasks.workunit.client.1.smithi019.stdout:6/989: dread de/d26/f39 [0,4194304] 0 2024-03-16T09:01:24.467 INFO:tasks.workunit.client.1.smithi019.stdout:6/990: fdatasync de/d26/d32/dc1/fca 0 2024-03-16T09:01:24.468 INFO:tasks.workunit.client.1.smithi019.stdout:6/991: truncate de/d26/d32/d97/d66/ddc/ffd 428922 0 2024-03-16T09:01:24.468 INFO:tasks.workunit.client.1.smithi019.stdout:3/857: write d3/d1b/d40/dc3/fea [2016334,33893] 0 2024-03-16T09:01:24.469 INFO:tasks.workunit.client.1.smithi019.stdout:6/992: rename de/d1c/f56 to de/d26/d32/d97/d66/da4/f138 0 2024-03-16T09:01:24.470 INFO:tasks.workunit.client.1.smithi019.stdout:6/993: write de/d26/d32/d97/d66/d72/f109 [2527391,46016] 0 2024-03-16T09:01:24.470 INFO:tasks.workunit.client.1.smithi019.stdout:7/805: dwrite d9/f74 [0,4194304] 0 2024-03-16T09:01:24.470 INFO:tasks.workunit.client.1.smithi019.stdout:1/862: dwrite d16/d31/d76/f99 [0,4194304] 0 2024-03-16T09:01:24.474 INFO:tasks.workunit.client.1.smithi019.stdout:6/994: creat de/d26/d32/d97/d66/ddc/f139 x:0 0 0 2024-03-16T09:01:24.475 INFO:tasks.workunit.client.1.smithi019.stdout:4/843: dwrite d4/d73/d103/fdb [0,4194304] 0 2024-03-16T09:01:24.475 INFO:tasks.workunit.client.1.smithi019.stdout:4/844: truncate d4/d6/fcc 2152882 0 2024-03-16T09:01:24.475 INFO:tasks.workunit.client.1.smithi019.stdout:2/947: dwrite d5/d111/f125 [0,4194304] 0 2024-03-16T09:01:24.476 INFO:tasks.workunit.client.1.smithi019.stdout:9/836: dwrite d1/d7/de/d24/d44/d76/d9c/dcc/fd7 [0,4194304] 0 2024-03-16T09:01:24.476 INFO:tasks.workunit.client.1.smithi019.stdout:7/806: unlink d9/db/d12/d5b/f65 0 2024-03-16T09:01:24.476 INFO:tasks.workunit.client.1.smithi019.stdout:7/807: fsync d9/d25/d28/d48/dfc/d13/d18/d30/d75/da4/fc2 0 2024-03-16T09:01:24.476 INFO:tasks.workunit.client.1.smithi019.stdout:7/808: fsync d9/d25/d28/d48/dfc/f59 0 2024-03-16T09:01:24.478 INFO:tasks.workunit.client.0.smithi007.stdout:7/941: rmdir d0/d19/d1f/d5d/d9d/df7/d7a 39 2024-03-16T09:01:24.481 INFO:tasks.workunit.client.1.smithi019.stdout:1/863: dread d16/f5d [0,4194304] 0 2024-03-16T09:01:24.481 INFO:tasks.workunit.client.1.smithi019.stdout:1/864: fdatasync d16/d1f/d4f/f60 0 2024-03-16T09:01:24.481 INFO:tasks.workunit.client.1.smithi019.stdout:5/888: dwrite d6/dd/daf/fbe [0,4194304] 0 2024-03-16T09:01:24.482 INFO:tasks.workunit.client.1.smithi019.stdout:6/995: symlink de/d26/d32/d97/dae/l13a 0 2024-03-16T09:01:24.483 INFO:tasks.workunit.client.1.smithi019.stdout:2/948: mkdir d5/d6/d104/dba/dfc/d74/d14c 0 2024-03-16T09:01:24.485 INFO:tasks.workunit.client.1.smithi019.stdout:2/949: mkdir d5/d6/d104/df7/d6f/da6/d83/d14d 0 2024-03-16T09:01:24.485 INFO:tasks.workunit.client.1.smithi019.stdout:2/950: truncate d5/d6/d104/df7/d6f/da6/fc0 599498 0 2024-03-16T09:01:24.485 INFO:tasks.workunit.client.1.smithi019.stdout:6/996: dread de/d26/d32/f9c [0,4194304] 0 2024-03-16T09:01:24.487 INFO:tasks.workunit.client.1.smithi019.stdout:2/951: creat d5/d6/d104/dba/dfc/d78/f14e x:0 0 0 2024-03-16T09:01:24.487 INFO:tasks.workunit.client.1.smithi019.stdout:0/941: dwrite d7/d8/d32/d66/d6d/d8c/db8/f35 [4194304,4194304] 0 2024-03-16T09:01:24.487 INFO:tasks.workunit.client.1.smithi019.stdout:0/942: readlink d7/l96 0 2024-03-16T09:01:24.487 INFO:tasks.workunit.client.1.smithi019.stdout:0/943: write d7/d8/d32/d4a/d61/d68/f125 [116237,71669] 0 2024-03-16T09:01:24.488 INFO:tasks.workunit.client.1.smithi019.stdout:2/952: mknod d5/d6/d104/dba/dfc/d78/c14f 0 2024-03-16T09:01:24.489 INFO:tasks.workunit.client.1.smithi019.stdout:9/837: truncate d1/d7/de/d24/d5b/f6e 1082418 0 2024-03-16T09:01:24.490 INFO:tasks.workunit.client.1.smithi019.stdout:9/838: readlink d1/d7/d67/d77/l10c 0 2024-03-16T09:01:24.490 INFO:tasks.workunit.client.1.smithi019.stdout:9/839: chown d1/d7/d86/f10a 36 1 2024-03-16T09:01:24.493 INFO:tasks.workunit.client.1.smithi019.stdout:1/865: creat d16/f108 x:0 0 0 2024-03-16T09:01:24.494 INFO:tasks.workunit.client.1.smithi019.stdout:5/889: creat d6/dd/d47/dd0/f129 x:0 0 0 2024-03-16T09:01:24.496 INFO:tasks.workunit.client.1.smithi019.stdout:4/845: mknod d4/d6/d15/d1f/c119 0 2024-03-16T09:01:24.497 INFO:tasks.workunit.client.0.smithi007.stdout:7/942: dwrite d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d134/f147 [0,4194304] 0 2024-03-16T09:01:24.497 INFO:tasks.workunit.client.1.smithi019.stdout:6/997: mkdir de/d26/d13b 0 2024-03-16T09:01:24.499 INFO:tasks.workunit.client.0.smithi007.stdout:7/943: rmdir d0/dc/d38/d4e 39 2024-03-16T09:01:24.500 INFO:tasks.workunit.client.1.smithi019.stdout:0/944: truncate d7/d8/dec/def/d48/d10c/f10e 98831 0 2024-03-16T09:01:24.505 INFO:tasks.workunit.client.1.smithi019.stdout:2/953: dread d5/de/dab/fe2 [0,4194304] 0 2024-03-16T09:01:24.505 INFO:tasks.workunit.client.1.smithi019.stdout:9/840: dread d1/d7/de/d24/d44/d60/d74/f80 [0,4194304] 0 2024-03-16T09:01:24.506 INFO:tasks.workunit.client.1.smithi019.stdout:1/866: symlink d16/dcb/daf/db2/d78/l109 0 2024-03-16T09:01:24.507 INFO:tasks.workunit.client.1.smithi019.stdout:2/954: mknod d5/d6/d104/dba/de8/c150 0 2024-03-16T09:01:24.509 INFO:tasks.workunit.client.1.smithi019.stdout:4/846: creat d4/d1b/d49/d58/d80/f11a x:0 0 0 2024-03-16T09:01:24.509 INFO:tasks.workunit.client.1.smithi019.stdout:4/847: fsync d4/da3/f10b 0 2024-03-16T09:01:24.509 INFO:tasks.workunit.client.1.smithi019.stdout:1/867: dread d16/d1f/d43/f100 [0,4194304] 0 2024-03-16T09:01:24.511 INFO:tasks.workunit.client.1.smithi019.stdout:9/841: mkdir d1/d7/de/d24/d44/d76/d9c/dcf/d11e 0 2024-03-16T09:01:24.511 INFO:tasks.workunit.client.1.smithi019.stdout:9/842: chown d1/d7/d21/f7b 0 1 2024-03-16T09:01:24.512 INFO:tasks.workunit.client.1.smithi019.stdout:2/955: creat d5/d6/d104/df7/d6f/da6/d83/d14d/f151 x:0 0 0 2024-03-16T09:01:24.512 INFO:tasks.workunit.client.1.smithi019.stdout:2/956: dread - d5/d5a/de6/f130 zero size 2024-03-16T09:01:24.512 INFO:tasks.workunit.client.1.smithi019.stdout:2/957: chown d5/d6/d2b/d2f/d35/d4b/dca/l113 76083 1 2024-03-16T09:01:24.512 INFO:tasks.workunit.client.1.smithi019.stdout:2/958: dread - d5/d5a/d80/f118 zero size 2024-03-16T09:01:24.513 INFO:tasks.workunit.client.1.smithi019.stdout:9/843: dread d1/d7/f3e [0,4194304] 0 2024-03-16T09:01:24.514 INFO:tasks.workunit.client.1.smithi019.stdout:9/844: write d1/d7/de/d24/d44/d60/d74/fad [1409215,94053] 0 2024-03-16T09:01:24.514 INFO:tasks.workunit.client.1.smithi019.stdout:0/945: dread d7/d1d/d63/d89/d9e/fc7 [0,4194304] 0 2024-03-16T09:01:24.517 INFO:tasks.workunit.client.1.smithi019.stdout:5/890: dread d6/dd/d47/d70/db2/d66/f48 [0,4194304] 0 2024-03-16T09:01:24.519 INFO:tasks.workunit.client.1.smithi019.stdout:2/959: dread d5/d2e/f75 [0,4194304] 0 2024-03-16T09:01:24.519 INFO:tasks.workunit.client.1.smithi019.stdout:2/960: readlink d5/d6/d2b/d53/le4 0 2024-03-16T09:01:24.521 INFO:tasks.workunit.client.1.smithi019.stdout:5/891: creat d6/d1f/d77/d10a/f12a x:0 0 0 2024-03-16T09:01:24.522 INFO:tasks.workunit.client.1.smithi019.stdout:2/961: link d5/d6/d104/df7/d6f/da6/fc0 d5/d6/d104/df7/d10a/f152 0 2024-03-16T09:01:24.523 INFO:tasks.workunit.client.0.smithi007.stdout:7/944: dwrite d0/d19/d1f/d5d/d9d/df7/d3a/daf/fdb [0,4194304] 0 2024-03-16T09:01:24.524 INFO:tasks.workunit.client.1.smithi019.stdout:2/962: mknod d5/d6/d116/c153 0 2024-03-16T09:01:24.524 INFO:tasks.workunit.client.1.smithi019.stdout:5/892: creat d6/f12b x:0 0 0 2024-03-16T09:01:24.525 INFO:tasks.workunit.client.1.smithi019.stdout:7/809: truncate d9/f74 2735177 0 2024-03-16T09:01:24.525 INFO:tasks.workunit.client.1.smithi019.stdout:2/963: truncate d5/d6/f28 1829635 0 2024-03-16T09:01:24.526 INFO:tasks.workunit.client.1.smithi019.stdout:9/845: dread d1/d7/de/d24/d44/d76/f5f [0,4194304] 0 2024-03-16T09:01:24.527 INFO:tasks.workunit.client.1.smithi019.stdout:6/998: dwrite de/d1c/d9a/d9e/fb5 [0,4194304] 0 2024-03-16T09:01:24.527 INFO:tasks.workunit.client.1.smithi019.stdout:6/999: chown de/d26/d32/d97/d66/d72/db7/lc7 27445972 1 2024-03-16T09:01:24.528 INFO:tasks.workunit.client.1.smithi019.stdout:9/846: creat d1/d7/f11f x:0 0 0 2024-03-16T09:01:24.529 INFO:tasks.workunit.client.0.smithi007.stdout:7/945: rmdir d0/d19/d1f/d5d/d9d/df8/d121 39 2024-03-16T09:01:24.530 INFO:tasks.workunit.client.1.smithi019.stdout:4/848: read d4/d73/d103/d4f/da0/fc6 [1940901,85540] 0 2024-03-16T09:01:24.531 INFO:tasks.workunit.client.0.smithi007.stdout:7/946: mknod d0/d19/d1f/d5d/d9d/df7/d7a/c149 0 2024-03-16T09:01:24.532 INFO:tasks.workunit.client.1.smithi019.stdout:4/849: creat d4/d73/d103/d3d/f11b x:0 0 0 2024-03-16T09:01:24.532 INFO:tasks.workunit.client.1.smithi019.stdout:4/850: getdents d4/d73/d8a/dd6/d10c 0 2024-03-16T09:01:24.532 INFO:tasks.workunit.client.1.smithi019.stdout:4/851: readlink d4/d1b/lff 0 2024-03-16T09:01:24.532 INFO:tasks.workunit.client.0.smithi007.stdout:7/947: mkdir d0/dc/d5a/dba/dd9/d14a 0 2024-03-16T09:01:24.533 INFO:tasks.workunit.client.0.smithi007.stdout:7/948: chown d0/d19/d1f/d5d/d9d/df7/d3a/daf/dc2 44507 1 2024-03-16T09:01:24.535 INFO:tasks.workunit.client.0.smithi007.stdout:7/949: unlink d0/d19/d1f/d5d/d9d/df7/l9f 0 2024-03-16T09:01:24.536 INFO:tasks.workunit.client.1.smithi019.stdout:2/964: dread d5/d6/d2b/f1e [0,4194304] 0 2024-03-16T09:01:24.536 INFO:tasks.workunit.client.1.smithi019.stdout:1/868: dwrite d16/d31/d76/f99 [0,4194304] 0 2024-03-16T09:01:24.536 INFO:tasks.workunit.client.1.smithi019.stdout:0/946: dwrite d7/d8/dec/d108/fe8 [0,4194304] 0 2024-03-16T09:01:24.537 INFO:tasks.workunit.client.0.smithi007.stdout:7/950: mknod d0/dc/d38/d4e/d12e/c14b 0 2024-03-16T09:01:24.537 INFO:tasks.workunit.client.0.smithi007.stdout:7/951: write d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f8e [1475843,63346] 0 2024-03-16T09:01:24.537 INFO:tasks.workunit.client.0.smithi007.stdout:7/952: fdatasync d0/dc/d38/d4e/f68 0 2024-03-16T09:01:24.537 INFO:tasks.workunit.client.0.smithi007.stdout:7/953: write d0/d19/d1f/d5d/d9d/df7/d3a/f96 [1819053,51997] 0 2024-03-16T09:01:24.538 INFO:tasks.workunit.client.1.smithi019.stdout:1/869: mknod d16/d1f/c10a 0 2024-03-16T09:01:24.538 INFO:tasks.workunit.client.1.smithi019.stdout:2/965: symlink d5/d111/d128/l154 0 2024-03-16T09:01:24.538 INFO:tasks.workunit.client.0.smithi007.stdout:4/941: sync 2024-03-16T09:01:24.538 INFO:tasks.workunit.client.0.smithi007.stdout:6/926: sync 2024-03-16T09:01:24.538 INFO:tasks.workunit.client.0.smithi007.stdout:1/879: sync 2024-03-16T09:01:24.539 INFO:tasks.workunit.client.0.smithi007.stdout:1/880: chown d9/dd/d1b/d2d/d30/d45/la8 4339963 1 2024-03-16T09:01:24.539 INFO:tasks.workunit.client.0.smithi007.stdout:1/881: fdatasync d9/dd/d1b/f10f 0 2024-03-16T09:01:24.539 INFO:tasks.workunit.client.0.smithi007.stdout:7/954: write d0/dc/d38/d4e/f50 [857505,129421] 0 2024-03-16T09:01:24.541 INFO:tasks.workunit.client.1.smithi019.stdout:0/947: symlink d7/d8/d32/d4a/d61/dbf/l12e 0 2024-03-16T09:01:24.542 INFO:tasks.workunit.client.1.smithi019.stdout:2/966: rename d5/d2e/f75 to d5/d2e/d4f/f155 0 2024-03-16T09:01:24.544 INFO:tasks.workunit.client.0.smithi007.stdout:6/927: mkdir d2/d13/d62/d131 0 2024-03-16T09:01:24.548 INFO:tasks.workunit.client.0.smithi007.stdout:7/955: creat d0/d19/d1f/d5d/d9d/df7/d3a/daf/f14c x:0 0 0 2024-03-16T09:01:24.551 INFO:tasks.workunit.client.1.smithi019.stdout:2/967: link d5/d6/d2b/f18 d5/d6/d104/dba/dfc/f156 0 2024-03-16T09:01:24.551 INFO:tasks.workunit.client.1.smithi019.stdout:1/870: creat d16/d1f/d4f/df1/f10b x:0 0 0 2024-03-16T09:01:24.553 INFO:tasks.workunit.client.1.smithi019.stdout:9/847: dread d1/d7/d67/d77/fd3 [0,4194304] 0 2024-03-16T09:01:24.555 INFO:tasks.workunit.client.1.smithi019.stdout:1/871: rename d16/d1f/d4f/l63 to d16/d31/d71/d89/l10c 0 2024-03-16T09:01:24.555 INFO:tasks.workunit.client.1.smithi019.stdout:1/872: dread - d16/dcb/daf/db2/d78/fe5 zero size 2024-03-16T09:01:24.555 INFO:tasks.workunit.client.1.smithi019.stdout:4/852: dread d4/d64/d99/fb9 [0,4194304] 0 2024-03-16T09:01:24.557 INFO:tasks.workunit.client.1.smithi019.stdout:9/848: truncate d1/d7/d67/d77/fd3 2779777 0 2024-03-16T09:01:24.557 INFO:tasks.workunit.client.1.smithi019.stdout:2/968: mknod d5/d6/d104/dba/dfc/d146/c157 0 2024-03-16T09:01:24.558 INFO:tasks.workunit.client.1.smithi019.stdout:1/873: mkdir d16/d1f/d3c/d79/db4/dee/d10d 0 2024-03-16T09:01:24.559 INFO:tasks.workunit.client.1.smithi019.stdout:4/853: creat d4/d6/f11c x:0 0 0 2024-03-16T09:01:24.562 INFO:tasks.workunit.client.1.smithi019.stdout:9/849: mkdir d1/d7/de/d24/d5b/dce/df8/df3/d120 0 2024-03-16T09:01:24.562 INFO:tasks.workunit.client.1.smithi019.stdout:9/850: chown d1/d7/d67/c6f 11617 1 2024-03-16T09:01:24.569 INFO:tasks.workunit.client.1.smithi019.stdout:0/948: truncate d7/d8/dec/def/f34 1214206 0 2024-03-16T09:01:24.572 INFO:tasks.workunit.client.1.smithi019.stdout:0/949: link d7/d8/d32/d66/d6d/d8c/db8/l4f d7/d8/dec/d108/d107/l12f 0 2024-03-16T09:01:24.572 INFO:tasks.workunit.client.1.smithi019.stdout:2/969: write d5/d2e/f9a [206682,38775] 0 2024-03-16T09:01:24.573 INFO:tasks.workunit.client.1.smithi019.stdout:2/970: creat d5/d6/d104/df7/f158 x:0 0 0 2024-03-16T09:01:24.575 INFO:tasks.workunit.client.1.smithi019.stdout:7/810: dwrite d9/d25/d28/d48/dfc/d13/d18/fac [8388608,4194304] 0 2024-03-16T09:01:24.575 INFO:tasks.workunit.client.1.smithi019.stdout:4/854: write d4/d6/d15/f2e [4170731,93329] 0 2024-03-16T09:01:24.577 INFO:tasks.workunit.client.1.smithi019.stdout:2/971: dread d5/d6/d104/dba/dfc/d74/f106 [0,4194304] 0 2024-03-16T09:01:24.579 INFO:tasks.workunit.client.0.smithi007.stdout:1/882: dwrite d9/dd/d1b/d2d/f6a [0,4194304] 0 2024-03-16T09:01:24.579 INFO:tasks.workunit.client.0.smithi007.stdout:1/883: write d9/dd/d1b/d28/d4c/f63 [3644966,11097] 0 2024-03-16T09:01:24.579 INFO:tasks.workunit.client.1.smithi019.stdout:4/855: rename d4/d6/d15/l25 to d4/d73/d103/l11d 0 2024-03-16T09:01:24.580 INFO:tasks.workunit.client.1.smithi019.stdout:4/856: write d4/d64/d99/f7b [4447525,65270] 0 2024-03-16T09:01:24.581 INFO:tasks.workunit.client.0.smithi007.stdout:7/956: dwrite d0/dc/d5a/f74 [0,4194304] 0 2024-03-16T09:01:24.581 INFO:tasks.workunit.client.0.smithi007.stdout:1/884: unlink d9/d6b/d7e/ffe 0 2024-03-16T09:01:24.582 INFO:tasks.workunit.client.0.smithi007.stdout:6/928: dwrite d2/d13/d3f/d50/d78/d81/fef [0,4194304] 0 2024-03-16T09:01:24.582 INFO:tasks.workunit.client.0.smithi007.stdout:4/942: dwrite d4/d14/d1c/f29 [4194304,4194304] 0 2024-03-16T09:01:24.582 INFO:tasks.workunit.client.0.smithi007.stdout:6/929: creat d2/d13/d3f/d50/d78/d81/f132 x:0 0 0 2024-03-16T09:01:24.583 INFO:tasks.workunit.client.0.smithi007.stdout:6/930: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d4e/f64 4534783 0 2024-03-16T09:01:24.583 INFO:tasks.workunit.client.0.smithi007.stdout:7/957: mknod d0/d19/d1f/d69/dc7/c14d 0 2024-03-16T09:01:24.583 INFO:tasks.workunit.client.1.smithi019.stdout:9/851: dread d1/d7/d67/f82 [0,4194304] 0 2024-03-16T09:01:24.583 INFO:tasks.workunit.client.1.smithi019.stdout:4/857: creat d4/d1b/d49/d58/d80/d91/f11e x:0 0 0 2024-03-16T09:01:24.583 INFO:tasks.workunit.client.1.smithi019.stdout:4/858: fsync d4/d6/d15/d9a/fb1 0 2024-03-16T09:01:24.584 INFO:tasks.workunit.client.0.smithi007.stdout:1/885: truncate d9/d6b/d7e/db0/dde/d11b/d7b/db4/fce 201051 0 2024-03-16T09:01:24.591 INFO:tasks.workunit.client.1.smithi019.stdout:7/811: write d9/f23 [3847400,115674] 0 2024-03-16T09:01:24.594 INFO:tasks.workunit.client.0.smithi007.stdout:4/943: rename d4/d14/d64/d68/d82/dac/f114 to d4/d14/d64/d9f/f14c 0 2024-03-16T09:01:24.596 INFO:tasks.workunit.client.0.smithi007.stdout:7/958: dread d0/d19/d1f/d5d/f67 [0,4194304] 0 2024-03-16T09:01:24.599 INFO:tasks.workunit.client.0.smithi007.stdout:6/931: link d2/d13/d4b/f55 d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/ddc/f133 0 2024-03-16T09:01:24.599 INFO:tasks.workunit.client.0.smithi007.stdout:6/932: dread - d2/d10/d3d/d6d/dbf/f112 zero size 2024-03-16T09:01:24.599 INFO:tasks.workunit.client.0.smithi007.stdout:6/933: write d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c/f115 [109304,29253] 0 2024-03-16T09:01:24.601 INFO:tasks.workunit.client.1.smithi019.stdout:2/972: dwrite d5/de/f55 [0,4194304] 0 2024-03-16T09:01:24.602 INFO:tasks.workunit.client.0.smithi007.stdout:1/886: mknod d9/d6b/d7e/db0/dde/d11b/d7b/dba/dda/c11e 0 2024-03-16T09:01:24.602 INFO:tasks.workunit.client.1.smithi019.stdout:1/874: dwrite d16/d1f/d43/f47 [0,4194304] 0 2024-03-16T09:01:24.604 INFO:tasks.workunit.client.0.smithi007.stdout:4/944: creat d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/d127/f14d x:0 0 0 2024-03-16T09:01:24.607 INFO:tasks.workunit.client.0.smithi007.stdout:7/959: link d0/d19/d1f/d5d/d9d/df7/d7a/dc9/le5 d0/d19/d1f/d5d/d9d/df7/d3a/daf/l14e 0 2024-03-16T09:01:24.608 INFO:tasks.workunit.client.1.smithi019.stdout:1/875: rename d16/d1f/d3c/d79 to d16/d1f/d73/ddf/d10e 0 2024-03-16T09:01:24.608 INFO:tasks.workunit.client.1.smithi019.stdout:1/876: write d16/d31/d76/d92/ffa [1013128,94618] 0 2024-03-16T09:01:24.610 INFO:tasks.workunit.client.1.smithi019.stdout:2/973: creat d5/d5a/f159 x:0 0 0 2024-03-16T09:01:24.611 INFO:tasks.workunit.client.1.smithi019.stdout:1/877: getdents d16/d31/d71/d89 0 2024-03-16T09:01:24.611 INFO:tasks.workunit.client.1.smithi019.stdout:1/878: chown f14 144732 1 2024-03-16T09:01:24.611 INFO:tasks.workunit.client.1.smithi019.stdout:1/879: fsync d16/d1f/f41 0 2024-03-16T09:01:24.611 INFO:tasks.workunit.client.1.smithi019.stdout:1/880: write d16/d1f/d3c/feb [359760,42980] 0 2024-03-16T09:01:24.612 INFO:tasks.workunit.client.0.smithi007.stdout:6/934: mknod d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/c134 0 2024-03-16T09:01:24.612 INFO:tasks.workunit.client.0.smithi007.stdout:6/935: fsync d2/d13/d62/d91/f95 0 2024-03-16T09:01:24.613 INFO:tasks.workunit.client.0.smithi007.stdout:4/945: truncate d4/f40 3998269 0 2024-03-16T09:01:24.613 INFO:tasks.workunit.client.0.smithi007.stdout:4/946: creat d4/dcb/dcd/f14e x:0 0 0 2024-03-16T09:01:24.614 INFO:tasks.workunit.client.1.smithi019.stdout:2/974: truncate d5/de/da2/fc6 653841 0 2024-03-16T09:01:24.616 INFO:tasks.workunit.client.0.smithi007.stdout:1/887: dread d9/dd/f14 [0,4194304] 0 2024-03-16T09:01:24.616 INFO:tasks.workunit.client.1.smithi019.stdout:7/812: dwrite d9/db/d12/f73 [0,4194304] 0 2024-03-16T09:01:24.618 INFO:tasks.workunit.client.0.smithi007.stdout:7/960: unlink d0/d19/lae 0 2024-03-16T09:01:24.618 INFO:tasks.workunit.client.0.smithi007.stdout:6/936: getdents d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d11c 0 2024-03-16T09:01:24.618 INFO:tasks.workunit.client.0.smithi007.stdout:6/937: truncate d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/fd3 116628 0 2024-03-16T09:01:24.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/813: rmdir d9/db 39 2024-03-16T09:01:24.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/814: write d9/d25/d28/d48/dfc/d13/d9a/dab/ff5 [387351,111214] 0 2024-03-16T09:01:24.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/815: creat d9/db/f100 x:0 0 0 2024-03-16T09:01:24.623 INFO:tasks.workunit.client.1.smithi019.stdout:4/859: dwrite d4/d73/fb0 [0,4194304] 0 2024-03-16T09:01:24.623 INFO:tasks.workunit.client.1.smithi019.stdout:4/860: read - d4/d1b/d49/d6e/f100 zero size 2024-03-16T09:01:24.624 INFO:tasks.workunit.client.0.smithi007.stdout:4/947: rename d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b/f71 to d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/f14f 0 2024-03-16T09:01:24.626 INFO:tasks.workunit.client.0.smithi007.stdout:7/961: write d0/d19/d1f/f82 [497134,84890] 0 2024-03-16T09:01:24.637 INFO:tasks.workunit.client.0.smithi007.stdout:6/938: write d2/f7 [1333513,102240] 0 2024-03-16T09:01:24.638 INFO:tasks.workunit.client.0.smithi007.stdout:6/939: getdents d2/d13/d62/d122 0 2024-03-16T09:01:24.638 INFO:tasks.workunit.client.0.smithi007.stdout:6/940: stat d2/d10/d3d/d6d/dbf 0 2024-03-16T09:01:24.639 INFO:tasks.workunit.client.1.smithi019.stdout:4/861: dread d4/d6/d15/da2/fae [0,4194304] 0 2024-03-16T09:01:24.639 INFO:tasks.workunit.client.1.smithi019.stdout:4/862: fsync d4/d1b/f28 0 2024-03-16T09:01:24.639 INFO:tasks.workunit.client.1.smithi019.stdout:4/863: readlink d4/d73/d103/d4f/l5c 0 2024-03-16T09:01:24.639 INFO:tasks.workunit.client.1.smithi019.stdout:4/864: chown d4/d1b/d49/d58/d80/d91/fd9 24 1 2024-03-16T09:01:24.641 INFO:tasks.workunit.client.0.smithi007.stdout:6/941: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a 39 2024-03-16T09:01:24.647 INFO:tasks.workunit.client.0.smithi007.stdout:7/962: read d0/dc/d38/d4e/fd5 [3520481,82462] 0 2024-03-16T09:01:24.647 INFO:tasks.workunit.client.0.smithi007.stdout:7/963: dread - d0/d19/d1f/d5d/d9d/df8/d121/f142 zero size 2024-03-16T09:01:24.647 INFO:tasks.workunit.client.0.smithi007.stdout:7/964: fsync d0/dc/d5a/dba/dd9/f12d 0 2024-03-16T09:01:24.649 INFO:tasks.workunit.client.1.smithi019.stdout:2/975: dwrite d5/d6/f13f [0,4194304] 0 2024-03-16T09:01:24.651 INFO:tasks.workunit.client.1.smithi019.stdout:2/976: truncate d5/d2e/f9a 3312705 0 2024-03-16T09:01:24.651 INFO:tasks.workunit.client.1.smithi019.stdout:2/977: dread - d5/d6/d2b/f103 zero size 2024-03-16T09:01:24.651 INFO:tasks.workunit.client.1.smithi019.stdout:2/978: write d5/d6/d104/dba/f137 [577261,90754] 0 2024-03-16T09:01:24.658 INFO:tasks.workunit.client.1.smithi019.stdout:1/881: dwrite d16/d18/f23 [0,4194304] 0 2024-03-16T09:01:24.658 INFO:tasks.workunit.client.1.smithi019.stdout:1/882: fsync d16/d31/d7b/fd6 0 2024-03-16T09:01:24.659 INFO:tasks.workunit.client.1.smithi019.stdout:7/816: dwrite d9/d25/d28/d48/dfc/d13/d18/fa3 [0,4194304] 0 2024-03-16T09:01:24.659 INFO:tasks.workunit.client.1.smithi019.stdout:7/817: dread - d9/d25/d28/d48/ff7 zero size 2024-03-16T09:01:24.659 INFO:tasks.workunit.client.0.smithi007.stdout:1/888: dwrite d9/f35 [0,4194304] 0 2024-03-16T09:01:24.659 INFO:tasks.workunit.client.0.smithi007.stdout:6/942: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c/ff0 [4194304,4194304] 0 2024-03-16T09:01:24.659 INFO:tasks.workunit.client.1.smithi019.stdout:1/883: link d16/f2f d16/dcb/daf/db2/f10f 0 2024-03-16T09:01:24.661 INFO:tasks.workunit.client.0.smithi007.stdout:6/943: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/ddc/f135 x:0 0 0 2024-03-16T09:01:24.661 INFO:tasks.workunit.client.0.smithi007.stdout:6/944: write d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/fd9 [837593,51470] 0 2024-03-16T09:01:24.661 INFO:tasks.workunit.client.0.smithi007.stdout:4/948: dwrite d4/d14/d64/d68/dbf/f138 [0,4194304] 0 2024-03-16T09:01:24.661 INFO:tasks.workunit.client.1.smithi019.stdout:4/865: dwrite d4/d6/d15/f67 [4194304,4194304] 0 2024-03-16T09:01:24.661 INFO:tasks.workunit.client.1.smithi019.stdout:4/866: write d4/d73/d103/d4f/f8b [932807,95709] 0 2024-03-16T09:01:24.662 INFO:tasks.workunit.client.0.smithi007.stdout:4/949: read d4/d14/d64/fb5 [636810,62085] 0 2024-03-16T09:01:24.662 INFO:tasks.workunit.client.0.smithi007.stdout:4/950: chown d4/ff 897490 1 2024-03-16T09:01:24.662 INFO:tasks.workunit.client.0.smithi007.stdout:4/951: chown d4/d14/d1c/d5b/f5e 0 1 2024-03-16T09:01:24.663 INFO:tasks.workunit.client.0.smithi007.stdout:6/945: rename d2/d13/d3f/d50/d78/dbe/dcb/d1e/cb8 to d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/c136 0 2024-03-16T09:01:24.663 INFO:tasks.workunit.client.1.smithi019.stdout:7/818: unlink d9/d25/d28/fa6 0 2024-03-16T09:01:24.664 INFO:tasks.workunit.client.1.smithi019.stdout:1/884: creat d16/dcb/f110 x:0 0 0 2024-03-16T09:01:24.666 INFO:tasks.workunit.client.1.smithi019.stdout:7/819: mknod d9/d25/d28/d48/dfc/d13/d18/c101 0 2024-03-16T09:01:24.668 INFO:tasks.workunit.client.0.smithi007.stdout:7/965: dwrite d0/d19/d1f/d5d/d9d/df7/fcf [0,4194304] 0 2024-03-16T09:01:24.668 INFO:tasks.workunit.client.0.smithi007.stdout:7/966: chown d0/d19/d1f/d5d/d9d/df7/d3a/daf/f104 4320560 1 2024-03-16T09:01:24.675 INFO:tasks.workunit.client.1.smithi019.stdout:4/867: dread d4/d6/d15/f67 [4194304,4194304] 0 2024-03-16T09:01:24.675 INFO:tasks.workunit.client.1.smithi019.stdout:4/868: write d4/d1b/f7e [4442589,15618] 0 2024-03-16T09:01:24.675 INFO:tasks.workunit.client.0.smithi007.stdout:6/946: link d2/c3 d2/d13/d4b/d94/dea/df7/c137 0 2024-03-16T09:01:24.675 INFO:tasks.workunit.client.0.smithi007.stdout:1/889: write d9/d6b/d7e/f92 [1366601,109495] 0 2024-03-16T09:01:24.676 INFO:tasks.workunit.client.1.smithi019.stdout:7/820: write d9/db/fb1 [5610665,109581] 0 2024-03-16T09:01:24.677 INFO:tasks.workunit.client.0.smithi007.stdout:7/967: stat d0/d19/d1f/d69/cd0 0 2024-03-16T09:01:24.677 INFO:tasks.workunit.client.1.smithi019.stdout:4/869: unlink d4/d6/d15/f67 0 2024-03-16T09:01:24.677 INFO:tasks.workunit.client.1.smithi019.stdout:4/870: write d4/d6/d15/d1f/fbd [785929,99283] 0 2024-03-16T09:01:24.677 INFO:tasks.workunit.client.1.smithi019.stdout:4/871: chown d4/d9e/ce4 4604939 1 2024-03-16T09:01:24.679 INFO:tasks.workunit.client.0.smithi007.stdout:4/952: link d4/d99/fbc d4/f150 0 2024-03-16T09:01:24.679 INFO:tasks.workunit.client.0.smithi007.stdout:4/953: fsync d4/d14/d64/d68/ff3 0 2024-03-16T09:01:24.680 INFO:tasks.workunit.client.0.smithi007.stdout:6/947: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/dc2/d138 0 2024-03-16T09:01:24.683 INFO:tasks.workunit.client.0.smithi007.stdout:6/948: mknod d2/d13/d3f/d50/d78/dbe/dcb/d1e/c139 0 2024-03-16T09:01:24.685 INFO:tasks.workunit.client.0.smithi007.stdout:1/890: getdents d9/d6b/d7e/d91 0 2024-03-16T09:01:24.686 INFO:tasks.workunit.client.0.smithi007.stdout:7/968: rmdir d0/d19/d1f/d5d/d9d/df7/d3a/daf 39 2024-03-16T09:01:24.687 INFO:tasks.workunit.client.0.smithi007.stdout:7/969: read - d0/d19/d1f/d5d/d9d/df7/d73/f127 zero size 2024-03-16T09:01:24.688 INFO:tasks.workunit.client.1.smithi019.stdout:7/821: read d9/d25/d28/d48/dfc/d13/f94 [3513984,14966] 0 2024-03-16T09:01:24.691 INFO:tasks.workunit.client.1.smithi019.stdout:7/822: mkdir d9/db/d102 0 2024-03-16T09:01:24.691 INFO:tasks.workunit.client.0.smithi007.stdout:6/949: rename d2/d10/d3d/f6c to d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d11c/f13a 0 2024-03-16T09:01:24.694 INFO:tasks.workunit.client.1.smithi019.stdout:1/885: dwrite d16/d1f/fb6 [0,4194304] 0 2024-03-16T09:01:24.694 INFO:tasks.workunit.client.1.smithi019.stdout:1/886: read - d16/d1f/d73/ddf/d10e/d96/fa0 zero size 2024-03-16T09:01:24.694 INFO:tasks.workunit.client.1.smithi019.stdout:4/872: dread d4/d73/d103/d4f/da0/fc6 [0,4194304] 0 2024-03-16T09:01:24.694 INFO:tasks.workunit.client.1.smithi019.stdout:4/873: truncate d4/d73/d103/d4f/fbc 610396 0 2024-03-16T09:01:24.695 INFO:tasks.workunit.client.1.smithi019.stdout:4/874: symlink d4/d73/d8a/dd6/d10c/l11f 0 2024-03-16T09:01:24.696 INFO:tasks.workunit.client.0.smithi007.stdout:4/954: read d4/d14/d64/d68/d82/f8d [873179,54825] 0 2024-03-16T09:01:24.696 INFO:tasks.workunit.client.0.smithi007.stdout:4/955: chown d4/d14/d64/d68/d82/f113 25131153 1 2024-03-16T09:01:24.697 INFO:tasks.workunit.client.0.smithi007.stdout:4/956: dread - d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d102/d103/f12f zero size 2024-03-16T09:01:24.697 INFO:tasks.workunit.client.0.smithi007.stdout:4/957: readlink d4/d14/d64/lce 0 2024-03-16T09:01:24.697 INFO:tasks.workunit.client.0.smithi007.stdout:4/958: chown d4/d14/d64/d9f/f4c 27 1 2024-03-16T09:01:24.697 INFO:tasks.workunit.client.0.smithi007.stdout:4/959: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/fe 20750657 1 2024-03-16T09:01:24.697 INFO:tasks.workunit.client.0.smithi007.stdout:4/960: write d4/d14/d64/d68/d82/dac/dfc/d8f/ff2 [839638,72949] 0 2024-03-16T09:01:24.698 INFO:tasks.workunit.client.1.smithi019.stdout:4/875: symlink d4/d6c/d89/l120 0 2024-03-16T09:01:24.698 INFO:tasks.workunit.client.1.smithi019.stdout:4/876: stat d4/d1b/l45 0 2024-03-16T09:01:24.700 INFO:tasks.workunit.client.1.smithi019.stdout:4/877: symlink d4/d105/l121 0 2024-03-16T09:01:24.700 INFO:tasks.workunit.client.1.smithi019.stdout:4/878: rename d4/d1b/d49/d58/d80/d91/df4 to d4/d1b/d49/d58/d80/d91/df4/d122 22 2024-03-16T09:01:24.700 INFO:tasks.workunit.client.1.smithi019.stdout:4/879: fdatasync f2 0 2024-03-16T09:01:24.700 INFO:tasks.workunit.client.1.smithi019.stdout:4/880: chown d4/d73/d103/d4f/fbc 13794 1 2024-03-16T09:01:24.701 INFO:tasks.workunit.client.1.smithi019.stdout:4/881: mknod d4/d73/d103/d3d/c123 0 2024-03-16T09:01:24.702 INFO:tasks.workunit.client.1.smithi019.stdout:4/882: creat d4/d6c/d79/f124 x:0 0 0 2024-03-16T09:01:24.703 INFO:tasks.workunit.client.0.smithi007.stdout:4/961: getdents d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d102 0 2024-03-16T09:01:24.704 INFO:tasks.workunit.client.1.smithi019.stdout:4/883: unlink d4/d73/d8a/dd6/ffb 0 2024-03-16T09:01:24.706 INFO:tasks.workunit.client.1.smithi019.stdout:7/823: write d9/db/d12/f73 [2863208,9456] 0 2024-03-16T09:01:24.707 INFO:tasks.workunit.client.0.smithi007.stdout:4/962: mknod d4/dcb/d115/d141/c151 0 2024-03-16T09:01:24.708 INFO:tasks.workunit.client.1.smithi019.stdout:7/824: creat d9/d25/d28/d48/dfc/d13/d9a/f103 x:0 0 0 2024-03-16T09:01:24.708 INFO:tasks.workunit.client.1.smithi019.stdout:1/887: write d16/d1f/d73/ddf/d10e/f91 [187782,52198] 0 2024-03-16T09:01:24.708 INFO:tasks.workunit.client.0.smithi007.stdout:4/963: truncate d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/f10d 1132289 0 2024-03-16T09:01:24.710 INFO:tasks.workunit.client.1.smithi019.stdout:7/825: creat d9/db/d12/d5b/dc3/dec/f104 x:0 0 0 2024-03-16T09:01:24.712 INFO:tasks.workunit.client.0.smithi007.stdout:1/891: dwrite d9/dd/d1b/d2d/f10e [0,4194304] 0 2024-03-16T09:01:24.712 INFO:tasks.workunit.client.1.smithi019.stdout:7/826: mknod d9/d25/d28/d48/dfc/d13/d18/d30/d9e/c105 0 2024-03-16T09:01:24.715 INFO:tasks.workunit.client.1.smithi019.stdout:7/827: mknod d9/d25/d28/d48/dfc/d13/d18/d30/d3e/da5/c106 0 2024-03-16T09:01:24.718 INFO:tasks.workunit.client.1.smithi019.stdout:7/828: mknod d9/d25/d28/d48/dfc/d13/d18/d30/d9e/dd7/c107 0 2024-03-16T09:01:24.718 INFO:tasks.workunit.client.1.smithi019.stdout:7/829: chown d9/db/fa7 1065091 1 2024-03-16T09:01:24.719 INFO:tasks.workunit.client.0.smithi007.stdout:4/964: dread d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/f14f [0,4194304] 0 2024-03-16T09:01:24.719 INFO:tasks.workunit.client.0.smithi007.stdout:4/965: fsync d4/d14/d1c/d5b/f74 0 2024-03-16T09:01:24.719 INFO:tasks.workunit.client.1.smithi019.stdout:3/858: sync 2024-03-16T09:01:24.719 INFO:tasks.workunit.client.1.smithi019.stdout:7/830: mkdir d9/d25/d108 0 2024-03-16T09:01:24.720 INFO:tasks.workunit.client.1.smithi019.stdout:7/831: getdents d9/d25/d28/d48/dfc/d13/d18/d30/d75/de1 0 2024-03-16T09:01:24.722 INFO:tasks.workunit.client.1.smithi019.stdout:7/832: symlink d9/d25/d93/l109 0 2024-03-16T09:01:24.724 INFO:tasks.workunit.client.1.smithi019.stdout:4/884: dwrite d4/d1b/d49/d58/d80/d91/fd9 [4194304,4194304] 0 2024-03-16T09:01:24.724 INFO:tasks.workunit.client.1.smithi019.stdout:4/885: stat d4/d6/d15/da2/fae 0 2024-03-16T09:01:24.726 INFO:tasks.workunit.client.1.smithi019.stdout:7/833: link d9/d25/d28/d48/dfc/d13/d18/d30/d3e/lcc d9/d25/d28/d48/d8d/dba/dad/db9/l10a 0 2024-03-16T09:01:24.730 INFO:tasks.workunit.client.1.smithi019.stdout:7/834: rename d9/ceb to d9/d25/d28/d48/c10b 0 2024-03-16T09:01:24.734 INFO:tasks.workunit.client.1.smithi019.stdout:4/886: dread d4/d6/d15/d1f/fb8 [0,4194304] 0 2024-03-16T09:01:24.735 INFO:tasks.workunit.client.1.smithi019.stdout:4/887: mknod d4/d64/d9b/c125 0 2024-03-16T09:01:24.744 INFO:tasks.workunit.client.0.smithi007.stdout:1/892: dwrite f5 [4194304,4194304] 0 2024-03-16T09:01:24.744 INFO:tasks.workunit.client.0.smithi007.stdout:1/893: truncate d9/dd/d5e/d89/f10d 660123 0 2024-03-16T09:01:24.744 INFO:tasks.workunit.client.1.smithi019.stdout:3/859: dwrite d3/d1b/d88/fa5 [4194304,4194304] 0 2024-03-16T09:01:24.745 INFO:tasks.workunit.client.0.smithi007.stdout:4/966: dwrite d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d102/d103/f12f [0,4194304] 0 2024-03-16T09:01:24.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/967: read d4/d14/d64/d68/d82/dac/dfc/d8f/da8/f20 [1763978,68061] 0 2024-03-16T09:01:24.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/968: readlink d4/d14/d64/d9f/lec 0 2024-03-16T09:01:24.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/969: creat d4/d14/d64/d9f/f152 x:0 0 0 2024-03-16T09:01:24.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/970: fdatasync d4/d14/d64/d68/d82/dac/dfc/d8f/da8/f70 0 2024-03-16T09:01:24.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/971: chown d4/d14/l57 216040 1 2024-03-16T09:01:24.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/972: readlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/lb7 0 2024-03-16T09:01:24.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/973: write d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/f14f [5845322,26531] 0 2024-03-16T09:01:24.746 INFO:tasks.workunit.client.0.smithi007.stdout:4/974: readlink d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/l9e 0 2024-03-16T09:01:24.748 INFO:tasks.workunit.client.0.smithi007.stdout:4/975: dread d4/d14/d1c/d5b/f8e [0,4194304] 0 2024-03-16T09:01:24.751 INFO:tasks.workunit.client.0.smithi007.stdout:4/976: mknod d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d9d/dbb/dd2/c153 0 2024-03-16T09:01:24.752 INFO:tasks.workunit.client.1.smithi019.stdout:4/888: dwrite d4/f35 [4194304,4194304] 0 2024-03-16T09:01:24.756 INFO:tasks.workunit.client.0.smithi007.stdout:4/977: dread d4/d14/f19 [4194304,4194304] 0 2024-03-16T09:01:24.760 INFO:tasks.workunit.client.0.smithi007.stdout:4/978: link d4/d14/d64/d68/dbf/f140 d4/dcb/f154 0 2024-03-16T09:01:24.760 INFO:tasks.workunit.client.0.smithi007.stdout:4/979: chown d4/d14/d1c/d5b/f8e 31473326 1 2024-03-16T09:01:24.761 INFO:tasks.workunit.client.0.smithi007.stdout:4/980: mknod d4/d14/d64/d9f/d9a/c155 0 2024-03-16T09:01:24.761 INFO:tasks.workunit.client.1.smithi019.stdout:4/889: write d4/d64/d9b/fe8 [2731782,75619] 0 2024-03-16T09:01:24.762 INFO:tasks.workunit.client.1.smithi019.stdout:4/890: creat d4/d1b/dc1/f126 x:0 0 0 2024-03-16T09:01:24.770 INFO:tasks.workunit.client.0.smithi007.stdout:1/894: dwrite d9/dd/d1b/d58/f66 [0,4194304] 0 2024-03-16T09:01:24.770 INFO:tasks.workunit.client.0.smithi007.stdout:1/895: fsync d9/dd/d1b/d2d/f10e 0 2024-03-16T09:01:24.773 INFO:tasks.workunit.client.0.smithi007.stdout:1/896: truncate d9/dd/d1b/dd8/ff3 2530730 0 2024-03-16T09:01:24.773 INFO:tasks.workunit.client.0.smithi007.stdout:1/897: fsync d9/dd/d1b/d2d/d30/fc3 0 2024-03-16T09:01:24.774 INFO:tasks.workunit.client.0.smithi007.stdout:1/898: mknod d9/d6b/d7e/db0/dde/d11b/c11f 0 2024-03-16T09:01:24.776 INFO:tasks.workunit.client.0.smithi007.stdout:1/899: rename d9/d6b/d7e/cf6 to d9/d6b/d7e/db0/dde/d11b/d7b/dba/c120 0 2024-03-16T09:01:24.777 INFO:tasks.workunit.client.0.smithi007.stdout:1/900: symlink d9/dd/d5e/d89/l121 0 2024-03-16T09:01:24.789 INFO:tasks.workunit.client.1.smithi019.stdout:4/891: dwrite d4/d1b/d49/d58/d80/d91/fba [0,4194304] 0 2024-03-16T09:01:24.793 INFO:tasks.workunit.client.1.smithi019.stdout:4/892: dread d4/d1b/d49/f7d [0,4194304] 0 2024-03-16T09:01:24.793 INFO:tasks.workunit.client.1.smithi019.stdout:4/893: read d4/d64/d9b/fe8 [2137628,68324] 0 2024-03-16T09:01:24.793 INFO:tasks.workunit.client.1.smithi019.stdout:4/894: readlink d4/d73/d103/l11d 0 2024-03-16T09:01:24.794 INFO:tasks.workunit.client.0.smithi007.stdout:4/981: dwrite d4/d14/d64/d68/d82/dac/fe1 [0,4194304] 0 2024-03-16T09:01:24.794 INFO:tasks.workunit.client.0.smithi007.stdout:4/982: chown d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84 1692266 1 2024-03-16T09:01:24.794 INFO:tasks.workunit.client.0.smithi007.stdout:1/901: dwrite d9/f35 [4194304,4194304] 0 2024-03-16T09:01:24.794 INFO:tasks.workunit.client.1.smithi019.stdout:4/895: rename d4/d6/d15/f60 to d4/d1b/f127 0 2024-03-16T09:01:24.795 INFO:tasks.workunit.client.1.smithi019.stdout:4/896: chown d4/d6c/d79/df7 105094011 1 2024-03-16T09:01:24.795 INFO:tasks.workunit.client.1.smithi019.stdout:4/897: write d4/d73/d103/d3d/f101 [732474,128093] 0 2024-03-16T09:01:24.797 INFO:tasks.workunit.client.0.smithi007.stdout:4/983: symlink d4/d14/d1c/l156 0 2024-03-16T09:01:24.799 INFO:tasks.workunit.client.1.smithi019.stdout:4/898: rename d4/d73/d103/d3d to d4/d73/d8a/d128 0 2024-03-16T09:01:24.799 INFO:tasks.workunit.client.1.smithi019.stdout:4/899: write d4/d6/d15/d9a/fb1 [57027,12124] 0 2024-03-16T09:01:24.799 INFO:tasks.workunit.client.0.smithi007.stdout:4/984: dread d4/dcb/dcd/fe7 [0,4194304] 0 2024-03-16T09:01:24.799 INFO:tasks.workunit.client.0.smithi007.stdout:4/985: fdatasync d4/f106 0 2024-03-16T09:01:24.799 INFO:tasks.workunit.client.0.smithi007.stdout:1/902: rename d9/dd/d1b/d28/f50 to d9/dd/d1b/f122 0 2024-03-16T09:01:24.800 INFO:tasks.workunit.client.0.smithi007.stdout:1/903: fdatasync d9/dd/d5e/fd3 0 2024-03-16T09:01:24.800 INFO:tasks.workunit.client.0.smithi007.stdout:4/986: truncate d4/d14/fb2 16348 0 2024-03-16T09:01:24.800 INFO:tasks.workunit.client.0.smithi007.stdout:4/987: write d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/ffb [871626,81540] 0 2024-03-16T09:01:24.801 INFO:tasks.workunit.client.1.smithi019.stdout:4/900: unlink d4/d73/d103/d4f/cc9 0 2024-03-16T09:01:24.801 INFO:tasks.workunit.client.1.smithi019.stdout:4/901: write d4/d73/d8a/d128/f46 [4324170,24754] 0 2024-03-16T09:01:24.801 INFO:tasks.workunit.client.1.smithi019.stdout:5/893: sync 2024-03-16T09:01:24.801 INFO:tasks.workunit.client.1.smithi019.stdout:0/950: sync 2024-03-16T09:01:24.801 INFO:tasks.workunit.client.1.smithi019.stdout:0/951: readlink d7/d8/d32/d66/d6d/d8c/db8/l42 0 2024-03-16T09:01:24.801 INFO:tasks.workunit.client.1.smithi019.stdout:0/952: chown d7/cd8 1 1 2024-03-16T09:01:24.801 INFO:tasks.workunit.client.1.smithi019.stdout:5/894: creat d6/dd/d47/d70/db2/d4c/f12c x:0 0 0 2024-03-16T09:01:24.802 INFO:tasks.workunit.client.1.smithi019.stdout:5/895: truncate d6/dd/daf/fe2 640672 0 2024-03-16T09:01:24.802 INFO:tasks.workunit.client.1.smithi019.stdout:0/953: creat d7/d8/dec/d108/da3/f130 x:0 0 0 2024-03-16T09:01:24.803 INFO:tasks.workunit.client.1.smithi019.stdout:0/954: write d7/f128 [367599,4101] 0 2024-03-16T09:01:24.803 INFO:tasks.workunit.client.0.smithi007.stdout:1/904: unlink d9/dd/d1b/d28/f60 0 2024-03-16T09:01:24.804 INFO:tasks.workunit.client.1.smithi019.stdout:5/896: creat d6/dd/d47/d70/dfb/f12d x:0 0 0 2024-03-16T09:01:24.805 INFO:tasks.workunit.client.0.smithi007.stdout:4/988: mknod d4/d14/d64/d68/d82/c157 0 2024-03-16T09:01:24.806 INFO:tasks.workunit.client.1.smithi019.stdout:0/955: unlink d7/d8/d32/d4a/lbe 0 2024-03-16T09:01:24.807 INFO:tasks.workunit.client.1.smithi019.stdout:5/897: truncate d6/dd/d37/f6b 3538378 0 2024-03-16T09:01:24.809 INFO:tasks.workunit.client.0.smithi007.stdout:4/989: mkdir d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/d134/d158 0 2024-03-16T09:01:24.810 INFO:tasks.workunit.client.1.smithi019.stdout:5/898: rename d6/dd/daf/ce3 to d6/dd/daf/c12e 0 2024-03-16T09:01:24.811 INFO:tasks.workunit.client.0.smithi007.stdout:1/905: symlink d9/dd/d1b/l123 0 2024-03-16T09:01:24.811 INFO:tasks.workunit.client.0.smithi007.stdout:4/990: dread d4/dcb/fd8 [0,4194304] 0 2024-03-16T09:01:24.811 INFO:tasks.workunit.client.0.smithi007.stdout:4/991: fsync d4/d14/d64/d9f/f53 0 2024-03-16T09:01:24.813 INFO:tasks.workunit.client.0.smithi007.stdout:1/906: dread d9/dd/d5e/d89/ff1 [0,4194304] 0 2024-03-16T09:01:24.815 INFO:tasks.workunit.client.0.smithi007.stdout:1/907: unlink d9/dd/c31 0 2024-03-16T09:01:24.815 INFO:tasks.workunit.client.0.smithi007.stdout:1/908: fsync d9/d6b/d7e/db0/dde/d11b/d7b/d90/f103 0 2024-03-16T09:01:24.815 INFO:tasks.workunit.client.0.smithi007.stdout:1/909: fdatasync d9/dd/d1b/d28/d4c/feb 0 2024-03-16T09:01:24.816 INFO:tasks.workunit.client.0.smithi007.stdout:1/910: creat d9/dd/d1b/d58/f124 x:0 0 0 2024-03-16T09:01:24.817 INFO:tasks.workunit.client.1.smithi019.stdout:4/902: dwrite d4/d73/d103/d4f/f8b [0,4194304] 0 2024-03-16T09:01:24.817 INFO:tasks.workunit.client.0.smithi007.stdout:1/911: truncate d9/dd/d1b/d2d/d30/d45/fbd 3435939 0 2024-03-16T09:01:24.818 INFO:tasks.workunit.client.0.smithi007.stdout:1/912: link d9/dd/d1b/dd8/lf2 d9/dd/d5e/d89/l125 0 2024-03-16T09:01:24.821 INFO:tasks.workunit.client.1.smithi019.stdout:4/903: chown d4/d64/laf 6771 1 2024-03-16T09:01:24.821 INFO:tasks.workunit.client.1.smithi019.stdout:4/904: readlink d4/d1b/d49/l4a 0 2024-03-16T09:01:24.822 INFO:tasks.workunit.client.0.smithi007.stdout:4/992: read d4/f54 [2661920,88093] 0 2024-03-16T09:01:24.831 INFO:tasks.workunit.client.0.smithi007.stdout:4/993: rename d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d93/d127 to d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d78/d80/d84/d89/d134/d158/d159 0 2024-03-16T09:01:24.831 INFO:tasks.workunit.client.0.smithi007.stdout:4/994: read d4/d14/d64/d68/d82/dac/dfc/faa [112996,120511] 0 2024-03-16T09:01:24.831 INFO:tasks.workunit.client.1.smithi019.stdout:4/905: dread d4/d73/d103/f4e [0,4194304] 0 2024-03-16T09:01:24.835 INFO:tasks.workunit.client.0.smithi007.stdout:4/995: creat d4/d14/d64/d68/d82/dac/dfc/de8/f15a x:0 0 0 2024-03-16T09:01:24.835 INFO:tasks.workunit.client.0.smithi007.stdout:4/996: chown d4/d14/d64/d68/d82/dac/lf5 388 1 2024-03-16T09:01:24.835 INFO:tasks.workunit.client.0.smithi007.stdout:4/997: dread - d4/f106 zero size 2024-03-16T09:01:24.837 INFO:tasks.workunit.client.0.smithi007.stdout:7/970: sync 2024-03-16T09:01:24.837 INFO:tasks.workunit.client.0.smithi007.stdout:6/950: sync 2024-03-16T09:01:24.837 INFO:tasks.workunit.client.0.smithi007.stdout:6/951: chown d2/d13/l7e 4 1 2024-03-16T09:01:24.838 INFO:tasks.workunit.client.0.smithi007.stdout:7/971: creat d0/d19/d1f/d76/f14f x:0 0 0 2024-03-16T09:01:24.838 INFO:tasks.workunit.client.0.smithi007.stdout:7/972: fsync d0/d19/d1f/d5d/d9d/df7/d3a/db3/fcc 0 2024-03-16T09:01:24.838 INFO:tasks.workunit.client.0.smithi007.stdout:6/952: write d2/d13/d4b/f55 [185309,128085] 0 2024-03-16T09:01:24.839 INFO:tasks.workunit.client.0.smithi007.stdout:1/913: dread d9/f62 [0,4194304] 0 2024-03-16T09:01:24.839 INFO:tasks.workunit.client.0.smithi007.stdout:7/973: rename d0/d19/d1f/d69/f87 to d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d139/d12b/d146/f150 0 2024-03-16T09:01:24.840 INFO:tasks.workunit.client.0.smithi007.stdout:6/953: creat d2/d13/d3f/d50/d78/dbe/dcb/d3b/f13b x:0 0 0 2024-03-16T09:01:24.842 INFO:tasks.workunit.client.0.smithi007.stdout:7/974: rmdir d0/d19/d1f/d5d/d9d/df7/d7a/d9a/d108/d134/dfd 39 2024-03-16T09:01:24.843 INFO:tasks.workunit.client.0.smithi007.stdout:6/954: mknod d2/d10/c13c 0 2024-03-16T09:01:24.843 INFO:tasks.workunit.client.0.smithi007.stdout:6/955: dread - d2/d10/fe0 zero size 2024-03-16T09:01:24.843 INFO:tasks.workunit.client.0.smithi007.stdout:6/956: chown d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/f11b 42 1 2024-03-16T09:01:24.843 INFO:tasks.workunit.client.0.smithi007.stdout:6/957: creat d2/d13/d4b/d94/dea/df7/f13d x:0 0 0 2024-03-16T09:01:24.844 INFO:tasks.workunit.client.0.smithi007.stdout:1/914: getdents d9/dd/d1b/d28 0 2024-03-16T09:01:24.848 INFO:tasks.workunit.client.0.smithi007.stdout:4/998: dread d4/d14/d1c/f32 [0,4194304] 0 2024-03-16T09:01:24.848 INFO:tasks.workunit.client.0.smithi007.stdout:4/999: fdatasync d4/d14/d64/d68/d82/dac/dfc/d8f/da8/d6b/dff/f13b 0 2024-03-16T09:01:24.848 INFO:tasks.workunit.client.0.smithi007.stdout:1/915: unlink d9/dd/d1b/d58/la9 0 2024-03-16T09:01:24.849 INFO:tasks.workunit.client.0.smithi007.stdout:1/916: symlink d9/d6b/d7e/db0/dde/d11b/d7b/l126 0 2024-03-16T09:01:24.849 INFO:tasks.workunit.client.0.smithi007.stdout:1/917: readlink d9/dd/d5e/l87 0 2024-03-16T09:01:24.849 INFO:tasks.workunit.client.0.smithi007.stdout:1/918: creat d9/dd/d5e/f127 x:0 0 0 2024-03-16T09:01:24.850 INFO:tasks.workunit.client.1.smithi019.stdout:5/899: dwrite d6/dd/d47/d70/db2/d4c/f6c [0,4194304] 0 2024-03-16T09:01:24.851 INFO:tasks.workunit.client.0.smithi007.stdout:6/958: rename d2/d10/d3d/d6d/dbf to d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/d13e 0 2024-03-16T09:01:24.852 INFO:tasks.workunit.client.1.smithi019.stdout:5/900: getdents d6/d1f/d77 0 2024-03-16T09:01:24.852 INFO:tasks.workunit.client.1.smithi019.stdout:4/906: dwrite d4/d9e/f10a [0,4194304] 0 2024-03-16T09:01:24.853 INFO:tasks.workunit.client.1.smithi019.stdout:4/907: fdatasync d4/d1b/d49/d6e/f110 0 2024-03-16T09:01:24.854 INFO:tasks.workunit.client.0.smithi007.stdout:7/975: getdents d0/d19/d1f/d76 0 2024-03-16T09:01:24.855 INFO:tasks.workunit.client.0.smithi007.stdout:7/976: write d0/dc/d5a/fdc [38781,100220] 0 2024-03-16T09:01:24.855 INFO:tasks.workunit.client.1.smithi019.stdout:5/901: rename d6/dd/d47/d70/db2/d4c/fce to d6/dd/d47/d7a/d105/f12f 0 2024-03-16T09:01:24.859 INFO:tasks.workunit.client.1.smithi019.stdout:4/908: symlink d4/d1b/d49/d58/d80/l129 0 2024-03-16T09:01:24.859 INFO:tasks.workunit.client.1.smithi019.stdout:5/902: mkdir d6/d130 0 2024-03-16T09:01:24.859 INFO:tasks.workunit.client.1.smithi019.stdout:5/903: write d6/dd/d47/d70/db2/ffa [1352936,92908] 0 2024-03-16T09:01:24.861 INFO:tasks.workunit.client.1.smithi019.stdout:4/909: rename d4/d73/d103/d4f/fbc to d4/d6c/d89/deb/f12a 0 2024-03-16T09:01:24.861 INFO:tasks.workunit.client.1.smithi019.stdout:4/910: chown d4/d64/d9b/da9 0 1 2024-03-16T09:01:24.862 INFO:tasks.workunit.client.0.smithi007.stdout:7/977: rename d0/d19/l36 to d0/d19/d1f/d5d/d9d/df7/d3a/daf/dc2/l151 0 2024-03-16T09:01:24.862 INFO:tasks.workunit.client.0.smithi007.stdout:6/959: dread d2/d13/d3f/d50/d78/dbe/dcb/f25 [0,4194304] 0 2024-03-16T09:01:24.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/904: symlink d6/dd/d47/d56/d87/l131 0 2024-03-16T09:01:24.863 INFO:tasks.workunit.client.1.smithi019.stdout:4/911: symlink d4/d73/l12b 0 2024-03-16T09:01:24.864 INFO:tasks.workunit.client.1.smithi019.stdout:5/905: mkdir d6/dd/d47/d7a/db9/d112/d132 0 2024-03-16T09:01:24.864 INFO:tasks.workunit.client.0.smithi007.stdout:7/978: rename d0/dc/d38/f138 to d0/dc/d5a/dba/dd9/f152 0 2024-03-16T09:01:24.865 INFO:tasks.workunit.client.1.smithi019.stdout:5/906: mknod d6/dd/d37/c133 0 2024-03-16T09:01:24.865 INFO:tasks.workunit.client.1.smithi019.stdout:5/907: dread - d6/dd/d61/f119 zero size 2024-03-16T09:01:24.866 INFO:tasks.workunit.client.0.smithi007.stdout:7/979: mknod d0/dc/d5a/c153 0 2024-03-16T09:01:24.866 INFO:tasks.workunit.client.0.smithi007.stdout:7/980: chown d0/d19/d1f/d5d/d9d/df7/d7a/f84 2635 1 2024-03-16T09:01:24.866 INFO:tasks.workunit.client.1.smithi019.stdout:5/908: truncate d6/dd/d47/d56/d87/fc3 1664462 0 2024-03-16T09:01:24.867 INFO:tasks.workunit.client.0.smithi007.stdout:7/981: link d0/d19/d1f/d76/c94 d0/dc/c154 0 2024-03-16T09:01:24.869 INFO:tasks.workunit.client.0.smithi007.stdout:7/982: mknod d0/d19/d1f/d5d/d9d/d132/c155 0 2024-03-16T09:01:24.869 INFO:tasks.workunit.client.0.smithi007.stdout:7/983: readlink d0/dc/lf 0 2024-03-16T09:01:24.869 INFO:tasks.workunit.client.0.smithi007.stdout:7/984: readlink d0/l9b 0 2024-03-16T09:01:24.871 INFO:tasks.workunit.client.0.smithi007.stdout:1/919: dwrite d9/dd/d1b/d2d/d30/d45/f114 [0,4194304] 0 2024-03-16T09:01:24.871 INFO:tasks.workunit.client.0.smithi007.stdout:1/920: chown d9/d6b/d7e/db0/dde/d11b/d7b/d90 425 1 2024-03-16T09:01:24.871 INFO:tasks.workunit.client.0.smithi007.stdout:1/921: truncate d9/dd/d1b/f10f 315488 0 2024-03-16T09:01:24.871 INFO:tasks.workunit.client.0.smithi007.stdout:7/985: rmdir d0/d19/d1f/d5d/d9d/df8 39 2024-03-16T09:01:24.871 INFO:tasks.workunit.client.0.smithi007.stdout:7/986: chown d0/d19/d1f/d5d/d9d/df7/d7a/dc9/c80 620 1 2024-03-16T09:01:24.874 INFO:tasks.workunit.client.0.smithi007.stdout:1/922: mknod d9/d6b/d7e/dac/dbb/c128 0 2024-03-16T09:01:24.875 INFO:tasks.workunit.client.0.smithi007.stdout:7/987: getdents d0/d19/d1f/d5d/d9d/df8/d121 0 2024-03-16T09:01:24.875 INFO:tasks.workunit.client.0.smithi007.stdout:7/988: write d0/d19/d1f/d5d/d9d/df7/d3a/daf/f14c [75362,117084] 0 2024-03-16T09:01:24.875 INFO:tasks.workunit.client.0.smithi007.stdout:7/989: write d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f98 [768233,58985] 0 2024-03-16T09:01:24.877 INFO:tasks.workunit.client.0.smithi007.stdout:7/990: symlink d0/d19/d1f/d5d/d9d/df7/d7a/l156 0 2024-03-16T09:01:24.877 INFO:tasks.workunit.client.0.smithi007.stdout:6/960: write d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/f98 [336346,97370] 0 2024-03-16T09:01:24.878 INFO:tasks.workunit.client.0.smithi007.stdout:7/991: creat d0/d19/f157 x:0 0 0 2024-03-16T09:01:24.885 INFO:tasks.workunit.client.0.smithi007.stdout:7/992: dread d0/d19/d1f/d5d/d9d/df7/f7f [0,4194304] 0 2024-03-16T09:01:24.889 INFO:tasks.workunit.client.1.smithi019.stdout:5/909: dwrite d6/dd/f1a [4194304,4194304] 0 2024-03-16T09:01:24.889 INFO:tasks.workunit.client.1.smithi019.stdout:5/910: write d6/f9 [2890059,42928] 0 2024-03-16T09:01:24.889 INFO:tasks.workunit.client.0.smithi007.stdout:1/923: write d9/dd/d5e/f65 [6466463,17116] 0 2024-03-16T09:01:24.889 INFO:tasks.workunit.client.0.smithi007.stdout:1/924: fsync d9/d6b/d7e/db0/dde/d11b/f80 0 2024-03-16T09:01:24.890 INFO:tasks.workunit.client.0.smithi007.stdout:1/925: fdatasync d9/dd/d1b/f117 0 2024-03-16T09:01:24.893 INFO:tasks.workunit.client.0.smithi007.stdout:1/926: symlink d9/dd/d1b/d28/l129 0 2024-03-16T09:01:24.896 INFO:tasks.workunit.client.0.smithi007.stdout:1/927: creat d9/dd/d1b/dd8/d112/f12a x:0 0 0 2024-03-16T09:01:24.904 INFO:tasks.workunit.client.0.smithi007.stdout:1/928: dread d9/dd/d1b/f2c [0,4194304] 0 2024-03-16T09:01:24.906 INFO:tasks.workunit.client.0.smithi007.stdout:1/929: unlink d9/d6b/d7e/db0/dde/d11b/d7b/l10b 0 2024-03-16T09:01:24.906 INFO:tasks.workunit.client.0.smithi007.stdout:1/930: readlink d9/def/l100 0 2024-03-16T09:01:24.910 INFO:tasks.workunit.client.0.smithi007.stdout:6/961: dwrite d2/d13/d3f/d50/d72/fb4 [0,4194304] 0 2024-03-16T09:01:24.910 INFO:tasks.workunit.client.0.smithi007.stdout:7/993: dwrite d0/dc/d5a/dba/dd9/d93/fe7 [0,4194304] 0 2024-03-16T09:01:24.911 INFO:tasks.workunit.client.0.smithi007.stdout:6/962: creat d2/d13/d62/f13f x:0 0 0 2024-03-16T09:01:24.913 INFO:tasks.workunit.client.0.smithi007.stdout:7/994: getdents d0/d19/d1f/d69/dc7 0 2024-03-16T09:01:24.916 INFO:tasks.workunit.client.0.smithi007.stdout:6/963: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d140 0 2024-03-16T09:01:24.918 INFO:tasks.workunit.client.1.smithi019.stdout:9/852: sync 2024-03-16T09:01:24.918 INFO:tasks.workunit.client.1.smithi019.stdout:9/853: creat d1/d7/de/d24/d44/d76/d9c/f121 x:0 0 0 2024-03-16T09:01:24.918 INFO:tasks.workunit.client.1.smithi019.stdout:9/854: write d1/d7/f81 [2297172,77340] 0 2024-03-16T09:01:24.918 INFO:tasks.workunit.client.1.smithi019.stdout:2/979: sync 2024-03-16T09:01:24.918 INFO:tasks.workunit.client.0.smithi007.stdout:6/964: rename d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/l99 to d2/d13/d4b/d94/dea/l141 0 2024-03-16T09:01:24.920 INFO:tasks.workunit.client.1.smithi019.stdout:2/980: dread d5/f4d [0,4194304] 0 2024-03-16T09:01:24.922 INFO:tasks.workunit.client.0.smithi007.stdout:6/965: rename d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d11c/c127 to d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/d13e/c142 0 2024-03-16T09:01:24.924 INFO:tasks.workunit.client.0.smithi007.stdout:1/931: dwrite d9/dd/d1b/d2d/d30/d45/f76 [0,4194304] 0 2024-03-16T09:01:24.924 INFO:tasks.workunit.client.1.smithi019.stdout:2/981: creat d5/de/dab/f15a x:0 0 0 2024-03-16T09:01:24.924 INFO:tasks.workunit.client.1.smithi019.stdout:2/982: dread - d5/de/dab/f10c zero size 2024-03-16T09:01:24.925 INFO:tasks.workunit.client.1.smithi019.stdout:2/983: truncate d5/d6/d104/dba/dfc/f156 4668372 0 2024-03-16T09:01:24.927 INFO:tasks.workunit.client.0.smithi007.stdout:6/966: rmdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/d123 0 2024-03-16T09:01:24.927 INFO:tasks.workunit.client.0.smithi007.stdout:6/967: fdatasync d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/ff4 0 2024-03-16T09:01:24.928 INFO:tasks.workunit.client.1.smithi019.stdout:2/984: dread d5/d2e/d4f/f57 [0,4194304] 0 2024-03-16T09:01:24.928 INFO:tasks.workunit.client.1.smithi019.stdout:2/985: creat d5/de/dab/f15b x:0 0 0 2024-03-16T09:01:24.929 INFO:tasks.workunit.client.0.smithi007.stdout:1/932: stat d9/dd/d1b/dd8/lf2 0 2024-03-16T09:01:24.929 INFO:tasks.workunit.client.0.smithi007.stdout:6/968: dread d2/d82/fa2 [0,4194304] 0 2024-03-16T09:01:24.930 INFO:tasks.workunit.client.0.smithi007.stdout:1/933: dread d9/f12 [0,4194304] 0 2024-03-16T09:01:24.930 INFO:tasks.workunit.client.0.smithi007.stdout:1/934: creat d9/dd/d5e/f12b x:0 0 0 2024-03-16T09:01:24.932 INFO:tasks.workunit.client.1.smithi019.stdout:2/986: mkdir d5/d6/d104/df7/d6f/da6/d15c 0 2024-03-16T09:01:24.932 INFO:tasks.workunit.client.1.smithi019.stdout:2/987: chown d5/d6/d104/dba/dfc/d74/d14c 72 1 2024-03-16T09:01:24.933 INFO:tasks.workunit.client.0.smithi007.stdout:7/995: dwrite d0/d19/d1f/d5d/d9d/df7/d7a/fa8 [0,4194304] 0 2024-03-16T09:01:24.934 INFO:tasks.workunit.client.0.smithi007.stdout:6/969: creat d2/d13/d3f/d50/d78/dbe/dcb/f143 x:0 0 0 2024-03-16T09:01:24.938 INFO:tasks.workunit.client.1.smithi019.stdout:5/911: unlink d6/dd/d47/f64 0 2024-03-16T09:01:24.938 INFO:tasks.workunit.client.1.smithi019.stdout:2/988: creat d5/d6/d104/df7/d6f/da6/d83/f15d x:0 0 0 2024-03-16T09:01:24.942 INFO:tasks.workunit.client.0.smithi007.stdout:1/935: creat d9/dd/d1b/d28/f12c x:0 0 0 2024-03-16T09:01:24.942 INFO:tasks.workunit.client.0.smithi007.stdout:1/936: fdatasync d9/f8b 0 2024-03-16T09:01:24.942 INFO:tasks.workunit.client.0.smithi007.stdout:1/937: chown d9/dd/d1b/d28/d4c/feb 98065534 1 2024-03-16T09:01:24.945 INFO:tasks.workunit.client.1.smithi019.stdout:2/989: rename d5/de/dab/f15a to d5/d6/d2b/dde/f15e 0 2024-03-16T09:01:24.945 INFO:tasks.workunit.client.1.smithi019.stdout:2/990: write d5/f13 [5176854,76722] 0 2024-03-16T09:01:24.945 INFO:tasks.workunit.client.1.smithi019.stdout:2/991: chown d5/d6/d2b/d89/ld8 6 1 2024-03-16T09:01:24.947 INFO:tasks.workunit.client.0.smithi007.stdout:6/970: creat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/d13e/f144 x:0 0 0 2024-03-16T09:01:24.948 INFO:tasks.workunit.client.0.smithi007.stdout:1/938: symlink d9/d6b/d7e/db0/dde/d11b/l12d 0 2024-03-16T09:01:24.950 INFO:tasks.workunit.client.1.smithi019.stdout:2/992: truncate d5/d2e/f3d 3943570 0 2024-03-16T09:01:24.951 INFO:tasks.workunit.client.0.smithi007.stdout:1/939: mknod d9/dd/d1b/dd8/d112/c12e 0 2024-03-16T09:01:24.952 INFO:tasks.workunit.client.0.smithi007.stdout:7/996: dwrite d0/dc/d38/d4e/f50 [0,4194304] 0 2024-03-16T09:01:24.953 INFO:tasks.workunit.client.1.smithi019.stdout:2/993: getdents d5/d6/d104/dba/dfc/d78 0 2024-03-16T09:01:24.956 INFO:tasks.workunit.client.0.smithi007.stdout:1/940: mkdir d9/d6b/d7e/db0/dde/d11b/d7b/dba/dda/d12f 0 2024-03-16T09:01:24.958 INFO:tasks.workunit.client.0.smithi007.stdout:1/941: dread d9/dd/d1b/d2d/d30/f6d [0,4194304] 0 2024-03-16T09:01:24.963 INFO:tasks.workunit.client.1.smithi019.stdout:2/994: creat d5/d6/d2b/d112/f15f x:0 0 0 2024-03-16T09:01:24.967 INFO:tasks.workunit.client.0.smithi007.stdout:7/997: link d0/d19/d1f/f4b d0/d19/d1f/d5d/d9d/df7/d3a/db3/f158 0 2024-03-16T09:01:24.973 INFO:tasks.workunit.client.1.smithi019.stdout:9/855: creat d1/d7/de/d24/d44/d76/d9c/dcf/d11e/f122 x:0 0 0 2024-03-16T09:01:24.976 INFO:tasks.workunit.client.1.smithi019.stdout:5/912: creat d6/dd/df3/f134 x:0 0 0 2024-03-16T09:01:24.976 INFO:tasks.workunit.client.1.smithi019.stdout:5/913: chown d6/dd/d47/d56 0 1 2024-03-16T09:01:24.976 INFO:tasks.workunit.client.1.smithi019.stdout:5/914: getdents d6/dd/d47/d7a/d105 0 2024-03-16T09:01:24.977 INFO:tasks.workunit.client.0.smithi007.stdout:1/942: dwrite d9/d6b/d7e/dac/f10c [0,4194304] 0 2024-03-16T09:01:24.977 INFO:tasks.workunit.client.0.smithi007.stdout:1/943: fsync d9/dd/d1b/d28/f46 0 2024-03-16T09:01:24.977 INFO:tasks.workunit.client.0.smithi007.stdout:1/944: fsync d9/dd/d5e/fd3 0 2024-03-16T09:01:24.977 INFO:tasks.workunit.client.0.smithi007.stdout:1/945: chown d9/dd/d1b/dd8/lf2 33487591 1 2024-03-16T09:01:24.977 INFO:tasks.workunit.client.0.smithi007.stdout:7/998: dread d0/d19/d1f/d5d/d9d/df7/d73/f5c [0,4194304] 0 2024-03-16T09:01:24.979 INFO:tasks.workunit.client.1.smithi019.stdout:5/915: symlink d6/dd/d47/d56/d87/l135 0 2024-03-16T09:01:24.980 INFO:tasks.workunit.client.1.smithi019.stdout:5/916: chown d6/d19/c123 847333 1 2024-03-16T09:01:24.982 INFO:tasks.workunit.client.0.smithi007.stdout:1/946: unlink d9/dd/d1b/d28/d4c/cf5 0 2024-03-16T09:01:24.982 INFO:tasks.workunit.client.0.smithi007.stdout:1/947: chown d9/c78 1690 1 2024-03-16T09:01:24.989 INFO:tasks.workunit.client.0.smithi007.stdout:1/948: unlink d9/dd/f59 0 2024-03-16T09:01:24.991 INFO:tasks.workunit.client.0.smithi007.stdout:1/949: mkdir d9/d6b/d7e/db0/dde/d11b/d7b/db4/d130 0 2024-03-16T09:01:24.994 INFO:tasks.workunit.client.0.smithi007.stdout:1/950: symlink d9/dd/l131 0 2024-03-16T09:01:24.994 INFO:tasks.workunit.client.0.smithi007.stdout:1/951: write d9/d6b/fe8 [101125,130721] 0 2024-03-16T09:01:24.996 INFO:tasks.workunit.client.0.smithi007.stdout:1/952: symlink d9/l132 0 2024-03-16T09:01:24.997 INFO:tasks.workunit.client.0.smithi007.stdout:1/953: mknod d9/d6b/d7e/db0/dde/d11b/d102/c133 0 2024-03-16T09:01:24.999 INFO:tasks.workunit.client.0.smithi007.stdout:1/954: mkdir d9/dd/d1b/d28/d134 0 2024-03-16T09:01:24.999 INFO:tasks.workunit.client.0.smithi007.stdout:1/955: chown d9/dd/d1b/d58/cae 359 1 2024-03-16T09:01:25.004 INFO:tasks.workunit.client.1.smithi019.stdout:3/860: sync 2024-03-16T09:01:25.004 INFO:tasks.workunit.client.1.smithi019.stdout:0/956: sync 2024-03-16T09:01:25.004 INFO:tasks.workunit.client.1.smithi019.stdout:3/861: fsync d3/d1b/d40/fc6 0 2024-03-16T09:01:25.005 INFO:tasks.workunit.client.1.smithi019.stdout:1/888: sync 2024-03-16T09:01:25.005 INFO:tasks.workunit.client.1.smithi019.stdout:7/835: sync 2024-03-16T09:01:25.005 INFO:tasks.workunit.client.1.smithi019.stdout:4/912: sync 2024-03-16T09:01:25.005 INFO:tasks.workunit.client.1.smithi019.stdout:7/836: chown d9/d25/d28/f9c 1780267 1 2024-03-16T09:01:25.005 INFO:tasks.workunit.client.1.smithi019.stdout:4/913: chown d4/d64/d99 105728286 1 2024-03-16T09:01:25.006 INFO:tasks.workunit.client.1.smithi019.stdout:3/862: link d3/d8b/dae/d3f/faf d3/d8b/dca/f116 0 2024-03-16T09:01:25.010 INFO:tasks.workunit.client.1.smithi019.stdout:0/957: getdents d7/d8/dec/def 0 2024-03-16T09:01:25.010 INFO:tasks.workunit.client.1.smithi019.stdout:0/958: fsync d7/d8/d32/d4a/d61/d68/f94 0 2024-03-16T09:01:25.010 INFO:tasks.workunit.client.1.smithi019.stdout:1/889: mkdir d16/dcb/daf/db2/d111 0 2024-03-16T09:01:25.010 INFO:tasks.workunit.client.1.smithi019.stdout:7/837: mknod d9/d25/d28/d48/dfc/d13/c10c 0 2024-03-16T09:01:25.011 INFO:tasks.workunit.client.1.smithi019.stdout:7/838: write d9/d25/d28/d48/dfc/d13/d18/d30/d75/de1/f7a [673070,66658] 0 2024-03-16T09:01:25.011 INFO:tasks.workunit.client.1.smithi019.stdout:7/839: chown d9/db/d12/d5b/f38 5197 1 2024-03-16T09:01:25.011 INFO:tasks.workunit.client.1.smithi019.stdout:7/840: dread - d9/db/d12/fea zero size 2024-03-16T09:01:25.011 INFO:tasks.workunit.client.1.smithi019.stdout:7/841: dread - d9/d25/d28/d48/ff7 zero size 2024-03-16T09:01:25.011 INFO:tasks.workunit.client.1.smithi019.stdout:4/914: creat d4/d73/d8a/dd6/d10c/f12c x:0 0 0 2024-03-16T09:01:25.011 INFO:tasks.workunit.client.1.smithi019.stdout:9/856: dwrite d1/f3 [0,4194304] 0 2024-03-16T09:01:25.012 INFO:tasks.workunit.client.1.smithi019.stdout:3/863: mkdir d3/d8b/da8/d117 0 2024-03-16T09:01:25.012 INFO:tasks.workunit.client.1.smithi019.stdout:1/890: creat d16/d1f/d73/ddf/d10e/d96/dac/f112 x:0 0 0 2024-03-16T09:01:25.015 INFO:tasks.workunit.client.0.smithi007.stdout:1/956: dwrite d9/d6b/f118 [0,4194304] 0 2024-03-16T09:01:25.015 INFO:tasks.workunit.client.1.smithi019.stdout:5/917: dwrite d6/d1f/f5a [0,4194304] 0 2024-03-16T09:01:25.015 INFO:tasks.workunit.client.0.smithi007.stdout:7/999: dwrite d0/d19/d1f/d5d/d9d/df7/d7a/dc9/f7b [0,4194304] 0 2024-03-16T09:01:25.017 INFO:tasks.workunit.client.1.smithi019.stdout:4/915: mknod d4/d73/d8a/dd6/d10c/c12d 0 2024-03-16T09:01:25.019 INFO:tasks.workunit.client.0.smithi007.stdout:1/957: link d9/d6b/d7e/cfd d9/d6b/d7e/db0/dde/c135 0 2024-03-16T09:01:25.020 INFO:tasks.workunit.client.1.smithi019.stdout:1/891: truncate d16/d1f/d4f/f58 3100994 0 2024-03-16T09:01:25.025 INFO:tasks.workunit.client.1.smithi019.stdout:3/864: dread d3/d1b/d40/d4a/f98 [0,4194304] 0 2024-03-16T09:01:25.027 INFO:tasks.workunit.client.1.smithi019.stdout:7/842: rmdir d9/db/d12/dcf 0 2024-03-16T09:01:25.027 INFO:tasks.workunit.client.1.smithi019.stdout:7/843: fsync d9/f3f 0 2024-03-16T09:01:25.029 INFO:tasks.workunit.client.1.smithi019.stdout:3/865: creat d3/d8b/dae/d3f/d111/f118 x:0 0 0 2024-03-16T09:01:25.029 INFO:tasks.workunit.client.1.smithi019.stdout:7/844: getdents d9/d25/d28/d48/d8d/dba/dad 0 2024-03-16T09:01:25.031 INFO:tasks.workunit.client.1.smithi019.stdout:9/857: dwrite d1/d7/de/f2a [0,4194304] 0 2024-03-16T09:01:25.031 INFO:tasks.workunit.client.1.smithi019.stdout:4/916: write d4/d6/d15/da2/fae [2603237,84129] 0 2024-03-16T09:01:25.031 INFO:tasks.workunit.client.1.smithi019.stdout:3/866: link d3/d8b/fa6 d3/d8b/dae/d5a/d80/f119 0 2024-03-16T09:01:25.032 INFO:tasks.workunit.client.1.smithi019.stdout:7/845: mknod d9/db/d102/c10d 0 2024-03-16T09:01:25.032 INFO:tasks.workunit.client.1.smithi019.stdout:7/846: read - d9/d25/d28/f60 zero size 2024-03-16T09:01:25.034 INFO:tasks.workunit.client.1.smithi019.stdout:0/959: dwrite d7/d1d/d2e/f3c [0,4194304] 0 2024-03-16T09:01:25.036 INFO:tasks.workunit.client.1.smithi019.stdout:3/867: creat d3/d11/f11a x:0 0 0 2024-03-16T09:01:25.042 INFO:tasks.workunit.client.1.smithi019.stdout:0/960: truncate d7/d8/dec/d108/f9f 3841497 0 2024-03-16T09:01:25.043 INFO:tasks.workunit.client.1.smithi019.stdout:3/868: creat d3/d89/f11b x:0 0 0 2024-03-16T09:01:25.044 INFO:tasks.workunit.client.1.smithi019.stdout:9/858: rename d1/d98 to d1/d7/de/d24/d44/d60/d74/d101/df0/d123 0 2024-03-16T09:01:25.044 INFO:tasks.workunit.client.1.smithi019.stdout:9/859: truncate d1/d7/dc0/de2/ff4 639889 0 2024-03-16T09:01:25.046 INFO:tasks.workunit.client.1.smithi019.stdout:0/961: creat d7/d8/d32/d4a/d61/d68/dd0/f131 x:0 0 0 2024-03-16T09:01:25.047 INFO:tasks.workunit.client.1.smithi019.stdout:3/869: unlink d3/d11/f49 0 2024-03-16T09:01:25.048 INFO:tasks.workunit.client.1.smithi019.stdout:3/870: chown d3/d8b/dae/d6a/ffb 63 1 2024-03-16T09:01:25.048 INFO:tasks.workunit.client.1.smithi019.stdout:3/871: write d3/d1b/d40/d4a/f98 [887097,53675] 0 2024-03-16T09:01:25.049 INFO:tasks.workunit.client.1.smithi019.stdout:7/847: rename d9/db/c45 to d9/d25/d28/d48/dfc/c10e 0 2024-03-16T09:01:25.049 INFO:tasks.workunit.client.1.smithi019.stdout:7/848: chown d9/db/f2c 20027187 1 2024-03-16T09:01:25.050 INFO:tasks.workunit.client.1.smithi019.stdout:9/860: mknod d1/d7/de/d24/d44/d112/c124 0 2024-03-16T09:01:25.051 INFO:tasks.workunit.client.1.smithi019.stdout:9/861: fdatasync d1/d7/d67/d77/fe0 0 2024-03-16T09:01:25.051 INFO:tasks.workunit.client.1.smithi019.stdout:9/862: write d1/d7/de/d24/d44/d76/d9c/fb6 [417543,42968] 0 2024-03-16T09:01:25.051 INFO:tasks.workunit.client.1.smithi019.stdout:9/863: write d1/d7/f102 [26667,108344] 0 2024-03-16T09:01:25.052 INFO:tasks.workunit.client.1.smithi019.stdout:3/872: creat d3/d1b/d40/d4a/db7/d110/f11c x:0 0 0 2024-03-16T09:01:25.053 INFO:tasks.workunit.client.1.smithi019.stdout:7/849: creat d9/d25/d108/f10f x:0 0 0 2024-03-16T09:01:25.054 INFO:tasks.workunit.client.0.smithi007.stdout:1/958: dwrite d9/dd/d1b/fd6 [0,4194304] 0 2024-03-16T09:01:25.054 INFO:tasks.workunit.client.1.smithi019.stdout:9/864: mknod d1/d117/c125 0 2024-03-16T09:01:25.055 INFO:tasks.workunit.client.1.smithi019.stdout:0/962: getdents d7/d8/dec/def/da1/da7 0 2024-03-16T09:01:25.056 INFO:tasks.workunit.client.1.smithi019.stdout:0/963: fdatasync d7/f2a 0 2024-03-16T09:01:25.056 INFO:tasks.workunit.client.1.smithi019.stdout:4/917: dwrite d4/d73/d103/d4f/da0/fc6 [0,4194304] 0 2024-03-16T09:01:25.056 INFO:tasks.workunit.client.1.smithi019.stdout:4/918: truncate d4/d64/d9b/fb7 4595074 0 2024-03-16T09:01:25.058 INFO:tasks.workunit.client.1.smithi019.stdout:3/873: mknod d3/d1b/d40/dc3/c11d 0 2024-03-16T09:01:25.058 INFO:tasks.workunit.client.1.smithi019.stdout:3/874: creat d3/d8b/da8/f11e x:0 0 0 2024-03-16T09:01:25.058 INFO:tasks.workunit.client.1.smithi019.stdout:3/875: chown d3/f51 31 1 2024-03-16T09:01:25.062 INFO:tasks.workunit.client.1.smithi019.stdout:7/850: unlink d9/d25/d28/d48/dfc/d13/d18/d30/d75/cef 0 2024-03-16T09:01:25.063 INFO:tasks.workunit.client.1.smithi019.stdout:7/851: chown d9/d25/d28/c51 367265 1 2024-03-16T09:01:25.063 INFO:tasks.workunit.client.1.smithi019.stdout:7/852: chown d9/d25/d28/d48/dfc/d13/d18/d30/d9e 69150060 1 2024-03-16T09:01:25.065 INFO:tasks.workunit.client.1.smithi019.stdout:9/865: dread d1/f3 [0,4194304] 0 2024-03-16T09:01:25.069 INFO:tasks.workunit.client.1.smithi019.stdout:4/919: creat d4/d6/d15/f12e x:0 0 0 2024-03-16T09:01:25.071 INFO:tasks.workunit.client.1.smithi019.stdout:9/866: creat d1/d7/de/d24/d44/d60/d74/de3/f126 x:0 0 0 2024-03-16T09:01:25.072 INFO:tasks.workunit.client.0.smithi007.stdout:1/959: dwrite d9/dd/d1b/d2d/d30/f4f [0,4194304] 0 2024-03-16T09:01:25.072 INFO:tasks.workunit.client.1.smithi019.stdout:7/853: dread d9/f74 [0,4194304] 0 2024-03-16T09:01:25.073 INFO:tasks.workunit.client.1.smithi019.stdout:4/920: mknod d4/d1b/d49/d58/d80/d91/df4/c12f 0 2024-03-16T09:01:25.074 INFO:tasks.workunit.client.0.smithi007.stdout:1/960: dread d9/dd/d1b/f117 [0,4194304] 0 2024-03-16T09:01:25.075 INFO:tasks.workunit.client.1.smithi019.stdout:3/876: dwrite d3/d8b/dae/d6a/f70 [0,4194304] 0 2024-03-16T09:01:25.075 INFO:tasks.workunit.client.1.smithi019.stdout:9/867: creat d1/d7/de/d24/d44/d60/d74/d101/f127 x:0 0 0 2024-03-16T09:01:25.076 INFO:tasks.workunit.client.0.smithi007.stdout:1/961: mkdir d9/dd/d1b/d28/d4c/de4/d136 0 2024-03-16T09:01:25.076 INFO:tasks.workunit.client.0.smithi007.stdout:1/962: fsync d9/dd/f10a 0 2024-03-16T09:01:25.076 INFO:tasks.workunit.client.1.smithi019.stdout:9/868: dread d1/d7/de/d24/d44/d60/d8b/dc2/fff [0,4194304] 0 2024-03-16T09:01:25.077 INFO:tasks.workunit.client.1.smithi019.stdout:4/921: mknod d4/d6/d15/d9a/c130 0 2024-03-16T09:01:25.081 INFO:tasks.workunit.client.1.smithi019.stdout:3/877: symlink d3/d8b/da8/d117/l11f 0 2024-03-16T09:01:25.082 INFO:tasks.workunit.client.1.smithi019.stdout:9/869: rmdir d1/d7/d86/ddb 39 2024-03-16T09:01:25.084 INFO:tasks.workunit.client.1.smithi019.stdout:4/922: symlink d4/d6c/d79/df2/l131 0 2024-03-16T09:01:25.085 INFO:tasks.workunit.client.1.smithi019.stdout:3/878: mknod d3/d8b/dae/d5a/d80/c120 0 2024-03-16T09:01:25.085 INFO:tasks.workunit.client.1.smithi019.stdout:3/879: fsync d3/d8b/dae/d5a/d75/d7a/fdf 0 2024-03-16T09:01:25.088 INFO:tasks.workunit.client.1.smithi019.stdout:4/923: symlink d4/d1b/d49/d58/l132 0 2024-03-16T09:01:25.089 INFO:tasks.workunit.client.1.smithi019.stdout:4/924: readlink d4/d1b/l45 0 2024-03-16T09:01:25.089 INFO:tasks.workunit.client.1.smithi019.stdout:4/925: truncate d4/d1b/f7e 9216761 0 2024-03-16T09:01:25.089 INFO:tasks.workunit.client.1.smithi019.stdout:4/926: stat d4/d1b/d49/d58/d80/d91/le9 0 2024-03-16T09:01:25.090 INFO:tasks.workunit.client.1.smithi019.stdout:3/880: link d3/d1b/d40/c71 d3/d89/df9/c121 0 2024-03-16T09:01:25.091 INFO:tasks.workunit.client.1.smithi019.stdout:9/870: rmdir d1/d7/dc0/de2 39 2024-03-16T09:01:25.091 INFO:tasks.workunit.client.1.smithi019.stdout:9/871: readlink d1/d7/d67/d77/l10c 0 2024-03-16T09:01:25.094 INFO:tasks.workunit.client.1.smithi019.stdout:4/927: rename d4/d73/d8a/d128/f46 to d4/d73/d103/f133 0 2024-03-16T09:01:25.094 INFO:tasks.workunit.client.1.smithi019.stdout:4/928: truncate d4/d64/d99/fb9 948816 0 2024-03-16T09:01:25.095 INFO:tasks.workunit.client.1.smithi019.stdout:4/929: creat d4/d1b/d49/d58/d80/d91/f134 x:0 0 0 2024-03-16T09:01:25.095 INFO:tasks.workunit.client.1.smithi019.stdout:4/930: dread d4/d6/d15/fe2 [0,4194304] 0 2024-03-16T09:01:25.097 INFO:tasks.workunit.client.1.smithi019.stdout:7/854: dwrite f4 [0,4194304] 0 2024-03-16T09:01:25.097 INFO:tasks.workunit.client.0.smithi007.stdout:1/963: dwrite d9/dd/d1b/d2d/d30/f8e [0,4194304] 0 2024-03-16T09:01:25.098 INFO:tasks.workunit.client.0.smithi007.stdout:1/964: readlink d9/dd/d1b/d28/l129 0 2024-03-16T09:01:25.098 INFO:tasks.workunit.client.1.smithi019.stdout:4/931: mkdir d4/d6/d15/d9a/d135 0 2024-03-16T09:01:25.099 INFO:tasks.workunit.client.0.smithi007.stdout:1/965: stat d9/d6b/d7e/db0/dde/d11b/d7b/lad 0 2024-03-16T09:01:25.100 INFO:tasks.workunit.client.0.smithi007.stdout:1/966: symlink d9/dd/d1b/d28/d4c/de4/d136/l137 0 2024-03-16T09:01:25.101 INFO:tasks.workunit.client.1.smithi019.stdout:9/872: read d1/d7/d21/da2/fb8 [454610,101768] 0 2024-03-16T09:01:25.101 INFO:tasks.workunit.client.0.smithi007.stdout:1/967: truncate d9/f13 1114614 0 2024-03-16T09:01:25.101 INFO:tasks.workunit.client.0.smithi007.stdout:1/968: truncate d9/dd/de0/f110 665134 0 2024-03-16T09:01:25.106 INFO:tasks.workunit.client.1.smithi019.stdout:7/855: dread d9/d25/d28/d48/dfc/d13/d18/d30/d3e/f62 [0,4194304] 0 2024-03-16T09:01:25.107 INFO:tasks.workunit.client.1.smithi019.stdout:7/856: mknod d9/db/d12/d5b/dc3/c110 0 2024-03-16T09:01:25.109 INFO:tasks.workunit.client.0.smithi007.stdout:1/969: dread d9/f19 [0,4194304] 0 2024-03-16T09:01:25.109 INFO:tasks.workunit.client.1.smithi019.stdout:3/881: dwrite d3/d8b/dae/d5a/f5c [0,4194304] 0 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:4/932: write d4/d64/d99/f31 [2666434,86860] 0 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:9/873: write d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/f4e [2433652,117495] 0 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:9/874: chown d1/d7/de/d24/d44/d76 3 1 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:4/933: fsync d4/d64/d99/f3f 0 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:4/934: chown d4/d6/d15/l2b 112 1 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:4/935: readlink d4/d6c/l77 0 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:9/875: write d1/d7/de/d24/d44/d76/d9c/f121 [852426,106672] 0 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:3/882: creat d3/d8b/dae/dd6/f122 x:0 0 0 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:3/883: chown d3/d8b/dae/d6a/f10b 175526 1 2024-03-16T09:01:25.112 INFO:tasks.workunit.client.1.smithi019.stdout:7/857: dread d9/db/d12/f26 [0,4194304] 0 2024-03-16T09:01:25.113 INFO:tasks.workunit.client.1.smithi019.stdout:4/936: rename d4/d1b/d49/d58/c10f to d4/d1b/d49/d58/c136 0 2024-03-16T09:01:25.114 INFO:tasks.workunit.client.1.smithi019.stdout:4/937: write d4/d6/d15/da2/fbe [871155,84493] 0 2024-03-16T09:01:25.114 INFO:tasks.workunit.client.1.smithi019.stdout:7/858: rmdir d9/d25/d28/d48/dfc/d13/d18/d30/d3e/da5 39 2024-03-16T09:01:25.117 INFO:tasks.workunit.client.1.smithi019.stdout:7/859: link d9/d25/d28/d48/dfc/c10e d9/c111 0 2024-03-16T09:01:25.118 INFO:tasks.workunit.client.1.smithi019.stdout:7/860: dread - d9/d25/d28/d48/dfc/d13/d18/d30/d75/da4/fc2 zero size 2024-03-16T09:01:25.118 INFO:tasks.workunit.client.1.smithi019.stdout:7/861: read d9/db/f2c [51138,7147] 0 2024-03-16T09:01:25.118 INFO:tasks.workunit.client.1.smithi019.stdout:7/862: write d9/db/f2c [974427,40901] 0 2024-03-16T09:01:25.118 INFO:tasks.workunit.client.1.smithi019.stdout:7/863: readlink d9/d25/d28/d48/dfc/d13/l8e 0 2024-03-16T09:01:25.119 INFO:tasks.workunit.client.1.smithi019.stdout:7/864: getdents d9/d25/d28/d48/dfc/d13/d18/d30 0 2024-03-16T09:01:25.119 INFO:tasks.workunit.client.1.smithi019.stdout:7/865: chown d9/d25/d28/d48/d8d/dba/dad/dd8/fde 10 1 2024-03-16T09:01:25.119 INFO:tasks.workunit.client.1.smithi019.stdout:7/866: chown d9/db/d12/f55 62 1 2024-03-16T09:01:25.121 INFO:tasks.workunit.client.1.smithi019.stdout:7/867: mkdir d9/d25/d28/d48/dfc/d13/d18/d30/d3e/da5/d112 0 2024-03-16T09:01:25.123 INFO:tasks.workunit.client.1.smithi019.stdout:7/868: getdents d9/d25/d28/d48/dfc/d13/d18/d30/d9e/dd7 0 2024-03-16T09:01:25.124 INFO:tasks.workunit.client.1.smithi019.stdout:9/876: write d1/d7/de/d24/f6d [3515624,124053] 0 2024-03-16T09:01:25.124 INFO:tasks.workunit.client.0.smithi007.stdout:1/970: dread d9/d6b/d7e/d91/fa3 [0,4194304] 0 2024-03-16T09:01:25.125 INFO:tasks.workunit.client.0.smithi007.stdout:1/971: write d9/d6b/d7e/f9a [487008,99009] 0 2024-03-16T09:01:25.125 INFO:tasks.workunit.client.1.smithi019.stdout:7/869: mkdir d9/db/d102/d113 0 2024-03-16T09:01:25.127 INFO:tasks.workunit.client.1.smithi019.stdout:7/870: creat d9/d25/d28/d48/ddd/f114 x:0 0 0 2024-03-16T09:01:25.127 INFO:tasks.workunit.client.1.smithi019.stdout:7/871: getdents d9/d25/d28/d48/dfc/d13/d18/d30/d3e/da5/d112 0 2024-03-16T09:01:25.128 INFO:tasks.workunit.client.0.smithi007.stdout:1/972: rename d9/dd/d1b/f94 to d9/d6b/d7e/db0/dde/d11b/d7b/db4/f138 0 2024-03-16T09:01:25.128 INFO:tasks.workunit.client.0.smithi007.stdout:1/973: creat d9/dd/d5e/f139 x:0 0 0 2024-03-16T09:01:25.130 INFO:tasks.workunit.client.0.smithi007.stdout:1/974: unlink d9/d6b/d7e/db0/dde/d11b/f80 0 2024-03-16T09:01:25.131 INFO:tasks.workunit.client.1.smithi019.stdout:7/872: rmdir d9/d25/d28/d48/dfc/d13/d18/d30/d9e 39 2024-03-16T09:01:25.133 INFO:tasks.workunit.client.0.smithi007.stdout:1/975: rename d9/dd/l68 to d9/d6b/d7e/dac/dbb/d105/l13a 0 2024-03-16T09:01:25.133 INFO:tasks.workunit.client.0.smithi007.stdout:1/976: chown d9/dd/cea 223 1 2024-03-16T09:01:25.137 INFO:tasks.workunit.client.1.smithi019.stdout:9/877: dread d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/f7a [0,4194304] 0 2024-03-16T09:01:25.141 INFO:tasks.workunit.client.1.smithi019.stdout:9/878: mkdir d1/d7/de/d24/d44/d112/d128 0 2024-03-16T09:01:25.141 INFO:tasks.workunit.client.1.smithi019.stdout:3/884: dwrite d3/d8b/dae/d6a/f7e [0,4194304] 0 2024-03-16T09:01:25.145 INFO:tasks.workunit.client.0.smithi007.stdout:1/977: write d9/dd/fff [988719,124590] 0 2024-03-16T09:01:25.152 INFO:tasks.workunit.client.1.smithi019.stdout:4/938: dwrite d4/d73/d8a/dd6/d10c/f12c [0,4194304] 0 2024-03-16T09:01:25.153 INFO:tasks.workunit.client.0.smithi007.stdout:6/971: sync 2024-03-16T09:01:25.156 INFO:tasks.workunit.client.1.smithi019.stdout:4/939: rename d4/d1b/d49/d58/d80/d91/fba to d4/d6c/d79/df2/f137 0 2024-03-16T09:01:25.158 INFO:tasks.workunit.client.1.smithi019.stdout:4/940: unlink d4/d6/d15/fe2 0 2024-03-16T09:01:25.191 INFO:tasks.workunit.client.1.smithi019.stdout:9/879: dwrite d1/d7/de/d24/d44/d60/d74/d101/df0/d123/fc5 [0,4194304] 0 2024-03-16T09:01:25.191 INFO:tasks.workunit.client.1.smithi019.stdout:9/880: chown d1/d7/de/d24/d44/d76/d10f 374655401 1 2024-03-16T09:01:25.191 INFO:tasks.workunit.client.1.smithi019.stdout:9/881: chown d1/d7/de/d24/d44/d60/d74/d101 10516 1 2024-03-16T09:01:25.192 INFO:tasks.workunit.client.1.smithi019.stdout:7/873: dwrite d9/d25/d28/d48/ffb [0,4194304] 0 2024-03-16T09:01:25.192 INFO:tasks.workunit.client.1.smithi019.stdout:3/885: dwrite d3/d89/fbc [0,4194304] 0 2024-03-16T09:01:25.193 INFO:tasks.workunit.client.1.smithi019.stdout:9/882: rmdir d1/d7/de/d24/d44/d112 39 2024-03-16T09:01:25.193 INFO:tasks.workunit.client.1.smithi019.stdout:4/941: dwrite d4/d1b/f8f [0,4194304] 0 2024-03-16T09:01:25.193 INFO:tasks.workunit.client.0.smithi007.stdout:1/978: dwrite d9/dd/d1b/f2c [0,4194304] 0 2024-03-16T09:01:25.193 INFO:tasks.workunit.client.0.smithi007.stdout:6/972: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d15/f92 [0,4194304] 0 2024-03-16T09:01:25.193 INFO:tasks.workunit.client.0.smithi007.stdout:6/973: fdatasync d2/d10/d3d/d12a/f9e 0 2024-03-16T09:01:25.193 INFO:tasks.workunit.client.0.smithi007.stdout:1/979: fsync d9/d6b/fe8 0 2024-03-16T09:01:25.194 INFO:tasks.workunit.client.0.smithi007.stdout:6/974: stat d2/d13/d3f/d50/d78/dbe/dcb/d1e/d6f 0 2024-03-16T09:01:25.194 INFO:tasks.workunit.client.0.smithi007.stdout:1/980: fdatasync d9/f85 0 2024-03-16T09:01:25.194 INFO:tasks.workunit.client.0.smithi007.stdout:1/981: write d9/f19 [849806,15150] 0 2024-03-16T09:01:25.194 INFO:tasks.workunit.client.0.smithi007.stdout:6/975: write d2/d29/fab [1228224,56543] 0 2024-03-16T09:01:25.194 INFO:tasks.workunit.client.1.smithi019.stdout:4/942: read d4/d6/d15/d1f/fbd [110699,32689] 0 2024-03-16T09:01:25.194 INFO:tasks.workunit.client.1.smithi019.stdout:4/943: readlink d4/d6c/d79/df7/l10e 0 2024-03-16T09:01:25.195 INFO:tasks.workunit.client.1.smithi019.stdout:7/874: mkdir d9/d25/d28/d48/dfc/d13/d18/d115 0 2024-03-16T09:01:25.197 INFO:tasks.workunit.client.0.smithi007.stdout:6/976: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d3b/d74/d145 0 2024-03-16T09:01:25.199 INFO:tasks.workunit.client.0.smithi007.stdout:1/982: mknod d9/d6b/d7e/db0/dde/d11b/d7b/db4/c13b 0 2024-03-16T09:01:25.200 INFO:tasks.workunit.client.1.smithi019.stdout:4/944: mkdir d4/d6/df9/d138 0 2024-03-16T09:01:25.200 INFO:tasks.workunit.client.1.smithi019.stdout:4/945: truncate d4/d1b/d49/f53 277132 0 2024-03-16T09:01:25.200 INFO:tasks.workunit.client.1.smithi019.stdout:4/946: creat d4/da3/f139 x:0 0 0 2024-03-16T09:01:25.201 INFO:tasks.workunit.client.0.smithi007.stdout:6/977: link d2/d13/d24/f6a d2/d13/d62/d91/f146 0 2024-03-16T09:01:25.201 INFO:tasks.workunit.client.0.smithi007.stdout:1/983: readlink d9/dd/d1b/d2d/l72 0 2024-03-16T09:01:25.202 INFO:tasks.workunit.client.1.smithi019.stdout:3/886: dread d3/d8b/dae/d6a/fb9 [0,4194304] 0 2024-03-16T09:01:25.202 INFO:tasks.workunit.client.1.smithi019.stdout:3/887: readlink d3/d8b/dae/d6a/l78 0 2024-03-16T09:01:25.202 INFO:tasks.workunit.client.1.smithi019.stdout:7/875: rename d9/d25/d28/d48/ddd/c80 to d9/d25/d28/d48/dfc/d13/d9a/c116 0 2024-03-16T09:01:25.203 INFO:tasks.workunit.client.0.smithi007.stdout:6/978: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/d13e/d147 0 2024-03-16T09:01:25.203 INFO:tasks.workunit.client.0.smithi007.stdout:6/979: chown d2/d10/f36 1912 1 2024-03-16T09:01:25.203 INFO:tasks.workunit.client.0.smithi007.stdout:1/984: creat d9/d6b/d7e/db0/dde/d11b/d7b/d90/f13c x:0 0 0 2024-03-16T09:01:25.204 INFO:tasks.workunit.client.1.smithi019.stdout:4/947: getdents d4/d6/d15/da2 0 2024-03-16T09:01:25.204 INFO:tasks.workunit.client.1.smithi019.stdout:4/948: chown d4/d6c/d89/deb 0 1 2024-03-16T09:01:25.204 INFO:tasks.workunit.client.1.smithi019.stdout:4/949: fdatasync d4/d6/d15/d9a/fb1 0 2024-03-16T09:01:25.205 INFO:tasks.workunit.client.1.smithi019.stdout:3/888: mknod d3/d1b/d40/d4a/db7/d110/c123 0 2024-03-16T09:01:25.206 INFO:tasks.workunit.client.0.smithi007.stdout:6/980: rename d2/d13/d3f/d50/d78/d81/l118 to d2/d13/d4b/d94/dea/l148 0 2024-03-16T09:01:25.206 INFO:tasks.workunit.client.1.smithi019.stdout:2/995: sync 2024-03-16T09:01:25.206 INFO:tasks.workunit.client.1.smithi019.stdout:0/964: sync 2024-03-16T09:01:25.207 INFO:tasks.workunit.client.1.smithi019.stdout:1/892: sync 2024-03-16T09:01:25.207 INFO:tasks.workunit.client.1.smithi019.stdout:5/918: sync 2024-03-16T09:01:25.207 INFO:tasks.workunit.client.1.smithi019.stdout:2/996: readlink d5/d6/d104/dba/d148/l14a 0 2024-03-16T09:01:25.207 INFO:tasks.workunit.client.1.smithi019.stdout:0/965: write d7/d8/d32/d4a/d61/d68/f125 [1128231,21869] 0 2024-03-16T09:01:25.207 INFO:tasks.workunit.client.0.smithi007.stdout:6/981: symlink d2/d13/d4b/d94/dea/l149 0 2024-03-16T09:01:25.208 INFO:tasks.workunit.client.0.smithi007.stdout:6/982: chown d2/d13/d3f/d50/d78/dbe/dcb/d3b/f87 800 1 2024-03-16T09:01:25.208 INFO:tasks.workunit.client.1.smithi019.stdout:5/919: mkdir d6/dd/d47/d7a/d136 0 2024-03-16T09:01:25.208 INFO:tasks.workunit.client.1.smithi019.stdout:1/893: symlink d16/d1f/d73/ddf/d10e/db4/dee/d10d/l113 0 2024-03-16T09:01:25.209 INFO:tasks.workunit.client.1.smithi019.stdout:1/894: read d16/d1f/f74 [3125605,18156] 0 2024-03-16T09:01:25.209 INFO:tasks.workunit.client.1.smithi019.stdout:1/895: dread - d16/dcb/f110 zero size 2024-03-16T09:01:25.209 INFO:tasks.workunit.client.1.smithi019.stdout:2/997: mknod d5/de/da2/c160 0 2024-03-16T09:01:25.209 INFO:tasks.workunit.client.1.smithi019.stdout:2/998: stat d5/d6/f119 0 2024-03-16T09:01:25.209 INFO:tasks.workunit.client.1.smithi019.stdout:2/999: creat d5/d6/d2b/d2f/d35/d4b/f161 x:0 0 0 2024-03-16T09:01:25.209 INFO:tasks.workunit.client.1.smithi019.stdout:1/896: creat d16/d31/d76/f114 x:0 0 0 2024-03-16T09:01:25.210 INFO:tasks.workunit.client.1.smithi019.stdout:1/897: symlink d16/dcb/daf/db2/d78/db1/l115 0 2024-03-16T09:01:25.211 INFO:tasks.workunit.client.1.smithi019.stdout:1/898: truncate d16/dcb/dd4/fe6 246760 0 2024-03-16T09:01:25.211 INFO:tasks.workunit.client.1.smithi019.stdout:1/899: write d16/d31/d76/f114 [30113,97418] 0 2024-03-16T09:01:25.211 INFO:tasks.workunit.client.1.smithi019.stdout:1/900: chown d16/d1f/d73/ddf/d10e/db4/dee/d10d 2664968 1 2024-03-16T09:01:25.213 INFO:tasks.workunit.client.1.smithi019.stdout:1/901: truncate d16/d31/f45 2038149 0 2024-03-16T09:01:25.215 INFO:tasks.workunit.client.0.smithi007.stdout:1/985: read d9/f62 [872883,15315] 0 2024-03-16T09:01:25.217 INFO:tasks.workunit.client.0.smithi007.stdout:1/986: symlink d9/def/l13d 0 2024-03-16T09:01:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:1/987: getdents d9/dd/d1b/d28/d4c/de4/d136 0 2024-03-16T09:01:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:1/988: fdatasync d9/dd/d1b/d2d/d30/fc3 0 2024-03-16T09:01:25.219 INFO:tasks.workunit.client.0.smithi007.stdout:1/989: read - d9/d6b/d7e/dac/dbb/fd4 zero size 2024-03-16T09:01:25.220 INFO:tasks.workunit.client.0.smithi007.stdout:6/983: write d2/d13/d3f/d50/f53 [637280,79894] 0 2024-03-16T09:01:25.220 INFO:tasks.workunit.client.1.smithi019.stdout:3/889: dread d3/d8b/dae/d6a/f7b [0,4194304] 0 2024-03-16T09:01:25.220 INFO:tasks.workunit.client.1.smithi019.stdout:3/890: stat d3/d8b/dae/d5a/d75/d7a/fdf 0 2024-03-16T09:01:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:1/990: mkdir d9/dd/d1b/d2d/d30/d13e 0 2024-03-16T09:01:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:1/991: fsync d9/dd/d1b/d2d/d30/d45/f76 0 2024-03-16T09:01:25.221 INFO:tasks.workunit.client.0.smithi007.stdout:1/992: creat d9/dd/d1b/dd8/d112/f13f x:0 0 0 2024-03-16T09:01:25.222 INFO:tasks.workunit.client.1.smithi019.stdout:7/876: write d9/d25/d28/d48/dfc/d13/fb8 [508381,61195] 0 2024-03-16T09:01:25.222 INFO:tasks.workunit.client.1.smithi019.stdout:3/891: read d3/d8b/dae/d5a/d75/d7a/fa0 [461405,5934] 0 2024-03-16T09:01:25.223 INFO:tasks.workunit.client.1.smithi019.stdout:5/920: dread d6/dd/f1a [4194304,4194304] 0 2024-03-16T09:01:25.224 INFO:tasks.workunit.client.1.smithi019.stdout:9/883: dwrite d1/d7/d21/ffb [0,4194304] 0 2024-03-16T09:01:25.226 INFO:tasks.workunit.client.1.smithi019.stdout:7/877: getdents d9/db/d12/d5b 0 2024-03-16T09:01:25.226 INFO:tasks.workunit.client.1.smithi019.stdout:7/878: fsync d9/d25/d28/d48/dfc/d13/d18/d30/d9e/fc1 0 2024-03-16T09:01:25.230 INFO:tasks.workunit.client.1.smithi019.stdout:4/950: dwrite d4/d6c/f7c [4194304,4194304] 0 2024-03-16T09:01:25.231 INFO:tasks.workunit.client.1.smithi019.stdout:9/884: truncate d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/f22 8201664 0 2024-03-16T09:01:25.234 INFO:tasks.workunit.client.1.smithi019.stdout:3/892: rmdir d3/d1b/d40/d4a/db7 39 2024-03-16T09:01:25.234 INFO:tasks.workunit.client.1.smithi019.stdout:3/893: dread - d3/d8b/da8/dcc/fec zero size 2024-03-16T09:01:25.234 INFO:tasks.workunit.client.1.smithi019.stdout:3/894: stat d3/d8b/dae/d5a/f61 0 2024-03-16T09:01:25.237 INFO:tasks.workunit.client.1.smithi019.stdout:7/879: mknod d9/d25/c117 0 2024-03-16T09:01:25.237 INFO:tasks.workunit.client.1.smithi019.stdout:7/880: readlink d9/d25/d28/d48/dfc/d13/l9b 0 2024-03-16T09:01:25.237 INFO:tasks.workunit.client.0.smithi007.stdout:1/993: dread d9/dd/f14 [0,4194304] 0 2024-03-16T09:01:25.239 INFO:tasks.workunit.client.1.smithi019.stdout:9/885: creat d1/d7/de/d24/d44/d60/d74/f129 x:0 0 0 2024-03-16T09:01:25.239 INFO:tasks.workunit.client.1.smithi019.stdout:9/886: write d1/d7/de/d24/d5b/dce/df8/d59/f95 [470931,64426] 0 2024-03-16T09:01:25.239 INFO:tasks.workunit.client.1.smithi019.stdout:9/887: write d1/d7/de/d24/d44/d60/d74/de3/ff5 [650109,84475] 0 2024-03-16T09:01:25.240 INFO:tasks.workunit.client.0.smithi007.stdout:1/994: dread d9/dd/d1b/f96 [0,4194304] 0 2024-03-16T09:01:25.240 INFO:tasks.workunit.client.0.smithi007.stdout:1/995: read d9/d6b/d7e/db0/dde/d11b/fab [172947,107571] 0 2024-03-16T09:01:25.240 INFO:tasks.workunit.client.0.smithi007.stdout:1/996: read d9/d6b/fe8 [724790,97081] 0 2024-03-16T09:01:25.240 INFO:tasks.workunit.client.1.smithi019.stdout:9/888: mkdir d1/d7/de/d24/d5b/dce/df8/df3/d12a 0 2024-03-16T09:01:25.242 INFO:tasks.workunit.client.0.smithi007.stdout:1/997: creat d9/d6b/d7e/db0/dde/d11b/d7b/dba/dda/f140 x:0 0 0 2024-03-16T09:01:25.242 INFO:tasks.workunit.client.0.smithi007.stdout:1/998: creat d9/dd/d1b/f141 x:0 0 0 2024-03-16T09:01:25.248 INFO:tasks.workunit.client.1.smithi019.stdout:7/881: write d9/d25/f54 [2057165,60036] 0 2024-03-16T09:01:25.251 INFO:tasks.workunit.client.1.smithi019.stdout:0/966: dwrite d7/d1d/d2e/fc5 [0,4194304] 0 2024-03-16T09:01:25.261 INFO:tasks.workunit.client.1.smithi019.stdout:1/902: dwrite d16/dcb/daf/db2/f8e [0,4194304] 0 2024-03-16T09:01:25.263 INFO:tasks.workunit.client.1.smithi019.stdout:5/921: dwrite d6/dd/f82 [0,4194304] 0 2024-03-16T09:01:25.263 INFO:tasks.workunit.client.1.smithi019.stdout:5/922: truncate d6/dd/d37/fa6 800733 0 2024-03-16T09:01:25.263 INFO:tasks.workunit.client.1.smithi019.stdout:5/923: read d6/d1f/f3f [82292,86262] 0 2024-03-16T09:01:25.263 INFO:tasks.workunit.client.1.smithi019.stdout:5/924: creat d6/d1f/f137 x:0 0 0 2024-03-16T09:01:25.267 INFO:tasks.workunit.client.1.smithi019.stdout:9/889: dread d1/d92/fa0 [0,4194304] 0 2024-03-16T09:01:25.270 INFO:tasks.workunit.client.1.smithi019.stdout:5/925: rmdir d6/dd/d47/d56/d87/d118 0 2024-03-16T09:01:25.270 INFO:tasks.workunit.client.1.smithi019.stdout:5/926: fdatasync d6/dd/d47/d70/db2/d66/fd1 0 2024-03-16T09:01:25.276 INFO:tasks.workunit.client.1.smithi019.stdout:9/890: dread d1/d7/de/d24/d44/d76/f5f [0,4194304] 0 2024-03-16T09:01:25.276 INFO:tasks.workunit.client.1.smithi019.stdout:9/891: chown d1/d7/de/d24/d5b/dce/df8/df3 56195 1 2024-03-16T09:01:25.276 INFO:tasks.workunit.client.1.smithi019.stdout:9/892: fdatasync d1/d7/de/d24/d44/d60/d74/f80 0 2024-03-16T09:01:25.278 INFO:tasks.workunit.client.1.smithi019.stdout:9/893: rename d1/d7/de/d24/d44/d112 to d1/d7/de/d24/d44/d60/d74/d101/df0/d12b 0 2024-03-16T09:01:25.282 INFO:tasks.workunit.client.1.smithi019.stdout:4/951: dwrite d4/d6/f39 [8388608,4194304] 0 2024-03-16T09:01:25.285 INFO:tasks.workunit.client.1.smithi019.stdout:4/952: link d4/d64/c7f d4/d73/d8a/c13a 0 2024-03-16T09:01:25.302 INFO:tasks.workunit.client.0.smithi007.stdout:1/999: dwrite d9/f12 [0,4194304] 0 2024-03-16T09:01:25.302 INFO:tasks.workunit.client.1.smithi019.stdout:3/895: dwrite d3/d8b/da8/f11e [0,4194304] 0 2024-03-16T09:01:25.302 INFO:tasks.workunit.client.1.smithi019.stdout:7/882: dwrite d9/d25/d28/d48/d8d/dba/fc9 [0,4194304] 0 2024-03-16T09:01:25.304 INFO:tasks.workunit.client.1.smithi019.stdout:0/967: dwrite d7/d8/d32/fcb [0,4194304] 0 2024-03-16T09:01:25.305 INFO:tasks.workunit.client.0.smithi007.stdout:6/984: dwrite d2/d13/d3f/d50/d78/dbe/dcb/d3b/d93/d8e/d49/fd3 [0,4194304] 0 2024-03-16T09:01:25.305 INFO:tasks.workunit.client.0.smithi007.stdout:6/985: readlink d2/d13/d62/l130 0 2024-03-16T09:01:25.305 INFO:tasks.workunit.client.1.smithi019.stdout:5/927: dwrite d6/dd/d61/fa9 [0,4194304] 0 2024-03-16T09:01:25.306 INFO:tasks.workunit.client.1.smithi019.stdout:9/894: dwrite d1/d7/d21/f7b [0,4194304] 0 2024-03-16T09:01:25.306 INFO:tasks.workunit.client.1.smithi019.stdout:3/896: symlink d3/d8b/dae/d5a/d80/l124 0 2024-03-16T09:01:25.308 INFO:tasks.workunit.client.0.smithi007.stdout:6/986: symlink d2/d13/d4b/d94/dea/l14a 0 2024-03-16T09:01:25.308 INFO:tasks.workunit.client.0.smithi007.stdout:6/987: chown d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/d13e/c142 31610031 1 2024-03-16T09:01:25.309 INFO:tasks.workunit.client.0.smithi007.stdout:6/988: rename d2/d13/d24/c3e to d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/d9c/c14b 0 2024-03-16T09:01:25.310 INFO:tasks.workunit.client.0.smithi007.stdout:6/989: symlink d2/d13/d62/d122/l14c 0 2024-03-16T09:01:25.311 INFO:tasks.workunit.client.1.smithi019.stdout:7/883: symlink d9/d25/d28/d48/dfc/d13/d18/d30/d3e/da5/d112/l118 0 2024-03-16T09:01:25.314 INFO:tasks.workunit.client.0.smithi007.stdout:6/990: rename d2/d29 to d2/d82/d14d 0 2024-03-16T09:01:25.320 INFO:tasks.workunit.client.1.smithi019.stdout:7/884: dread d9/d25/d28/f69 [0,4194304] 0 2024-03-16T09:01:25.322 INFO:tasks.workunit.client.1.smithi019.stdout:0/968: mkdir d7/d8/d32/d4a/d61/dbf/d132 0 2024-03-16T09:01:25.322 INFO:tasks.workunit.client.1.smithi019.stdout:9/895: dwrite d1/d7/de/d24/d44/d76/fa5 [0,4194304] 0 2024-03-16T09:01:25.325 INFO:tasks.workunit.client.1.smithi019.stdout:5/928: creat d6/dd/d47/d70/db2/d89/d9e/f138 x:0 0 0 2024-03-16T09:01:25.328 INFO:tasks.workunit.client.1.smithi019.stdout:3/897: unlink d3/d8b/dae/d5a/d75/d7a/f109 0 2024-03-16T09:01:25.329 INFO:tasks.workunit.client.1.smithi019.stdout:0/969: dread d7/f6a [0,4194304] 0 2024-03-16T09:01:25.329 INFO:tasks.workunit.client.1.smithi019.stdout:0/970: creat d7/d8/d32/d4a/d61/d68/dd0/f133 x:0 0 0 2024-03-16T09:01:25.330 INFO:tasks.workunit.client.1.smithi019.stdout:7/885: truncate d9/db/f87 3348575 0 2024-03-16T09:01:25.330 INFO:tasks.workunit.client.0.smithi007.stdout:6/991: dwrite d2/f7 [0,4194304] 0 2024-03-16T09:01:25.334 INFO:tasks.workunit.client.1.smithi019.stdout:5/929: symlink d6/dd/d47/d70/l139 0 2024-03-16T09:01:25.334 INFO:tasks.workunit.client.1.smithi019.stdout:5/930: truncate d6/dd/d47/d70/db2/d4c/f12c 149925 0 2024-03-16T09:01:25.335 INFO:tasks.workunit.client.1.smithi019.stdout:3/898: creat d3/d1b/d40/de9/f125 x:0 0 0 2024-03-16T09:01:25.337 INFO:tasks.workunit.client.1.smithi019.stdout:0/971: creat d7/d1d/d63/d89/d9e/f134 x:0 0 0 2024-03-16T09:01:25.339 INFO:tasks.workunit.client.1.smithi019.stdout:7/886: creat d9/db/d12/d5b/f119 x:0 0 0 2024-03-16T09:01:25.339 INFO:tasks.workunit.client.1.smithi019.stdout:5/931: symlink d6/dd/dde/l13a 0 2024-03-16T09:01:25.343 INFO:tasks.workunit.client.1.smithi019.stdout:7/887: creat d9/d25/d28/d48/d8d/dba/dad/f11a x:0 0 0 2024-03-16T09:01:25.343 INFO:tasks.workunit.client.1.smithi019.stdout:7/888: fdatasync d9/d25/d28/d48/ffb 0 2024-03-16T09:01:25.347 INFO:tasks.workunit.client.1.smithi019.stdout:9/896: dwrite d1/d92/fc1 [0,4194304] 0 2024-03-16T09:01:25.347 INFO:tasks.workunit.client.1.smithi019.stdout:3/899: dread d3/d8b/dae/d6a/f70 [0,4194304] 0 2024-03-16T09:01:25.347 INFO:tasks.workunit.client.1.smithi019.stdout:9/897: chown d1/c37 0 1 2024-03-16T09:01:25.349 INFO:tasks.workunit.client.1.smithi019.stdout:9/898: getdents d1/d7/d21/da2 0 2024-03-16T09:01:25.350 INFO:tasks.workunit.client.1.smithi019.stdout:9/899: link d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/f58 d1/d7/de/d24/d5b/dce/df8/df3/f12c 0 2024-03-16T09:01:25.350 INFO:tasks.workunit.client.1.smithi019.stdout:9/900: symlink d1/d7/de/d24/d44/d60/d74/d101/df0/d12b/d128/l12d 0 2024-03-16T09:01:25.352 INFO:tasks.workunit.client.1.smithi019.stdout:9/901: read d1/d7/de/d24/d44/d76/d9c/f121 [713932,66954] 0 2024-03-16T09:01:25.353 INFO:tasks.workunit.client.1.smithi019.stdout:9/902: getdents d1/d7/de/d24/d44/d60/d74/de3 0 2024-03-16T09:01:25.353 INFO:tasks.workunit.client.1.smithi019.stdout:9/903: chown d1/d7/de/d24/d44/d60/l107 6 1 2024-03-16T09:01:25.368 INFO:tasks.workunit.client.0.smithi007.stdout:6/992: dwrite d2/f46 [4194304,4194304] 0 2024-03-16T09:01:25.368 INFO:tasks.workunit.client.1.smithi019.stdout:5/932: dwrite d6/dd/d47/d70/db2/d89/f96 [0,4194304] 0 2024-03-16T09:01:25.369 INFO:tasks.workunit.client.1.smithi019.stdout:5/933: chown d6/dd/d37/cbc 45 1 2024-03-16T09:01:25.369 INFO:tasks.workunit.client.1.smithi019.stdout:9/904: dwrite d1/d7/dc0/de2/ff4 [0,4194304] 0 2024-03-16T09:01:25.371 INFO:tasks.workunit.client.1.smithi019.stdout:5/934: link d6/dd/d47/l110 d6/dd/d47/d70/dfb/l13b 0 2024-03-16T09:01:25.371 INFO:tasks.workunit.client.1.smithi019.stdout:5/935: truncate d6/dd/dde/df1/f111 843167 0 2024-03-16T09:01:25.376 INFO:tasks.workunit.client.1.smithi019.stdout:5/936: mknod d6/dd/d47/d70/db2/d89/d9e/c13c 0 2024-03-16T09:01:25.377 INFO:tasks.workunit.client.1.smithi019.stdout:0/972: rename d7/d8/d32/d4a/d61/d68/dd0 to d7/d8/d32/d4a/d61/dbf/d135 0 2024-03-16T09:01:25.379 INFO:tasks.workunit.client.1.smithi019.stdout:5/937: creat d6/dd/d47/d7a/db9/f13d x:0 0 0 2024-03-16T09:01:25.379 INFO:tasks.workunit.client.1.smithi019.stdout:5/938: write d6/dd/f5c [4824894,28020] 0 2024-03-16T09:01:25.379 INFO:tasks.workunit.client.0.smithi007.stdout:6/993: dread d2/d13/d3f/d50/d78/d81/f8c [0,4194304] 0 2024-03-16T09:01:25.381 INFO:tasks.workunit.client.1.smithi019.stdout:7/889: rename d9/d25/d28/d48/dfc/d13/d9a/ca1 to d9/d25/d28/d48/dfc/d13/c11b 0 2024-03-16T09:01:25.385 INFO:tasks.workunit.client.1.smithi019.stdout:5/939: mknod d6/dd/d47/dd0/c13e 0 2024-03-16T09:01:25.385 INFO:tasks.workunit.client.1.smithi019.stdout:5/940: fsync d6/dd/d61/fd4 0 2024-03-16T09:01:25.385 INFO:tasks.workunit.client.1.smithi019.stdout:5/941: fdatasync d6/d19/d34/fdf 0 2024-03-16T09:01:25.385 INFO:tasks.workunit.client.1.smithi019.stdout:5/942: chown d6/dd/d47/d70/db2/d4c/ccb 818754650 1 2024-03-16T09:01:25.386 INFO:tasks.workunit.client.1.smithi019.stdout:9/905: dwrite d1/f6a [0,4194304] 0 2024-03-16T09:01:25.387 INFO:tasks.workunit.client.1.smithi019.stdout:0/973: dread d7/d8/f88 [0,4194304] 0 2024-03-16T09:01:25.387 INFO:tasks.workunit.client.1.smithi019.stdout:5/943: mknod d6/dd/d37/d54/c13f 0 2024-03-16T09:01:25.387 INFO:tasks.workunit.client.1.smithi019.stdout:5/944: chown d6/d1f/d78/cb0 282816 1 2024-03-16T09:01:25.388 INFO:tasks.workunit.client.1.smithi019.stdout:5/945: stat d6/dd/f1a 0 2024-03-16T09:01:25.390 INFO:tasks.workunit.client.1.smithi019.stdout:9/906: creat d1/d7/de/d24/d44/d60/d74/d101/df0/d12b/d128/f12e x:0 0 0 2024-03-16T09:01:25.390 INFO:tasks.workunit.client.1.smithi019.stdout:9/907: dread - d1/d7/f11f zero size 2024-03-16T09:01:25.390 INFO:tasks.workunit.client.1.smithi019.stdout:9/908: dread - d1/d7/d86/f10a zero size 2024-03-16T09:01:25.390 INFO:tasks.workunit.client.1.smithi019.stdout:0/974: mkdir d7/d8/d32/d66/d6d/d7c/d136 0 2024-03-16T09:01:25.391 INFO:tasks.workunit.client.1.smithi019.stdout:9/909: creat d1/d7/de/f12f x:0 0 0 2024-03-16T09:01:25.392 INFO:tasks.workunit.client.1.smithi019.stdout:9/910: creat d1/d7/d86/d119/f130 x:0 0 0 2024-03-16T09:01:25.395 INFO:tasks.workunit.client.1.smithi019.stdout:4/953: sync 2024-03-16T09:01:25.395 INFO:tasks.workunit.client.1.smithi019.stdout:1/903: sync 2024-03-16T09:01:25.395 INFO:tasks.workunit.client.1.smithi019.stdout:4/954: chown d4/d6/d15/d1f/l94 629 1 2024-03-16T09:01:25.396 INFO:tasks.workunit.client.1.smithi019.stdout:1/904: dread - d16/d31/d71/d89/dce/fe2 zero size 2024-03-16T09:01:25.396 INFO:tasks.workunit.client.1.smithi019.stdout:1/905: fdatasync d16/f5d 0 2024-03-16T09:01:25.396 INFO:tasks.workunit.client.1.smithi019.stdout:1/906: stat d16/d1f/d43/f4c 0 2024-03-16T09:01:25.396 INFO:tasks.workunit.client.1.smithi019.stdout:1/907: write d16/d1f/d73/ddf/d10e/d96/dac/fea [378580,15673] 0 2024-03-16T09:01:25.396 INFO:tasks.workunit.client.1.smithi019.stdout:4/955: rmdir d4/d73/d103 39 2024-03-16T09:01:25.398 INFO:tasks.workunit.client.1.smithi019.stdout:1/908: mknod d16/d1f/d73/ddf/d10e/d96/c116 0 2024-03-16T09:01:25.399 INFO:tasks.workunit.client.1.smithi019.stdout:1/909: symlink d16/d1f/d73/ddf/d10e/db4/l117 0 2024-03-16T09:01:25.402 INFO:tasks.workunit.client.1.smithi019.stdout:1/910: mkdir d16/dcb/daf/db2/d78/d118 0 2024-03-16T09:01:25.402 INFO:tasks.workunit.client.1.smithi019.stdout:1/911: write d16/d1f/d73/ddf/d10e/f91 [1858864,103466] 0 2024-03-16T09:01:25.402 INFO:tasks.workunit.client.1.smithi019.stdout:1/912: write d16/d1f/d73/ddf/d10e/f91 [3065823,24981] 0 2024-03-16T09:01:25.403 INFO:tasks.workunit.client.1.smithi019.stdout:1/913: truncate d16/d31/d71/d89/ff2 594621 0 2024-03-16T09:01:25.403 INFO:tasks.workunit.client.1.smithi019.stdout:1/914: fdatasync d16/d1f/d73/ddf/d10e/fba 0 2024-03-16T09:01:25.403 INFO:tasks.workunit.client.1.smithi019.stdout:1/915: truncate d16/d31/d71/d89/ff2 1540844 0 2024-03-16T09:01:25.403 INFO:tasks.workunit.client.1.smithi019.stdout:1/916: fsync d16/d1f/d73/ddf/d10e/fd3 0 2024-03-16T09:01:25.403 INFO:tasks.workunit.client.1.smithi019.stdout:1/917: creat d16/d1f/f119 x:0 0 0 2024-03-16T09:01:25.403 INFO:tasks.workunit.client.1.smithi019.stdout:1/918: write d16/d31/d76/f95 [849475,118686] 0 2024-03-16T09:01:25.404 INFO:tasks.workunit.client.1.smithi019.stdout:1/919: symlink d16/dcb/dd4/l11a 0 2024-03-16T09:01:25.404 INFO:tasks.workunit.client.1.smithi019.stdout:1/920: creat d16/d1f/d4f/d8a/f11b x:0 0 0 2024-03-16T09:01:25.404 INFO:tasks.workunit.client.1.smithi019.stdout:1/921: truncate d16/d1f/d73/fa7 119642 0 2024-03-16T09:01:25.480 INFO:tasks.workunit.client.1.smithi019.stdout:7/890: dwrite d9/d25/d28/d48/dfc/f59 [4194304,4194304] 0 2024-03-16T09:01:25.480 INFO:tasks.workunit.client.1.smithi019.stdout:9/911: dwrite d1/d7/de/d24/d44/d60/d74/f80 [0,4194304] 0 2024-03-16T09:01:25.494 INFO:tasks.workunit.client.1.smithi019.stdout:9/912: write d1/d7/de/d24/ff2 [4655436,51527] 0 2024-03-16T09:01:25.496 INFO:tasks.workunit.client.1.smithi019.stdout:9/913: rename d1/d7/d21/da2/c100 to d1/d7/de/d24/d44/d60/d74/d101/df0/c131 0 2024-03-16T09:01:25.496 INFO:tasks.workunit.client.1.smithi019.stdout:9/914: dread - d1/d7/de/d24/d44/d60/d74/d101/f127 zero size 2024-03-16T09:01:25.497 INFO:tasks.workunit.client.1.smithi019.stdout:9/915: mknod d1/d7/de/d24/d44/d60/c132 0 2024-03-16T09:01:25.497 INFO:tasks.workunit.client.1.smithi019.stdout:9/916: chown d1/d7/f11f 3441 1 2024-03-16T09:01:25.497 INFO:tasks.workunit.client.1.smithi019.stdout:9/917: mkdir d1/d7/de/d24/d5b/dce/df8/df3/d133 0 2024-03-16T09:01:25.498 INFO:tasks.workunit.client.1.smithi019.stdout:9/918: mknod d1/d7/de/d24/d44/d60/d74/de3/c134 0 2024-03-16T09:01:25.504 INFO:tasks.workunit.client.1.smithi019.stdout:0/975: dwrite d7/d8/d32/d4a/d61/d68/f90 [0,4194304] 0 2024-03-16T09:01:25.504 INFO:tasks.workunit.client.1.smithi019.stdout:0/976: read - d7/d8/dec/d108/da3/f130 zero size 2024-03-16T09:01:25.507 INFO:tasks.workunit.client.1.smithi019.stdout:0/977: creat d7/d8/d32/d66/d6d/d7c/d136/f137 x:0 0 0 2024-03-16T09:01:25.507 INFO:tasks.workunit.client.1.smithi019.stdout:0/978: creat d7/d8/d32/d66/d6d/d7c/f138 x:0 0 0 2024-03-16T09:01:25.507 INFO:tasks.workunit.client.1.smithi019.stdout:0/979: fsync d7/d8/dec/d108/fe8 0 2024-03-16T09:01:25.515 INFO:tasks.workunit.client.1.smithi019.stdout:7/891: dwrite d9/d25/d28/d48/dfc/d13/f94 [0,4194304] 0 2024-03-16T09:01:25.515 INFO:tasks.workunit.client.1.smithi019.stdout:9/919: dwrite d1/d7/d67/fe4 [0,4194304] 0 2024-03-16T09:01:25.516 INFO:tasks.workunit.client.1.smithi019.stdout:4/956: dwrite d4/d1b/d49/f72 [0,4194304] 0 2024-03-16T09:01:25.516 INFO:tasks.workunit.client.1.smithi019.stdout:1/922: dwrite d16/d1f/d43/f47 [0,4194304] 0 2024-03-16T09:01:25.521 INFO:tasks.workunit.client.1.smithi019.stdout:9/920: mknod d1/d7/dc0/c135 0 2024-03-16T09:01:25.524 INFO:tasks.workunit.client.1.smithi019.stdout:1/923: dread d16/d1f/d4f/f57 [0,4194304] 0 2024-03-16T09:01:25.524 INFO:tasks.workunit.client.1.smithi019.stdout:0/980: dwrite d7/d8/d32/d4a/d61/d68/f125 [0,4194304] 0 2024-03-16T09:01:25.528 INFO:tasks.workunit.client.1.smithi019.stdout:9/921: dread d1/d7/de/d24/d44/d60/d74/de3/ff5 [0,4194304] 0 2024-03-16T09:01:25.528 INFO:tasks.workunit.client.1.smithi019.stdout:9/922: fdatasync d1/d7/d21/ffb 0 2024-03-16T09:01:25.533 INFO:tasks.workunit.client.1.smithi019.stdout:1/924: rename d16/d1f/d43/d55/l77 to d16/d31/d76/d92/l11c 0 2024-03-16T09:01:25.534 INFO:tasks.workunit.client.1.smithi019.stdout:0/981: creat d7/d8/d32/df5/f139 x:0 0 0 2024-03-16T09:01:25.534 INFO:tasks.workunit.client.1.smithi019.stdout:0/982: write d7/d8/d32/d4a/d61/dbf/d135/f133 [1025676,126514] 0 2024-03-16T09:01:25.534 INFO:tasks.workunit.client.1.smithi019.stdout:0/983: chown d7/d8/d32/d4a/l8d 813765642 1 2024-03-16T09:01:25.535 INFO:tasks.workunit.client.1.smithi019.stdout:9/923: mknod d1/d7/d67/d77/c136 0 2024-03-16T09:01:25.536 INFO:tasks.workunit.client.1.smithi019.stdout:4/957: dwrite d4/d6/f81 [4194304,4194304] 0 2024-03-16T09:01:25.536 INFO:tasks.workunit.client.1.smithi019.stdout:4/958: stat d4/d64/d99/fb9 0 2024-03-16T09:01:25.536 INFO:tasks.workunit.client.1.smithi019.stdout:7/892: dwrite d9/db/d12/f2f [4194304,4194304] 0 2024-03-16T09:01:25.536 INFO:tasks.workunit.client.1.smithi019.stdout:7/893: stat d9/df4/cfd 0 2024-03-16T09:01:25.536 INFO:tasks.workunit.client.1.smithi019.stdout:7/894: readlink d9/d25/d28/d48/dfc/d13/d18/d30/d75/da4/le8 0 2024-03-16T09:01:25.536 INFO:tasks.workunit.client.1.smithi019.stdout:0/984: rmdir d7/d1d/d63/d6e 39 2024-03-16T09:01:25.537 INFO:tasks.workunit.client.1.smithi019.stdout:9/924: creat d1/d7/dc0/f137 x:0 0 0 2024-03-16T09:01:25.538 INFO:tasks.workunit.client.1.smithi019.stdout:9/925: dread - d1/d7/de/d24/d44/d76/d9c/dcf/d11e/f122 zero size 2024-03-16T09:01:25.538 INFO:tasks.workunit.client.1.smithi019.stdout:9/926: readlink d1/d7/l3d 0 2024-03-16T09:01:25.538 INFO:tasks.workunit.client.1.smithi019.stdout:9/927: fdatasync d1/d7/de/d24/d44/d60/d74/d101/df0/d12b/d128/f12e 0 2024-03-16T09:01:25.539 INFO:tasks.workunit.client.1.smithi019.stdout:4/959: mkdir d4/d73/d103/d13b 0 2024-03-16T09:01:25.540 INFO:tasks.workunit.client.1.smithi019.stdout:4/960: dread d4/d6/d15/d9a/fb1 [0,4194304] 0 2024-03-16T09:01:25.544 INFO:tasks.workunit.client.1.smithi019.stdout:7/895: dread d9/db/f47 [0,4194304] 0 2024-03-16T09:01:25.546 INFO:tasks.workunit.client.1.smithi019.stdout:9/928: creat d1/d7/d67/f138 x:0 0 0 2024-03-16T09:01:25.546 INFO:tasks.workunit.client.1.smithi019.stdout:9/929: write d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/f7a [2710746,69168] 0 2024-03-16T09:01:25.546 INFO:tasks.workunit.client.1.smithi019.stdout:0/985: dread d7/d8/d32/d4a/d61/d68/f125 [0,4194304] 0 2024-03-16T09:01:25.546 INFO:tasks.workunit.client.1.smithi019.stdout:0/986: write d7/d1d/f101 [700614,86635] 0 2024-03-16T09:01:25.548 INFO:tasks.workunit.client.1.smithi019.stdout:9/930: rename d1/d7/f81 to d1/d7/de/da3/f139 0 2024-03-16T09:01:25.550 INFO:tasks.workunit.client.1.smithi019.stdout:9/931: rename d1/d7/de/d24/d5b/dce/lfc to d1/d7/de/d24/d44/d60/d74/d101/df0/d123/l13a 0 2024-03-16T09:01:25.550 INFO:tasks.workunit.client.1.smithi019.stdout:9/932: write d1/d7/de/d24/d44/d60/d74/d101/f127 [861718,24278] 0 2024-03-16T09:01:25.550 INFO:tasks.workunit.client.1.smithi019.stdout:3/900: sync 2024-03-16T09:01:25.551 INFO:tasks.workunit.client.1.smithi019.stdout:3/901: creat d3/d8b/da8/dcc/f126 x:0 0 0 2024-03-16T09:01:25.551 INFO:tasks.workunit.client.1.smithi019.stdout:3/902: truncate d3/d89/fcb 681343 0 2024-03-16T09:01:25.551 INFO:tasks.workunit.client.1.smithi019.stdout:3/903: dread - d3/d8b/dae/d3f/f7c zero size 2024-03-16T09:01:25.551 INFO:tasks.workunit.client.1.smithi019.stdout:3/904: creat d3/d11/f127 x:0 0 0 2024-03-16T09:01:25.552 INFO:tasks.workunit.client.1.smithi019.stdout:7/896: dread d9/d25/d28/d48/dfc/d13/d18/fac [4194304,4194304] 0 2024-03-16T09:01:25.555 INFO:tasks.workunit.client.1.smithi019.stdout:4/961: dwrite d4/d64/d9b/fb7 [4194304,4194304] 0 2024-03-16T09:01:25.555 INFO:tasks.workunit.client.1.smithi019.stdout:4/962: creat d4/d1b/d49/d6e/f13c x:0 0 0 2024-03-16T09:01:25.558 INFO:tasks.workunit.client.1.smithi019.stdout:7/897: rename d9/d25/d93/ff0 to d9/d25/d93/f11c 0 2024-03-16T09:01:25.560 INFO:tasks.workunit.client.1.smithi019.stdout:4/963: mkdir d4/d64/d99/d13d 0 2024-03-16T09:01:25.562 INFO:tasks.workunit.client.1.smithi019.stdout:3/905: rename d3/d1b/l79 to d3/d8b/da8/d117/l128 0 2024-03-16T09:01:25.563 INFO:tasks.workunit.client.1.smithi019.stdout:0/987: dwrite d7/d8/dec/d108/f9f [0,4194304] 0 2024-03-16T09:01:25.563 INFO:tasks.workunit.client.1.smithi019.stdout:4/964: creat d4/d73/d8a/f13e x:0 0 0 2024-03-16T09:01:25.563 INFO:tasks.workunit.client.1.smithi019.stdout:4/965: write d4/d1b/f28 [2082234,28396] 0 2024-03-16T09:01:25.565 INFO:tasks.workunit.client.1.smithi019.stdout:3/906: unlink d3/d8b/dae/d3f/d111/fb0 0 2024-03-16T09:01:25.566 INFO:tasks.workunit.client.1.smithi019.stdout:5/946: sync 2024-03-16T09:01:25.566 INFO:tasks.workunit.client.1.smithi019.stdout:9/933: getdents d1/d7/d67 0 2024-03-16T09:01:25.568 INFO:tasks.workunit.client.1.smithi019.stdout:4/966: rename d4/d6/c22 to d4/d6/d15/d1f/c13f 0 2024-03-16T09:01:25.570 INFO:tasks.workunit.client.1.smithi019.stdout:5/947: mknod d6/dd/d47/d7a/db9/c140 0 2024-03-16T09:01:25.570 INFO:tasks.workunit.client.1.smithi019.stdout:5/948: write d6/dd/d37/d54/fa2 [4536310,14170] 0 2024-03-16T09:01:25.571 INFO:tasks.workunit.client.1.smithi019.stdout:4/967: creat d4/d6/d15/d9a/f140 x:0 0 0 2024-03-16T09:01:25.572 INFO:tasks.workunit.client.1.smithi019.stdout:0/988: write d7/d1d/d2e/f3c [3524408,56864] 0 2024-03-16T09:01:25.573 INFO:tasks.workunit.client.1.smithi019.stdout:5/949: creat d6/d1f/d77/d84/f141 x:0 0 0 2024-03-16T09:01:25.573 INFO:tasks.workunit.client.1.smithi019.stdout:5/950: dread - d6/d19/d34/fdf zero size 2024-03-16T09:01:25.574 INFO:tasks.workunit.client.1.smithi019.stdout:4/968: truncate d4/d6/d15/d9a/fa4 1212367 0 2024-03-16T09:01:25.575 INFO:tasks.workunit.client.1.smithi019.stdout:9/934: dread d1/d7/f51 [0,4194304] 0 2024-03-16T09:01:25.575 INFO:tasks.workunit.client.1.smithi019.stdout:0/989: creat d7/d1d/d63/d6e/db0/d10f/f13a x:0 0 0 2024-03-16T09:01:25.575 INFO:tasks.workunit.client.1.smithi019.stdout:7/898: dwrite d9/db/f87 [0,4194304] 0 2024-03-16T09:01:25.576 INFO:tasks.workunit.client.1.smithi019.stdout:5/951: mkdir d6/dd/d47/d70/db2/d89/d142 0 2024-03-16T09:01:25.578 INFO:tasks.workunit.client.1.smithi019.stdout:7/899: symlink d9/d25/d28/d48/dfc/d13/d9a/l11d 0 2024-03-16T09:01:25.578 INFO:tasks.workunit.client.1.smithi019.stdout:7/900: truncate d9/db/f100 517399 0 2024-03-16T09:01:25.581 INFO:tasks.workunit.client.1.smithi019.stdout:9/935: rmdir d1/d7/de/d24/d44/d76/d9c 39 2024-03-16T09:01:25.583 INFO:tasks.workunit.client.1.smithi019.stdout:5/952: rename d6/dd/df3 to d6/dd/d47/d7a/db9/d143 0 2024-03-16T09:01:25.584 INFO:tasks.workunit.client.1.smithi019.stdout:4/969: read d4/d1b/d49/d58/d80/d91/fd0 [2286082,94253] 0 2024-03-16T09:01:25.586 INFO:tasks.workunit.client.1.smithi019.stdout:3/907: dwrite d3/d8b/dae/d3f/d111/f118 [0,4194304] 0 2024-03-16T09:01:25.587 INFO:tasks.workunit.client.1.smithi019.stdout:5/953: rename d6/dd/d37/f93 to d6/dd/d37/d76/d107/f144 0 2024-03-16T09:01:25.587 INFO:tasks.workunit.client.1.smithi019.stdout:5/954: write d6/d1f/d78/fe6 [966787,21957] 0 2024-03-16T09:01:25.589 INFO:tasks.workunit.client.1.smithi019.stdout:4/970: creat d4/d73/d103/d4f/f141 x:0 0 0 2024-03-16T09:01:25.595 INFO:tasks.workunit.client.1.smithi019.stdout:4/971: unlink d4/d1b/f5d 0 2024-03-16T09:01:25.609 INFO:tasks.workunit.client.0.smithi007.stdout:6/994: sync 2024-03-16T09:01:25.610 INFO:tasks.workunit.client.0.smithi007.stdout:6/995: mkdir d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/dc9/d14e 0 2024-03-16T09:01:25.610 INFO:tasks.workunit.client.0.smithi007.stdout:6/996: dread - d2/d13/d3f/d50/d78/dbe/dcb/d1e/d54/de3/d9a/f12f zero size 2024-03-16T09:01:25.610 INFO:tasks.workunit.client.0.smithi007.stdout:6/997: chown d2/d13/d62/l130 219 1 2024-03-16T09:01:25.612 INFO:tasks.workunit.client.1.smithi019.stdout:0/990: dwrite d7/d1d/d63/d6e/db0/d10f/f13a [0,4194304] 0 2024-03-16T09:01:25.621 INFO:tasks.workunit.client.1.smithi019.stdout:3/908: dwrite d3/d8b/da8/dcc/fec [0,4194304] 0 2024-03-16T09:01:25.621 INFO:tasks.workunit.client.1.smithi019.stdout:4/972: dwrite d4/d73/fde [0,4194304] 0 2024-03-16T09:01:25.621 INFO:tasks.workunit.client.1.smithi019.stdout:9/936: dwrite d1/d7/de/d24/d44/d60/fbf [0,4194304] 0 2024-03-16T09:01:25.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/901: dwrite d9/d25/d28/d48/dfc/d13/d18/d30/f97 [0,4194304] 0 2024-03-16T09:01:25.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/902: creat d9/d25/d93/f11e x:0 0 0 2024-03-16T09:01:25.622 INFO:tasks.workunit.client.1.smithi019.stdout:7/903: stat d9/fff 0 2024-03-16T09:01:25.630 INFO:tasks.workunit.client.1.smithi019.stdout:4/973: mknod d4/d73/d8a/dd6/c142 0 2024-03-16T09:01:25.633 INFO:tasks.workunit.client.1.smithi019.stdout:9/937: symlink d1/d92/l13b 0 2024-03-16T09:01:25.636 INFO:tasks.workunit.client.1.smithi019.stdout:4/974: symlink d4/d6/d15/d9a/l143 0 2024-03-16T09:01:25.637 INFO:tasks.workunit.client.1.smithi019.stdout:4/975: dread - d4/d1b/d49/d6e/f13c zero size 2024-03-16T09:01:25.637 INFO:tasks.workunit.client.1.smithi019.stdout:9/938: creat d1/d7/dc0/de2/f13c x:0 0 0 2024-03-16T09:01:25.639 INFO:tasks.workunit.client.1.smithi019.stdout:9/939: write d1/d7/de/d24/d44/d60/d74/fad [536661,33446] 0 2024-03-16T09:01:25.639 INFO:tasks.workunit.client.1.smithi019.stdout:9/940: truncate d1/d7/d67/d77/faa 290740 0 2024-03-16T09:01:25.639 INFO:tasks.workunit.client.1.smithi019.stdout:9/941: dread - d1/d7/d67/f138 zero size 2024-03-16T09:01:25.639 INFO:tasks.workunit.client.1.smithi019.stdout:4/976: mkdir d4/d73/d144 0 2024-03-16T09:01:25.641 INFO:tasks.workunit.client.1.smithi019.stdout:9/942: rename d1/d7/de/d24/d44/d60/d74/f129 to d1/d7/de/d24/d44/d76/f13d 0 2024-03-16T09:01:25.643 INFO:tasks.workunit.client.1.smithi019.stdout:4/977: mknod d4/d6/d15/d9a/c145 0 2024-03-16T09:01:25.644 INFO:tasks.workunit.client.1.smithi019.stdout:0/991: dwrite d7/d8/d32/d66/d6d/d8c/db8/d82/f6f [0,4194304] 0 2024-03-16T09:01:25.645 INFO:tasks.workunit.client.0.smithi007.stdout:6/998: dwrite d2/d10/d3d/d6d/fa1 [0,4194304] 0 2024-03-16T09:01:25.645 INFO:tasks.workunit.client.0.smithi007.stdout:6/999: fdatasync d2/d13/d62/f13f 0 2024-03-16T09:01:25.646 INFO:tasks.workunit.client.1.smithi019.stdout:7/904: dwrite d9/d25/d28/d48/dfc/d13/d18/d30/d75/de1/f7a [0,4194304] 0 2024-03-16T09:01:25.646 INFO:tasks.workunit.client.1.smithi019.stdout:7/905: readlink d9/d25/d28/d48/dfc/d13/d18/d30/d3e/lcc 0 2024-03-16T09:01:25.646 INFO:tasks.workunit.client.1.smithi019.stdout:3/909: dwrite d3/d8b/dae/d5a/d75/d7a/dd0/f104 [0,4194304] 0 2024-03-16T09:01:25.646 INFO:tasks.workunit.client.1.smithi019.stdout:9/943: unlink d1/d7/de/da3/f139 0 2024-03-16T09:01:25.647 INFO:tasks.workunit.client.1.smithi019.stdout:4/978: rename d4/d1b/d49/d58/d80/d8e/lc2 to d4/d1b/d49/d58/d80/l146 0 2024-03-16T09:01:25.647 INFO:tasks.workunit.client.1.smithi019.stdout:4/979: dread - d4/d6/d15/d9a/f140 zero size 2024-03-16T09:01:25.647 INFO:tasks.workunit.client.1.smithi019.stdout:4/980: fsync d4/d6c/d89/fdf 0 2024-03-16T09:01:25.647 INFO:tasks.workunit.client.1.smithi019.stdout:4/981: readlink d4/d6/d15/d9a/l143 0 2024-03-16T09:01:25.648 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm -rf -- ./tmp.u3zwYJdfBv 2024-03-16T09:01:25.650 INFO:tasks.workunit.client.1.smithi019.stdout:1/925: sync 2024-03-16T09:01:25.650 INFO:tasks.workunit.client.1.smithi019.stdout:1/926: readlink d16/l59 0 2024-03-16T09:01:25.652 INFO:tasks.workunit.client.1.smithi019.stdout:7/906: creat d9/d25/d28/d48/dfc/d13/d18/d30/d3e/f11f x:0 0 0 2024-03-16T09:01:25.652 INFO:tasks.workunit.client.1.smithi019.stdout:7/907: chown d9/d25/d28/d48/dfc/c27 80 1 2024-03-16T09:01:25.656 INFO:tasks.workunit.client.1.smithi019.stdout:9/944: rename d1/d7/de/l5c to d1/d7/de/d24/d5b/dce/df8/df3/l13e 0 2024-03-16T09:01:25.658 INFO:tasks.workunit.client.1.smithi019.stdout:4/982: creat d4/d6c/d79/df7/f147 x:0 0 0 2024-03-16T09:01:25.659 INFO:tasks.workunit.client.1.smithi019.stdout:7/908: symlink d9/d25/d28/d48/dfc/d13/l120 0 2024-03-16T09:01:25.659 INFO:tasks.workunit.client.1.smithi019.stdout:7/909: fsync d9/d25/d28/d48/ddd/fd9 0 2024-03-16T09:01:25.659 INFO:tasks.workunit.client.1.smithi019.stdout:7/910: readlink d9/d25/d28/d48/dfc/d13/d18/d30/d75/de1/le4 0 2024-03-16T09:01:25.660 INFO:tasks.workunit.client.1.smithi019.stdout:9/945: creat d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/f13f x:0 0 0 2024-03-16T09:01:25.660 INFO:tasks.workunit.client.1.smithi019.stdout:9/946: stat d1/d7/de/d24/d5b/c93 0 2024-03-16T09:01:25.661 INFO:tasks.workunit.client.1.smithi019.stdout:0/992: dwrite d7/d8/dec/def/f5a [0,4194304] 0 2024-03-16T09:01:25.661 INFO:tasks.workunit.client.1.smithi019.stdout:0/993: fsync d7/d8/d32/d66/d6d/d8c/db8/d82/ff0 0 2024-03-16T09:01:25.662 INFO:tasks.workunit.client.1.smithi019.stdout:4/983: unlink d4/d73/d103/l36 0 2024-03-16T09:01:25.664 INFO:tasks.workunit.client.1.smithi019.stdout:7/911: mknod d9/d25/d28/d48/dfc/d13/d18/d30/d75/da4/c121 0 2024-03-16T09:01:25.664 INFO:tasks.workunit.client.1.smithi019.stdout:7/912: stat d9/d25/d28/d48/ddd/f114 0 2024-03-16T09:01:25.665 INFO:tasks.workunit.client.1.smithi019.stdout:9/947: creat d1/d7/de/d24/d5b/dce/df8/df3/d133/f140 x:0 0 0 2024-03-16T09:01:25.666 INFO:tasks.workunit.client.1.smithi019.stdout:0/994: symlink d7/d8/d32/df5/l13b 0 2024-03-16T09:01:25.666 INFO:tasks.workunit.client.1.smithi019.stdout:0/995: stat d7/fe4 0 2024-03-16T09:01:25.667 INFO:tasks.workunit.client.1.smithi019.stdout:9/948: creat d1/d7/de/d24/d5b/dce/df8/dc9/dca/f141 x:0 0 0 2024-03-16T09:01:25.668 INFO:tasks.workunit.client.1.smithi019.stdout:3/910: dwrite d3/d89/fcb [0,4194304] 0 2024-03-16T09:01:25.669 INFO:tasks.workunit.client.1.smithi019.stdout:0/996: mkdir d7/d1d/d63/d89/d9e/d13c 0 2024-03-16T09:01:25.669 INFO:tasks.workunit.client.1.smithi019.stdout:0/997: creat d7/d8/d32/f13d x:0 0 0 2024-03-16T09:01:25.670 INFO:tasks.workunit.client.1.smithi019.stdout:9/949: symlink d1/d7/de/l142 0 2024-03-16T09:01:25.674 INFO:tasks.workunit.client.1.smithi019.stdout:4/984: write d4/d73/d103/d4f/f8b [699842,38345] 0 2024-03-16T09:01:25.677 INFO:tasks.workunit.client.1.smithi019.stdout:0/998: dread d7/d1d/d2e/fc5 [0,4194304] 0 2024-03-16T09:01:25.677 INFO:tasks.workunit.client.1.smithi019.stdout:9/950: mknod d1/d7/de/c143 0 2024-03-16T09:01:25.680 INFO:tasks.workunit.client.1.smithi019.stdout:0/999: rename d7/d8/d32/fa5 to d7/d1d/d63/d89/d9e/d121/f13e 0 2024-03-16T09:01:25.681 INFO:tasks.workunit.client.1.smithi019.stdout:9/951: creat d1/d7/de/d24/d5b/dce/df8/d59/f144 x:0 0 0 2024-03-16T09:01:25.686 INFO:tasks.workunit.client.1.smithi019.stdout:3/911: write d3/d89/fcb [4797245,112713] 0 2024-03-16T09:01:25.686 INFO:tasks.workunit.client.1.smithi019.stdout:3/912: fsync d3/d1b/d88/fa5 0 2024-03-16T09:01:25.687 INFO:tasks.workunit.client.1.smithi019.stdout:5/955: sync 2024-03-16T09:01:25.687 INFO:tasks.workunit.client.1.smithi019.stdout:5/956: dread - d6/dd/d47/fff zero size 2024-03-16T09:01:25.688 INFO:tasks.workunit.client.1.smithi019.stdout:5/957: getdents d6/dd/d37/d76/d107 0 2024-03-16T09:01:25.688 INFO:tasks.workunit.client.1.smithi019.stdout:5/958: readlink d6/d19/d34/dc2/lc9 0 2024-03-16T09:01:25.688 INFO:tasks.workunit.client.1.smithi019.stdout:5/959: fsync d6/dd/d37/fb4 0 2024-03-16T09:01:25.689 INFO:tasks.workunit.client.1.smithi019.stdout:7/913: dwrite d9/d25/d28/d48/dfc/d13/d9a/dab/ff5 [0,4194304] 0 2024-03-16T09:01:25.689 INFO:tasks.workunit.client.1.smithi019.stdout:7/914: write d9/d25/d28/d48/dfc/d13/d9a/dab/ff6 [270469,117764] 0 2024-03-16T09:01:25.692 INFO:tasks.workunit.client.1.smithi019.stdout:7/915: symlink d9/d25/d28/d48/dfc/d13/d18/d30/d75/de1/l122 0 2024-03-16T09:01:25.692 INFO:tasks.workunit.client.1.smithi019.stdout:7/916: fsync d9/d25/d28/d48/dfc/d13/d9a/f77 0 2024-03-16T09:01:25.700 INFO:tasks.workunit.client.1.smithi019.stdout:5/960: write d6/f35 [1238779,1951] 0 2024-03-16T09:01:25.700 INFO:tasks.workunit.client.1.smithi019.stdout:5/961: chown d6/dd/dde/df1/f111 0 1 2024-03-16T09:01:25.702 INFO:tasks.workunit.client.1.smithi019.stdout:5/962: mknod d6/dd/d47/d70/dfb/c145 0 2024-03-16T09:01:25.704 INFO:tasks.workunit.client.1.smithi019.stdout:5/963: mknod d6/dd/d47/d70/db2/d4c/c146 0 2024-03-16T09:01:25.705 INFO:tasks.workunit.client.1.smithi019.stdout:5/964: write d6/d19/f9b [2559373,34470] 0 2024-03-16T09:01:25.705 INFO:tasks.workunit.client.1.smithi019.stdout:5/965: rmdir d6/dd/d47/d70/db2/d89 39 2024-03-16T09:01:25.705 INFO:tasks.workunit.client.1.smithi019.stdout:5/966: mkdir d6/dd/dde/df1/d147 0 2024-03-16T09:01:25.705 INFO:tasks.workunit.client.1.smithi019.stdout:3/913: dwrite d3/d8b/dae/d3f/f77 [0,4194304] 0 2024-03-16T09:01:25.708 INFO:tasks.workunit.client.1.smithi019.stdout:3/914: link d3/d8b/dae/lab d3/d8b/dca/l129 0 2024-03-16T09:01:25.708 INFO:tasks.workunit.client.1.smithi019.stdout:7/917: dwrite d9/f3f [4194304,4194304] 0 2024-03-16T09:01:25.710 INFO:tasks.workunit.client.1.smithi019.stdout:3/915: rename d3/d8b/dae/d5a/d75/d7a/f87 to d3/d8b/dae/f12a 0 2024-03-16T09:01:25.712 INFO:tasks.workunit.client.1.smithi019.stdout:5/967: dread d6/dd/d47/d7a/db9/fbb [0,4194304] 0 2024-03-16T09:01:25.713 INFO:tasks.workunit.client.1.smithi019.stdout:3/916: rename d3/d8b/cce to d3/d11/c12b 0 2024-03-16T09:01:25.715 INFO:tasks.workunit.client.1.smithi019.stdout:5/968: mknod d6/dd/d47/d70/db2/d89/c148 0 2024-03-16T09:01:25.722 INFO:tasks.workunit.client.1.smithi019.stdout:3/917: write d3/d8b/da8/dcc/fec [3619222,12074] 0 2024-03-16T09:01:25.724 INFO:tasks.workunit.client.1.smithi019.stdout:3/918: mknod d3/d1b/d88/c12c 0 2024-03-16T09:01:25.724 INFO:tasks.workunit.client.1.smithi019.stdout:3/919: chown d3/d8b/dae/d5a/d80/lf7 131375100 1 2024-03-16T09:01:25.724 INFO:tasks.workunit.client.1.smithi019.stdout:7/918: dwrite f4 [4194304,4194304] 0 2024-03-16T09:01:25.725 INFO:tasks.workunit.client.1.smithi019.stdout:3/920: truncate d3/d8b/dae/f12a 3009126 0 2024-03-16T09:01:25.733 INFO:tasks.workunit.client.1.smithi019.stdout:5/969: dwrite d6/dd/dde/df1/f111 [0,4194304] 0 2024-03-16T09:01:25.733 INFO:tasks.workunit.client.1.smithi019.stdout:5/970: chown d6/dd/d47/d70/db2/d89/cef 23736 1 2024-03-16T09:01:25.733 INFO:tasks.workunit.client.1.smithi019.stdout:5/971: fsync d6/dd/d47/d70/db2/d66/fd1 0 2024-03-16T09:01:25.733 INFO:tasks.workunit.client.1.smithi019.stdout:5/972: readlink d6/dd/d37/d54/l68 0 2024-03-16T09:01:25.734 INFO:tasks.workunit.client.1.smithi019.stdout:7/919: dread d9/d25/d28/d48/dfc/d13/d9a/dab/f6b [0,4194304] 0 2024-03-16T09:01:25.738 INFO:tasks.workunit.client.1.smithi019.stdout:5/973: symlink d6/dd/d47/d56/d87/l149 0 2024-03-16T09:01:25.767 INFO:tasks.workunit.client.1.smithi019.stdout:7/920: dwrite d9/d25/d28/f60 [0,4194304] 0 2024-03-16T09:01:25.767 INFO:tasks.workunit.client.1.smithi019.stdout:5/974: dwrite d6/dd/d37/d54/f5f [0,4194304] 0 2024-03-16T09:01:25.767 INFO:tasks.workunit.client.1.smithi019.stdout:3/921: dwrite d3/d8b/dae/f50 [0,4194304] 0 2024-03-16T09:01:25.767 INFO:tasks.workunit.client.1.smithi019.stdout:3/922: creat d3/d8b/dae/d5a/d75/d7a/dd0/f12d x:0 0 0 2024-03-16T09:01:25.768 INFO:tasks.workunit.client.1.smithi019.stdout:3/923: chown d3/d8b/fa3 3 1 2024-03-16T09:01:25.770 INFO:tasks.workunit.client.1.smithi019.stdout:7/921: unlink d9/fe2 0 2024-03-16T09:01:25.772 INFO:tasks.workunit.client.1.smithi019.stdout:5/975: symlink d6/dd/daf/l14a 0 2024-03-16T09:01:25.775 INFO:tasks.workunit.client.1.smithi019.stdout:3/924: truncate d3/d1b/d40/dc3/fea 2781007 0 2024-03-16T09:01:25.776 INFO:tasks.workunit.client.1.smithi019.stdout:7/922: rmdir d9/d25/d28/d48/d8d/dba/dad/dd8 39 2024-03-16T09:01:25.780 INFO:tasks.workunit.client.1.smithi019.stdout:7/923: rename d9/l3a to d9/d25/d28/d48/d8d/l123 0 2024-03-16T09:01:25.784 INFO:tasks.workunit.client.1.smithi019.stdout:5/976: write d6/d1f/d78/f9a [4163779,79919] 0 2024-03-16T09:01:25.785 INFO:tasks.workunit.client.1.smithi019.stdout:7/924: symlink d9/d25/d28/d48/dfc/d13/l124 0 2024-03-16T09:01:25.786 INFO:tasks.workunit.client.1.smithi019.stdout:1/927: sync 2024-03-16T09:01:25.787 INFO:tasks.workunit.client.1.smithi019.stdout:1/928: truncate d16/d1f/f27 2410802 0 2024-03-16T09:01:25.790 INFO:tasks.workunit.client.1.smithi019.stdout:1/929: truncate d16/da1/fc1 827341 0 2024-03-16T09:01:25.790 INFO:tasks.workunit.client.1.smithi019.stdout:7/925: dread f8 [0,4194304] 0 2024-03-16T09:01:25.790 INFO:tasks.workunit.client.1.smithi019.stdout:7/926: write d9/d25/d28/f57 [804521,76167] 0 2024-03-16T09:01:25.793 INFO:tasks.workunit.client.1.smithi019.stdout:5/977: dread d6/f35 [0,4194304] 0 2024-03-16T09:01:25.794 INFO:tasks.workunit.client.1.smithi019.stdout:5/978: mknod d6/dd/d47/d56/c14b 0 2024-03-16T09:01:25.797 INFO:tasks.workunit.client.1.smithi019.stdout:5/979: getdents d6/dd/d47/d7a/db9 0 2024-03-16T09:01:25.800 INFO:tasks.workunit.client.1.smithi019.stdout:3/925: dwrite d3/d8b/dae/d5a/d80/ffe [0,4194304] 0 2024-03-16T09:01:25.800 INFO:tasks.workunit.client.1.smithi019.stdout:3/926: chown d3/d8b/dae/lab 7753 1 2024-03-16T09:01:25.803 INFO:tasks.workunit.client.1.smithi019.stdout:3/927: readlink d3/d1b/l4d 0 2024-03-16T09:01:25.806 INFO:tasks.workunit.client.1.smithi019.stdout:3/928: link d3/d11/c58 d3/d1b/d40/d4a/c12e 0 2024-03-16T09:01:25.806 INFO:tasks.workunit.client.1.smithi019.stdout:3/929: truncate d3/d1b/dc7/ff1 499478 0 2024-03-16T09:01:25.807 INFO:tasks.workunit.client.1.smithi019.stdout:3/930: creat d3/d1b/f12f x:0 0 0 2024-03-16T09:01:25.807 INFO:tasks.workunit.client.1.smithi019.stdout:3/931: creat d3/d1b/d40/de9/f130 x:0 0 0 2024-03-16T09:01:25.807 INFO:tasks.workunit.client.1.smithi019.stdout:3/932: dread - d3/d8b/fa6 zero size 2024-03-16T09:01:25.807 INFO:tasks.workunit.client.1.smithi019.stdout:3/933: write d3/d1b/d40/fc6 [991244,121009] 0 2024-03-16T09:01:25.808 INFO:tasks.workunit.client.1.smithi019.stdout:3/934: fdatasync d3/d8b/dae/d5a/d75/fde 0 2024-03-16T09:01:25.809 INFO:tasks.workunit.client.1.smithi019.stdout:1/930: dwrite d16/d1f/f119 [0,4194304] 0 2024-03-16T09:01:25.809 INFO:tasks.workunit.client.1.smithi019.stdout:1/931: chown d16/dcb/daf/db2/d111 69283 1 2024-03-16T09:01:25.816 INFO:tasks.workunit.client.1.smithi019.stdout:5/980: dwrite d6/dd/f1d [4194304,4194304] 0 2024-03-16T09:01:25.819 INFO:tasks.workunit.client.1.smithi019.stdout:5/981: unlink d6/dd/d47/d70/db2/d89/d9e/f138 0 2024-03-16T09:01:25.819 INFO:tasks.workunit.client.1.smithi019.stdout:5/982: creat d6/dd/d47/d70/dfb/f14c x:0 0 0 2024-03-16T09:01:25.822 INFO:tasks.workunit.client.1.smithi019.stdout:3/935: read d3/d8b/dae/d3f/f77 [3170496,120007] 0 2024-03-16T09:01:25.822 INFO:tasks.workunit.client.1.smithi019.stdout:4/985: sync 2024-03-16T09:01:25.823 INFO:tasks.workunit.client.1.smithi019.stdout:9/952: sync 2024-03-16T09:01:25.823 INFO:tasks.workunit.client.1.smithi019.stdout:9/953: chown d1/d7/de/d24/d44/d76/d9c 2022087 1 2024-03-16T09:01:25.824 INFO:tasks.workunit.client.1.smithi019.stdout:4/986: rename d4/d1b/d49/d58/cbf to d4/d73/d103/c148 0 2024-03-16T09:01:25.826 INFO:tasks.workunit.client.1.smithi019.stdout:9/954: unlink d1/d7/de/d24/d44/d60/d74/d101/df0/d12b/d128/l12d 0 2024-03-16T09:01:25.826 INFO:tasks.workunit.client.1.smithi019.stdout:9/955: fsync d1/d7/f3e 0 2024-03-16T09:01:25.826 INFO:tasks.workunit.client.1.smithi019.stdout:9/956: creat d1/d7/de/d24/d5b/dce/df8/dc9/dca/f145 x:0 0 0 2024-03-16T09:01:25.829 INFO:tasks.workunit.client.1.smithi019.stdout:4/987: unlink f2 0 2024-03-16T09:01:25.829 INFO:tasks.workunit.client.1.smithi019.stdout:1/932: dwrite d16/d31/d76/d92/ffa [0,4194304] 0 2024-03-16T09:01:25.830 INFO:tasks.workunit.client.1.smithi019.stdout:1/933: fdatasync f14 0 2024-03-16T09:01:25.830 INFO:tasks.workunit.client.1.smithi019.stdout:9/957: unlink d1/d7/de/d24/d44/d76/l49 0 2024-03-16T09:01:25.830 INFO:tasks.workunit.client.1.smithi019.stdout:9/958: write d1/d7/f3e [839438,22875] 0 2024-03-16T09:01:25.834 INFO:tasks.workunit.client.1.smithi019.stdout:1/934: creat d16/d1f/d43/d55/f11d x:0 0 0 2024-03-16T09:01:25.835 INFO:tasks.workunit.client.1.smithi019.stdout:9/959: rename d1/d7/de/d24/d44/d60/d8b/dc2/ccd to d1/d7/de/d24/d5b/dce/df8/dc9/dca/c146 0 2024-03-16T09:01:25.837 INFO:tasks.workunit.client.1.smithi019.stdout:4/988: rmdir d4/d73/d8a/dd6/d10c 39 2024-03-16T09:01:25.837 INFO:tasks.workunit.client.1.smithi019.stdout:4/989: readlink d4/d73/d103/d4f/le1 0 2024-03-16T09:01:25.838 INFO:tasks.workunit.client.1.smithi019.stdout:3/936: write d3/d8b/dae/d5a/f61 [3600464,95580] 0 2024-03-16T09:01:25.839 INFO:tasks.workunit.client.1.smithi019.stdout:4/990: creat d4/d64/d99/d13d/f149 x:0 0 0 2024-03-16T09:01:25.839 INFO:tasks.workunit.client.1.smithi019.stdout:4/991: write d4/d73/d103/d4f/fcf [1090734,121736] 0 2024-03-16T09:01:25.842 INFO:tasks.workunit.client.1.smithi019.stdout:4/992: unlink d4/d73/d8a/d128/de5/l116 0 2024-03-16T09:01:25.845 INFO:tasks.workunit.client.1.smithi019.stdout:4/993: getdents d4/d6/d15 0 2024-03-16T09:01:25.846 INFO:tasks.workunit.client.1.smithi019.stdout:5/983: dwrite d6/dd/d47/d7a/d105/f12f [0,4194304] 0 2024-03-16T09:01:25.849 INFO:tasks.workunit.client.1.smithi019.stdout:5/984: creat d6/dd/d47/d7a/db9/d112/d132/f14d x:0 0 0 2024-03-16T09:01:25.849 INFO:tasks.workunit.client.1.smithi019.stdout:5/985: dread - d6/d1f/d77/d10a/d11c/f120 zero size 2024-03-16T09:01:25.849 INFO:tasks.workunit.client.1.smithi019.stdout:5/986: dread - d6/dd/d47/d70/dfb/f14c zero size 2024-03-16T09:01:25.849 INFO:tasks.workunit.client.1.smithi019.stdout:5/987: fdatasync d6/d19/f4e 0 2024-03-16T09:01:25.849 INFO:tasks.workunit.client.1.smithi019.stdout:5/988: truncate d6/dd/d47/d70/db2/d89/fa7 927363 0 2024-03-16T09:01:25.851 INFO:tasks.workunit.client.1.smithi019.stdout:5/989: unlink d6/d19/f9b 0 2024-03-16T09:01:25.851 INFO:tasks.workunit.client.1.smithi019.stdout:3/937: dread d3/d8b/dae/d3f/d111/f118 [0,4194304] 0 2024-03-16T09:01:25.852 INFO:tasks.workunit.client.1.smithi019.stdout:3/938: symlink d3/d8b/dae/d6a/l131 0 2024-03-16T09:01:25.853 INFO:tasks.workunit.client.1.smithi019.stdout:3/939: truncate d3/d89/fcb 4785488 0 2024-03-16T09:01:25.853 INFO:tasks.workunit.client.1.smithi019.stdout:3/940: fdatasync d3/d11/f127 0 2024-03-16T09:01:25.853 INFO:tasks.workunit.client.1.smithi019.stdout:3/941: stat d3/d11/f11a 0 2024-03-16T09:01:25.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/990: write d6/f8 [641135,41159] 0 2024-03-16T09:01:25.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/991: write d6/d1f/f5a [4854999,81042] 0 2024-03-16T09:01:25.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/992: chown d6/dd/d47/d56/d87/l149 0 1 2024-03-16T09:01:25.863 INFO:tasks.workunit.client.1.smithi019.stdout:5/993: fsync d6/d1f/d78/fe6 0 2024-03-16T09:01:25.865 INFO:tasks.workunit.client.1.smithi019.stdout:5/994: mkdir d6/d19/d34/dc2/d14e 0 2024-03-16T09:01:25.866 INFO:tasks.workunit.client.1.smithi019.stdout:1/935: dwrite d16/d31/fe1 [0,4194304] 0 2024-03-16T09:01:25.868 INFO:tasks.workunit.client.1.smithi019.stdout:3/942: write d3/d8b/dae/f50 [1913726,127597] 0 2024-03-16T09:01:25.868 INFO:tasks.workunit.client.1.smithi019.stdout:3/943: write d3/d8b/fa6 [967029,65975] 0 2024-03-16T09:01:25.876 INFO:tasks.workunit.client.1.smithi019.stdout:3/944: mknod d3/d8b/dae/d5a/d75/c132 0 2024-03-16T09:01:25.877 INFO:tasks.workunit.client.1.smithi019.stdout:3/945: chown d3/d9b/fb6 39318879 1 2024-03-16T09:01:25.877 INFO:tasks.workunit.client.1.smithi019.stdout:5/995: dread d6/d19/f29 [4194304,4194304] 0 2024-03-16T09:01:25.877 INFO:tasks.workunit.client.1.smithi019.stdout:5/996: getdents d6/d1f/d77 0 2024-03-16T09:01:25.878 INFO:tasks.workunit.client.1.smithi019.stdout:3/946: unlink d3/d8b/dae/d6a/c90 0 2024-03-16T09:01:25.881 INFO:tasks.workunit.client.1.smithi019.stdout:3/947: mknod d3/d8b/dae/c133 0 2024-03-16T09:01:25.881 INFO:tasks.workunit.client.1.smithi019.stdout:3/948: write d3/d1b/d40/de9/f130 [137562,48904] 0 2024-03-16T09:01:25.887 INFO:tasks.workunit.client.1.smithi019.stdout:5/997: write d6/d1f/fa4 [3970620,67656] 0 2024-03-16T09:01:25.888 INFO:tasks.workunit.client.1.smithi019.stdout:5/998: write d6/dd/d47/d70/db2/ffa [1868775,10185] 0 2024-03-16T09:01:25.888 INFO:tasks.workunit.client.1.smithi019.stdout:1/936: dwrite d16/d1f/d4f/f58 [0,4194304] 0 2024-03-16T09:01:25.890 INFO:tasks.workunit.client.1.smithi019.stdout:3/949: mknod d3/d8b/dae/d5a/c134 0 2024-03-16T09:01:25.892 INFO:tasks.workunit.client.1.smithi019.stdout:5/999: creat d6/dd/d37/d76/f14f x:0 0 0 2024-03-16T09:01:25.893 INFO:tasks.workunit.client.1.smithi019.stdout:1/937: truncate d16/f9c 200871 0 2024-03-16T09:01:25.896 INFO:tasks.workunit.client.1.smithi019.stdout:3/950: symlink d3/d89/l135 0 2024-03-16T09:01:25.896 INFO:tasks.workunit.client.1.smithi019.stdout:3/951: truncate d3/d8b/dae/d5a/d75/d7a/dd0/f12d 637854 0 2024-03-16T09:01:25.896 INFO:tasks.workunit.client.1.smithi019.stdout:3/952: chown d3/d8b/dae/d3f/d111/l113 4 1 2024-03-16T09:01:25.898 INFO:tasks.workunit.client.1.smithi019.stdout:1/938: unlink d16/d31/d76/d92/l11c 0 2024-03-16T09:01:25.898 INFO:tasks.workunit.client.1.smithi019.stdout:1/939: readlink d16/ld1 0 2024-03-16T09:01:25.899 INFO:tasks.workunit.client.1.smithi019.stdout:1/940: write d16/d1f/f34 [2302982,92384] 0 2024-03-16T09:01:25.899 INFO:tasks.workunit.client.1.smithi019.stdout:1/941: write d16/d31/d76/f6b [4967486,27363] 0 2024-03-16T09:01:25.901 INFO:tasks.workunit.client.1.smithi019.stdout:3/953: link d3/d8b/dae/l96 d3/d8b/dae/d5a/l136 0 2024-03-16T09:01:25.903 INFO:tasks.workunit.client.1.smithi019.stdout:3/954: symlink d3/d8b/da8/dcc/l137 0 2024-03-16T09:01:25.907 INFO:tasks.workunit.client.1.smithi019.stdout:1/942: write d16/d31/d76/d92/ffa [3168663,58896] 0 2024-03-16T09:01:25.907 INFO:tasks.workunit.client.1.smithi019.stdout:3/955: symlink d3/d11/d100/l138 0 2024-03-16T09:01:25.908 INFO:tasks.workunit.client.1.smithi019.stdout:3/956: readlink d3/d8b/dae/l8a 0 2024-03-16T09:01:25.908 INFO:tasks.workunit.client.1.smithi019.stdout:1/943: readlink d16/d31/d71/d89/l10c 0 2024-03-16T09:01:25.909 INFO:tasks.workunit.client.1.smithi019.stdout:1/944: mknod d16/dcb/c11e 0 2024-03-16T09:01:25.918 INFO:tasks.workunit.client.1.smithi019.stdout:1/945: write d16/d18/f1b [2103534,59380] 0 2024-03-16T09:01:25.919 INFO:tasks.workunit.client.1.smithi019.stdout:1/946: write d16/f65 [1760415,10364] 0 2024-03-16T09:01:25.919 INFO:tasks.workunit.client.1.smithi019.stdout:1/947: chown d16/d31/d7b 451809 1 2024-03-16T09:01:25.924 INFO:tasks.workunit.client.1.smithi019.stdout:3/957: dwrite d3/d8b/dae/d5a/d75/d7a/dd0/f104 [0,4194304] 0 2024-03-16T09:01:25.927 INFO:tasks.workunit.client.1.smithi019.stdout:1/948: rename d16/d31/d71/df0 to d16/d1f/d73/ddf/d10e/d96/d11f 0 2024-03-16T09:01:25.938 INFO:tasks.workunit.client.1.smithi019.stdout:3/958: mknod d3/d9b/c139 0 2024-03-16T09:01:25.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:25 smithi019 ceph-mon[36407]: pgmap v26: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 299 MiB/s rd, 364 MiB/s wr, 483 op/s 2024-03-16T09:01:25.963 INFO:tasks.workunit.client.1.smithi019.stdout:3/959: dwrite d3/d1b/d40/d4a/db7/d110/f11c [0,4194304] 0 2024-03-16T09:01:25.963 INFO:tasks.workunit.client.1.smithi019.stdout:3/960: write d3/d1b/d40/d4a/f98 [2792095,29696] 0 2024-03-16T09:01:25.970 INFO:tasks.workunit.client.1.smithi019.stdout:1/949: dwrite d16/d1f/d43/f70 [0,4194304] 0 2024-03-16T09:01:25.975 INFO:tasks.workunit.client.1.smithi019.stdout:1/950: rename d16/d1f/c3e to d16/dcb/daf/db2/d78/db1/df9/c120 0 2024-03-16T09:01:25.976 INFO:tasks.workunit.client.1.smithi019.stdout:1/951: stat d16/c35 0 2024-03-16T09:01:25.976 INFO:tasks.workunit.client.1.smithi019.stdout:1/952: chown d16/d1f/d43/l52 0 1 2024-03-16T09:01:25.976 INFO:tasks.workunit.client.1.smithi019.stdout:1/953: mknod d16/dcb/daf/db2/d78/db1/c121 0 2024-03-16T09:01:25.978 INFO:tasks.workunit.client.1.smithi019.stdout:1/954: creat d16/dcb/daf/db2/d111/f122 x:0 0 0 2024-03-16T09:01:25.979 INFO:tasks.workunit.client.1.smithi019.stdout:3/961: dwrite d3/d8b/dae/d6a/ffb [0,4194304] 0 2024-03-16T09:01:25.979 INFO:tasks.workunit.client.1.smithi019.stdout:3/962: fdatasync d3/d1b/f12f 0 2024-03-16T09:01:25.980 INFO:tasks.workunit.client.1.smithi019.stdout:1/955: mkdir d16/d18/d123 0 2024-03-16T09:01:25.983 INFO:tasks.workunit.client.1.smithi019.stdout:3/963: mkdir d3/d8b/dae/dd6/d13a 0 2024-03-16T09:01:25.983 INFO:tasks.workunit.client.1.smithi019.stdout:3/964: chown d3/d11/l39 345 1 2024-03-16T09:01:25.984 INFO:tasks.workunit.client.1.smithi019.stdout:3/965: symlink d3/d1b/d40/d4a/l13b 0 2024-03-16T09:01:25.989 INFO:tasks.workunit.client.1.smithi019.stdout:1/956: dread fb [0,4194304] 0 2024-03-16T09:01:25.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:25 smithi007 ceph-mon[29470]: pgmap v26: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 299 MiB/s rd, 364 MiB/s wr, 483 op/s 2024-03-16T09:01:25.998 INFO:tasks.workunit.client.1.smithi019.stdout:1/957: read d16/f3d [254119,93743] 0 2024-03-16T09:01:26.006 INFO:tasks.workunit.client.1.smithi019.stdout:1/958: write d16/d1f/d43/f6f [456858,99566] 0 2024-03-16T09:01:26.006 INFO:tasks.workunit.client.1.smithi019.stdout:1/959: write d16/d84/fbf [813072,83266] 0 2024-03-16T09:01:26.007 INFO:tasks.workunit.client.1.smithi019.stdout:1/960: dread d16/f9c [0,4194304] 0 2024-03-16T09:01:26.007 INFO:tasks.workunit.client.1.smithi019.stdout:1/961: write d16/f22 [37721,67499] 0 2024-03-16T09:01:26.008 INFO:tasks.workunit.client.1.smithi019.stdout:1/962: symlink d16/d18/df4/l124 0 2024-03-16T09:01:26.021 INFO:tasks.workunit.client.1.smithi019.stdout:1/963: dwrite d16/d1f/d73/ddf/d10e/d96/dac/fc5 [0,4194304] 0 2024-03-16T09:01:26.021 INFO:tasks.workunit.client.1.smithi019.stdout:1/964: readlink d16/d31/d71/d89/dce/lda 0 2024-03-16T09:01:26.023 INFO:tasks.workunit.client.1.smithi019.stdout:1/965: rename d16/dcb/daf/db2/d111/f122 to d16/d1f/d73/ddf/f125 0 2024-03-16T09:01:26.025 INFO:tasks.workunit.client.1.smithi019.stdout:1/966: rename d16/d31/d76/l82 to d16/dcb/dd4/l126 0 2024-03-16T09:01:26.025 INFO:tasks.workunit.client.1.smithi019.stdout:1/967: read d16/d1f/d73/fa7 [114034,35156] 0 2024-03-16T09:01:26.025 INFO:tasks.workunit.client.1.smithi019.stdout:1/968: chown d16/d31/d7b/l81 0 1 2024-03-16T09:01:26.037 INFO:tasks.workunit.client.1.smithi019.stdout:7/927: sync 2024-03-16T09:01:26.039 INFO:tasks.workunit.client.1.smithi019.stdout:7/928: unlink d9/db/f87 0 2024-03-16T09:01:26.040 INFO:tasks.workunit.client.1.smithi019.stdout:7/929: link d9/d25/d28/d48/l81 d9/d25/d28/d48/dfc/d13/d9a/dab/l125 0 2024-03-16T09:01:26.040 INFO:tasks.workunit.client.1.smithi019.stdout:7/930: stat d9/db/d12/d5b/dc3/dec/f104 0 2024-03-16T09:01:26.041 INFO:tasks.workunit.client.1.smithi019.stdout:7/931: readlink d9/d25/d28/d48/dfc/d13/d18/d30/d9e/lf9 0 2024-03-16T09:01:26.041 INFO:tasks.workunit.client.1.smithi019.stdout:7/932: write d9/db/f2c [1924358,98646] 0 2024-03-16T09:01:26.042 INFO:tasks.workunit.client.1.smithi019.stdout:7/933: mkdir d9/db/d102/d113/d126 0 2024-03-16T09:01:26.044 INFO:tasks.workunit.client.1.smithi019.stdout:1/969: dwrite d16/f3d [0,4194304] 0 2024-03-16T09:01:26.045 INFO:tasks.workunit.client.1.smithi019.stdout:7/934: link d9/d25/d28/d48/dfc/d13/d9a/ccb d9/db/d102/d113/d126/c127 0 2024-03-16T09:01:26.048 INFO:tasks.workunit.client.1.smithi019.stdout:7/935: rename d9/faa to d9/d25/d28/d48/d8d/dba/dad/dd8/f128 0 2024-03-16T09:01:26.048 INFO:tasks.workunit.client.1.smithi019.stdout:7/936: creat d9/db/d12/d5b/f129 x:0 0 0 2024-03-16T09:01:26.054 INFO:tasks.workunit.client.1.smithi019.stdout:7/937: unlink d9/d25/d28/d48/dfc/d13/d18/d30/d75/de1/la2 0 2024-03-16T09:01:26.054 INFO:tasks.workunit.client.1.smithi019.stdout:1/970: dread d16/d1f/f2e [0,4194304] 0 2024-03-16T09:01:26.054 INFO:tasks.workunit.client.1.smithi019.stdout:1/971: creat d16/dcb/daf/db2/d111/f127 x:0 0 0 2024-03-16T09:01:26.054 INFO:tasks.workunit.client.1.smithi019.stdout:1/972: fsync d16/f42 0 2024-03-16T09:01:26.055 INFO:tasks.workunit.client.1.smithi019.stdout:7/938: mkdir d9/d25/d28/d48/d8d/dba/dad/dd8/d12a 0 2024-03-16T09:01:26.055 INFO:tasks.workunit.client.1.smithi019.stdout:7/939: chown d9/d25/d93/cd1 21639349 1 2024-03-16T09:01:26.057 INFO:tasks.workunit.client.1.smithi019.stdout:1/973: creat d16/d1f/d4f/f128 x:0 0 0 2024-03-16T09:01:26.062 INFO:tasks.workunit.client.1.smithi019.stdout:7/940: read d9/d25/d28/d48/dfc/d13/d18/d30/f97 [311812,50364] 0 2024-03-16T09:01:26.062 INFO:tasks.workunit.client.1.smithi019.stdout:7/941: readlink d9/d25/d28/d48/l81 0 2024-03-16T09:01:26.063 INFO:tasks.workunit.client.1.smithi019.stdout:7/942: fdatasync d9/db/d12/d5b/ff8 0 2024-03-16T09:01:26.063 INFO:tasks.workunit.client.1.smithi019.stdout:7/943: unlink d9/db/l89 0 2024-03-16T09:01:26.063 INFO:tasks.workunit.client.1.smithi019.stdout:7/944: write d9/d25/d28/d48/dfc/d13/d9a/dab/f6b [839504,123270] 0 2024-03-16T09:01:26.064 INFO:tasks.workunit.client.1.smithi019.stdout:1/974: dread d16/d31/f69 [0,4194304] 0 2024-03-16T09:01:26.073 INFO:tasks.workunit.client.1.smithi019.stdout:7/945: dread d9/db/d12/f55 [0,4194304] 0 2024-03-16T09:01:26.077 INFO:tasks.workunit.client.1.smithi019.stdout:1/975: write d16/d1f/f119 [4053047,90294] 0 2024-03-16T09:01:26.078 INFO:tasks.workunit.client.1.smithi019.stdout:1/976: fsync d16/d31/d71/f8d 0 2024-03-16T09:01:26.079 INFO:tasks.workunit.client.1.smithi019.stdout:1/977: mknod d16/dcb/c129 0 2024-03-16T09:01:26.081 INFO:tasks.workunit.client.1.smithi019.stdout:1/978: link c7 d16/d1f/d73/ddf/d10e/d96/d11f/c12a 0 2024-03-16T09:01:26.081 INFO:tasks.workunit.client.1.smithi019.stdout:1/979: write d16/d1f/d73/ddf/d10e/d96/dac/f112 [809252,69799] 0 2024-03-16T09:01:26.083 INFO:tasks.workunit.client.1.smithi019.stdout:1/980: dread d16/d31/d76/f95 [0,4194304] 0 2024-03-16T09:01:26.084 INFO:tasks.workunit.client.1.smithi019.stdout:1/981: creat d16/d1f/f12b x:0 0 0 2024-03-16T09:01:26.085 INFO:tasks.workunit.client.1.smithi019.stdout:1/982: mkdir d16/dcb/daf/db2/d111/d12c 0 2024-03-16T09:01:26.085 INFO:tasks.workunit.client.1.smithi019.stdout:1/983: creat d16/d84/f12d x:0 0 0 2024-03-16T09:01:26.088 INFO:tasks.workunit.client.1.smithi019.stdout:7/946: dwrite d9/d25/d28/d48/dfc/d13/d18/d30/d9e/dd7/fe6 [0,4194304] 0 2024-03-16T09:01:26.092 INFO:tasks.workunit.client.1.smithi019.stdout:7/947: unlink d9/d25/d28/d48/dfc/d13/d18/d30/d75/de1/ce3 0 2024-03-16T09:01:26.092 INFO:tasks.workunit.client.1.smithi019.stdout:7/948: creat d9/d25/d28/d48/d8d/dba/dad/f12b x:0 0 0 2024-03-16T09:01:26.094 INFO:tasks.workunit.client.1.smithi019.stdout:1/984: write d16/d1f/d43/f70 [3248462,89316] 0 2024-03-16T09:01:26.094 INFO:tasks.workunit.client.1.smithi019.stdout:1/985: fdatasync d16/d31/d76/f95 0 2024-03-16T09:01:26.112 INFO:tasks.workunit.client.1.smithi019.stdout:7/949: dwrite d9/d25/d28/d48/ff7 [0,4194304] 0 2024-03-16T09:01:26.112 INFO:tasks.workunit.client.1.smithi019.stdout:1/986: dwrite d16/f42 [0,4194304] 0 2024-03-16T09:01:26.115 INFO:tasks.workunit.client.1.smithi019.stdout:7/950: link d9/d25/d28/d48/d8d/dba/c96 d9/d25/d28/d48/dfc/d13/d18/d30/d3e/da5/c12c 0 2024-03-16T09:01:26.116 INFO:tasks.workunit.client.1.smithi019.stdout:1/987: symlink d16/d31/d71/l12e 0 2024-03-16T09:01:26.118 INFO:tasks.workunit.client.1.smithi019.stdout:7/951: truncate d9/d25/d28/d48/ffb 527099 0 2024-03-16T09:01:26.119 INFO:tasks.workunit.client.1.smithi019.stdout:1/988: symlink d16/d18/l12f 0 2024-03-16T09:01:26.121 INFO:tasks.workunit.client.1.smithi019.stdout:7/952: mkdir d9/d25/d28/d48/d8d/d12d 0 2024-03-16T09:01:26.124 INFO:tasks.workunit.client.1.smithi019.stdout:7/953: mknod d9/d25/d28/d48/d8d/dba/dad/dd8/c12e 0 2024-03-16T09:01:26.128 INFO:tasks.workunit.client.1.smithi019.stdout:1/989: rename d16/d1f/d73/ddf to d16/d31/d7b/dd9/d130 0 2024-03-16T09:01:26.128 INFO:tasks.workunit.client.1.smithi019.stdout:1/990: chown d16/d31/d71 23519 1 2024-03-16T09:01:26.128 INFO:tasks.workunit.client.1.smithi019.stdout:1/991: fsync d16/d1f/d4f/f60 0 2024-03-16T09:01:26.130 INFO:tasks.workunit.client.1.smithi019.stdout:7/954: creat d9/d25/d28/d48/dfc/d13/d18/d30/d9e/f12f x:0 0 0 2024-03-16T09:01:26.133 INFO:tasks.workunit.client.1.smithi019.stdout:7/955: creat d9/d25/d28/d48/dfc/d13/d18/d30/d9e/dd7/f130 x:0 0 0 2024-03-16T09:01:26.140 INFO:tasks.workunit.client.1.smithi019.stdout:1/992: getdents d16/d1f/d4f/d8a 0 2024-03-16T09:01:26.152 INFO:tasks.workunit.client.1.smithi019.stdout:1/993: read d16/f2f [7829174,49074] 0 2024-03-16T09:01:26.153 INFO:tasks.workunit.client.1.smithi019.stdout:7/956: dwrite d9/d25/d28/d48/dfc/d13/d18/d30/d9e/fc1 [0,4194304] 0 2024-03-16T09:01:26.153 INFO:tasks.workunit.client.1.smithi019.stdout:1/994: truncate d16/f3d 344974 0 2024-03-16T09:01:26.156 INFO:tasks.workunit.client.1.smithi019.stdout:7/957: mknod d9/d25/d28/d48/dfc/d13/d18/d30/d3e/da5/c131 0 2024-03-16T09:01:26.157 INFO:tasks.workunit.client.1.smithi019.stdout:1/995: truncate d16/d1f/ff3 2510508 0 2024-03-16T09:01:26.159 INFO:tasks.workunit.client.1.smithi019.stdout:1/996: mknod d16/d31/d76/d92/c131 0 2024-03-16T09:01:26.161 INFO:tasks.workunit.client.1.smithi019.stdout:1/997: rmdir d16/d31/d71/d89/dce 39 2024-03-16T09:01:26.164 INFO:tasks.workunit.client.1.smithi019.stdout:7/958: dread d9/d25/d28/d48/dfc/d13/f94 [0,4194304] 0 2024-03-16T09:01:26.169 INFO:tasks.workunit.client.1.smithi019.stdout:7/959: link d9/d25/d28/d48/dfc/d13/d18/d30/f97 d9/d25/d28/d48/dfc/d13/d18/d30/d9e/dd7/f132 0 2024-03-16T09:01:26.169 INFO:tasks.workunit.client.1.smithi019.stdout:7/960: creat d9/d25/d28/d48/d8d/dba/dad/f133 x:0 0 0 2024-03-16T09:01:26.169 INFO:tasks.workunit.client.1.smithi019.stdout:7/961: write f8 [2904897,76542] 0 2024-03-16T09:01:26.169 INFO:tasks.workunit.client.1.smithi019.stdout:7/962: write d9/d25/d28/d48/dfc/d13/d9a/dab/ff6 [472687,118658] 0 2024-03-16T09:01:26.171 INFO:tasks.workunit.client.1.smithi019.stdout:7/963: creat d9/db/d12/d5b/dc3/f134 x:0 0 0 2024-03-16T09:01:26.174 INFO:tasks.workunit.client.1.smithi019.stdout:7/964: link d9/d25/d28/d48/dfc/d13/d18/d30/d75/de1/le4 d9/db/d102/d113/d126/l135 0 2024-03-16T09:01:26.174 INFO:tasks.workunit.client.1.smithi019.stdout:7/965: fsync d9/d25/d28/d48/dfc/d13/d18/f5a 0 2024-03-16T09:01:26.175 INFO:tasks.workunit.client.1.smithi019.stdout:7/966: mknod d9/db/d102/c136 0 2024-03-16T09:01:26.176 INFO:tasks.workunit.client.1.smithi019.stdout:7/967: dread d9/d25/d28/d48/ddd/fd9 [0,4194304] 0 2024-03-16T09:01:26.176 INFO:tasks.workunit.client.1.smithi019.stdout:7/968: readlink d9/d25/d28/d48/ddd/lc6 0 2024-03-16T09:01:26.178 INFO:tasks.workunit.client.1.smithi019.stdout:7/969: unlink d9/d25/d28/d48/dfc/d13/d18/d30/d9e/dd7/f132 0 2024-03-16T09:01:26.179 INFO:tasks.workunit.client.1.smithi019.stdout:7/970: write d9/d25/d28/d48/dfc/d13/d18/d30/d9e/dd7/f130 [123911,113149] 0 2024-03-16T09:01:26.179 INFO:tasks.workunit.client.1.smithi019.stdout:7/971: truncate d9/d25/d28/d48/dfc/d13/d18/d30/d9e/f12f 938283 0 2024-03-16T09:01:26.181 INFO:tasks.workunit.client.1.smithi019.stdout:7/972: unlink d9/d25/d28/d48/dfc/d13/l9b 0 2024-03-16T09:01:26.181 INFO:tasks.workunit.client.1.smithi019.stdout:1/998: dwrite d16/d31/d76/f6b [0,4194304] 0 2024-03-16T09:01:26.184 INFO:tasks.workunit.client.1.smithi019.stdout:9/960: sync 2024-03-16T09:01:26.192 INFO:tasks.workunit.client.1.smithi019.stdout:7/973: read d9/d25/d28/d48/ff7 [1976991,112283] 0 2024-03-16T09:01:26.203 INFO:tasks.workunit.client.1.smithi019.stdout:1/999: dwrite d16/d1f/d3c/fe4 [0,4194304] 0 2024-03-16T09:01:26.212 INFO:tasks.workunit.client.1.smithi019.stdout:9/961: dwrite d1/d7/de/d24/d44/d60/f9f [0,4194304] 0 2024-03-16T09:01:26.212 INFO:tasks.workunit.client.1.smithi019.stdout:9/962: write d1/d7/de/d24/d5b/dce/df8/d59/f9d [677014,61415] 0 2024-03-16T09:01:26.212 INFO:tasks.workunit.client.1.smithi019.stdout:7/974: dwrite d9/d25/d28/f60 [4194304,4194304] 0 2024-03-16T09:01:26.216 INFO:tasks.workunit.client.1.smithi019.stdout:9/963: link d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/ceb d1/d7/de/d24/d44/d76/d9c/dcf/c147 0 2024-03-16T09:01:26.221 INFO:tasks.workunit.client.1.smithi019.stdout:7/975: rename d9/d25/d28/d48/fed to d9/d25/d28/d48/ddd/f137 0 2024-03-16T09:01:26.221 INFO:tasks.workunit.client.1.smithi019.stdout:7/976: chown d9/d25/d93/cd1 0 1 2024-03-16T09:01:26.228 INFO:tasks.workunit.client.1.smithi019.stdout:4/994: sync 2024-03-16T09:01:26.230 INFO:tasks.workunit.client.1.smithi019.stdout:3/966: sync 2024-03-16T09:01:26.231 INFO:tasks.workunit.client.1.smithi019.stdout:3/967: symlink d3/d8b/dae/d5a/d75/d7a/dd0/l13c 0 2024-03-16T09:01:26.232 INFO:tasks.workunit.client.1.smithi019.stdout:3/968: truncate d3/d8b/dae/d6a/f7b 292103 0 2024-03-16T09:01:26.232 INFO:tasks.workunit.client.1.smithi019.stdout:3/969: chown d3/d8b/dae/d5a/d75/c132 2002 1 2024-03-16T09:01:26.233 INFO:tasks.workunit.client.1.smithi019.stdout:3/970: truncate d3/d1b/d40/fb5 3799810 0 2024-03-16T09:01:26.240 INFO:tasks.workunit.client.1.smithi019.stdout:9/964: dwrite d1/d7/d67/f138 [0,4194304] 0 2024-03-16T09:01:26.241 INFO:tasks.workunit.client.1.smithi019.stdout:7/977: dwrite d9/d25/d28/d48/dfc/d13/d18/d30/d75/fa0 [0,4194304] 0 2024-03-16T09:01:26.244 INFO:tasks.workunit.client.1.smithi019.stdout:9/965: symlink d1/d7/dc0/l148 0 2024-03-16T09:01:26.244 INFO:tasks.workunit.client.1.smithi019.stdout:9/966: stat d1/d7/d21/fab 0 2024-03-16T09:01:26.248 INFO:tasks.workunit.client.1.smithi019.stdout:7/978: mkdir d9/db/d12/d138 0 2024-03-16T09:01:26.250 INFO:tasks.workunit.client.1.smithi019.stdout:7/979: symlink d9/d25/d28/d48/dfc/d13/d18/d30/d3e/l139 0 2024-03-16T09:01:26.253 INFO:tasks.workunit.client.1.smithi019.stdout:7/980: creat d9/d25/d28/d48/dfc/d13/d18/d30/d75/f13a x:0 0 0 2024-03-16T09:01:26.253 INFO:tasks.workunit.client.1.smithi019.stdout:9/967: dread d1/d7/de/d24/d44/d76/f9a [0,4194304] 0 2024-03-16T09:01:26.253 INFO:tasks.workunit.client.1.smithi019.stdout:9/968: fdatasync d1/d7/de/d24/d44/d60/d74/d101/f10e 0 2024-03-16T09:01:26.254 INFO:tasks.workunit.client.1.smithi019.stdout:7/981: creat d9/d25/d28/f13b x:0 0 0 2024-03-16T09:01:26.255 INFO:tasks.workunit.client.1.smithi019.stdout:9/969: rename d1/d7/de/d24/d44/d60/d74/d101/df0/d123/c104 to d1/d7/de/d24/d5b/dce/df8/dc9/dca/c149 0 2024-03-16T09:01:26.256 INFO:tasks.workunit.client.1.smithi019.stdout:7/982: symlink d9/df4/l13c 0 2024-03-16T09:01:26.257 INFO:tasks.workunit.client.1.smithi019.stdout:9/970: stat d1/d7/dc0/c10d 0 2024-03-16T09:01:26.257 INFO:tasks.workunit.client.1.smithi019.stdout:9/971: creat d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/f14a x:0 0 0 2024-03-16T09:01:26.260 INFO:tasks.workunit.client.1.smithi019.stdout:7/983: rename d9/d25/d28/d48/dfc/d13/d18/lc7 to d9/d25/d28/d48/dfc/d13/d18/d30/d9e/l13d 0 2024-03-16T09:01:26.260 INFO:tasks.workunit.client.1.smithi019.stdout:7/984: dread - d9/d25/d28/d48/dfc/d13/d18/d30/d3e/f11f zero size 2024-03-16T09:01:26.261 INFO:tasks.workunit.client.1.smithi019.stdout:9/972: readlink d1/d7/de/l8d 0 2024-03-16T09:01:26.261 INFO:tasks.workunit.client.1.smithi019.stdout:9/973: write d1/d7/d21/da2/f9e [5067294,124136] 0 2024-03-16T09:01:26.262 INFO:tasks.workunit.client.1.smithi019.stdout:7/985: mkdir d9/df4/d13e 0 2024-03-16T09:01:26.265 INFO:tasks.workunit.client.1.smithi019.stdout:9/974: creat d1/d7/de/d24/d5b/f14b x:0 0 0 2024-03-16T09:01:26.267 INFO:tasks.workunit.client.1.smithi019.stdout:7/986: rename d9/d25/d28/d48/d8d/dba/cbb to d9/d25/d93/c13f 0 2024-03-16T09:01:26.267 INFO:tasks.workunit.client.1.smithi019.stdout:7/987: fsync f8 0 2024-03-16T09:01:26.267 INFO:tasks.workunit.client.1.smithi019.stdout:7/988: write d9/d25/f54 [1953323,65371] 0 2024-03-16T09:01:26.268 INFO:tasks.workunit.client.1.smithi019.stdout:9/975: creat d1/d7/d67/f14c x:0 0 0 2024-03-16T09:01:26.269 INFO:tasks.workunit.client.1.smithi019.stdout:9/976: creat d1/d7/de/da3/f14d x:0 0 0 2024-03-16T09:01:26.269 INFO:tasks.workunit.client.1.smithi019.stdout:9/977: truncate d1/d7/d21/f87 714412 0 2024-03-16T09:01:26.274 INFO:tasks.workunit.client.1.smithi019.stdout:9/978: mknod d1/d7/dc0/de2/c14e 0 2024-03-16T09:01:26.275 INFO:tasks.workunit.client.1.smithi019.stdout:9/979: truncate d1/d7/de/d24/d5b/dce/df8/df3/f12c 5845401 0 2024-03-16T09:01:26.277 INFO:tasks.workunit.client.1.smithi019.stdout:9/980: symlink d1/d7/dc0/de2/l14f 0 2024-03-16T09:01:26.278 INFO:tasks.workunit.client.1.smithi019.stdout:9/981: unlink d1/d7/d67/f14c 0 2024-03-16T09:01:26.281 INFO:tasks.workunit.client.1.smithi019.stdout:9/982: dread d1/d7/de/d24/d44/d60/d74/d101/f127 [0,4194304] 0 2024-03-16T09:01:26.281 INFO:tasks.workunit.client.1.smithi019.stdout:9/983: truncate d1/d7/d67/d77/faa 833132 0 2024-03-16T09:01:26.517 INFO:tasks.workunit.client.1.smithi019.stdout:3/971: sync 2024-03-16T09:01:26.517 INFO:tasks.workunit.client.1.smithi019.stdout:4/995: sync 2024-03-16T09:01:26.517 INFO:tasks.workunit.client.1.smithi019.stdout:3/972: chown d3/d1b/l4d 496 1 2024-03-16T09:01:26.517 INFO:tasks.workunit.client.1.smithi019.stdout:4/996: fsync d4/d64/d99/f7b 0 2024-03-16T09:01:26.544 INFO:tasks.workunit.client.1.smithi019.stdout:3/973: dwrite d3/d8b/dae/d3f/fad [0,4194304] 0 2024-03-16T09:01:26.544 INFO:tasks.workunit.client.1.smithi019.stdout:3/974: fdatasync d3/d1b/d40/dc3/fea 0 2024-03-16T09:01:26.544 INFO:tasks.workunit.client.1.smithi019.stdout:3/975: readlink d3/d1b/d40/d4a/l94 0 2024-03-16T09:01:26.545 INFO:tasks.workunit.client.1.smithi019.stdout:3/976: fsync d3/d1b/d40/ff5 0 2024-03-16T09:01:26.545 INFO:tasks.workunit.client.1.smithi019.stdout:3/977: chown d3/d8b/dae/d3f/c55 32 1 2024-03-16T09:01:26.545 INFO:tasks.workunit.client.1.smithi019.stdout:3/978: write d3/d11/f127 [424001,76155] 0 2024-03-16T09:01:26.545 INFO:tasks.workunit.client.1.smithi019.stdout:4/997: dwrite d4/d64/d9b/fb7 [0,4194304] 0 2024-03-16T09:01:26.547 INFO:tasks.workunit.client.1.smithi019.stdout:3/979: creat d3/d8b/dae/dd6/d13a/f13d x:0 0 0 2024-03-16T09:01:26.547 INFO:tasks.workunit.client.1.smithi019.stdout:3/980: truncate d3/d1b/d40/f101 959298 0 2024-03-16T09:01:26.548 INFO:tasks.workunit.client.1.smithi019.stdout:4/998: creat d4/d64/d112/f14a x:0 0 0 2024-03-16T09:01:26.550 INFO:tasks.workunit.client.1.smithi019.stdout:3/981: unlink d3/ce 0 2024-03-16T09:01:26.552 INFO:tasks.workunit.client.1.smithi019.stdout:4/999: rename d4/da3/f10b to d4/d6/d15/d9a/d135/f14b 0 2024-03-16T09:01:26.553 INFO:tasks.workunit.client.1.smithi019.stdout:3/982: mknod d3/d1b/dc7/c13e 0 2024-03-16T09:01:26.555 INFO:tasks.workunit.client.1.smithi019.stdout:3/983: rename d3/d8b/dae/d6a/l99 to d3/d8b/l13f 0 2024-03-16T09:01:26.565 INFO:tasks.workunit.client.1.smithi019.stdout:3/984: read d3/d89/f47 [596917,77339] 0 2024-03-16T09:01:26.565 INFO:tasks.workunit.client.1.smithi019.stdout:3/985: write d3/d8b/da8/dcc/f126 [715519,62791] 0 2024-03-16T09:01:26.566 INFO:tasks.workunit.client.1.smithi019.stdout:3/986: dread d3/d8b/dae/d5a/d75/d7a/fdf [0,4194304] 0 2024-03-16T09:01:26.568 INFO:tasks.workunit.client.1.smithi019.stdout:3/987: symlink d3/d8b/dae/d3f/d111/l140 0 2024-03-16T09:01:26.568 INFO:tasks.workunit.client.1.smithi019.stdout:3/988: write d3/d8b/fa3 [916121,104635] 0 2024-03-16T09:01:26.568 INFO:tasks.workunit.client.1.smithi019.stdout:3/989: truncate d3/d8b/dae/d3f/faf 963068 0 2024-03-16T09:01:26.576 INFO:tasks.workunit.client.1.smithi019.stdout:3/990: dread d3/d8b/dae/d5a/f61 [0,4194304] 0 2024-03-16T09:01:26.576 INFO:tasks.workunit.client.1.smithi019.stdout:3/991: creat d3/d8b/dca/f141 x:0 0 0 2024-03-16T09:01:26.576 INFO:tasks.workunit.client.1.smithi019.stdout:3/992: truncate d3/d1b/f9e 1720963 0 2024-03-16T09:01:26.590 INFO:tasks.workunit.client.1.smithi019.stdout:3/993: dwrite d3/d89/f38 [0,4194304] 0 2024-03-16T09:01:26.595 INFO:tasks.workunit.client.1.smithi019.stdout:3/994: link d3/d1b/fdb d3/d1b/dc7/f142 0 2024-03-16T09:01:26.604 INFO:tasks.workunit.client.1.smithi019.stdout:3/995: dread d3/d11/f6f [0,4194304] 0 2024-03-16T09:01:26.605 INFO:tasks.workunit.client.1.smithi019.stdout:9/984: sync 2024-03-16T09:01:26.605 INFO:tasks.workunit.client.1.smithi019.stdout:7/989: sync 2024-03-16T09:01:26.605 INFO:tasks.workunit.client.1.smithi019.stdout:9/985: chown d1/d7/de/d24/d5b/dce/df8/dc9/f115 406814824 1 2024-03-16T09:01:26.605 INFO:tasks.workunit.client.1.smithi019.stdout:7/990: creat d9/d25/d28/d48/dfc/d13/d18/d30/d9e/f140 x:0 0 0 2024-03-16T09:01:26.606 INFO:tasks.workunit.client.1.smithi019.stdout:3/996: link d3/d8b/dae/d6a/ffb d3/d1b/d40/d4a/db7/f143 0 2024-03-16T09:01:26.608 INFO:tasks.workunit.client.1.smithi019.stdout:7/991: mkdir d9/d25/d28/d48/dfc/d13/d18/d30/d141 0 2024-03-16T09:01:26.612 INFO:tasks.workunit.client.1.smithi019.stdout:7/992: rename d9/db/d102/d113/d126/l135 to d9/d25/d28/d48/d8d/dba/l142 0 2024-03-16T09:01:26.613 INFO:tasks.workunit.client.1.smithi019.stdout:9/986: dread d1/d7/de/d24/d44/d60/d74/d101/df0/d123/dae/f7a [0,4194304] 0 2024-03-16T09:01:26.613 INFO:tasks.workunit.client.1.smithi019.stdout:9/987: chown d1/d7/de/d24/d44/d76 124797 1 2024-03-16T09:01:26.613 INFO:tasks.workunit.client.1.smithi019.stdout:9/988: readlink d1/d7/de/d24/d5b/dce/df8/df3/l106 0 2024-03-16T09:01:26.613 INFO:tasks.workunit.client.1.smithi019.stdout:9/989: write d1/d7/d67/d77/fd3 [2040365,2227] 0 2024-03-16T09:01:26.615 INFO:tasks.workunit.client.1.smithi019.stdout:9/990: write d1/d7/f45 [1305116,86652] 0 2024-03-16T09:01:26.615 INFO:tasks.workunit.client.1.smithi019.stdout:7/993: symlink d9/db/d12/d5b/dc3/dec/l143 0 2024-03-16T09:01:26.616 INFO:tasks.workunit.client.1.smithi019.stdout:9/991: mkdir d1/d7/de/d24/d44/d60/d150 0 2024-03-16T09:01:26.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/994: symlink d9/d25/d28/d48/dfc/d13/l144 0 2024-03-16T09:01:26.618 INFO:tasks.workunit.client.1.smithi019.stdout:7/995: write f8 [3080587,10048] 0 2024-03-16T09:01:26.623 INFO:tasks.workunit.client.1.smithi019.stdout:9/992: rename d1/d7/de/d24/d44/d60/d74/d101/df0/d123/fd1 to d1/d7/de/d24/d5b/dce/df8/df3/d120/f151 0 2024-03-16T09:01:26.625 INFO:tasks.workunit.client.1.smithi019.stdout:9/993: mkdir d1/d7/de/d24/d44/d76/d152 0 2024-03-16T09:01:26.625 INFO:tasks.workunit.client.1.smithi019.stdout:9/994: creat d1/d7/d67/f153 x:0 0 0 2024-03-16T09:01:26.642 INFO:tasks.workunit.client.1.smithi019.stdout:9/995: dwrite d1/d7/f45 [0,4194304] 0 2024-03-16T09:01:26.642 INFO:tasks.workunit.client.1.smithi019.stdout:3/997: dwrite d3/d8b/dca/f116 [0,4194304] 0 2024-03-16T09:01:26.643 INFO:tasks.workunit.client.1.smithi019.stdout:9/996: creat d1/f154 x:0 0 0 2024-03-16T09:01:26.645 INFO:tasks.workunit.client.1.smithi019.stdout:3/998: mknod d3/c144 0 2024-03-16T09:01:26.652 INFO:tasks.workunit.client.1.smithi019.stdout:9/997: dread d1/d7/f34 [4194304,4194304] 0 2024-03-16T09:01:26.655 INFO:tasks.workunit.client.1.smithi019.stdout:9/998: dread d1/d7/de/fd4 [0,4194304] 0 2024-03-16T09:01:26.655 INFO:tasks.workunit.client.1.smithi019.stdout:9/999: dread - d1/d7/de/d24/d5b/dce/df8/dfe/f108 zero size 2024-03-16T09:01:26.657 INFO:tasks.workunit.client.1.smithi019.stdout:3/999: dwrite d3/d1b/dc7/ff1 [0,4194304] 0 2024-03-16T09:01:26.767 INFO:tasks.workunit.client.1.smithi019.stdout:7/996: sync 2024-03-16T09:01:26.774 INFO:tasks.workunit.client.1.smithi019.stdout:7/997: dread d9/d25/d28/d48/dfc/d13/d18/fac [4194304,4194304] 0 2024-03-16T09:01:26.774 INFO:tasks.workunit.client.1.smithi019.stdout:7/998: chown d9/d25/d28/d48/dfc/d13/c11b 456869513 1 2024-03-16T09:01:26.774 INFO:tasks.workunit.client.1.smithi019.stdout:7/999: fdatasync d9/db/d12/d5b/dc3/f134 0 2024-03-16T09:01:26.776 INFO:tasks.workunit.client.1.smithi019.stderr:+ rm -rf -- ./tmp.ONg0mYJCmf 2024-03-16T09:01:27.414 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:27 smithi007 ceph-mon[29470]: pgmap v27: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 414 MiB/s rd, 510 MiB/s wr, 700 op/s 2024-03-16T09:01:27.415 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:27 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:27.415 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:27 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:27.415 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:27 smithi007 ceph-mon[29470]: Reconfiguring grafana.smithi007 (dependencies changed)... 2024-03-16T09:01:27.415 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:27 smithi007 ceph-mon[29470]: Reconfiguring daemon grafana.smithi007 on smithi007 2024-03-16T09:01:27.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:27 smithi019 ceph-mon[36407]: pgmap v27: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 414 MiB/s rd, 510 MiB/s wr, 700 op/s 2024-03-16T09:01:27.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:27 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:27.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:27 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:27.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:27 smithi019 ceph-mon[36407]: Reconfiguring grafana.smithi007 (dependencies changed)... 2024-03-16T09:01:27.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:27 smithi019 ceph-mon[36407]: Reconfiguring daemon grafana.smithi007 on smithi007 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.844+0000 7f7a3800c700 1 -- 172.21.15.7:0/2249293795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fbd00 msgr2=0x7f7a300fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.844+0000 7f7a3800c700 1 --2- 172.21.15.7:0/2249293795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fbd00 0x7f7a300fc110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c004940 tx=0x7f7a2c009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.844+0000 7f7a3800c700 1 -- 172.21.15.7:0/2249293795 shutdown_connections 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.844+0000 7f7a3800c700 1 --2- 172.21.15.7:0/2249293795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a300fcfb0 0x7f7a300fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.844+0000 7f7a3800c700 1 --2- 172.21.15.7:0/2249293795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fbd00 0x7f7a300fc110 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.844+0000 7f7a3800c700 1 -- 172.21.15.7:0/2249293795 >> 172.21.15.7:0/2249293795 conn(0x7f7a300f7440 msgr2=0x7f7a300f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.844+0000 7f7a3800c700 1 -- 172.21.15.7:0/2249293795 shutdown_connections 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.844+0000 7f7a3800c700 1 -- 172.21.15.7:0/2249293795 wait complete. 2024-03-16T09:01:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.845+0000 7f7a3800c700 1 Processor -- start 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.845+0000 7f7a3800c700 1 -- start start 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a3800c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a300fbd00 0x7f7a30103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a3800c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fcfb0 0x7f7a30102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a3800c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301041f0 con 0x7f7a300fbd00 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a3800c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30102850 con 0x7f7a300fcfb0 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a35da8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a300fbd00 0x7f7a30103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a35da8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a300fbd00 0x7f7a30103bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49626/0 (socket says 172.21.15.7:49626) 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a35da8700 1 -- 172.21.15.7:0/2991502194 learned_addr learned my addr 172.21.15.7:0/2991502194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a355a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fcfb0 0x7f7a30102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a35da8700 1 -- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fcfb0 msgr2=0x7f7a30102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a35da8700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fcfb0 0x7f7a30102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a35da8700 1 -- 172.21.15.7:0/2991502194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c005040 con 0x7f7a300fbd00 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a355a7700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fcfb0 0x7f7a30102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.846+0000 7f7a35da8700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a300fbd00 0x7f7a30103bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009150 tx=0x7f7a2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:28.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.847+0000 7f7a267fc700 1 -- 172.21.15.7:0/2991502194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a2c00e040 con 0x7f7a300fbd00 2024-03-16T09:01:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.847+0000 7f7a267fc700 1 -- 172.21.15.7:0/2991502194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a2c00bbf0 con 0x7f7a300fbd00 2024-03-16T09:01:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.847+0000 7f7a267fc700 1 -- 172.21.15.7:0/2991502194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a2c01c630 con 0x7f7a300fbd00 2024-03-16T09:01:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.847+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a30102ad0 con 0x7f7a300fbd00 2024-03-16T09:01:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.847+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a30103010 con 0x7f7a300fbd00 2024-03-16T09:01:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.848+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3004c710 con 0x7f7a300fbd00 2024-03-16T09:01:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.850+0000 7f7a267fc700 1 -- 172.21.15.7:0/2991502194 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7f7a2c00bd60 con 0x7f7a300fbd00 2024-03-16T09:01:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.850+0000 7f7a267fc700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f7a1c06b450 0x7f7a1c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.850+0000 7f7a267fc700 1 -- 172.21.15.7:0/2991502194 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f7a2c08af00 con 0x7f7a300fbd00 2024-03-16T09:01:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.850+0000 7f7a355a7700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f7a1c06b450 0x7f7a1c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:28.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.851+0000 7f7a355a7700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f7a1c06b450 0x7f7a1c06d900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a20004fd0 tx=0x7f7a200043c0 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:28.855+0000 7f7a267fc700 1 -- 172.21.15.7:0/2991502194 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f7a2c0586b0 con 0x7f7a300fbd00 2024-03-16T09:01:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.255+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a301012d0 con 0x7f7a1c06b450 2024-03-16T09:01:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.257+0000 7f7a267fc700 1 -- 172.21.15.7:0/2991502194 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f7a301012d0 con 0x7f7a1c06b450 2024-03-16T09:01:29.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f7a1c06b450 msgr2=0x7f7a1c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:29.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f7a1c06b450 0x7f7a1c06d900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a20004fd0 tx=0x7f7a200043c0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a300fbd00 msgr2=0x7f7a30103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:29.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a300fbd00 0x7f7a30103bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009150 tx=0x7f7a2c009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:29.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 shutdown_connections 2024-03-16T09:01:29.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f7a1c06b450 0x7f7a1c06d900 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a300fbd00 0x7f7a30103bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 --2- 172.21.15.7:0/2991502194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a300fcfb0 0x7f7a30102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.259+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 >> 172.21.15.7:0/2991502194 conn(0x7f7a300f7440 msgr2=0x7f7a300f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:29.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.260+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 shutdown_connections 2024-03-16T09:01:29.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.260+0000 7f7a3800c700 1 -- 172.21.15.7:0/2991502194 wait complete. 2024-03-16T09:01:29.277 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:01:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.393+0000 7fa016b67700 1 -- 172.21.15.7:0/1945691931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 msgr2=0x7fa0100fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.393+0000 7fa016b67700 1 --2- 172.21.15.7:0/1945691931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 0x7fa0100fe540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc004940 tx=0x7f9ffc009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.393+0000 7fa016b67700 1 -- 172.21.15.7:0/1945691931 shutdown_connections 2024-03-16T09:01:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.393+0000 7fa016b67700 1 --2- 172.21.15.7:0/1945691931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 0x7fa0100fe540 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.393+0000 7fa016b67700 1 --2- 172.21.15.7:0/1945691931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0100fcee0 0x7fa0100fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.393+0000 7fa016b67700 1 -- 172.21.15.7:0/1945691931 >> 172.21.15.7:0/1945691931 conn(0x7fa0100f8560 msgr2=0x7fa0100fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.393+0000 7fa016b67700 1 -- 172.21.15.7:0/1945691931 shutdown_connections 2024-03-16T09:01:29.397 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:29 smithi007 ceph-mon[29470]: pgmap v28: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 274 MiB/s rd, 338 MiB/s wr, 470 op/s 2024-03-16T09:01:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.394+0000 7fa016b67700 1 -- 172.21.15.7:0/1945691931 wait complete. 2024-03-16T09:01:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.394+0000 7fa016b67700 1 Processor -- start 2024-03-16T09:01:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.394+0000 7fa016b67700 1 -- start start 2024-03-16T09:01:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa016b67700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0100fcee0 0x7fa01010fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa016b67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 0x7fa01010b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa016b67700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0101103b0 con 0x7fa0100fe0d0 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa016b67700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01010bc70 con 0x7fa0100fcee0 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa014903700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0100fcee0 0x7fa01010fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa014903700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0100fcee0 0x7fa01010fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:38550/0 (socket says 172.21.15.7:38550) 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa014903700 1 -- 172.21.15.7:0/3468365328 learned_addr learned my addr 172.21.15.7:0/3468365328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa00ffff700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 0x7fa01010b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa014903700 1 -- 172.21.15.7:0/3468365328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 msgr2=0x7fa01010b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa014903700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 0x7fa01010b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa014903700 1 -- 172.21.15.7:0/3468365328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa004005040 con 0x7fa0100fcee0 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa00ffff700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 0x7fa01010b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.395+0000 7fa014903700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0100fcee0 0x7fa01010fd60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0040096d0 tx=0x7fa004009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.396+0000 7fa00d7fa700 1 -- 172.21.15.7:0/3468365328 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa00400e7e0 con 0x7fa0100fcee0 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.396+0000 7fa00d7fa700 1 -- 172.21.15.7:0/3468365328 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa00400f040 con 0x7fa0100fcee0 2024-03-16T09:01:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.396+0000 7fa00d7fa700 1 -- 172.21.15.7:0/3468365328 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa004013dc0 con 0x7fa0100fcee0 2024-03-16T09:01:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.396+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc005040 con 0x7fa0100fcee0 2024-03-16T09:01:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.396+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01010cb10 con 0x7fa0100fcee0 2024-03-16T09:01:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.397+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01010c330 con 0x7fa0100fcee0 2024-03-16T09:01:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.397+0000 7fa00d7fa700 1 -- 172.21.15.7:0/3468365328 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7fa00400e940 con 0x7fa0100fcee0 2024-03-16T09:01:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.398+0000 7fa00d7fa700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa00006b660 0x7fa00006db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.398+0000 7fa00d7fa700 1 -- 172.21.15.7:0/3468365328 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa004089fd0 con 0x7fa0100fcee0 2024-03-16T09:01:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.398+0000 7fa00ffff700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa00006b660 0x7fa00006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.399+0000 7fa00ffff700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa00006b660 0x7fa00006db10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00b0d0 tx=0x7f9ffc009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:29.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.404+0000 7fa00d7fa700 1 -- 172.21.15.7:0/3468365328 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fa004053c50 con 0x7fa0100fcee0 2024-03-16T09:01:29.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:29 smithi019 ceph-mon[36407]: pgmap v28: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 274 MiB/s rd, 338 MiB/s wr, 470 op/s 2024-03-16T09:01:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.739+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa01004bf70 con 0x7fa00006b660 2024-03-16T09:01:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.740+0000 7fa00d7fa700 1 -- 172.21.15.7:0/3468365328 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f9ffc007cc0 con 0x7fa00006b660 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.742+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa00006b660 msgr2=0x7fa00006db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.742+0000 7fa016b67700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa00006b660 0x7fa00006db10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00b0d0 tx=0x7f9ffc009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.742+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0100fcee0 msgr2=0x7fa01010fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.742+0000 7fa016b67700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0100fcee0 0x7fa01010fd60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0040096d0 tx=0x7fa004009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.743+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 shutdown_connections 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.743+0000 7fa016b67700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa00006b660 0x7fa00006db10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.743+0000 7fa016b67700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0100fcee0 0x7fa01010fd60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.743+0000 7fa016b67700 1 --2- 172.21.15.7:0/3468365328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0100fe0d0 0x7fa01010b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.743+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 >> 172.21.15.7:0/3468365328 conn(0x7fa0100f8560 msgr2=0x7fa0100f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.743+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 shutdown_connections 2024-03-16T09:01:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.743+0000 7fa016b67700 1 -- 172.21.15.7:0/3468365328 wait complete. 2024-03-16T09:01:29.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.881+0000 7f30d8a9b700 1 -- 172.21.15.7:0/3177084543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d4104fc0 msgr2=0x7f30d41073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:29.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.881+0000 7f30d8a9b700 1 --2- 172.21.15.7:0/3177084543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d4104fc0 0x7f30d41073a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009ce0 tx=0x7f30cc004d10 comp rx=0 tx=0).stop 2024-03-16T09:01:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.884+0000 7f30d8a9b700 1 -- 172.21.15.7:0/3177084543 shutdown_connections 2024-03-16T09:01:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.884+0000 7f30d8a9b700 1 --2- 172.21.15.7:0/3177084543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d41078e0 0x7f30d4109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.884+0000 7f30d8a9b700 1 --2- 172.21.15.7:0/3177084543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d4104fc0 0x7f30d41073a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.884+0000 7f30d8a9b700 1 -- 172.21.15.7:0/3177084543 >> 172.21.15.7:0/3177084543 conn(0x7f30d4069de0 msgr2=0x7f30d406c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.884+0000 7f30d8a9b700 1 -- 172.21.15.7:0/3177084543 shutdown_connections 2024-03-16T09:01:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.884+0000 7f30d8a9b700 1 -- 172.21.15.7:0/3177084543 wait complete. 2024-03-16T09:01:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.885+0000 7f30d8a9b700 1 Processor -- start 2024-03-16T09:01:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.885+0000 7f30d8a9b700 1 -- start start 2024-03-16T09:01:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.885+0000 7f30d8a9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d4104fc0 0x7f30d4118490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.885+0000 7f30d8a9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d41078e0 0x7f30d41189d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.885+0000 7f30d8a9b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d41190b0 con 0x7f30d4104fc0 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.885+0000 7f30d8a9b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4074210 con 0x7f30d41078e0 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.885+0000 7f30d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d41078e0 0x7f30d41189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.886+0000 7f30d1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d41078e0 0x7f30d41189d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:38558/0 (socket says 172.21.15.7:38558) 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.886+0000 7f30d1d9b700 1 -- 172.21.15.7:0/902576788 learned_addr learned my addr 172.21.15.7:0/902576788 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.886+0000 7f30d259c700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d4104fc0 0x7f30d4118490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.886+0000 7f30d259c700 1 -- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d41078e0 msgr2=0x7f30d41189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.886+0000 7f30d259c700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d41078e0 0x7f30d41189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.886+0000 7f30d259c700 1 -- 172.21.15.7:0/902576788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc009940 con 0x7f30d4104fc0 2024-03-16T09:01:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.886+0000 7f30d259c700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d4104fc0 0x7f30d4118490 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f30cc000f80 tx=0x7f30cc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:29.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.888+0000 7f30c2ffd700 1 -- 172.21.15.7:0/902576788 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30cc00e040 con 0x7f30d4104fc0 2024-03-16T09:01:29.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.888+0000 7f30c2ffd700 1 -- 172.21.15.7:0/902576788 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30cc008e20 con 0x7f30d4104fc0 2024-03-16T09:01:29.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.888+0000 7f30c2ffd700 1 -- 172.21.15.7:0/902576788 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30cc01c740 con 0x7f30d4104fc0 2024-03-16T09:01:29.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.888+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d4074490 con 0x7f30d4104fc0 2024-03-16T09:01:29.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.888+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d4074a50 con 0x7f30d4104fc0 2024-03-16T09:01:29.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.891+0000 7f30c2ffd700 1 -- 172.21.15.7:0/902576788 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7f30cc012410 con 0x7f30d4104fc0 2024-03-16T09:01:29.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.892+0000 7f30c2ffd700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f30bc06b720 0x7f30bc06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:29.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.892+0000 7f30c2ffd700 1 -- 172.21.15.7:0/902576788 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f30cc08bad0 con 0x7f30d4104fc0 2024-03-16T09:01:29.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.893+0000 7f30d1d9b700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f30bc06b720 0x7f30bc06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:29.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.894+0000 7f30d1d9b700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f30bc06b720 0x7f30bc06dbd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f30d4119580 tx=0x7f30c4009220 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:29.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.895+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b40052e0 con 0x7f30d4104fc0 2024-03-16T09:01:29.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:29.905+0000 7f30c2ffd700 1 -- 172.21.15.7:0/902576788 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f30cc055810 con 0x7f30d4104fc0 2024-03-16T09:01:30.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.286+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f30b4000bf0 con 0x7f30bc06b720 2024-03-16T09:01:30.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.297+0000 7f30c2ffd700 1 -- 172.21.15.7:0/902576788 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f30b4000bf0 con 0x7f30bc06b720 2024-03-16T09:01:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:01:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 starting - - - - 2024-03-16T09:01:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (8m) 37s ago 8m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:01:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (7m) 43s ago 7m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:01:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (6m) 37s ago 8m 90.7M - 9.4.7 954c08fa6188 4f8b149e57d8 2024-03-16T09:01:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (2m) 37s ago 2m 11.3M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:01:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (2m) 37s ago 2m 140M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:01:30.300 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (2m) 43s ago 2m 59.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (2m) 43s ago 2m 11.9M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:9283 running (9m) 37s ago 9m 385M - 17.2.7-648-g1df27247 0bdcfccf14ce 376995a75ba3 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283 running (60s) 43s ago 7m 491M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 35bb11552f05 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (9m) 37s ago 9m 45.8M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (7m) 43s ago 7m 41.4M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 starting - - - - 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (7m) 43s ago 7m 16.8M - 1.5.0 0da6a335fe13 9edea61eb01f 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (6m) 37s ago 6m 100M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (6m) 37s ago 6m 87.9M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (5m) 37s ago 5m 82.8M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (5m) 43s ago 5m 95.0M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (4m) 43s ago 4m 82.6M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:01:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (4m) 43s ago 4m 86.5M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:01:30.302 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (6m) 37s ago 7m 59.7M - 2.43.0 a07b618ecd1d b457178ffb05 2024-03-16T09:01:30.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f30bc06b720 msgr2=0x7f30bc06dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:30.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f30bc06b720 0x7f30bc06dbd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f30d4119580 tx=0x7f30c4009220 comp rx=0 tx=0).stop 2024-03-16T09:01:30.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d4104fc0 msgr2=0x7f30d4118490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:30.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d4104fc0 0x7f30d4118490 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f30cc000f80 tx=0x7f30cc004d10 comp rx=0 tx=0).stop 2024-03-16T09:01:30.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 shutdown_connections 2024-03-16T09:01:30.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f30bc06b720 0x7f30bc06dbd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d4104fc0 0x7f30d4118490 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 --2- 172.21.15.7:0/902576788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30d41078e0 0x7f30d41189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 >> 172.21.15.7:0/902576788 conn(0x7f30d4069de0 msgr2=0x7f30d406a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:30.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 shutdown_connections 2024-03-16T09:01:30.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.300+0000 7f30d8a9b700 1 -- 172.21.15.7:0/902576788 wait complete. 2024-03-16T09:01:30.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.417+0000 7f32c441e700 1 -- 172.21.15.7:0/336852997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 msgr2=0x7f32bc0fc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:30.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.417+0000 7f32c441e700 1 --2- 172.21.15.7:0/336852997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 0x7f32bc0fc710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f32b8004940 tx=0x7f32b8009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:30.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.417+0000 7f32c441e700 1 -- 172.21.15.7:0/336852997 shutdown_connections 2024-03-16T09:01:30.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.417+0000 7f32c441e700 1 --2- 172.21.15.7:0/336852997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32bc0fcc50 0x7f32bc0fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.417+0000 7f32c441e700 1 --2- 172.21.15.7:0/336852997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 0x7f32bc0fc710 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.417+0000 7f32c441e700 1 -- 172.21.15.7:0/336852997 >> 172.21.15.7:0/336852997 conn(0x7f32bc0f6c10 msgr2=0x7f32bc0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:30.420 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:30 smithi007 ceph-mon[29470]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:30.420 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:30 smithi007 ceph-mon[29470]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.418+0000 7f32c441e700 1 -- 172.21.15.7:0/336852997 shutdown_connections 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.418+0000 7f32c441e700 1 -- 172.21.15.7:0/336852997 wait complete. 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.418+0000 7f32c441e700 1 Processor -- start 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.418+0000 7f32c441e700 1 -- start start 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c441e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 0x7f32bc0fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c441e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32bc0fcc50 0x7f32bc112710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c441e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc112c50 con 0x7f32bc0fc300 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c441e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc112d90 con 0x7f32bc0fcc50 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c21ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 0x7f32bc0fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c19b9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32bc0fcc50 0x7f32bc112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c21ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 0x7f32bc0fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49694/0 (socket says 172.21.15.7:49694) 2024-03-16T09:01:30.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c21ba700 1 -- 172.21.15.7:0/3285864008 learned_addr learned my addr 172.21.15.7:0/3285864008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:01:30.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c21ba700 1 -- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32bc0fcc50 msgr2=0x7f32bc112710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:30.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c21ba700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32bc0fcc50 0x7f32bc112710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c21ba700 1 -- 172.21.15.7:0/3285864008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b8005040 con 0x7f32bc0fc300 2024-03-16T09:01:30.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c19b9700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32bc0fcc50 0x7f32bc112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:01:30.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.419+0000 7f32c21ba700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 0x7f32bc0fba20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32b8000c00 tx=0x7f32b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:30.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.420+0000 7f32b2ffd700 1 -- 172.21.15.7:0/3285864008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b800e070 con 0x7f32bc0fc300 2024-03-16T09:01:30.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.420+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc112fc0 con 0x7f32bc0fc300 2024-03-16T09:01:30.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.420+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc1134a0 con 0x7f32bc0fc300 2024-03-16T09:01:30.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.420+0000 7f32b2ffd700 1 -- 172.21.15.7:0/3285864008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32b801c400 con 0x7f32bc0fc300 2024-03-16T09:01:30.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.420+0000 7f32b2ffd700 1 -- 172.21.15.7:0/3285864008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b800b650 con 0x7f32bc0fc300 2024-03-16T09:01:30.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.422+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc04c710 con 0x7f32bc0fc300 2024-03-16T09:01:30.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.422+0000 7f32b2ffd700 1 -- 172.21.15.7:0/3285864008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7f32b801c570 con 0x7f32bc0fc300 2024-03-16T09:01:30.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.423+0000 7f32b2ffd700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f32a806b670 0x7f32a806db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:30.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.423+0000 7f32b2ffd700 1 -- 172.21.15.7:0/3285864008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f32b808b480 con 0x7f32bc0fc300 2024-03-16T09:01:30.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.423+0000 7f32c19b9700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f32a806b670 0x7f32a806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:30.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.424+0000 7f32c19b9700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f32a806b670 0x7f32a806db20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009b80 tx=0x7f32ac00b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:30.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.430+0000 7f32b2ffd700 1 -- 172.21.15.7:0/3285864008 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f32b8058cf0 con 0x7f32bc0fc300 2024-03-16T09:01:30.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:30 smithi019 ceph-mon[36407]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:30.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:30 smithi019 ceph-mon[36407]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:30.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.793+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f32bc113960 con 0x7f32bc0fc300 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.793+0000 7f32b2ffd700 1 -- 172.21.15.7:0/3285864008 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f32b8004490 con 0x7f32bc0fc300 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 1, 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:01:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 13, 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:01:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:01:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f32a806b670 msgr2=0x7f32a806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f32a806b670 0x7f32a806db20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009b80 tx=0x7f32ac00b040 comp rx=0 tx=0).stop 2024-03-16T09:01:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 msgr2=0x7f32bc0fba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 0x7f32bc0fba20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32b8000c00 tx=0x7f32b8009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 shutdown_connections 2024-03-16T09:01:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f32a806b670 0x7f32a806db20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32bc0fc300 0x7f32bc0fba20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 --2- 172.21.15.7:0/3285864008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32bc0fcc50 0x7f32bc112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 >> 172.21.15.7:0/3285864008 conn(0x7f32bc0f6c10 msgr2=0x7f32bc0f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 shutdown_connections 2024-03-16T09:01:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.796+0000 7f32c441e700 1 -- 172.21.15.7:0/3285864008 wait complete. 2024-03-16T09:01:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.925+0000 7f14293d8700 1 -- 172.21.15.7:0/3742270997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14240fed50 msgr2=0x7f14240fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.925+0000 7f14293d8700 1 --2- 172.21.15.7:0/3742270997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14240fed50 0x7f14240fae90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1418004940 tx=0x7f1418009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.925+0000 7f14293d8700 1 -- 172.21.15.7:0/3742270997 shutdown_connections 2024-03-16T09:01:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.925+0000 7f14293d8700 1 --2- 172.21.15.7:0/3742270997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14240fed50 0x7f14240fae90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.925+0000 7f14293d8700 1 --2- 172.21.15.7:0/3742270997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14240fe400 0x7f14240fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.925+0000 7f14293d8700 1 -- 172.21.15.7:0/3742270997 >> 172.21.15.7:0/3742270997 conn(0x7f14240f6ba0 msgr2=0x7f14240f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.925+0000 7f14293d8700 1 -- 172.21.15.7:0/3742270997 shutdown_connections 2024-03-16T09:01:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.925+0000 7f14293d8700 1 -- 172.21.15.7:0/3742270997 wait complete. 2024-03-16T09:01:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.926+0000 7f14293d8700 1 Processor -- start 2024-03-16T09:01:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.926+0000 7f14293d8700 1 -- start start 2024-03-16T09:01:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.926+0000 7f14293d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14240fe400 0x7f142410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.926+0000 7f14293d8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14240fed50 0x7f142410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.926+0000 7f14293d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142410e490 con 0x7f14240fe400 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.926+0000 7f14293d8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142410ece0 con 0x7f14240fed50 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f14227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14240fed50 0x7f142410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f14227fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14240fed50 0x7f142410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:38590/0 (socket says 172.21.15.7:38590) 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f14227fc700 1 -- 172.21.15.7:0/2496443859 learned_addr learned my addr 172.21.15.7:0/2496443859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f14227fc700 1 -- 172.21.15.7:0/2496443859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14240fe400 msgr2=0x7f142410d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f14227fc700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14240fe400 0x7f142410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f14227fc700 1 -- 172.21.15.7:0/2496443859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1418005040 con 0x7f14240fed50 2024-03-16T09:01:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f14227fc700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14240fed50 0x7f142410de90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1418013870 tx=0x7f1418009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f140b7fe700 1 -- 172.21.15.7:0/2496443859 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141800e070 con 0x7f14240fed50 2024-03-16T09:01:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f140b7fe700 1 -- 172.21.15.7:0/2496443859 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f141801c400 con 0x7f14240fed50 2024-03-16T09:01:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.927+0000 7f140b7fe700 1 -- 172.21.15.7:0/2496443859 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141800b650 con 0x7f14240fed50 2024-03-16T09:01:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.928+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142410ef60 con 0x7f14240fed50 2024-03-16T09:01:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.928+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142410f440 con 0x7f14240fed50 2024-03-16T09:01:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.929+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142405f5f0 con 0x7f14240fed50 2024-03-16T09:01:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.929+0000 7f140b7fe700 1 -- 172.21.15.7:0/2496443859 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7f141800b7b0 con 0x7f14240fed50 2024-03-16T09:01:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.930+0000 7f140b7fe700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f140c06fa70 0x7f140c071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:01:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.930+0000 7f140b7fe700 1 -- 172.21.15.7:0/2496443859 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f141808b950 con 0x7f14240fed50 2024-03-16T09:01:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.930+0000 7f1422ffd700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f140c06fa70 0x7f140c071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:01:30.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.931+0000 7f1422ffd700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f140c06fa70 0x7f140c071f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1414009fd0 tx=0x7f141400b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:01:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:30.935+0000 7f140b7fe700 1 -- 172.21.15.7:0/2496443859 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f1418059100 con 0x7f14240fed50 2024-03-16T09:01:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.304+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f142410f720 con 0x7f140c06fa70 2024-03-16T09:01:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.305+0000 7f140b7fe700 1 -- 172.21.15.7:0/2496443859 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f142410f720 con 0x7f140c06fa70 2024-03-16T09:01:31.307 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:01:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:01:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:01:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T09:01:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-03-16T09:01:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "1/2 daemons upgraded", 2024-03-16T09:01:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-03-16T09:01:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:01:31.308 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:01:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f140c06fa70 msgr2=0x7f140c071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f140c06fa70 0x7f140c071f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1414009fd0 tx=0x7f141400b040 comp rx=0 tx=0).stop 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14240fed50 msgr2=0x7f142410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14240fed50 0x7f142410de90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1418013870 tx=0x7f1418009e30 comp rx=0 tx=0).stop 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 shutdown_connections 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f140c06fa70 0x7f140c071f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14240fe400 0x7f142410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 --2- 172.21.15.7:0/2496443859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14240fed50 0x7f142410de90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 >> 172.21.15.7:0/2496443859 conn(0x7f14240f6ba0 msgr2=0x7f14240f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.309+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 shutdown_connections 2024-03-16T09:01:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:01:31.310+0000 7f14293d8700 1 -- 172.21.15.7:0/2496443859 wait complete. 2024-03-16T09:01:31.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:31 smithi019 ceph-mon[36407]: pgmap v29: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 327 MiB/s rd, 406 MiB/s wr, 621 op/s 2024-03-16T09:01:31.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:31 smithi019 ceph-mon[36407]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:31.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:31 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3285864008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:01:31.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:31 smithi007 ceph-mon[29470]: pgmap v29: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 327 MiB/s rd, 406 MiB/s wr, 621 op/s 2024-03-16T09:01:31.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:31 smithi007 ceph-mon[29470]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:31.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:31 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3285864008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:01:32.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:32 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:01:32.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:32 smithi019 ceph-mon[36407]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:32.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:32 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:01:32.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:32 smithi007 ceph-mon[29470]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:01:33.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:33 smithi019 ceph-mon[36407]: pgmap v30: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 185 MiB/s rd, 231 MiB/s wr, 385 op/s 2024-03-16T09:01:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:33 smithi007 ceph-mon[29470]: pgmap v30: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 185 MiB/s rd, 231 MiB/s wr, 385 op/s 2024-03-16T09:01:35.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:35 smithi019 ceph-mon[36407]: pgmap v31: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 185 MiB/s rd, 231 MiB/s wr, 391 op/s 2024-03-16T09:01:35.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:35 smithi007 ceph-mon[29470]: pgmap v31: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 185 MiB/s rd, 231 MiB/s wr, 391 op/s 2024-03-16T09:01:37.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:37 smithi007 ceph-mon[29470]: pgmap v32: 65 pgs: 65 active+clean; 2.4 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 168 MiB/s rd, 216 MiB/s wr, 472 op/s 2024-03-16T09:01:37.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:37 smithi019 ceph-mon[36407]: pgmap v32: 65 pgs: 65 active+clean; 2.4 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 168 MiB/s rd, 216 MiB/s wr, 472 op/s 2024-03-16T09:01:37.996 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-16T09:01:37.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-16T09:01:38.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:38 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:01:38.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:38 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:01:39.703 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-16T09:01:39.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-16T09:01:39.880 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:39 smithi007 ceph-mon[29470]: pgmap v33: 65 pgs: 65 active+clean; 2.4 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 53 MiB/s rd, 70 MiB/s wr, 255 op/s 2024-03-16T09:01:39.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:39 smithi019 ceph-mon[36407]: pgmap v33: 65 pgs: 65 active+clean; 2.4 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 53 MiB/s rd, 70 MiB/s wr, 255 op/s 2024-03-16T09:01:41.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:41 smithi007 ceph-mon[29470]: pgmap v34: 65 pgs: 65 active+clean; 731 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 53 MiB/s rd, 71 MiB/s wr, 378 op/s 2024-03-16T09:01:41.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:41 smithi019 ceph-mon[36407]: pgmap v34: 65 pgs: 65 active+clean; 731 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 53 MiB/s rd, 71 MiB/s wr, 378 op/s 2024-03-16T09:01:43.701 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:43 smithi007 ceph-mon[29470]: pgmap v35: 65 pgs: 65 active+clean; 731 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 31 KiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-03-16T09:01:43.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:43 smithi019 ceph-mon[36407]: pgmap v35: 65 pgs: 65 active+clean; 731 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 31 KiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-03-16T09:01:45.629 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:45 smithi007 ceph-mon[29470]: pgmap v36: 65 pgs: 65 active+clean; 585 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 49 KiB/s rd, 2.1 MiB/s wr, 267 op/s 2024-03-16T09:01:45.629 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:45 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:45.630 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:45 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:45.630 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:45 smithi007 ceph-mon[29470]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-03-16T09:01:45.630 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:45 smithi007 ceph-mon[29470]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-03-16T09:01:45.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:45 smithi019 ceph-mon[36407]: pgmap v36: 65 pgs: 65 active+clean; 585 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 49 KiB/s rd, 2.1 MiB/s wr, 267 op/s 2024-03-16T09:01:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:45 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:45 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:45 smithi019 ceph-mon[36407]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-03-16T09:01:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:45 smithi019 ceph-mon[36407]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-03-16T09:01:47.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:47 smithi019 ceph-mon[36407]: pgmap v37: 65 pgs: 65 active+clean; 288 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.8 MiB/s wr, 369 op/s 2024-03-16T09:01:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:47 smithi007 ceph-mon[29470]: pgmap v37: 65 pgs: 65 active+clean; 288 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 2.8 MiB/s wr, 369 op/s 2024-03-16T09:01:49.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:49 smithi019 ceph-mon[36407]: pgmap v38: 65 pgs: 65 active+clean; 288 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-03-16T09:01:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:49 smithi007 ceph-mon[29470]: pgmap v38: 65 pgs: 65 active+clean; 288 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-03-16T09:01:51.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:51 smithi019 ceph-mon[36407]: pgmap v39: 65 pgs: 65 active+clean; 294 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-03-16T09:01:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:51 smithi007 ceph-mon[29470]: pgmap v39: 65 pgs: 65 active+clean; 294 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-03-16T09:01:53.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:53 smithi019 ceph-mon[36407]: pgmap v40: 65 pgs: 65 active+clean; 294 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.4 MiB/s wr, 298 op/s 2024-03-16T09:01:53.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:53 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:01:53.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:53 smithi007 ceph-mon[29470]: pgmap v40: 65 pgs: 65 active+clean; 294 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.4 MiB/s wr, 298 op/s 2024-03-16T09:01:53.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:53 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:01:55.842 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:55 smithi007 ceph-mon[29470]: pgmap v41: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 1.9 MiB/s wr, 383 op/s 2024-03-16T09:01:55.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:55 smithi019 ceph-mon[36407]: pgmap v41: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 51 KiB/s rd, 1.9 MiB/s wr, 383 op/s 2024-03-16T09:01:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:56 smithi007 ceph-mon[29470]: pgmap v42: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 2.1 MiB/s wr, 431 op/s 2024-03-16T09:01:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:56 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:56 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:56 smithi007 ceph-mon[29470]: Reconfiguring node-exporter.smithi019 (dependencies changed)... 2024-03-16T09:01:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:56 smithi007 ceph-mon[29470]: Reconfiguring daemon node-exporter.smithi019 on smithi019 2024-03-16T09:01:57.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:56 smithi019 ceph-mon[36407]: pgmap v42: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 2.1 MiB/s wr, 431 op/s 2024-03-16T09:01:57.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:56 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:57.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:56 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:01:57.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:56 smithi019 ceph-mon[36407]: Reconfiguring node-exporter.smithi019 (dependencies changed)... 2024-03-16T09:01:57.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:56 smithi019 ceph-mon[36407]: Reconfiguring daemon node-exporter.smithi019 on smithi019 2024-03-16T09:01:59.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:01:58 smithi007 ceph-mon[29470]: pgmap v43: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-03-16T09:01:59.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:01:58 smithi019 ceph-mon[36407]: pgmap v43: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-03-16T09:02:01.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:00 smithi007 ceph-mon[29470]: pgmap v44: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.1 MiB/s wr, 472 op/s 2024-03-16T09:02:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/2507427811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fdff0 msgr2=0x7fa9c00fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 --2- 172.21.15.7:0/2507427811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fdff0 0x7fa9c00fe460 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8004940 tx=0x7fa9a8009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/2507427811 shutdown_connections 2024-03-16T09:02:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 --2- 172.21.15.7:0/2507427811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fdff0 0x7fa9c00fe460 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 --2- 172.21.15.7:0/2507427811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00fce80 0x7fa9c00fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/2507427811 >> 172.21.15.7:0/2507427811 conn(0x7fa9c00f8580 msgr2=0x7fa9c00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/2507427811 shutdown_connections 2024-03-16T09:02:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/2507427811 wait complete. 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.419+0000 7fa9c5dc8700 1 Processor -- start 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9c5dc8700 1 -- start start 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9c5dc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fce80 0x7fa9c0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9c5dc8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00fdff0 0x7fa9c0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9c5dc8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c00708c0 con 0x7fa9c00fce80 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9c5dc8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0070a30 con 0x7fa9c00fdff0 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fce80 0x7fa9c0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fce80 0x7fa9c0071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42194/0 (socket says 172.21.15.7:42194) 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/1599071975 learned_addr learned my addr 172.21.15.7:0/1599071975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9beffd700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00fdff0 0x7fa9c0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00fdff0 msgr2=0x7fa9c0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00fdff0 0x7fa9c0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9bf7fe700 1 -- 172.21.15.7:0/1599071975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8005040 con 0x7fa9c00fce80 2024-03-16T09:02:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.420+0000 7fa9bf7fe700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fce80 0x7fa9c0071e40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800eb40 tx=0x7fa9b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.421+0000 7fa9b7fff700 1 -- 172.21.15.7:0/1599071975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9b8007cc0 con 0x7fa9c00fce80 2024-03-16T09:02:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.421+0000 7fa9b7fff700 1 -- 172.21.15.7:0/1599071975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9b800f040 con 0x7fa9c00fce80 2024-03-16T09:02:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.421+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c0070cb0 con 0x7fa9c00fce80 2024-03-16T09:02:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.421+0000 7fa9b7fff700 1 -- 172.21.15.7:0/1599071975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9b800ca10 con 0x7fa9c00fce80 2024-03-16T09:02:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.421+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c01159c0 con 0x7fa9c00fce80 2024-03-16T09:02:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.421+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c005f5f0 con 0x7fa9c00fce80 2024-03-16T09:02:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.422+0000 7fa9b7fff700 1 -- 172.21.15.7:0/1599071975 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7fa9b800cc80 con 0x7fa9c00fce80 2024-03-16T09:02:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.422+0000 7fa9b7fff700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa9ac06b660 0x7fa9ac06db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.422+0000 7fa9b7fff700 1 -- 172.21.15.7:0/1599071975 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa9b8089e40 con 0x7fa9c00fce80 2024-03-16T09:02:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.423+0000 7fa9beffd700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa9ac06b660 0x7fa9ac06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.423+0000 7fa9beffd700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa9ac06b660 0x7fa9ac06db10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009b30 tx=0x7fa9a8009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:01.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.428+0000 7fa9b7fff700 1 -- 172.21.15.7:0/1599071975 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fa9b8050040 con 0x7fa9c00fce80 2024-03-16T09:02:01.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:00 smithi019 ceph-mon[36407]: pgmap v44: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.1 MiB/s wr, 472 op/s 2024-03-16T09:02:01.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.651+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9c01022c0 con 0x7fa9ac06b660 2024-03-16T09:02:01.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.652+0000 7fa9b7fff700 1 -- 172.21.15.7:0/1599071975 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fa9c01022c0 con 0x7fa9ac06b660 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.654+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa9ac06b660 msgr2=0x7fa9ac06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.654+0000 7fa9c5dc8700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa9ac06b660 0x7fa9ac06db10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009b30 tx=0x7fa9a8009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.654+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fce80 msgr2=0x7fa9c0071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.654+0000 7fa9c5dc8700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fce80 0x7fa9c0071e40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800eb40 tx=0x7fa9b800b040 comp rx=0 tx=0).stop 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.655+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 shutdown_connections 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.655+0000 7fa9c5dc8700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7fa9ac06b660 0x7fa9ac06db10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.655+0000 7fa9c5dc8700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9c00fce80 0x7fa9c0071e40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.655+0000 7fa9c5dc8700 1 --2- 172.21.15.7:0/1599071975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9c00fdff0 0x7fa9c0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.655+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 >> 172.21.15.7:0/1599071975 conn(0x7fa9c00f8580 msgr2=0x7fa9c00f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.655+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 shutdown_connections 2024-03-16T09:02:01.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.655+0000 7fa9c5dc8700 1 -- 172.21.15.7:0/1599071975 wait complete. 2024-03-16T09:02:01.667 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:02:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 -- 172.21.15.7:0/1950198498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 msgr2=0x7f9d680fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 --2- 172.21.15.7:0/1950198498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 0x7f9d680fd280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9d50004940 tx=0x7f9d50009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 -- 172.21.15.7:0/1950198498 shutdown_connections 2024-03-16T09:02:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 --2- 172.21.15.7:0/1950198498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d680fdfe0 0x7f9d680fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 --2- 172.21.15.7:0/1950198498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 0x7f9d680fd280 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 -- 172.21.15.7:0/1950198498 >> 172.21.15.7:0/1950198498 conn(0x7f9d680f8550 msgr2=0x7f9d680fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 -- 172.21.15.7:0/1950198498 shutdown_connections 2024-03-16T09:02:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 -- 172.21.15.7:0/1950198498 wait complete. 2024-03-16T09:02:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 Processor -- start 2024-03-16T09:02:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.759+0000 7f9d6e450700 1 -- start start 2024-03-16T09:02:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d6e450700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 0x7f9d6810fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d6e450700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d680fdfe0 0x7f9d68110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d6e450700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68110640 con 0x7f9d680fce70 2024-03-16T09:02:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d6e450700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d681107b0 con 0x7f9d680fdfe0 2024-03-16T09:02:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d67fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 0x7f9d6810fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d67fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 0x7f9d6810fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42212/0 (socket says 172.21.15.7:42212) 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d67fff700 1 -- 172.21.15.7:0/2392084881 learned_addr learned my addr 172.21.15.7:0/2392084881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d677fe700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d680fdfe0 0x7f9d68110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d67fff700 1 -- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d680fdfe0 msgr2=0x7f9d68110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d67fff700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d680fdfe0 0x7f9d68110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d67fff700 1 -- 172.21.15.7:0/2392084881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d50005040 con 0x7f9d680fce70 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d67fff700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 0x7f9d6810fb00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9d50013870 tx=0x7f9d50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d64ff9700 1 -- 172.21.15.7:0/2392084881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5000e070 con 0x7f9d680fce70 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d64ff9700 1 -- 172.21.15.7:0/2392084881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d50013de0 con 0x7f9d680fce70 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d681151b0 con 0x7f9d680fce70 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.760+0000 7f9d64ff9700 1 -- 172.21.15.7:0/2392084881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5001cea0 con 0x7f9d680fce70 2024-03-16T09:02:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.761+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d68115690 con 0x7f9d680fce70 2024-03-16T09:02:01.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.762+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d68101970 con 0x7f9d680fce70 2024-03-16T09:02:01.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.762+0000 7f9d64ff9700 1 -- 172.21.15.7:0/2392084881 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7f9d50013910 con 0x7f9d680fce70 2024-03-16T09:02:01.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.762+0000 7f9d64ff9700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f9d5406b670 0x7f9d5406db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.762+0000 7f9d64ff9700 1 -- 172.21.15.7:0/2392084881 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9d5008b970 con 0x7f9d680fce70 2024-03-16T09:02:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.762+0000 7f9d677fe700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f9d5406b670 0x7f9d5406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.763+0000 7f9d677fe700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f9d5406b670 0x7f9d5406db20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009b80 tx=0x7f9d5800b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:01.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:01.767+0000 7f9d64ff9700 1 -- 172.21.15.7:0/2392084881 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f9d50059120 con 0x7f9d680fce70 2024-03-16T09:02:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.021+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d6802b340 con 0x7f9d5406b670 2024-03-16T09:02:02.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.022+0000 7f9d64ff9700 1 -- 172.21.15.7:0/2392084881 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f9d6802b340 con 0x7f9d5406b670 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f9d5406b670 msgr2=0x7f9d5406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f9d5406b670 0x7f9d5406db20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009b80 tx=0x7f9d5800b040 comp rx=0 tx=0).stop 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 msgr2=0x7f9d6810fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 0x7f9d6810fb00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9d50013870 tx=0x7f9d50009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 shutdown_connections 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f9d5406b670 0x7f9d5406db20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d680fce70 0x7f9d6810fb00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 --2- 172.21.15.7:0/2392084881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d680fdfe0 0x7f9d68110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 >> 172.21.15.7:0/2392084881 conn(0x7f9d680f8550 msgr2=0x7f9d680f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 shutdown_connections 2024-03-16T09:02:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.025+0000 7f9d6e450700 1 -- 172.21.15.7:0/2392084881 wait complete. 2024-03-16T09:02:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.137+0000 7f8534468700 1 -- 172.21.15.7:0/2019273633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 msgr2=0x7f852c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.137+0000 7f8534468700 1 --2- 172.21.15.7:0/2019273633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 0x7f852c102340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f851c004940 tx=0x7f851c009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.137+0000 7f8534468700 1 -- 172.21.15.7:0/2019273633 shutdown_connections 2024-03-16T09:02:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.137+0000 7f8534468700 1 --2- 172.21.15.7:0/2019273633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 0x7f852c102340 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.137+0000 7f8534468700 1 --2- 172.21.15.7:0/2019273633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c066280 0x7f852c0ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.137+0000 7f8534468700 1 -- 172.21.15.7:0/2019273633 >> 172.21.15.7:0/2019273633 conn(0x7f852c0f7430 msgr2=0x7f852c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.137+0000 7f8534468700 1 -- 172.21.15.7:0/2019273633 shutdown_connections 2024-03-16T09:02:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.137+0000 7f8534468700 1 -- 172.21.15.7:0/2019273633 wait complete. 2024-03-16T09:02:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8534468700 1 Processor -- start 2024-03-16T09:02:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8534468700 1 -- start start 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8534468700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c066280 0x7f852c10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8534468700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 0x7f852c10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8534468700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c10e4f0 con 0x7f852c0fff60 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8534468700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c10ed40 con 0x7f852c066280 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8531a03700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 0x7f852c10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8532204700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c066280 0x7f852c10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8531a03700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 0x7f852c10def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42222/0 (socket says 172.21.15.7:42222) 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.138+0000 7f8531a03700 1 -- 172.21.15.7:0/1613279347 learned_addr learned my addr 172.21.15.7:0/1613279347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8532204700 1 -- 172.21.15.7:0/1613279347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 msgr2=0x7f852c10def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8532204700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 0x7f852c10def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8532204700 1 -- 172.21.15.7:0/1613279347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8528005040 con 0x7f852c066280 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8531a03700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 0x7f852c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8532204700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c066280 0x7f852c10d9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8528009fd0 tx=0x7f8528009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8522ffd700 1 -- 172.21.15.7:0/1613279347 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8528012570 con 0x7f852c066280 2024-03-16T09:02:02.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8522ffd700 1 -- 172.21.15.7:0/1613279347 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f852800f040 con 0x7f852c066280 2024-03-16T09:02:02.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c005040 con 0x7f852c066280 2024-03-16T09:02:02.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8522ffd700 1 -- 172.21.15.7:0/1613279347 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8528013c00 con 0x7f852c066280 2024-03-16T09:02:02.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.139+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c10f3d0 con 0x7f852c066280 2024-03-16T09:02:02.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.140+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c05f5f0 con 0x7f852c066280 2024-03-16T09:02:02.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.141+0000 7f8522ffd700 1 -- 172.21.15.7:0/1613279347 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7f85280126d0 con 0x7f852c066280 2024-03-16T09:02:02.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.141+0000 7f8522ffd700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f851806b660 0x7f851806db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.141+0000 7f8522ffd700 1 -- 172.21.15.7:0/1613279347 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8528089fb0 con 0x7f852c066280 2024-03-16T09:02:02.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.141+0000 7f8531a03700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f851806b660 0x7f851806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.142+0000 7f8531a03700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f851806b660 0x7f851806db10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f851c00b0d0 tx=0x7f851c009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.146+0000 7f8522ffd700 1 -- 172.21.15.7:0/1613279347 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f8528057660 con 0x7f852c066280 2024-03-16T09:02:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.361+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f852c10f750 con 0x7f851806b660 2024-03-16T09:02:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.371+0000 7f8522ffd700 1 -- 172.21.15.7:0/1613279347 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f852c10f750 con 0x7f851806b660 2024-03-16T09:02:02.372 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:02:02.372 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 starting - - - - 2024-03-16T09:02:02.372 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (9m) 69s ago 9m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:02:02.372 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (8m) 75s ago 8m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 starting - - - - 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (3m) 69s ago 3m 11.3M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (3m) 69s ago 3m 140M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (3m) 75s ago 3m 59.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (3m) 75s ago 3m 11.9M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:9283 running (10m) 69s ago 10m 385M - 17.2.7-648-g1df27247 0bdcfccf14ce 376995a75ba3 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283 running (92s) 75s ago 8m 491M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 35bb11552f05 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (10m) 69s ago 10m 45.8M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (7m) 75s ago 7m 41.4M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 starting - - - - 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (8m) 75s ago 8m 16.8M - 1.5.0 0da6a335fe13 9edea61eb01f 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 69s ago 7m 100M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (6m) 69s ago 6m 87.9M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:02:02.373 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (6m) 69s ago 6m 82.8M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:02:02.374 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (5m) 75s ago 5m 95.0M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:02:02.374 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (5m) 75s ago 5m 82.6M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:02:02.374 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (5m) 75s ago 5m 86.5M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:02:02.374 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 starting - - - - 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.373+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f851806b660 msgr2=0x7f851806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.373+0000 7f8534468700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f851806b660 0x7f851806db10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f851c00b0d0 tx=0x7f851c009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c066280 msgr2=0x7f852c10d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c066280 0x7f852c10d9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8528009fd0 tx=0x7f8528009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 shutdown_connections 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7f851806b660 0x7f851806db10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f852c066280 0x7f852c10d9b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 --2- 172.21.15.7:0/1613279347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f852c0fff60 0x7f852c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 >> 172.21.15.7:0/1613279347 conn(0x7f852c0f7430 msgr2=0x7f852c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 shutdown_connections 2024-03-16T09:02:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.374+0000 7f8534468700 1 -- 172.21.15.7:0/1613279347 wait complete. 2024-03-16T09:02:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.475+0000 7ff406422700 1 -- 172.21.15.7:0/1705084890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fdfe0 msgr2=0x7ff4000fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.475+0000 7ff406422700 1 --2- 172.21.15.7:0/1705084890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fdfe0 0x7ff4000fe450 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8004940 tx=0x7ff3e8009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.475+0000 7ff406422700 1 -- 172.21.15.7:0/1705084890 shutdown_connections 2024-03-16T09:02:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.475+0000 7ff406422700 1 --2- 172.21.15.7:0/1705084890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fdfe0 0x7ff4000fe450 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.475+0000 7ff406422700 1 --2- 172.21.15.7:0/1705084890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4000fce70 0x7ff4000fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.475+0000 7ff406422700 1 -- 172.21.15.7:0/1705084890 >> 172.21.15.7:0/1705084890 conn(0x7ff4000f8550 msgr2=0x7ff4000fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.475+0000 7ff406422700 1 -- 172.21.15.7:0/1705084890 shutdown_connections 2024-03-16T09:02:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.475+0000 7ff406422700 1 -- 172.21.15.7:0/1705084890 wait complete. 2024-03-16T09:02:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff406422700 1 Processor -- start 2024-03-16T09:02:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff406422700 1 -- start start 2024-03-16T09:02:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff406422700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fce70 0x7ff40010d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff406422700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4000fdfe0 0x7ff40010ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff406422700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40010e3d0 con 0x7ff4000fce70 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff406422700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40010e540 con 0x7ff4000fdfe0 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fce70 0x7ff40010d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fce70 0x7ff40010d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59406/0 (socket says 172.21.15.7:59406) 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff3fffff700 1 -- 172.21.15.7:0/262593380 learned_addr learned my addr 172.21.15.7:0/262593380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.476+0000 7ff3ff7fe700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4000fdfe0 0x7ff40010ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff3fffff700 1 -- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4000fdfe0 msgr2=0x7ff40010ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff3fffff700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4000fdfe0 0x7ff40010ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff3fffff700 1 -- 172.21.15.7:0/262593380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e8005040 con 0x7ff4000fce70 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff3fffff700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fce70 0x7ff40010d890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000eb40 tx=0x7ff3f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff3fcff9700 1 -- 172.21.15.7:0/262593380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f0007cc0 con 0x7ff4000fce70 2024-03-16T09:02:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff3fcff9700 1 -- 172.21.15.7:0/262593380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3f000f040 con 0x7ff4000fce70 2024-03-16T09:02:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff400112f40 con 0x7ff4000fce70 2024-03-16T09:02:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff3fcff9700 1 -- 172.21.15.7:0/262593380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f000ca10 con 0x7ff4000fce70 2024-03-16T09:02:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.477+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4001133f0 con 0x7ff4000fce70 2024-03-16T09:02:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.478+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40004c710 con 0x7ff4000fce70 2024-03-16T09:02:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.478+0000 7ff3fcff9700 1 -- 172.21.15.7:0/262593380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7ff3f000cc80 con 0x7ff4000fce70 2024-03-16T09:02:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.479+0000 7ff3fcff9700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff3ec06b670 0x7ff3ec06db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.479+0000 7ff3fcff9700 1 -- 172.21.15.7:0/262593380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff3f0089e00 con 0x7ff4000fce70 2024-03-16T09:02:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.479+0000 7ff3ff7fe700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff3ec06b670 0x7ff3ec06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.479+0000 7ff3ff7fe700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff3ec06b670 0x7ff3ec06db20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009b30 tx=0x7ff3e8009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:02.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.483+0000 7ff3fcff9700 1 -- 172.21.15.7:0/262593380 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff3f0050040 con 0x7ff4000fce70 2024-03-16T09:02:02.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.789+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff4001135e0 con 0x7ff4000fce70 2024-03-16T09:02:02.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.790+0000 7ff3fcff9700 1 -- 172.21.15.7:0/262593380 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7ff3f00575e0 con 0x7ff4000fce70 2024-03-16T09:02:02.791 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:02:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 1, 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:02:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:02:02.793 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 13, 2024-03-16T09:02:02.793 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:02:02.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:02:02.793 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:02:02.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff3ec06b670 msgr2=0x7ff3ec06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff3ec06b670 0x7ff3ec06db20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009b30 tx=0x7ff3e8009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fce70 msgr2=0x7ff40010d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fce70 0x7ff40010d890 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000eb40 tx=0x7ff3f000b040 comp rx=0 tx=0).stop 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 shutdown_connections 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff3ec06b670 0x7ff3ec06db20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000fce70 0x7ff40010d890 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 --2- 172.21.15.7:0/262593380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4000fdfe0 0x7ff40010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.793+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 >> 172.21.15.7:0/262593380 conn(0x7ff4000f8550 msgr2=0x7ff4000f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.794+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 shutdown_connections 2024-03-16T09:02:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.794+0000 7ff406422700 1 -- 172.21.15.7:0/262593380 wait complete. 2024-03-16T09:02:02.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2217158866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fcf80 msgr2=0x7ff2c40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 --2- 172.21.15.7:0/2217158866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fcf80 0x7ff2c40fd3f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4004940 tx=0x7ff2b4009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:02.902 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:02 smithi007 ceph-mon[29470]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2217158866 shutdown_connections 2024-03-16T09:02:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 --2- 172.21.15.7:0/2217158866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fcf80 0x7ff2c40fd3f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 --2- 172.21.15.7:0/2217158866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c40fbcd0 0x7ff2c40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2217158866 >> 172.21.15.7:0/2217158866 conn(0x7ff2c40f7430 msgr2=0x7ff2c40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2217158866 shutdown_connections 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2217158866 wait complete. 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 Processor -- start 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.901+0000 7ff2c8f10700 1 -- start start 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c8f10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fbcd0 0x7ff2c41106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c8f10700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c40fcf80 0x7ff2c4112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c8f10700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c4066350 con 0x7ff2c40fbcd0 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c8f10700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c4066490 con 0x7ff2c40fcf80 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c40fcf80 0x7ff2c4112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c1d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c40fcf80 0x7ff2c4112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:43886/0 (socket says 172.21.15.7:43886) 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c1d9b700 1 -- 172.21.15.7:0/2173827366 learned_addr learned my addr 172.21.15.7:0/2173827366 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c259c700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fbcd0 0x7ff2c41106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c1d9b700 1 -- 172.21.15.7:0/2173827366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fbcd0 msgr2=0x7ff2c41106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c1d9b700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fbcd0 0x7ff2c41106a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c1d9b700 1 -- 172.21.15.7:0/2173827366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b4005040 con 0x7ff2c40fcf80 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.902+0000 7ff2c259c700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fbcd0 0x7ff2c41106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.903+0000 7ff2c1d9b700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c40fcf80 0x7ff2c4112bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4013870 tx=0x7ff2b4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.903+0000 7ff2baffd700 1 -- 172.21.15.7:0/2173827366 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2b4021070 con 0x7ff2c40fcf80 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.903+0000 7ff2baffd700 1 -- 172.21.15.7:0/2173827366 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2b4009430 con 0x7ff2c40fcf80 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.903+0000 7ff2baffd700 1 -- 172.21.15.7:0/2173827366 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2b400e070 con 0x7ff2c40fcf80 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.903+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c4113130 con 0x7ff2c40fcf80 2024-03-16T09:02:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.904+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c4113670 con 0x7ff2c40fcf80 2024-03-16T09:02:02.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.904+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c405f5f0 con 0x7ff2c40fcf80 2024-03-16T09:02:02.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.905+0000 7ff2baffd700 1 -- 172.21.15.7:0/2173827366 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) v1 ==== 89190+0+0 (secure 0 0 0) 0x7ff2b401c750 con 0x7ff2c40fcf80 2024-03-16T09:02:02.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.905+0000 7ff2baffd700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff2b006b600 0x7ff2b006dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:02.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.905+0000 7ff2baffd700 1 -- 172.21.15.7:0/2173827366 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff2b408aff0 con 0x7ff2c40fcf80 2024-03-16T09:02:02.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.906+0000 7ff2c259c700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff2b006b600 0x7ff2b006dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:02.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.906+0000 7ff2c259c700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff2b006b600 0x7ff2b006dab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4065fd0 tx=0x7ff2ac00b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:02.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:02.911+0000 7ff2baffd700 1 -- 172.21.15.7:0/2173827366 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff2b4054e50 con 0x7ff2c40fcf80 2024-03-16T09:02:02.947 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:02 smithi019 ceph-mon[36407]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:03.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.145+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 --> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2c41012a0 con 0x7ff2b006b600 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.147+0000 7ff2baffd700 1 -- 172.21.15.7:0/2173827366 <== mgr.24383 v2:172.21.15.19:6828/3745729358 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff2c41012a0 con 0x7ff2b006b600 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "1/2 daemons upgraded", 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:02:03.148 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:02:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff2b006b600 msgr2=0x7ff2b006dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff2b006b600 0x7ff2b006dab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4065fd0 tx=0x7ff2ac00b040 comp rx=0 tx=0).stop 2024-03-16T09:02:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c40fcf80 msgr2=0x7ff2c4112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c40fcf80 0x7ff2c4112bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4013870 tx=0x7ff2b4009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 shutdown_connections 2024-03-16T09:02:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:6828/3745729358,v1:172.21.15.19:6829/3745729358] conn(0x7ff2b006b600 0x7ff2b006dab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c40fbcd0 0x7ff2c41106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 --2- 172.21.15.7:0/2173827366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2c40fcf80 0x7ff2c4112bf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 >> 172.21.15.7:0/2173827366 conn(0x7ff2c40f7430 msgr2=0x7ff2c40f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 shutdown_connections 2024-03-16T09:02:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:03.149+0000 7ff2c8f10700 1 -- 172.21.15.7:0/2173827366 wait complete. 2024-03-16T09:02:03.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:03 smithi019 ceph-mon[36407]: pgmap v45: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.4 MiB/s wr, 322 op/s 2024-03-16T09:02:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:03 smithi019 ceph-mon[36407]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:03 smithi019 ceph-mon[36407]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:03 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/262593380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:02:03.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:03 smithi007 ceph-mon[29470]: pgmap v45: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.4 MiB/s wr, 322 op/s 2024-03-16T09:02:03.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:03 smithi007 ceph-mon[29470]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:03.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:03 smithi007 ceph-mon[29470]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:03.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:03 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/262593380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:02:05.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:04 smithi019 ceph-mon[36407]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:04 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:04 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:04 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T09:02:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:04 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T09:02:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:04 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:02:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:04 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:05.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:04 smithi007 ceph-mon[29470]: from='client.24471 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:05.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:04 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:05.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:04 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:05.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:04 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T09:02:05.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:04 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T09:02:05.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:04 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:02:05.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:04 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: Upgrade: Need to upgrade myself (mgr.smithi019.iuqxhi) 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: Upgrade: Need to upgrade myself (mgr.smithi019.iuqxhi) 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: pgmap v46: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.1 MiB/s wr, 348 op/s 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.uamwtp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.uamwtp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:02:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:05 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:02:06.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T09:02:06.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T09:02:06.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:02:06.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: Upgrade: Need to upgrade myself (mgr.smithi019.iuqxhi) 2024-03-16T09:02:06.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: Upgrade: Need to upgrade myself (mgr.smithi019.iuqxhi) 2024-03-16T09:02:06.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: pgmap v46: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.1 MiB/s wr, 348 op/s 2024-03-16T09:02:06.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:06.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.uamwtp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:02:06.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.uamwtp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:02:06.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:02:06.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:05 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:02:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:06 smithi007 ceph-mon[29470]: Upgrade: Updating mgr.smithi007.uamwtp 2024-03-16T09:02:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:06 smithi007 ceph-mon[29470]: Deploying daemon mgr.smithi007.uamwtp on smithi007 2024-03-16T09:02:07.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:06 smithi019 ceph-mon[36407]: Upgrade: Updating mgr.smithi007.uamwtp 2024-03-16T09:02:07.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:06 smithi019 ceph-mon[36407]: Deploying daemon mgr.smithi007.uamwtp on smithi007 2024-03-16T09:02:07.970 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:07 smithi007 ceph-mon[29470]: pgmap v47: 65 pgs: 65 active+clean; 315 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 2.3 MiB/s wr, 297 op/s 2024-03-16T09:02:08.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:07 smithi019 ceph-mon[36407]: pgmap v47: 65 pgs: 65 active+clean; 315 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 2.3 MiB/s wr, 297 op/s 2024-03-16T09:02:09.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:08 smithi019 ceph-mon[36407]: pgmap v48: 65 pgs: 65 active+clean; 315 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-03-16T09:02:09.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:08 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:02:09.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:08 smithi007 ceph-mon[29470]: pgmap v48: 65 pgs: 65 active+clean; 315 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-03-16T09:02:09.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:08 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:02:10.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:09 smithi019 ceph-mon[36407]: mgrmap e25: smithi019.iuqxhi(active, since 90s), standbys: smithi007.uamwtp 2024-03-16T09:02:10.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:09 smithi007 ceph-mon[29470]: mgrmap e25: smithi019.iuqxhi(active, since 90s), standbys: smithi007.uamwtp 2024-03-16T09:02:11.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:10 smithi019 ceph-mon[36407]: pgmap v49: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 3.4 MiB/s wr, 452 op/s 2024-03-16T09:02:11.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:10 smithi007 ceph-mon[29470]: pgmap v49: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 3.4 MiB/s wr, 452 op/s 2024-03-16T09:02:12.223 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:11 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:12.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:11 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:12.224 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:11 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:12.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:11 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:12.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:11 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:12.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:11 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:13.913 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:13 smithi007 ceph-mon[29470]: pgmap v50: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 2.7 MiB/s wr, 303 op/s 2024-03-16T09:02:13.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:13 smithi019 ceph-mon[36407]: pgmap v50: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 2.7 MiB/s wr, 303 op/s 2024-03-16T09:02:16.130 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:15 smithi007 ceph-mon[29470]: pgmap v51: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 60 KiB/s rd, 3.5 MiB/s wr, 437 op/s 2024-03-16T09:02:16.130 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:15 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:16.131 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:15 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:16.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:15 smithi019 ceph-mon[36407]: pgmap v51: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 60 KiB/s rd, 3.5 MiB/s wr, 437 op/s 2024-03-16T09:02:16.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:15 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:16.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:15 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:17.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:17 smithi019 ceph-mon[36407]: pgmap v52: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 3.4 MiB/s wr, 555 op/s 2024-03-16T09:02:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:17 smithi007 ceph-mon[29470]: pgmap v52: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 3.4 MiB/s wr, 555 op/s 2024-03-16T09:02:20.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:19 smithi019 ceph-mon[36407]: pgmap v53: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 2.7 MiB/s wr, 521 op/s 2024-03-16T09:02:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:19 smithi007 ceph-mon[29470]: pgmap v53: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 2.7 MiB/s wr, 521 op/s 2024-03-16T09:02:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:20 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.uamwtp/crt"}]: dispatch 2024-03-16T09:02:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:20 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:02:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:20 smithi019 ceph-mon[36407]: Standby manager daemon smithi007.uamwtp restarted 2024-03-16T09:02:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:20 smithi019 ceph-mon[36407]: Standby manager daemon smithi007.uamwtp started 2024-03-16T09:02:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:20 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.uamwtp/key"}]: dispatch 2024-03-16T09:02:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:20 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:02:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:20 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.uamwtp/crt"}]: dispatch 2024-03-16T09:02:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:20 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:02:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:20 smithi007 ceph-mon[29470]: Standby manager daemon smithi007.uamwtp restarted 2024-03-16T09:02:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:20 smithi007 ceph-mon[29470]: Standby manager daemon smithi007.uamwtp started 2024-03-16T09:02:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:20 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi007.uamwtp/key"}]: dispatch 2024-03-16T09:02:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:20 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:02:22.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:21 smithi019 ceph-mon[36407]: pgmap v54: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 70 KiB/s rd, 3.5 MiB/s wr, 681 op/s 2024-03-16T09:02:22.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:21 smithi019 ceph-mon[36407]: mgrmap e26: smithi019.iuqxhi(active, since 102s), standbys: smithi007.uamwtp 2024-03-16T09:02:22.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:21 smithi007 ceph-mon[29470]: pgmap v54: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 70 KiB/s rd, 3.5 MiB/s wr, 681 op/s 2024-03-16T09:02:22.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:21 smithi007 ceph-mon[29470]: mgrmap e26: smithi019.iuqxhi(active, since 102s), standbys: smithi007.uamwtp 2024-03-16T09:02:24.098 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:23 smithi007 ceph-mon[29470]: pgmap v55: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 437 op/s 2024-03-16T09:02:24.098 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:23 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:02:24.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:23 smithi019 ceph-mon[36407]: pgmap v55: 65 pgs: 65 active+clean; 311 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 437 op/s 2024-03-16T09:02:24.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:23 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:02:26.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:25 smithi019 ceph-mon[36407]: pgmap v56: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 2.2 MiB/s wr, 464 op/s 2024-03-16T09:02:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:25 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:25 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:25 smithi007 ceph-mon[29470]: pgmap v56: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 2.2 MiB/s wr, 464 op/s 2024-03-16T09:02:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:25 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:25 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:27.410 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:27 smithi007 ceph-mon[29470]: pgmap v57: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 2.2 MiB/s wr, 470 op/s 2024-03-16T09:02:27.410 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:27 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:27.410 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:27 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:27.630 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:27 smithi019 ceph-mon[36407]: pgmap v57: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 2.2 MiB/s wr, 470 op/s 2024-03-16T09:02:27.631 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:27 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:27.631 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:27 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:29.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:29 smithi019 ceph-mon[36407]: pgmap v58: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.5 MiB/s wr, 327 op/s 2024-03-16T09:02:29.730 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:29 smithi007 ceph-mon[29470]: pgmap v58: 65 pgs: 65 active+clean; 313 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.5 MiB/s wr, 327 op/s 2024-03-16T09:02:31.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr fail", "who": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr fail", "who": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: osdmap e48: 6 total, 6 up, 6 in 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd='[{"prefix": "mgr fail", "who": "smithi019.iuqxhi"}]': finished 2024-03-16T09:02:31.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: mgrmap e27: smithi007.uamwtp(active, starting, since 0.00604747s) 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:02:31.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:02:31.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:02:31.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:02:31.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: Manager daemon smithi007.uamwtp is now available 2024-03-16T09:02:31.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:31.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:31.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:02:31.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:02:31.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T09:02:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 172.21.15.19:0/694085090' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr fail", "who": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "mgr fail", "who": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: osdmap e48: 6 total, 6 up, 6 in 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24383 ' entity='mgr.smithi019.iuqxhi' cmd='[{"prefix": "mgr fail", "who": "smithi019.iuqxhi"}]': finished 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: mgrmap e27: smithi007.uamwtp(active, starting, since 0.00604747s) 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:02:31.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: Manager daemon smithi007.uamwtp is now available 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:02:31.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T09:02:32.917 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:32 smithi019 ceph-mon[36407]: [16/Mar/2024:09:02:31] ENGINE Bus STARTING 2024-03-16T09:02:32.917 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:32 smithi019 ceph-mon[36407]: [16/Mar/2024:09:02:31] ENGINE Serving on http://172.21.15.7:8765 2024-03-16T09:02:32.917 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:32 smithi019 ceph-mon[36407]: [16/Mar/2024:09:02:31] ENGINE Serving on https://172.21.15.7:7150 2024-03-16T09:02:32.917 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:32 smithi019 ceph-mon[36407]: [16/Mar/2024:09:02:31] ENGINE Bus STARTED 2024-03-16T09:02:32.917 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:32 smithi019 ceph-mon[36407]: mgrmap e28: smithi007.uamwtp(active, since 1.00841s) 2024-03-16T09:02:32.917 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:32 smithi019 ceph-mon[36407]: pgmap v3: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:02:32.917 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:32 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:32.917 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:32 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:33.028 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:32 smithi007 ceph-mon[29470]: [16/Mar/2024:09:02:31] ENGINE Bus STARTING 2024-03-16T09:02:33.028 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:32 smithi007 ceph-mon[29470]: [16/Mar/2024:09:02:31] ENGINE Serving on http://172.21.15.7:8765 2024-03-16T09:02:33.028 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:32 smithi007 ceph-mon[29470]: [16/Mar/2024:09:02:31] ENGINE Serving on https://172.21.15.7:7150 2024-03-16T09:02:33.028 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:32 smithi007 ceph-mon[29470]: [16/Mar/2024:09:02:31] ENGINE Bus STARTED 2024-03-16T09:02:33.028 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:32 smithi007 ceph-mon[29470]: mgrmap e28: smithi007.uamwtp(active, since 1.00841s) 2024-03-16T09:02:33.028 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:32 smithi007 ceph-mon[29470]: pgmap v3: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:02:33.028 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:32 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:33.028 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:32 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.281+0000 7f27f5b45700 1 -- 172.21.15.7:0/3994947023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fcee0 msgr2=0x7f27f00fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.281+0000 7f27f5b45700 1 --2- 172.21.15.7:0/3994947023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fcee0 0x7f27f00fd310 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f27e0004940 tx=0x7f27e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.281+0000 7f27f5b45700 1 -- 172.21.15.7:0/3994947023 shutdown_connections 2024-03-16T09:02:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.281+0000 7f27f5b45700 1 --2- 172.21.15.7:0/3994947023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f00fe0d0 0x7f27f00fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.281+0000 7f27f5b45700 1 --2- 172.21.15.7:0/3994947023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fcee0 0x7f27f00fd310 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.281+0000 7f27f5b45700 1 -- 172.21.15.7:0/3994947023 >> 172.21.15.7:0/3994947023 conn(0x7f27f00f8560 msgr2=0x7f27f00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.281+0000 7f27f5b45700 1 -- 172.21.15.7:0/3994947023 shutdown_connections 2024-03-16T09:02:33.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.281+0000 7f27f5b45700 1 -- 172.21.15.7:0/3994947023 wait complete. 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.282+0000 7f27f5b45700 1 Processor -- start 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.282+0000 7f27f5b45700 1 -- start start 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.282+0000 7f27f5b45700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f00fcee0 0x7f27f010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.282+0000 7f27f5b45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fe0d0 0x7f27f010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.282+0000 7f27f5b45700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f010e530 con 0x7f27f00fe0d0 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.282+0000 7f27f5b45700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f010ed80 con 0x7f27f00fcee0 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fe0d0 0x7f27f010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27ef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f00fcee0 0x7f27f010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27ef7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f00fcee0 0x7f27f010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:34964/0 (socket says 172.21.15.7:34964) 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27ef7fe700 1 -- 172.21.15.7:0/1976996435 learned_addr learned my addr 172.21.15.7:0/1976996435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27ef7fe700 1 -- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fe0d0 msgr2=0x7f27f010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27ef7fe700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fe0d0 0x7f27f010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27ef7fe700 1 -- 172.21.15.7:0/1976996435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0005040 con 0x7f27f00fcee0 2024-03-16T09:02:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27eeffd700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fe0d0 0x7f27f010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:02:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27ef7fe700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f00fcee0 0x7f27f010d9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27e0000c00 tx=0x7f27e0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.283+0000 7f27d7fff700 1 -- 172.21.15.7:0/1976996435 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27e000e030 con 0x7f27f00fcee0 2024-03-16T09:02:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.284+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f010f000 con 0x7f27f00fcee0 2024-03-16T09:02:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.284+0000 7f27d7fff700 1 -- 172.21.15.7:0/1976996435 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27e001c400 con 0x7f27f00fcee0 2024-03-16T09:02:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.284+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f010f540 con 0x7f27f00fcee0 2024-03-16T09:02:33.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.284+0000 7f27d7fff700 1 -- 172.21.15.7:0/1976996435 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27e001cea0 con 0x7f27f00fcee0 2024-03-16T09:02:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.285+0000 7f27d7fff700 1 -- 172.21.15.7:0/1976996435 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 29) v1 ==== 46048+0+0 (secure 0 0 0) 0x7f27e001c940 con 0x7f27f00fcee0 2024-03-16T09:02:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.285+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f005f5f0 con 0x7f27f00fcee0 2024-03-16T09:02:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.285+0000 7f27d7fff700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f27d8038dc0 0x7f27d803b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.285+0000 7f27d7fff700 1 -- 172.21.15.7:0/1976996435 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f27e004df30 con 0x7f27f00fcee0 2024-03-16T09:02:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.285+0000 7f27eeffd700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f27d8038dc0 0x7f27d803b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.286+0000 7f27eeffd700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f27d8038dc0 0x7f27d803b270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27e4004600 tx=0x7f27e400b040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.291+0000 7f27d7fff700 1 -- 172.21.15.7:0/1976996435 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f27e0023e40 con 0x7f27f00fcee0 2024-03-16T09:02:33.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.559+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27f010f820 con 0x7f27d8038dc0 2024-03-16T09:02:33.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.560+0000 7f27d7fff700 1 -- 172.21.15.7:0/1976996435 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f27f010f820 con 0x7f27d8038dc0 2024-03-16T09:02:33.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f27d8038dc0 msgr2=0x7f27d803b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:33.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f27d8038dc0 0x7f27d803b270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f27e4004600 tx=0x7f27e400b040 comp rx=0 tx=0).stop 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f00fcee0 msgr2=0x7f27f010d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f00fcee0 0x7f27f010d9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27e0000c00 tx=0x7f27e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 shutdown_connections 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f27d8038dc0 0x7f27d803b270 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27f00fcee0 0x7f27f010d9f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 --2- 172.21.15.7:0/1976996435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f00fe0d0 0x7f27f010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 >> 172.21.15.7:0/1976996435 conn(0x7f27f00f8560 msgr2=0x7f27f00f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.562+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 shutdown_connections 2024-03-16T09:02:33.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.563+0000 7f27f5b45700 1 -- 172.21.15.7:0/1976996435 wait complete. 2024-03-16T09:02:33.575 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:02:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.677+0000 7f1be901a700 1 -- 172.21.15.7:0/355750805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 msgr2=0x7f1be4102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.677+0000 7f1be901a700 1 --2- 172.21.15.7:0/355750805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 0x7f1be4102360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8004940 tx=0x7f1bd8009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.677+0000 7f1be901a700 1 -- 172.21.15.7:0/355750805 shutdown_connections 2024-03-16T09:02:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.677+0000 7f1be901a700 1 --2- 172.21.15.7:0/355750805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 0x7f1be4102360 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.677+0000 7f1be901a700 1 --2- 172.21.15.7:0/355750805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4066e70 0x7f1be40ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.677+0000 7f1be901a700 1 -- 172.21.15.7:0/355750805 >> 172.21.15.7:0/355750805 conn(0x7f1be40f7430 msgr2=0x7f1be40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.677+0000 7f1be901a700 1 -- 172.21.15.7:0/355750805 shutdown_connections 2024-03-16T09:02:33.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.677+0000 7f1be901a700 1 -- 172.21.15.7:0/355750805 wait complete. 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.678+0000 7f1be901a700 1 Processor -- start 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.678+0000 7f1be901a700 1 -- start start 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.678+0000 7f1be901a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4066e70 0x7f1be410d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.678+0000 7f1be901a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 0x7f1be410de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.678+0000 7f1be901a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be410e410 con 0x7f1be40fff80 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.678+0000 7f1be901a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be410e580 con 0x7f1be4066e70 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 0x7f1be410de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 0x7f1be410de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59130/0 (socket says 172.21.15.7:59130) 2024-03-16T09:02:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be259c700 1 -- 172.21.15.7:0/2361234530 learned_addr learned my addr 172.21.15.7:0/2361234530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be2d9d700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4066e70 0x7f1be410d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be259c700 1 -- 172.21.15.7:0/2361234530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4066e70 msgr2=0x7f1be410d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be259c700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4066e70 0x7f1be410d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be259c700 1 -- 172.21.15.7:0/2361234530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8005040 con 0x7f1be40fff80 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be2d9d700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4066e70 0x7f1be410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.679+0000 7f1be259c700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 0x7f1be410de10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8013870 tx=0x7f1bd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.680+0000 7f1bcb7fe700 1 -- 172.21.15.7:0/2361234530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bd8021070 con 0x7f1be40fff80 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.680+0000 7f1bcb7fe700 1 -- 172.21.15.7:0/2361234530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bd8009430 con 0x7f1be40fff80 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.680+0000 7f1bcb7fe700 1 -- 172.21.15.7:0/2361234530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bd800e070 con 0x7f1be40fff80 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.680+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be40fb820 con 0x7f1be40fff80 2024-03-16T09:02:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.680+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be40fbd60 con 0x7f1be40fff80 2024-03-16T09:02:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.681+0000 7f1bcb7fe700 1 -- 172.21.15.7:0/2361234530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 29) v1 ==== 46048+0+0 (secure 0 0 0) 0x7f1bd8026410 con 0x7f1be40fff80 2024-03-16T09:02:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.681+0000 7f1bcb7fe700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1bcc03d160 0x7f1bcc03f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.681+0000 7f1bcb7fe700 1 -- 172.21.15.7:0/2361234530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1bd804de40 con 0x7f1be40fff80 2024-03-16T09:02:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.681+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be404c710 con 0x7f1be40fff80 2024-03-16T09:02:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.682+0000 7f1be2d9d700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1bcc03d160 0x7f1bcc03f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.682+0000 7f1be2d9d700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1bcc03d160 0x7f1bcc03f610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009b80 tx=0x7f1bd400b040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:33.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.687+0000 7f1bcb7fe700 1 -- 172.21.15.7:0/2361234530 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f1bd8026800 con 0x7f1be40fff80 2024-03-16T09:02:33.868 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:33 smithi019 ceph-mon[36407]: pgmap v4: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:02:33.868 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:33 smithi019 ceph-mon[36407]: mgrmap e29: smithi007.uamwtp(active, since 2s) 2024-03-16T09:02:33.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.972+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1be40f8f70 con 0x7f1bcc03d160 2024-03-16T09:02:33.975 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:33 smithi007 ceph-mon[29470]: pgmap v4: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:02:33.975 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:33 smithi007 ceph-mon[29470]: mgrmap e29: smithi007.uamwtp(active, since 2s) 2024-03-16T09:02:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.974+0000 7f1bcb7fe700 1 -- 172.21.15.7:0/2361234530 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f1be40f8f70 con 0x7f1bcc03d160 2024-03-16T09:02:33.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1bcc03d160 msgr2=0x7f1bcc03f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:33.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1bcc03d160 0x7f1bcc03f610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009b80 tx=0x7f1bd400b040 comp rx=0 tx=0).stop 2024-03-16T09:02:33.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 msgr2=0x7f1be410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 0x7f1be410de10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8013870 tx=0x7f1bd8009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 shutdown_connections 2024-03-16T09:02:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1bcc03d160 0x7f1bcc03f610 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4066e70 0x7f1be410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 --2- 172.21.15.7:0/2361234530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be40fff80 0x7f1be410de10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 >> 172.21.15.7:0/2361234530 conn(0x7f1be40f7430 msgr2=0x7f1be40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 shutdown_connections 2024-03-16T09:02:33.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:33.978+0000 7f1be901a700 1 -- 172.21.15.7:0/2361234530 wait complete. 2024-03-16T09:02:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.147+0000 7f2ccb275700 1 -- 172.21.15.7:0/544850726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc406f9d0 msgr2=0x7f2cc406e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.147+0000 7f2ccb275700 1 --2- 172.21.15.7:0/544850726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc406f9d0 0x7f2cc406e460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0004940 tx=0x7f2cc0009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.147+0000 7f2ccb275700 1 -- 172.21.15.7:0/544850726 shutdown_connections 2024-03-16T09:02:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.147+0000 7f2ccb275700 1 --2- 172.21.15.7:0/544850726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc406e9a0 0x7f2cc406ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.147+0000 7f2ccb275700 1 --2- 172.21.15.7:0/544850726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc406f9d0 0x7f2cc406e460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.147+0000 7f2ccb275700 1 -- 172.21.15.7:0/544850726 >> 172.21.15.7:0/544850726 conn(0x7f2cc4069de0 msgr2=0x7f2cc406c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.147+0000 7f2ccb275700 1 -- 172.21.15.7:0/544850726 shutdown_connections 2024-03-16T09:02:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.147+0000 7f2ccb275700 1 -- 172.21.15.7:0/544850726 wait complete. 2024-03-16T09:02:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.148+0000 7f2ccb275700 1 Processor -- start 2024-03-16T09:02:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.148+0000 7f2ccb275700 1 -- start start 2024-03-16T09:02:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.148+0000 7f2ccb275700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc406e9a0 0x7f2cc411c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.148+0000 7f2ccb275700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc406f9d0 0x7f2cc411cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.149+0000 7f2ccb275700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc411d290 con 0x7f2cc406f9d0 2024-03-16T09:02:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.149+0000 7f2ccb275700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4121c30 con 0x7f2cc406e9a0 2024-03-16T09:02:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.149+0000 7f2cc9011700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc406e9a0 0x7f2cc411c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.149+0000 7f2cc9011700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc406e9a0 0x7f2cc411c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:35020/0 (socket says 172.21.15.7:35020) 2024-03-16T09:02:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.149+0000 7f2cc9011700 1 -- 172.21.15.7:0/3413213004 learned_addr learned my addr 172.21.15.7:0/3413213004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.149+0000 7f2cc8810700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc406f9d0 0x7f2cc411cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.150+0000 7f2cc9011700 1 -- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc406f9d0 msgr2=0x7f2cc411cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.150+0000 7f2cc9011700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc406f9d0 0x7f2cc411cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.150+0000 7f2cc9011700 1 -- 172.21.15.7:0/3413213004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc0005040 con 0x7f2cc406e9a0 2024-03-16T09:02:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.150+0000 7f2cc9011700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc406e9a0 0x7f2cc411c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009150 tx=0x7f2cc0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.151+0000 7f2cb9ffb700 1 -- 172.21.15.7:0/3413213004 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cc000e040 con 0x7f2cc406e9a0 2024-03-16T09:02:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.151+0000 7f2cb9ffb700 1 -- 172.21.15.7:0/3413213004 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cc000bbf0 con 0x7f2cc406e9a0 2024-03-16T09:02:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.151+0000 7f2cb9ffb700 1 -- 172.21.15.7:0/3413213004 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cc001c630 con 0x7f2cc406e9a0 2024-03-16T09:02:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.151+0000 7f2ccb275700 1 -- 172.21.15.7:0/3413213004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc4121e30 con 0x7f2cc406e9a0 2024-03-16T09:02:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.151+0000 7f2ccb275700 1 -- 172.21.15.7:0/3413213004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc4122250 con 0x7f2cc406e9a0 2024-03-16T09:02:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.152+0000 7f2cb9ffb700 1 -- 172.21.15.7:0/3413213004 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 29) v1 ==== 46048+0+0 (secure 0 0 0) 0x7f2cc000bd60 con 0x7f2cc406e9a0 2024-03-16T09:02:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.152+0000 7f2cb9ffb700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f2cb0038df0 0x7f2cb003b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.152+0000 7f2cb9ffb700 1 -- 172.21.15.7:0/3413213004 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2cc004ded0 con 0x7f2cc406e9a0 2024-03-16T09:02:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.153+0000 7f2cc8810700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f2cb0038df0 0x7f2cb003b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.153+0000 7f2caf7fe700 1 -- 172.21.15.7:0/3413213004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca40052b0 con 0x7f2cc406e9a0 2024-03-16T09:02:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.153+0000 7f2cc8810700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f2cb0038df0 0x7f2cb003b2a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009bf0 tx=0x7f2cb40091a0 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.162+0000 7f2cb9ffb700 1 -- 172.21.15.7:0/3413213004 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f2cc0011550 con 0x7f2cc406e9a0 2024-03-16T09:02:34.250 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-16T09:02:34.250 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-16T09:02:34.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.395+0000 7f2caf7fe700 1 -- 172.21.15.7:0/3413213004 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ca4000bc0 con 0x7f2cb0038df0 2024-03-16T09:02:34.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.407+0000 7f2cb9ffb700 1 -- 172.21.15.7:0/3413213004 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2ca4000bc0 con 0x7f2cb0038df0 2024-03-16T09:02:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:02:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (69s) 9s ago 9m 20.0M - 0.25.0 c8568f914cd2 d4d6445aa339 2024-03-16T09:02:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (9m) 9s ago 9m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:02:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (8m) 19s ago 8m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:02:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (50s) 9s ago 9m 79.7M - 9.4.7 954c08fa6188 32f01fbd2bf4 2024-03-16T09:02:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (4m) 9s ago 4m 12.0M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:02:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (3m) 9s ago 3m 168M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (3m) 19s ago 3m 173M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (3m) 19s ago 3m 12.5M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (24s) 9s ago 10m 62.5M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283 running (2m) 19s ago 8m 500M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 35bb11552f05 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (10m) 9s ago 10m 48.0M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (8m) 19s ago 8m 41.4M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (89s) 9s ago 9m 16.0M - 1.5.0 0da6a335fe13 500fda412d5a 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (31s) 19s ago 8m 7948k - 1.5.0 0da6a335fe13 73b5400665fb 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 9s ago 7m 192M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (7m) 9s ago 7m 220M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (6m) 9s ago 6m 168M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (6m) 19s ago 6m 245M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (5m) 19s ago 5m 231M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:02:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (5m) 19s ago 5m 207M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:02:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (38s) 9s ago 9m 44.0M - 2.43.0 a07b618ecd1d a7fc642830dd 2024-03-16T09:02:34.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 -- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f2cb0038df0 msgr2=0x7f2cb003b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f2cb0038df0 0x7f2cb003b2a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009bf0 tx=0x7f2cb40091a0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 -- 172.21.15.7:0/3413213004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc406e9a0 msgr2=0x7f2cc411c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc406e9a0 0x7f2cc411c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009150 tx=0x7f2cc0009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 -- 172.21.15.7:0/3413213004 shutdown_connections 2024-03-16T09:02:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f2cb0038df0 0x7f2cb003b2a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cc406e9a0 0x7f2cc411c6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 --2- 172.21.15.7:0/3413213004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc406f9d0 0x7f2cc411cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 -- 172.21.15.7:0/3413213004 >> 172.21.15.7:0/3413213004 conn(0x7f2cc4069de0 msgr2=0x7f2cc406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 -- 172.21.15.7:0/3413213004 shutdown_connections 2024-03-16T09:02:34.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.409+0000 7f2caf7fe700 1 -- 172.21.15.7:0/3413213004 wait complete. 2024-03-16T09:02:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 -- 172.21.15.7:0/546208574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fe430 msgr2=0x7f83d40fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 --2- 172.21.15.7:0/546208574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fe430 0x7f83d40fe840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f83c8004940 tx=0x7f83c8009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 -- 172.21.15.7:0/546208574 shutdown_connections 2024-03-16T09:02:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 --2- 172.21.15.7:0/546208574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fed80 0x7f83d40faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 --2- 172.21.15.7:0/546208574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fe430 0x7f83d40fe840 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 -- 172.21.15.7:0/546208574 >> 172.21.15.7:0/546208574 conn(0x7f83d40f6bd0 msgr2=0x7f83d40f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 -- 172.21.15.7:0/546208574 shutdown_connections 2024-03-16T09:02:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 -- 172.21.15.7:0/546208574 wait complete. 2024-03-16T09:02:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 Processor -- start 2024-03-16T09:02:34.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.513+0000 7f83dad46700 1 -- start start 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83dad46700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fe430 0x7f83d410d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83dad46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fed80 0x7f83d410dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83dad46700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d410e3e0 con 0x7f83d40fed80 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83dad46700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d410e550 con 0x7f83d40fe430 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fed80 0x7f83d410dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fed80 0x7f83d410dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59170/0 (socket says 172.21.15.7:59170) 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83d3fff700 1 -- 172.21.15.7:0/20031122 learned_addr learned my addr 172.21.15.7:0/20031122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83d8ae2700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fe430 0x7f83d410d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83d3fff700 1 -- 172.21.15.7:0/20031122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fe430 msgr2=0x7f83d410d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83d3fff700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fe430 0x7f83d410d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83d3fff700 1 -- 172.21.15.7:0/20031122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8005040 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.514+0000 7f83d3fff700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fed80 0x7f83d410dde0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f83c0002a80 tx=0x7f83c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.515+0000 7f83d17fa700 1 -- 172.21.15.7:0/20031122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c0007cc0 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.515+0000 7f83d17fa700 1 -- 172.21.15.7:0/20031122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83c000f040 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.515+0000 7f83d17fa700 1 -- 172.21.15.7:0/20031122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c000dac0 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.515+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d4112fb0 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.515+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d4113540 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.516+0000 7f83d17fa700 1 -- 172.21.15.7:0/20031122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 29) v1 ==== 46048+0+0 (secure 0 0 0) 0x7f83c000dc20 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.516+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d404c710 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.516+0000 7f83d17fa700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f83c4038d50 0x7f83c403b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.516+0000 7f83d17fa700 1 -- 172.21.15.7:0/20031122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f83c004c5a0 con 0x7f83d40fed80 2024-03-16T09:02:34.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.516+0000 7f83d8ae2700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f83c4038d50 0x7f83c403b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.516+0000 7f83d8ae2700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f83c4038d50 0x7f83c403b200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b3b0 tx=0x7f83c8009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:34.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.521+0000 7f83d17fa700 1 -- 172.21.15.7:0/20031122 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f83c001ab90 con 0x7f83d40fed80 2024-03-16T09:02:34.657 DEBUG:teuthology.parallel:result is None 2024-03-16T09:02:34.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.778+0000 7f83d17fa700 1 -- 172.21.15.7:0/20031122 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 30) v1 ==== 46104+0+0 (secure 0 0 0) 0x7f83c0011d50 con 0x7f83d40fed80 2024-03-16T09:02:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.824+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83d41058e0 con 0x7f83d40fed80 2024-03-16T09:02:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.825+0000 7f83d17fa700 1 -- 172.21.15.7:0/20031122 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f83c0011500 con 0x7f83d40fed80 2024-03-16T09:02:34.827 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:02:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:02:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:02:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:02:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:02:34.827 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 12, 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:02:34.828 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:02:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f83c4038d50 msgr2=0x7f83c403b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f83c4038d50 0x7f83c403b200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b3b0 tx=0x7f83c8009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fed80 msgr2=0x7f83d410dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fed80 0x7f83d410dde0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f83c0002a80 tx=0x7f83c00046b0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 shutdown_connections 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f83c4038d50 0x7f83c403b200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d40fe430 0x7f83d410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 --2- 172.21.15.7:0/20031122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83d40fed80 0x7f83d410dde0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.828+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 >> 172.21.15.7:0/20031122 conn(0x7f83d40f6bd0 msgr2=0x7f83d40f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.829+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 shutdown_connections 2024-03-16T09:02:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.829+0000 7f83dad46700 1 -- 172.21.15.7:0/20031122 wait complete. 2024-03-16T09:02:34.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:34 smithi019 ceph-mon[36407]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:34.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.976+0000 7f66f280f700 1 -- 172.21.15.7:0/401675539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fed90 msgr2=0x7f66ec0faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.976+0000 7f66f280f700 1 --2- 172.21.15.7:0/401675539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fed90 0x7f66ec0faed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f66e0004940 tx=0x7f66e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.976+0000 7f66f280f700 1 -- 172.21.15.7:0/401675539 shutdown_connections 2024-03-16T09:02:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.976+0000 7f66f280f700 1 --2- 172.21.15.7:0/401675539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fed90 0x7f66ec0faed0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.976+0000 7f66f280f700 1 --2- 172.21.15.7:0/401675539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ec0fe440 0x7f66ec0fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.976+0000 7f66f280f700 1 -- 172.21.15.7:0/401675539 >> 172.21.15.7:0/401675539 conn(0x7f66ec0f6be0 msgr2=0x7f66ec0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66f280f700 1 -- 172.21.15.7:0/401675539 shutdown_connections 2024-03-16T09:02:34.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66f280f700 1 -- 172.21.15.7:0/401675539 wait complete. 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66f280f700 1 Processor -- start 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66f280f700 1 -- start start 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66f280f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fe440 0x7f66ec0fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66f280f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ec0fed90 0x7f66ec0fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66f280f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec0fc640 con 0x7f66ec0fe440 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66f280f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec0fc7b0 con 0x7f66ec0fed90 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.977+0000 7f66ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fe440 0x7f66ec0fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fe440 0x7f66ec0fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59184/0 (socket says 172.21.15.7:59184) 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66ebfff700 1 -- 172.21.15.7:0/3844710235 learned_addr learned my addr 172.21.15.7:0/3844710235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66eb7fe700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ec0fed90 0x7f66ec0fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66ebfff700 1 -- 172.21.15.7:0/3844710235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ec0fed90 msgr2=0x7f66ec0fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66ebfff700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ec0fed90 0x7f66ec0fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66ebfff700 1 -- 172.21.15.7:0/3844710235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0005040 con 0x7f66ec0fe440 2024-03-16T09:02:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66ebfff700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fe440 0x7f66ec0fda50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00b790 tx=0x7f66dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66e8ff9700 1 -- 172.21.15.7:0/3844710235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66dc00f6f0 con 0x7f66ec0fe440 2024-03-16T09:02:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66e8ff9700 1 -- 172.21.15.7:0/3844710235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66dc00fd20 con 0x7f66ec0fe440 2024-03-16T09:02:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66e8ff9700 1 -- 172.21.15.7:0/3844710235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66dc0103a0 con 0x7f66ec0fe440 2024-03-16T09:02:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ec0fca90 con 0x7f66ec0fe440 2024-03-16T09:02:34.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.978+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ec115a10 con 0x7f66ec0fe440 2024-03-16T09:02:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.979+0000 7f66e8ff9700 1 -- 172.21.15.7:0/3844710235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 30) v1 ==== 46104+0+0 (secure 0 0 0) 0x7f66dc010500 con 0x7f66ec0fe440 2024-03-16T09:02:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.979+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ec04c710 con 0x7f66ec0fe440 2024-03-16T09:02:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.979+0000 7f66e8ff9700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f66d4038df0 0x7f66d403b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:02:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.979+0000 7f66e8ff9700 1 -- 172.21.15.7:0/3844710235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f66dc04c830 con 0x7f66ec0fe440 2024-03-16T09:02:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.979+0000 7f66eb7fe700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f66d4038df0 0x7f66d403b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:02:34.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.980+0000 7f66eb7fe700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f66d4038df0 0x7f66d403b2a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66e000b970 tx=0x7f66e0009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:02:34.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:34.984+0000 7f66e8ff9700 1 -- 172.21.15.7:0/3844710235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f66dc01a400 con 0x7f66ec0fe440 2024-03-16T09:02:35.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:34 smithi007 ceph-mon[29470]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:35.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.261+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66ec102490 con 0x7f66d4038df0 2024-03-16T09:02:35.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.263+0000 7f66e8ff9700 1 -- 172.21.15.7:0/3844710235 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f66ec102490 con 0x7f66d4038df0 2024-03-16T09:02:35.265 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/2 daemons upgraded", 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:02:35.266 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:02:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f66d4038df0 msgr2=0x7f66d403b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f66d4038df0 0x7f66d403b2a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66e000b970 tx=0x7f66e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:02:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fe440 msgr2=0x7f66ec0fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:02:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fe440 0x7f66ec0fda50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00b790 tx=0x7f66dc009250 comp rx=0 tx=0).stop 2024-03-16T09:02:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 shutdown_connections 2024-03-16T09:02:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f66d4038df0 0x7f66d403b2a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0fe440 0x7f66ec0fda50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 --2- 172.21.15.7:0/3844710235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ec0fed90 0x7f66ec0fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:02:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 >> 172.21.15.7:0/3844710235 conn(0x7f66ec0f6be0 msgr2=0x7f66ec0f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:02:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 shutdown_connections 2024-03-16T09:02:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:02:35.267+0000 7f66f280f700 1 -- 172.21.15.7:0/3844710235 wait complete. 2024-03-16T09:02:36.044 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:35 smithi019 ceph-mon[36407]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:36.044 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:35 smithi019 ceph-mon[36407]: from='client.24509 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:36.044 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:35 smithi019 ceph-mon[36407]: pgmap v5: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:02:36.045 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:35 smithi019 ceph-mon[36407]: mgrmap e30: smithi007.uamwtp(active, since 4s) 2024-03-16T09:02:36.045 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:35 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/20031122' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:02:36.045 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:35 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:36.045 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:35 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:35 smithi007 ceph-mon[29470]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:35 smithi007 ceph-mon[29470]: from='client.24509 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:35 smithi007 ceph-mon[29470]: pgmap v5: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:02:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:35 smithi007 ceph-mon[29470]: mgrmap e30: smithi007.uamwtp(active, since 4s) 2024-03-16T09:02:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:35 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/20031122' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:02:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:35 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:36.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:35 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:36.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:36 smithi019 ceph-mon[36407]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:36 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:36.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:36 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:37.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:36 smithi007 ceph-mon[29470]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:02:37.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:36 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:37.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:36 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:37.358 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-16T09:02:37.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-16T09:02:37.711 DEBUG:teuthology.parallel:result is None 2024-03-16T09:02:37.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-16T09:02:37.753 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-16T09:02:37.753 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-16T09:02:37.785 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-16T09:02:37.786 DEBUG:teuthology.parallel:result is None 2024-03-16T09:02:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:37 smithi007 ceph-mon[29470]: pgmap v6: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:02:38.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:37 smithi019 ceph-mon[36407]: pgmap v6: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:02:40.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:39 smithi019 ceph-mon[36407]: pgmap v7: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 60 KiB/s rd, 1.3 MiB/s wr, 309 op/s 2024-03-16T09:02:40.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:39 smithi019 ceph-mon[36407]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T09:02:40.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:39 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/3768591657' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T09:02:40.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:39 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/3768591657' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:02:40.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:39 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/3768591657' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T09:02:40.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:39 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/3768591657' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:02:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:39 smithi007 ceph-mon[29470]: pgmap v7: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 60 KiB/s rd, 1.3 MiB/s wr, 309 op/s 2024-03-16T09:02:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:39 smithi007 ceph-mon[29470]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T09:02:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:39 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/3768591657' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T09:02:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:39 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/3768591657' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:02:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:39 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/3768591657' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T09:02:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:39 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/3768591657' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:02:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:40 smithi007 ceph-mon[29470]: mgrmap e31: smithi007.uamwtp(active, since 9s), standbys: smithi019.iuqxhi 2024-03-16T09:02:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:40 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.iuqxhi", "id": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:02:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:40 smithi019 ceph-mon[36407]: mgrmap e31: smithi007.uamwtp(active, since 9s), standbys: smithi019.iuqxhi 2024-03-16T09:02:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:40 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.iuqxhi", "id": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:02:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:41 smithi019 ceph-mon[36407]: pgmap v8: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 999 KiB/s wr, 240 op/s 2024-03-16T09:02:42.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:41 smithi007 ceph-mon[29470]: pgmap v8: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 999 KiB/s wr, 240 op/s 2024-03-16T09:02:43.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:42 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:43.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:42 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:43.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:42 smithi019 ceph-mon[36407]: pgmap v9: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 919 KiB/s wr, 246 op/s 2024-03-16T09:02:43.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:42 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:43.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:42 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:43.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:42 smithi007 ceph-mon[29470]: pgmap v9: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 44 KiB/s rd, 919 KiB/s wr, 246 op/s 2024-03-16T09:02:44.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:44 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:44 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:44.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:44 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:44.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:44 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:45.324 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:45 smithi007 ceph-mon[29470]: pgmap v10: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 842 KiB/s wr, 226 op/s 2024-03-16T09:02:45.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:45 smithi019 ceph-mon[36407]: pgmap v10: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 842 KiB/s wr, 226 op/s 2024-03-16T09:02:46.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:46 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:02:46.364 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:46 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:02:47.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:47 smithi019 ceph-mon[36407]: pgmap v11: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 842 KiB/s wr, 226 op/s 2024-03-16T09:02:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:47 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:47 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:47 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:02:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:47 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:02:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:47 smithi007 ceph-mon[29470]: pgmap v11: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 842 KiB/s wr, 226 op/s 2024-03-16T09:02:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:47 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:47 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:47 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:02:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:47 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:02:48.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T09:02:48.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T09:02:48.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:02:48.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:02:48.930 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:02:48.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:02:48.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:02:48.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:02:48.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.931 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:48 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:48 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:49.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:49 smithi019 ceph-mon[36407]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-03-16T09:02:49.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:49 smithi019 ceph-mon[36407]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-03-16T09:02:49.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:49 smithi019 ceph-mon[36407]: pgmap v12: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 842 KiB/s wr, 226 op/s 2024-03-16T09:02:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:49 smithi007 ceph-mon[29470]: Reconfiguring prometheus.smithi007 (dependencies changed)... 2024-03-16T09:02:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:49 smithi007 ceph-mon[29470]: Reconfiguring daemon prometheus.smithi007 on smithi007 2024-03-16T09:02:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:49 smithi007 ceph-mon[29470]: pgmap v12: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 842 KiB/s wr, 226 op/s 2024-03-16T09:02:52.147 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:51 smithi007 ceph-mon[29470]: pgmap v13: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 5.1 KiB/s rd, 93 KiB/s wr, 46 op/s 2024-03-16T09:02:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:51 smithi019 ceph-mon[36407]: pgmap v13: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 5.1 KiB/s rd, 93 KiB/s wr, 46 op/s 2024-03-16T09:02:54.132 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:53 smithi019 ceph-mon[36407]: pgmap v14: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 5.1 KiB/s rd, 93 KiB/s wr, 46 op/s 2024-03-16T09:02:54.132 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:53 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:54.132 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:53 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:54.133 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:53 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:02:54.133 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:53 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:53 smithi007 ceph-mon[29470]: pgmap v14: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 5.1 KiB/s rd, 93 KiB/s wr, 46 op/s 2024-03-16T09:02:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:53 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:53 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:53 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:02:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:53 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:55.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:54 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:02:55.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:54 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:02:56.192 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:55 smithi019 ceph-mon[36407]: pgmap v15: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-03-16T09:02:56.193 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:55 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:02:56.193 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:55 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:02:56.193 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:55 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:02:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:55 smithi007 ceph-mon[29470]: pgmap v15: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-03-16T09:02:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:55 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:02:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:55 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:02:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:55 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:02:57.081 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:56 smithi019 ceph-mon[36407]: Upgrade: Updating mgr.smithi019.iuqxhi 2024-03-16T09:02:57.081 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:56 smithi019 ceph-mon[36407]: Deploying daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T09:02:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:56 smithi007 ceph-mon[29470]: Upgrade: Updating mgr.smithi019.iuqxhi 2024-03-16T09:02:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:56 smithi007 ceph-mon[29470]: Deploying daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T09:02:57.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:57 smithi019 ceph-mon[36407]: pgmap v16: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-03-16T09:02:58.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:57 smithi007 ceph-mon[29470]: pgmap v16: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-03-16T09:02:59.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:59 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:59.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:59 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:59.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:59 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:59.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:02:59 smithi007 ceph-mon[29470]: pgmap v17: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-03-16T09:02:59.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:59 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:59.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:59 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:02:59.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:59 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:02:59.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:02:59 smithi019 ceph-mon[36407]: pgmap v17: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 426 B/s wr, 0 op/s 2024-03-16T09:03:01.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:01 smithi007 ceph-mon[29470]: pgmap v18: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:01.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:03:01.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:01.963 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:02.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:01 smithi019 ceph-mon[36407]: pgmap v18: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:02.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:03:02.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:02.125 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:04.105 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:03 smithi019 ceph-mon[36407]: pgmap v19: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:04.106 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:03 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:04.106 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:03 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:03 smithi007 ceph-mon[29470]: pgmap v19: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:03 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:03 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:05 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:05 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:05 smithi019 ceph-mon[36407]: pgmap v20: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.390+0000 7ff34bf74700 1 -- 172.21.15.7:0/3293444028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fdfe0 msgr2=0x7ff3440fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.390+0000 7ff34bf74700 1 --2- 172.21.15.7:0/3293444028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fdfe0 0x7ff3440fe450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff338004940 tx=0x7ff338009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.390+0000 7ff34bf74700 1 -- 172.21.15.7:0/3293444028 shutdown_connections 2024-03-16T09:03:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.390+0000 7ff34bf74700 1 --2- 172.21.15.7:0/3293444028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fdfe0 0x7ff3440fe450 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.390+0000 7ff34bf74700 1 --2- 172.21.15.7:0/3293444028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3440fce70 0x7ff3440fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.390+0000 7ff34bf74700 1 -- 172.21.15.7:0/3293444028 >> 172.21.15.7:0/3293444028 conn(0x7ff3440f8550 msgr2=0x7ff3440fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:05.393 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:05 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:05.393 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:05 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:05.393 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:05 smithi007 ceph-mon[29470]: pgmap v20: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.390+0000 7ff34bf74700 1 -- 172.21.15.7:0/3293444028 shutdown_connections 2024-03-16T09:03:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.390+0000 7ff34bf74700 1 -- 172.21.15.7:0/3293444028 wait complete. 2024-03-16T09:03:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff34bf74700 1 Processor -- start 2024-03-16T09:03:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff34bf74700 1 -- start start 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff34bf74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fce70 0x7ff344071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff34bf74700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3440fdfe0 0x7ff344070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff34bf74700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff344072260 con 0x7ff3440fce70 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff34bf74700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff344070850 con 0x7ff3440fdfe0 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff349d10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fce70 0x7ff344071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff349d10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fce70 0x7ff344071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42004/0 (socket says 172.21.15.7:42004) 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff349d10700 1 -- 172.21.15.7:0/3428649827 learned_addr learned my addr 172.21.15.7:0/3428649827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff34950f700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3440fdfe0 0x7ff344070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff349d10700 1 -- 172.21.15.7:0/3428649827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3440fdfe0 msgr2=0x7ff344070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff349d10700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3440fdfe0 0x7ff344070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.391+0000 7ff349d10700 1 -- 172.21.15.7:0/3428649827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338005040 con 0x7ff3440fce70 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.392+0000 7ff349d10700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fce70 0x7ff344071c60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff33400ec30 tx=0x7ff33400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.392+0000 7ff3427fc700 1 -- 172.21.15.7:0/3428649827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff334007b30 con 0x7ff3440fce70 2024-03-16T09:03:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.392+0000 7ff3427fc700 1 -- 172.21.15.7:0/3428649827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff33400f040 con 0x7ff3440fce70 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.392+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff344070b00 con 0x7ff3440fce70 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.392+0000 7ff3427fc700 1 -- 172.21.15.7:0/3428649827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff33400cc90 con 0x7ff3440fce70 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.392+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff344071040 con 0x7ff3440fce70 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.393+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34404c710 con 0x7ff3440fce70 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.394+0000 7ff3427fc700 1 -- 172.21.15.7:0/3428649827 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 31) v1 ==== 91655+0+0 (secure 0 0 0) 0x7ff334007c90 con 0x7ff3440fce70 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.394+0000 7ff3427fc700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff32c06d910 0x7ff32c06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.394+0000 7ff3427fc700 1 -- 172.21.15.7:0/3428649827 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff33408d530 con 0x7ff3440fce70 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.394+0000 7ff34950f700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff32c06d910 0x7ff32c06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.395+0000 7ff34950f700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff32c06d910 0x7ff32c06fdc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff338009b30 tx=0x7ff338009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:05.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.399+0000 7ff3427fc700 1 -- 172.21.15.7:0/3428649827 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff33405ab50 con 0x7ff3440fce70 2024-03-16T09:03:05.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.621+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff344102280 con 0x7ff32c06d910 2024-03-16T09:03:05.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.622+0000 7ff3427fc700 1 -- 172.21.15.7:0/3428649827 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff344102280 con 0x7ff32c06d910 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff32c06d910 msgr2=0x7ff32c06fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff32c06d910 0x7ff32c06fdc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff338009b30 tx=0x7ff338009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fce70 msgr2=0x7ff344071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fce70 0x7ff344071c60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff33400ec30 tx=0x7ff33400b040 comp rx=0 tx=0).stop 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 shutdown_connections 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff32c06d910 0x7ff32c06fdc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3440fce70 0x7ff344071c60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 --2- 172.21.15.7:0/3428649827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3440fdfe0 0x7ff344070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.624+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 >> 172.21.15.7:0/3428649827 conn(0x7ff3440f8550 msgr2=0x7ff3440f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.625+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 shutdown_connections 2024-03-16T09:03:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.625+0000 7ff34bf74700 1 -- 172.21.15.7:0/3428649827 wait complete. 2024-03-16T09:03:05.637 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:03:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 -- 172.21.15.7:0/100657812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fdff0 msgr2=0x7f0e3c0fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 --2- 172.21.15.7:0/100657812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fdff0 0x7f0e3c0fe460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0e30004940 tx=0x7f0e30009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 -- 172.21.15.7:0/100657812 shutdown_connections 2024-03-16T09:03:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 --2- 172.21.15.7:0/100657812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fdff0 0x7f0e3c0fe460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 --2- 172.21.15.7:0/100657812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e3c0fce80 0x7f0e3c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 -- 172.21.15.7:0/100657812 >> 172.21.15.7:0/100657812 conn(0x7f0e3c0f8580 msgr2=0x7f0e3c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 -- 172.21.15.7:0/100657812 shutdown_connections 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 -- 172.21.15.7:0/100657812 wait complete. 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 Processor -- start 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.728+0000 7f0e43f2a700 1 -- start start 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e43f2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fce80 0x7f0e3c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e43f2a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e3c0fdff0 0x7f0e3c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e43f2a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c070950 con 0x7f0e3c0fce80 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e43f2a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c070ac0 con 0x7f0e3c0fdff0 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e41cc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fce80 0x7f0e3c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e41cc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fce80 0x7f0e3c071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42024/0 (socket says 172.21.15.7:42024) 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e41cc6700 1 -- 172.21.15.7:0/1384778845 learned_addr learned my addr 172.21.15.7:0/1384778845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e414c5700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e3c0fdff0 0x7f0e3c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e41cc6700 1 -- 172.21.15.7:0/1384778845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e3c0fdff0 msgr2=0x7f0e3c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e41cc6700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e3c0fdff0 0x7f0e3c070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e41cc6700 1 -- 172.21.15.7:0/1384778845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30005040 con 0x7f0e3c0fce80 2024-03-16T09:03:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e41cc6700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fce80 0x7f0e3c071e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400b790 tx=0x7f0e34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e3a7fc700 1 -- 172.21.15.7:0/1384778845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e3400f6f0 con 0x7f0e3c0fce80 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e3a7fc700 1 -- 172.21.15.7:0/1384778845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e3400fd20 con 0x7f0e3c0fce80 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e3a7fc700 1 -- 172.21.15.7:0/1384778845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e340103a0 con 0x7f0e3c0fce80 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.729+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c070da0 con 0x7f0e3c0fce80 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.730+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c115ae0 con 0x7f0e3c0fce80 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.731+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c05f5f0 con 0x7f0e3c0fce80 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.731+0000 7f0e3a7fc700 1 -- 172.21.15.7:0/1384778845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 31) v1 ==== 91655+0+0 (secure 0 0 0) 0x7f0e34011400 con 0x7f0e3c0fce80 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.732+0000 7f0e3a7fc700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0e2406d900 0x7f0e2406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.732+0000 7f0e3a7fc700 1 -- 172.21.15.7:0/1384778845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0e3408cc10 con 0x7f0e3c0fce80 2024-03-16T09:03:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.732+0000 7f0e414c5700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0e2406d900 0x7f0e2406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.732+0000 7f0e414c5700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0e2406d900 0x7f0e2406fdb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000b3b0 tx=0x7f0e30009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.736+0000 7f0e3a7fc700 1 -- 172.21.15.7:0/1384778845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f0e34056930 con 0x7f0e3c0fce80 2024-03-16T09:03:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.958+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e3c1022b0 con 0x7f0e2406d900 2024-03-16T09:03:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.959+0000 7f0e3a7fc700 1 -- 172.21.15.7:0/1384778845 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f0e3c1022b0 con 0x7f0e2406d900 2024-03-16T09:03:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0e2406d900 msgr2=0x7f0e2406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0e2406d900 0x7f0e2406fdb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000b3b0 tx=0x7f0e30009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fce80 msgr2=0x7f0e3c071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fce80 0x7f0e3c071e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400b790 tx=0x7f0e34009250 comp rx=0 tx=0).stop 2024-03-16T09:03:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 shutdown_connections 2024-03-16T09:03:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0e2406d900 0x7f0e2406fdb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c0fce80 0x7f0e3c071e40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 --2- 172.21.15.7:0/1384778845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e3c0fdff0 0x7f0e3c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 >> 172.21.15.7:0/1384778845 conn(0x7f0e3c0f8580 msgr2=0x7f0e3c0f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 shutdown_connections 2024-03-16T09:03:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:05.961+0000 7f0e43f2a700 1 -- 172.21.15.7:0/1384778845 wait complete. 2024-03-16T09:03:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.066+0000 7f22d82be700 1 -- 172.21.15.7:0/1478456995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fe440 msgr2=0x7f22d00fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.066+0000 7f22d82be700 1 --2- 172.21.15.7:0/1478456995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fe440 0x7f22d00fe850 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f22cc004940 tx=0x7f22cc009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 -- 172.21.15.7:0/1478456995 shutdown_connections 2024-03-16T09:03:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 --2- 172.21.15.7:0/1478456995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22d00fed90 0x7f22d00faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 --2- 172.21.15.7:0/1478456995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fe440 0x7f22d00fe850 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 -- 172.21.15.7:0/1478456995 >> 172.21.15.7:0/1478456995 conn(0x7f22d00f6be0 msgr2=0x7f22d00f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 -- 172.21.15.7:0/1478456995 shutdown_connections 2024-03-16T09:03:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 -- 172.21.15.7:0/1478456995 wait complete. 2024-03-16T09:03:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 Processor -- start 2024-03-16T09:03:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 -- start start 2024-03-16T09:03:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22d00fe440 0x7f22d00fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fed90 0x7f22d00fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00fc640 con 0x7f22d00fed90 2024-03-16T09:03:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d82be700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00fc7b0 con 0x7f22d00fe440 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d5859700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fed90 0x7f22d00fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.067+0000 7f22d5859700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fed90 0x7f22d00fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42048/0 (socket says 172.21.15.7:42048) 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22d5859700 1 -- 172.21.15.7:0/3274576917 learned_addr learned my addr 172.21.15.7:0/3274576917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22d605a700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22d00fe440 0x7f22d00fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22d5859700 1 -- 172.21.15.7:0/3274576917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22d00fe440 msgr2=0x7f22d00fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22d5859700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22d00fe440 0x7f22d00fda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22d5859700 1 -- 172.21.15.7:0/3274576917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc005040 con 0x7f22d00fed90 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22d5859700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fed90 0x7f22d00fc100 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b790 tx=0x7f22c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22c6ffd700 1 -- 172.21.15.7:0/3274576917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22c000f770 con 0x7f22d00fed90 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22c6ffd700 1 -- 172.21.15.7:0/3274576917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22c000fda0 con 0x7f22d00fed90 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d00fca90 con 0x7f22d00fed90 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22c6ffd700 1 -- 172.21.15.7:0/3274576917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22c000d310 con 0x7f22d00fed90 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.068+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d0115a40 con 0x7f22d00fed90 2024-03-16T09:03:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.069+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d004c710 con 0x7f22d00fed90 2024-03-16T09:03:06.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.070+0000 7f22c6ffd700 1 -- 172.21.15.7:0/3274576917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) v1 ==== 91655+0+0 (secure 0 0 0) 0x7f22c0010420 con 0x7f22d00fed90 2024-03-16T09:03:06.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.070+0000 7f22c6ffd700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f22bc06d910 0x7f22bc06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.070+0000 7f22c6ffd700 1 -- 172.21.15.7:0/3274576917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f22c008cf40 con 0x7f22d00fed90 2024-03-16T09:03:06.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.070+0000 7f22d605a700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f22bc06d910 0x7f22bc06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.071+0000 7f22d605a700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f22bc06d910 0x7f22bc06fdc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b3b0 tx=0x7f22cc009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.075+0000 7f22c6ffd700 1 -- 172.21.15.7:0/3274576917 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f22c005a6d0 con 0x7f22d00fed90 2024-03-16T09:03:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:06 smithi019 ceph-mon[36407]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:06 smithi019 ceph-mon[36407]: Standby manager daemon smithi019.iuqxhi restarted 2024-03-16T09:03:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:06 smithi019 ceph-mon[36407]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T09:03:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:06 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/2811886295' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T09:03:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:06 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/2811886295' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:03:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:06 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/2811886295' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T09:03:06.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:06 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/2811886295' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:03:06.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.296+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f22d0102490 con 0x7f22bc06d910 2024-03-16T09:03:06.299 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:06 smithi007 ceph-mon[29470]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:06.299 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:06 smithi007 ceph-mon[29470]: Standby manager daemon smithi019.iuqxhi restarted 2024-03-16T09:03:06.299 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:06 smithi007 ceph-mon[29470]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T09:03:06.299 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:06 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/2811886295' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T09:03:06.299 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:06 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/2811886295' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:03:06.299 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:06 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/2811886295' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T09:03:06.299 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:06 smithi007 ceph-mon[29470]: from='mgr.? 172.21.15.19:0/2811886295' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.308+0000 7f22c6ffd700 1 -- 172.21.15.7:0/3274576917 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f22d0102490 con 0x7f22bc06d910 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (101s) 2s ago 9m 28.7M - 0.25.0 c8568f914cd2 d4d6445aa339 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (10m) 2s ago 10m 6669k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (9m) 4s ago 9m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (82s) 2s ago 9m 80.0M - 9.4.7 954c08fa6188 32f01fbd2bf4 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (4m) 2s ago 4m 12.5M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (4m) 2s ago 4m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (4m) 4s ago 4m 121M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:03:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (4m) 4s ago 4m 13.0M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (55s) 2s ago 11m 493M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (8s) 4s ago 9m 64.1M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (11m) 2s ago 11m 52.4M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (8m) 4s ago 8m 42.5M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (2m) 2s ago 10m 15.8M - 1.5.0 0da6a335fe13 500fda412d5a 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (63s) 4s ago 9m 13.3M - 1.5.0 0da6a335fe13 73b5400665fb 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (8m) 2s ago 8m 205M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (7m) 2s ago 7m 233M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (7m) 2s ago 7m 181M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (6m) 4s ago 6m 262M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (6m) 4s ago 6m 243M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (6m) 4s ago 6m 220M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:03:06.311 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (13s) 2s ago 9m 40.4M - 2.43.0 a07b618ecd1d 3afa866ef81c 2024-03-16T09:03:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f22bc06d910 msgr2=0x7f22bc06fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f22bc06d910 0x7f22bc06fdc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b3b0 tx=0x7f22cc009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fed90 msgr2=0x7f22d00fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fed90 0x7f22d00fc100 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22c000b790 tx=0x7f22c0009250 comp rx=0 tx=0).stop 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 shutdown_connections 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f22bc06d910 0x7f22bc06fdc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22d00fe440 0x7f22d00fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 --2- 172.21.15.7:0/3274576917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d00fed90 0x7f22d00fc100 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 >> 172.21.15.7:0/3274576917 conn(0x7f22d00f6be0 msgr2=0x7f22d00f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 shutdown_connections 2024-03-16T09:03:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.310+0000 7f22d82be700 1 -- 172.21.15.7:0/3274576917 wait complete. 2024-03-16T09:03:06.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 -- 172.21.15.7:0/3552479977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c066280 msgr2=0x7fe27c0ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 --2- 172.21.15.7:0/3552479977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c066280 0x7fe27c0ffa20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe278004940 tx=0x7fe278009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:06.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 -- 172.21.15.7:0/3552479977 shutdown_connections 2024-03-16T09:03:06.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 --2- 172.21.15.7:0/3552479977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe27c0fff60 0x7fe27c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 --2- 172.21.15.7:0/3552479977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c066280 0x7fe27c0ffa20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 -- 172.21.15.7:0/3552479977 >> 172.21.15.7:0/3552479977 conn(0x7fe27c0f7430 msgr2=0x7fe27c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 -- 172.21.15.7:0/3552479977 shutdown_connections 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 -- 172.21.15.7:0/3552479977 wait complete. 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 Processor -- start 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 -- start start 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe27c066280 0x7fe27c0fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0fff60 0x7fe27c0fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0fbe60 con 0x7fe27c0fff60 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.414+0000 7fe284096700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0fbfa0 con 0x7fe27c066280 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe281631700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0fff60 0x7fe27c0fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe281631700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0fff60 0x7fe27c0fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42066/0 (socket says 172.21.15.7:42066) 2024-03-16T09:03:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe281631700 1 -- 172.21.15.7:0/3390803228 learned_addr learned my addr 172.21.15.7:0/3390803228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe281e32700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe27c066280 0x7fe27c0fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe281631700 1 -- 172.21.15.7:0/3390803228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe27c066280 msgr2=0x7fe27c0fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe281631700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe27c066280 0x7fe27c0fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe281631700 1 -- 172.21.15.7:0/3390803228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe278005040 con 0x7fe27c0fff60 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe281631700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0fff60 0x7fe27c0fb920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe27000eb70 tx=0x7fe27000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe26e7fc700 1 -- 172.21.15.7:0/3390803228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe270007cc0 con 0x7fe27c0fff60 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe26e7fc700 1 -- 172.21.15.7:0/3390803228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe27000f040 con 0x7fe27c0fff60 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c0fc220 con 0x7fe27c0fff60 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe26e7fc700 1 -- 172.21.15.7:0/3390803228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe27000ca90 con 0x7fe27c0fff60 2024-03-16T09:03:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.415+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c0fc760 con 0x7fe27c0fff60 2024-03-16T09:03:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.416+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c05f5f0 con 0x7fe27c0fff60 2024-03-16T09:03:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.417+0000 7fe26e7fc700 1 -- 172.21.15.7:0/3390803228 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) v1 ==== 91655+0+0 (secure 0 0 0) 0x7fe27000cbf0 con 0x7fe27c0fff60 2024-03-16T09:03:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.417+0000 7fe26e7fc700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe26806d9d0 0x7fe26806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.417+0000 7fe26e7fc700 1 -- 172.21.15.7:0/3390803228 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe27008cbb0 con 0x7fe27c0fff60 2024-03-16T09:03:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.417+0000 7fe281e32700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe26806d9d0 0x7fe26806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.418+0000 7fe281e32700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe26806d9d0 0x7fe26806fe80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe278013740 tx=0x7fe278009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.422+0000 7fe26e7fc700 1 -- 172.21.15.7:0/3390803228 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fe270056680 con 0x7fe27c0fff60 2024-03-16T09:03:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.721+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe27c04bed0 con 0x7fe27c0fff60 2024-03-16T09:03:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.721+0000 7fe26e7fc700 1 -- 172.21.15.7:0/3390803228 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe270059d40 con 0x7fe27c0fff60 2024-03-16T09:03:06.723 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:03:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 12, 2024-03-16T09:03:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:03:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:03:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:03:06.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe26806d9d0 msgr2=0x7fe26806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe26806d9d0 0x7fe26806fe80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe278013740 tx=0x7fe278009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0fff60 msgr2=0x7fe27c0fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0fff60 0x7fe27c0fb920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe27000eb70 tx=0x7fe27000b040 comp rx=0 tx=0).stop 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 shutdown_connections 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe26806d9d0 0x7fe26806fe80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe27c066280 0x7fe27c0fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 --2- 172.21.15.7:0/3390803228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe27c0fff60 0x7fe27c0fb920 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 >> 172.21.15.7:0/3390803228 conn(0x7fe27c0f7430 msgr2=0x7fe27c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.724+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 shutdown_connections 2024-03-16T09:03:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.725+0000 7fe284096700 1 -- 172.21.15.7:0/3390803228 wait complete. 2024-03-16T09:03:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.832+0000 7f1da9702700 1 -- 172.21.15.7:0/1532750457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 msgr2=0x7f1da40fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.832+0000 7f1da9702700 1 --2- 172.21.15.7:0/1532750457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 0x7f1da40fe460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1d94004940 tx=0x7f1d94009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.832+0000 7f1da9702700 1 -- 172.21.15.7:0/1532750457 shutdown_connections 2024-03-16T09:03:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.832+0000 7f1da9702700 1 --2- 172.21.15.7:0/1532750457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 0x7f1da40fe460 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.832+0000 7f1da9702700 1 --2- 172.21.15.7:0/1532750457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1da40fce80 0x7f1da40fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.832+0000 7f1da9702700 1 -- 172.21.15.7:0/1532750457 >> 172.21.15.7:0/1532750457 conn(0x7f1da40f8580 msgr2=0x7f1da40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.832+0000 7f1da9702700 1 -- 172.21.15.7:0/1532750457 shutdown_connections 2024-03-16T09:03:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.832+0000 7f1da9702700 1 -- 172.21.15.7:0/1532750457 wait complete. 2024-03-16T09:03:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da9702700 1 Processor -- start 2024-03-16T09:03:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da9702700 1 -- start start 2024-03-16T09:03:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da9702700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1da40fce80 0x7f1da410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da9702700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 0x7f1da410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da9702700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da410e550 con 0x7f1da40fdff0 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da9702700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da410eda0 con 0x7f1da40fce80 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 0x7f1da410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 0x7f1da410df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42092/0 (socket says 172.21.15.7:42092) 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da27fc700 1 -- 172.21.15.7:0/3052704807 learned_addr learned my addr 172.21.15.7:0/3052704807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da2ffd700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1da40fce80 0x7f1da410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da27fc700 1 -- 172.21.15.7:0/3052704807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1da40fce80 msgr2=0x7f1da410da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da27fc700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1da40fce80 0x7f1da410da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da27fc700 1 -- 172.21.15.7:0/3052704807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94005040 con 0x7f1da40fdff0 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.833+0000 7f1da2ffd700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1da40fce80 0x7f1da410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.834+0000 7f1da27fc700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 0x7f1da410df50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b3b0 tx=0x7f1d94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.834+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/3052704807 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9400e040 con 0x7f1da40fdff0 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.834+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/3052704807 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d94004290 con 0x7f1da40fdff0 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.834+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da410f020 con 0x7f1da40fdff0 2024-03-16T09:03:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.834+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da410f560 con 0x7f1da40fdff0 2024-03-16T09:03:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.834+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/3052704807 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9401c630 con 0x7f1da40fdff0 2024-03-16T09:03:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.835+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d900052e0 con 0x7f1da40fdff0 2024-03-16T09:03:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.835+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/3052704807 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) v1 ==== 91655+0+0 (secure 0 0 0) 0x7f1d9400bbf0 con 0x7f1da40fdff0 2024-03-16T09:03:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.836+0000 7f1d8b7fe700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1d8c06d900 0x7f1d8c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.836+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/3052704807 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1d9408e300 con 0x7f1da40fdff0 2024-03-16T09:03:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.836+0000 7f1da2ffd700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1d8c06d900 0x7f1d8c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.836+0000 7f1da2ffd700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1d8c06d900 0x7f1d8c06fdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009fd0 tx=0x7f1d98009ee0 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:06.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:06.845+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/3052704807 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f1d9405ba40 con 0x7f1da40fdff0 2024-03-16T09:03:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.079+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d90000bf0 con 0x7f1d8c06d900 2024-03-16T09:03:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.080+0000 7f1d8b7fe700 1 -- 172.21.15.7:0/3052704807 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f1d90000bf0 con 0x7f1d8c06d900 2024-03-16T09:03:07.082 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:03:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:03:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:03:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T09:03:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:03:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-03-16T09:03:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:03:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/2 daemons upgraded", 2024-03-16T09:03:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-16T09:03:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:03:07.083 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:03:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1d8c06d900 msgr2=0x7f1d8c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1d8c06d900 0x7f1d8c06fdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009fd0 tx=0x7f1d98009ee0 comp rx=0 tx=0).stop 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 msgr2=0x7f1da410df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 0x7f1da410df50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b3b0 tx=0x7f1d94009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 shutdown_connections 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f1d8c06d900 0x7f1d8c06fdb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1da40fce80 0x7f1da410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 --2- 172.21.15.7:0/3052704807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1da40fdff0 0x7f1da410df50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 >> 172.21.15.7:0/3052704807 conn(0x7f1da40f8580 msgr2=0x7f1da40f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 shutdown_connections 2024-03-16T09:03:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:07.082+0000 7f1da9702700 1 -- 172.21.15.7:0/3052704807 wait complete. 2024-03-16T09:03:07.388 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:07.388 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: mgrmap e32: smithi007.uamwtp(active, since 35s), standbys: smithi019.iuqxhi 2024-03-16T09:03:07.388 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:07.388 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.388 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.388 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:03:07.388 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all mgr 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.uamwtp"}]: dispatch 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.uamwtp"}]': finished 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi019.iuqxhi"}]: dispatch 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi019.iuqxhi"}]': finished 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all rgw 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all rbd-mirror 2024-03-16T09:03:07.389 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all ceph-exporter 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all iscsi 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all nfs 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.390 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all nvmeof 2024-03-16T09:03:07.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3390803228' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.391 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:07 smithi007 ceph-mon[29470]: pgmap v21: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:07.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:07.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: mgrmap e32: smithi007.uamwtp(active, since 35s), standbys: smithi019.iuqxhi 2024-03-16T09:03:07.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:07.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:03:07.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all mgr 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.uamwtp"}]: dispatch 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi007.uamwtp"}]': finished 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi019.iuqxhi"}]: dispatch 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi019.iuqxhi"}]': finished 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all rgw 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all rbd-mirror 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all ceph-exporter 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all iscsi 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all nfs 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all nvmeof 2024-03-16T09:03:07.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:07.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3390803228' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:07.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:07 smithi019 ceph-mon[36407]: pgmap v21: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:08.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:08 smithi019 ceph-mon[36407]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:08.982 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:08 smithi007 ceph-mon[29470]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:09.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:09 smithi007 ceph-mon[29470]: Upgrade: Updating node-exporter.smithi007 (1/2) 2024-03-16T09:03:09.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:09 smithi007 ceph-mon[29470]: Deploying daemon node-exporter.smithi007 on smithi007 2024-03-16T09:03:09.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:09 smithi007 ceph-mon[29470]: pgmap v22: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:09.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:09 smithi019 ceph-mon[36407]: Upgrade: Updating node-exporter.smithi007 (1/2) 2024-03-16T09:03:09.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:09 smithi019 ceph-mon[36407]: Deploying daemon node-exporter.smithi007 on smithi007 2024-03-16T09:03:09.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:09 smithi019 ceph-mon[36407]: pgmap v22: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:11 smithi007 ceph-mon[29470]: pgmap v23: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:11 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:11 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:12.186 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:11 smithi019 ceph-mon[36407]: pgmap v23: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:12.186 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:11 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:12.187 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:11 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:14.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:13 smithi019 ceph-mon[36407]: Upgrade: Updating node-exporter.smithi019 (2/2) 2024-03-16T09:03:14.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:13 smithi019 ceph-mon[36407]: Deploying daemon node-exporter.smithi019 on smithi019 2024-03-16T09:03:14.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:13 smithi019 ceph-mon[36407]: pgmap v24: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:13 smithi007 ceph-mon[29470]: Upgrade: Updating node-exporter.smithi019 (2/2) 2024-03-16T09:03:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:13 smithi007 ceph-mon[29470]: Deploying daemon node-exporter.smithi019 on smithi019 2024-03-16T09:03:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:13 smithi007 ceph-mon[29470]: pgmap v24: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:16.112 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:15 smithi019 ceph-mon[36407]: pgmap v25: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:16.112 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:15 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:16.112 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:15 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:16.112 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:15 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:16.140 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:15 smithi007 ceph-mon[29470]: pgmap v25: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:16.140 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:15 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:16.140 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:15 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:16.140 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:15 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:16.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:03:16.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:03:18.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:17 smithi019 ceph-mon[36407]: pgmap v26: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:18.224 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:17 smithi007 ceph-mon[29470]: pgmap v26: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:19 smithi007 ceph-mon[29470]: pgmap v27: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:19.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:19.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:19 smithi019 ceph-mon[36407]: pgmap v27: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:20.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:20 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:20.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:20 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:20.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:20 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:20.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:20 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:21.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:21 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:21.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:21 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:21.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:21 smithi007 ceph-mon[29470]: pgmap v28: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:21.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:21 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:21.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:21 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:21.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:21 smithi019 ceph-mon[36407]: pgmap v28: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:22.543 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:22 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:22.543 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:22 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:22.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:22 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:22.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:22 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:23.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:23 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:23.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:23 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:23.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:23 smithi007 ceph-mon[29470]: pgmap v29: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:23.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:23 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:23.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:23 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:23.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:23 smithi019 ceph-mon[36407]: pgmap v29: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-16T09:03:26.169 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: pgmap v30: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.170 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.171 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:25 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: pgmap v30: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:26.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:25 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:27.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:27 smithi007 ceph-mon[29470]: pgmap v31: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:27.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:27 smithi019 ceph-mon[36407]: pgmap v31: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:28.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:28 smithi019 ceph-mon[36407]: Upgrade: Updating prometheus.smithi007 2024-03-16T09:03:28.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:28 smithi019 ceph-mon[36407]: Deploying daemon prometheus.smithi007 on smithi007 2024-03-16T09:03:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:28 smithi007 ceph-mon[29470]: Upgrade: Updating prometheus.smithi007 2024-03-16T09:03:28.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:28 smithi007 ceph-mon[29470]: Deploying daemon prometheus.smithi007 on smithi007 2024-03-16T09:03:29.766 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:29 smithi007 ceph-mon[29470]: pgmap v32: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:29 smithi019 ceph-mon[36407]: pgmap v32: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:32.211 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:31 smithi007 ceph-mon[29470]: pgmap v33: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:32.211 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:03:32.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:31 smithi019 ceph-mon[36407]: pgmap v33: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:32.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:03:34.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:33 smithi019 ceph-mon[36407]: pgmap v34: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:34.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:33 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:34.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:33 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:34.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:33 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:34.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:33 smithi007 ceph-mon[29470]: pgmap v34: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:34.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:33 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:34.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:33 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:34.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:33 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:35.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:35 smithi007 ceph-mon[29470]: pgmap v35: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:36.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:35 smithi019 ceph-mon[36407]: pgmap v35: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.191+0000 7f546d48a700 1 -- 172.21.15.7:0/87704070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 msgr2=0x7f5468109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.191+0000 7f546d48a700 1 --2- 172.21.15.7:0/87704070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 0x7f5468109cc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f545c004940 tx=0x7f545c009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.191+0000 7f546d48a700 1 -- 172.21.15.7:0/87704070 shutdown_connections 2024-03-16T09:03:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.191+0000 7f546d48a700 1 --2- 172.21.15.7:0/87704070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 0x7f5468109cc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.191+0000 7f546d48a700 1 --2- 172.21.15.7:0/87704070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468104fc0 0x7f54681073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.191+0000 7f546d48a700 1 -- 172.21.15.7:0/87704070 >> 172.21.15.7:0/87704070 conn(0x7f5468069de0 msgr2=0x7f546806c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.191+0000 7f546d48a700 1 -- 172.21.15.7:0/87704070 shutdown_connections 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.191+0000 7f546d48a700 1 -- 172.21.15.7:0/87704070 wait complete. 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f546d48a700 1 Processor -- start 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f546d48a700 1 -- start start 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f546d48a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468104fc0 0x7f546806f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f546d48a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 0x7f546811af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f546d48a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546811b490 con 0x7f54681078e0 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f546d48a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546811b5d0 con 0x7f5468104fc0 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f54667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 0x7f546811af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f54667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 0x7f546811af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34256/0 (socket says 172.21.15.7:34256) 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f54667fc700 1 -- 172.21.15.7:0/3347830598 learned_addr learned my addr 172.21.15.7:0/3347830598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f5466ffd700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468104fc0 0x7f546806f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f54667fc700 1 -- 172.21.15.7:0/3347830598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468104fc0 msgr2=0x7f546806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f54667fc700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468104fc0 0x7f546806f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f54667fc700 1 -- 172.21.15.7:0/3347830598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c005040 con 0x7f54681078e0 2024-03-16T09:03:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.192+0000 7f54667fc700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 0x7f546811af50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f545c00b970 tx=0x7f545c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.193+0000 7f544f7fe700 1 -- 172.21.15.7:0/3347830598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f545c00e040 con 0x7f54681078e0 2024-03-16T09:03:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.193+0000 7f544f7fe700 1 -- 172.21.15.7:0/3347830598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f545c00bbf0 con 0x7f54681078e0 2024-03-16T09:03:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.193+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546811b7d0 con 0x7f54681078e0 2024-03-16T09:03:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.193+0000 7f544f7fe700 1 -- 172.21.15.7:0/3347830598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f545c01c630 con 0x7f54681078e0 2024-03-16T09:03:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.193+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546811bd90 con 0x7f54681078e0 2024-03-16T09:03:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.194+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546804c710 con 0x7f54681078e0 2024-03-16T09:03:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.194+0000 7f544f7fe700 1 -- 172.21.15.7:0/3347830598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) v1 ==== 91655+0+0 (secure 0 0 0) 0x7f545c00bd60 con 0x7f54681078e0 2024-03-16T09:03:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.195+0000 7f544f7fe700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f545006d910 0x7f545006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.195+0000 7f544f7fe700 1 -- 172.21.15.7:0/3347830598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f545c08ee00 con 0x7f54681078e0 2024-03-16T09:03:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.195+0000 7f5466ffd700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f545006d910 0x7f545006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.195+0000 7f5466ffd700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f545006d910 0x7f545006fdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5458004990 tx=0x7f5458009ee0 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.203+0000 7f544f7fe700 1 -- 172.21.15.7:0/3347830598 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f545c05c540 con 0x7f54681078e0 2024-03-16T09:03:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.427+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f546806b920 con 0x7f545006d910 2024-03-16T09:03:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.428+0000 7f544f7fe700 1 -- 172.21.15.7:0/3347830598 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f546806b920 con 0x7f545006d910 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f545006d910 msgr2=0x7f545006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f545006d910 0x7f545006fdc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5458004990 tx=0x7f5458009ee0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 msgr2=0x7f546811af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 0x7f546811af50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f545c00b970 tx=0x7f545c009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 shutdown_connections 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f545006d910 0x7f545006fdc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468104fc0 0x7f546806f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 --2- 172.21.15.7:0/3347830598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54681078e0 0x7f546811af50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.430+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 >> 172.21.15.7:0/3347830598 conn(0x7f5468069de0 msgr2=0x7f546806a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.431+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 shutdown_connections 2024-03-16T09:03:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.431+0000 7f546d48a700 1 -- 172.21.15.7:0/3347830598 wait complete. 2024-03-16T09:03:37.443 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.534+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/1905562279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 msgr2=0x7f5a040fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.534+0000 7f5a0b6bb700 1 --2- 172.21.15.7:0/1905562279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 0x7f5a040fc0f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a00004940 tx=0x7f5a00009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.534+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/1905562279 shutdown_connections 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.534+0000 7f5a0b6bb700 1 --2- 172.21.15.7:0/1905562279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a040fcf90 0x7f5a040fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.534+0000 7f5a0b6bb700 1 --2- 172.21.15.7:0/1905562279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 0x7f5a040fc0f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.534+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/1905562279 >> 172.21.15.7:0/1905562279 conn(0x7f5a040f7440 msgr2=0x7f5a040f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.534+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/1905562279 shutdown_connections 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.534+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/1905562279 wait complete. 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a0b6bb700 1 Processor -- start 2024-03-16T09:03:37.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a0b6bb700 1 -- start start 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a0b6bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 0x7f5a0410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a0b6bb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a040fcf90 0x7f5a0410dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a0b6bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0410e5a0 con 0x7f5a040fbce0 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a0b6bb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0410eda0 con 0x7f5a040fcf90 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a09457700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 0x7f5a0410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a09457700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 0x7f5a0410da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34282/0 (socket says 172.21.15.7:34282) 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a09457700 1 -- 172.21.15.7:0/446724464 learned_addr learned my addr 172.21.15.7:0/446724464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a08c56700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a040fcf90 0x7f5a0410dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a09457700 1 -- 172.21.15.7:0/446724464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a040fcf90 msgr2=0x7f5a0410dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a09457700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a040fcf90 0x7f5a0410dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a09457700 1 -- 172.21.15.7:0/446724464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a00005040 con 0x7f5a040fbce0 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.535+0000 7f5a09457700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 0x7f5a0410da60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000b3b0 tx=0x7f5a00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.536+0000 7f59f9ffb700 1 -- 172.21.15.7:0/446724464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a0000e040 con 0x7f5a040fbce0 2024-03-16T09:03:37.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.536+0000 7f59f9ffb700 1 -- 172.21.15.7:0/446724464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a0000bcc0 con 0x7f5a040fbce0 2024-03-16T09:03:37.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.536+0000 7f59f9ffb700 1 -- 172.21.15.7:0/446724464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a0001c630 con 0x7f5a040fbce0 2024-03-16T09:03:37.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.536+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0410f020 con 0x7f5a040fbce0 2024-03-16T09:03:37.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.536+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0410f560 con 0x7f5a040fbce0 2024-03-16T09:03:37.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.537+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a04101b20 con 0x7f5a040fbce0 2024-03-16T09:03:37.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.537+0000 7f59f9ffb700 1 -- 172.21.15.7:0/446724464 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) v1 ==== 91655+0+0 (secure 0 0 0) 0x7f5a00013980 con 0x7f5a040fbce0 2024-03-16T09:03:37.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.538+0000 7f59f9ffb700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f59f006d900 0x7f59f006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.538+0000 7f59f9ffb700 1 -- 172.21.15.7:0/446724464 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5a0008dc90 con 0x7f5a040fbce0 2024-03-16T09:03:37.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.538+0000 7f5a08c56700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f59f006d900 0x7f59f006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.538+0000 7f5a08c56700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f59f006d900 0x7f59f006fdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f59f4004990 tx=0x7f59f4009ee0 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:37.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.542+0000 7f59f9ffb700 1 -- 172.21.15.7:0/446724464 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f5a0005b330 con 0x7f5a040fbce0 2024-03-16T09:03:37.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:37 smithi007 ceph-mon[29470]: pgmap v36: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.786+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a04104940 con 0x7f59f006d900 2024-03-16T09:03:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.788+0000 7f59f9ffb700 1 -- 172.21.15.7:0/446724464 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f5a04104940 con 0x7f59f006d900 2024-03-16T09:03:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f59f006d900 msgr2=0x7f59f006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f59f006d900 0x7f59f006fdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f59f4004990 tx=0x7f59f4009ee0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 msgr2=0x7f5a0410da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 0x7f5a0410da60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000b3b0 tx=0x7f5a00009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 shutdown_connections 2024-03-16T09:03:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f59f006d900 0x7f59f006fdb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a040fbce0 0x7f5a0410da60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 --2- 172.21.15.7:0/446724464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a040fcf90 0x7f5a0410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 >> 172.21.15.7:0/446724464 conn(0x7f5a040f7440 msgr2=0x7f5a04066490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 shutdown_connections 2024-03-16T09:03:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.790+0000 7f5a0b6bb700 1 -- 172.21.15.7:0/446724464 wait complete. 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.894+0000 7f70e73fb700 1 -- 172.21.15.7:0/1664894777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fc2b0 msgr2=0x7f70e00fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.894+0000 7f70e73fb700 1 --2- 172.21.15.7:0/1664894777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fc2b0 0x7f70e00fc6c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f70dc004940 tx=0x7f70dc009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.894+0000 7f70e73fb700 1 -- 172.21.15.7:0/1664894777 shutdown_connections 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.894+0000 7f70e73fb700 1 --2- 172.21.15.7:0/1664894777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e00fcc00 0x7f70e00fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.894+0000 7f70e73fb700 1 --2- 172.21.15.7:0/1664894777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fc2b0 0x7f70e00fc6c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.894+0000 7f70e73fb700 1 -- 172.21.15.7:0/1664894777 >> 172.21.15.7:0/1664894777 conn(0x7f70e00f6ba0 msgr2=0x7f70e00f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.894+0000 7f70e73fb700 1 -- 172.21.15.7:0/1664894777 shutdown_connections 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.894+0000 7f70e73fb700 1 -- 172.21.15.7:0/1664894777 wait complete. 2024-03-16T09:03:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e73fb700 1 Processor -- start 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e73fb700 1 -- start start 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e73fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e00fc2b0 0x7f70e00ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e73fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fcc00 0x7f70e00fe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e73fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00fe940 con 0x7f70e00fcc00 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e73fb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00fea80 con 0x7f70e00fc2b0 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e4996700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fcc00 0x7f70e00fe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e4996700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fcc00 0x7f70e00fe3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34302/0 (socket says 172.21.15.7:34302) 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e4996700 1 -- 172.21.15.7:0/1562586643 learned_addr learned my addr 172.21.15.7:0/1562586643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e5197700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e00fc2b0 0x7f70e00ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e4996700 1 -- 172.21.15.7:0/1562586643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e00fc2b0 msgr2=0x7f70e00ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e4996700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e00fc2b0 0x7f70e00ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.895+0000 7f70e4996700 1 -- 172.21.15.7:0/1562586643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc005040 con 0x7f70e00fcc00 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.896+0000 7f70e4996700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fcc00 0x7f70e00fe3b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f70d0002a80 tx=0x7f70d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.896+0000 7f70d5ffb700 1 -- 172.21.15.7:0/1562586643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d0007cc0 con 0x7f70e00fcc00 2024-03-16T09:03:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.896+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e00fed60 con 0x7f70e00fcc00 2024-03-16T09:03:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.896+0000 7f70d5ffb700 1 -- 172.21.15.7:0/1562586643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70d000f040 con 0x7f70e00fcc00 2024-03-16T09:03:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.896+0000 7f70d5ffb700 1 -- 172.21.15.7:0/1562586643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d000dac0 con 0x7f70e00fcc00 2024-03-16T09:03:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.896+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e0115a80 con 0x7f70e00fcc00 2024-03-16T09:03:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.897+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e0101c00 con 0x7f70e00fcc00 2024-03-16T09:03:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.898+0000 7f70d5ffb700 1 -- 172.21.15.7:0/1562586643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) v1 ==== 91655+0+0 (secure 0 0 0) 0x7f70d000dc20 con 0x7f70e00fcc00 2024-03-16T09:03:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.898+0000 7f70d5ffb700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f70cc06d900 0x7f70cc06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.898+0000 7f70d5ffb700 1 -- 172.21.15.7:0/1562586643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f70d008ccd0 con 0x7f70e00fcc00 2024-03-16T09:03:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.899+0000 7f70e5197700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f70cc06d900 0x7f70cc06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.899+0000 7f70e5197700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f70cc06d900 0x7f70cc06fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70dc004890 tx=0x7f70dc009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:37.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:37.903+0000 7f70d5ffb700 1 -- 172.21.15.7:0/1562586643 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f70d00567a0 con 0x7f70e00fcc00 2024-03-16T09:03:37.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:37 smithi019 ceph-mon[36407]: pgmap v36: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:38.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.118+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f70e005e400 con 0x7f70cc06d900 2024-03-16T09:03:38.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.128+0000 7f70d5ffb700 1 -- 172.21.15.7:0/1562586643 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f70e005e400 con 0x7f70cc06d900 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (2m) 17s ago 10m 28.7M - 0.25.0 c8568f914cd2 d4d6445aa339 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (10m) 17s ago 10m 6669k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (9m) 20s ago 9m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (114s) 17s ago 10m 79.1M - 9.4.7 954c08fa6188 32f01fbd2bf4 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (5m) 17s ago 5m 12.6M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (4m) 17s ago 4m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (5m) 20s ago 5m 121M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:03:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (4m) 20s ago 4m 13.1M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (87s) 17s ago 11m 495M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (39s) 20s ago 9m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (11m) 17s ago 12m 52.4M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (9m) 20s ago 9m 43.7M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (27s) 17s ago 10m 6807k - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (23s) 20s ago 9m 3565k - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (8m) 17s ago 8m 204M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (8m) 17s ago 8m 233M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (7m) 17s ago 7m 181M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (7m) 20s ago 7m 262M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:03:38.132 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (7m) 20s ago 7m 244M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:03:38.133 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (6m) 20s ago 6m 221M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:03:38.133 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 starting - - - - 2024-03-16T09:03:38.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f70cc06d900 msgr2=0x7f70cc06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f70cc06d900 0x7f70cc06fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f70dc004890 tx=0x7f70dc009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fcc00 msgr2=0x7f70e00fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fcc00 0x7f70e00fe3b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f70d0002a80 tx=0x7f70d00046b0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 shutdown_connections 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f70cc06d900 0x7f70cc06fdb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e00fc2b0 0x7f70e00ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 --2- 172.21.15.7:0/1562586643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e00fcc00 0x7f70e00fe3b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 >> 172.21.15.7:0/1562586643 conn(0x7f70e00f6ba0 msgr2=0x7f70e00fb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 shutdown_connections 2024-03-16T09:03:38.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.131+0000 7f70e73fb700 1 -- 172.21.15.7:0/1562586643 wait complete. 2024-03-16T09:03:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 -- 172.21.15.7:0/1600804758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff968066b70 msgr2=0x7ff9680ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 --2- 172.21.15.7:0/1600804758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff968066b70 0x7ff9680ff9e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff95c004940 tx=0x7ff95c009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:38.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 -- 172.21.15.7:0/1600804758 shutdown_connections 2024-03-16T09:03:38.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 --2- 172.21.15.7:0/1600804758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9680fff20 0x7ff968102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 --2- 172.21.15.7:0/1600804758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff968066b70 0x7ff9680ff9e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 -- 172.21.15.7:0/1600804758 >> 172.21.15.7:0/1600804758 conn(0x7ff9680f7450 msgr2=0x7ff9680f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:38.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 -- 172.21.15.7:0/1600804758 shutdown_connections 2024-03-16T09:03:38.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 -- 172.21.15.7:0/1600804758 wait complete. 2024-03-16T09:03:38.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 Processor -- start 2024-03-16T09:03:38.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.233+0000 7ff96efba700 1 -- start start 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff96efba700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff968066b70 0x7ff9680fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff96efba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9680fff20 0x7ff9680fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff96efba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680fbe90 con 0x7ff9680fff20 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff96efba700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680fc000 con 0x7ff968066b70 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff967fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9680fff20 0x7ff9680fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff967fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9680fff20 0x7ff9680fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34324/0 (socket says 172.21.15.7:34324) 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff967fff700 1 -- 172.21.15.7:0/3448541700 learned_addr learned my addr 172.21.15.7:0/3448541700 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff96cd56700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff968066b70 0x7ff9680fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff967fff700 1 -- 172.21.15.7:0/3448541700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff968066b70 msgr2=0x7ff9680fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff967fff700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff968066b70 0x7ff9680fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff967fff700 1 -- 172.21.15.7:0/3448541700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c005040 con 0x7ff9680fff20 2024-03-16T09:03:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff967fff700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9680fff20 0x7ff9680fb8e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c830 tx=0x7ff954004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:38.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff9657fa700 1 -- 172.21.15.7:0/3448541700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff95400a760 con 0x7ff9680fff20 2024-03-16T09:03:38.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff9657fa700 1 -- 172.21.15.7:0/3448541700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff95400ad90 con 0x7ff9680fff20 2024-03-16T09:03:38.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9680fc2e0 con 0x7ff9680fff20 2024-03-16T09:03:38.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.234+0000 7ff9657fa700 1 -- 172.21.15.7:0/3448541700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9540113a0 con 0x7ff9680fff20 2024-03-16T09:03:38.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.235+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff968115a80 con 0x7ff9680fff20 2024-03-16T09:03:38.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.235+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96805f5f0 con 0x7ff9680fff20 2024-03-16T09:03:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.236+0000 7ff9657fa700 1 -- 172.21.15.7:0/3448541700 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) v1 ==== 91655+0+0 (secure 0 0 0) 0x7ff954012400 con 0x7ff9680fff20 2024-03-16T09:03:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.236+0000 7ff9657fa700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff95806d900 0x7ff95806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.236+0000 7ff9657fa700 1 -- 172.21.15.7:0/3448541700 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff95408cf80 con 0x7ff9680fff20 2024-03-16T09:03:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.236+0000 7ff96cd56700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff95806d900 0x7ff95806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.237+0000 7ff96cd56700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff95806d900 0x7ff95806fdb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009fd0 tx=0x7ff95c009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.241+0000 7ff9657fa700 1 -- 172.21.15.7:0/3448541700 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff954056be0 con 0x7ff9680fff20 2024-03-16T09:03:38.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.574+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff9681058d0 con 0x7ff9680fff20 2024-03-16T09:03:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.574+0000 7ff9657fa700 1 -- 172.21.15.7:0/3448541700 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff95405a2a0 con 0x7ff9680fff20 2024-03-16T09:03:38.577 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:03:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:03:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:03:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:03:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:03:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 12, 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:03:38.578 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:03:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff95806d900 msgr2=0x7ff95806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff95806d900 0x7ff95806fdb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009fd0 tx=0x7ff95c009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9680fff20 msgr2=0x7ff9680fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9680fff20 0x7ff9680fb8e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c830 tx=0x7ff954004470 comp rx=0 tx=0).stop 2024-03-16T09:03:38.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 shutdown_connections 2024-03-16T09:03:38.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff95806d900 0x7ff95806fdb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff968066b70 0x7ff9680fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 --2- 172.21.15.7:0/3448541700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9680fff20 0x7ff9680fb8e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 >> 172.21.15.7:0/3448541700 conn(0x7ff9680f7450 msgr2=0x7ff9680f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:38.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 shutdown_connections 2024-03-16T09:03:38.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.577+0000 7ff96efba700 1 -- 172.21.15.7:0/3448541700 wait complete. 2024-03-16T09:03:38.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/3484982073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e0066ef0 msgr2=0x7fb1e00ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 --2- 172.21.15.7:0/3484982073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e0066ef0 0x7fb1e00ff9f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0004940 tx=0x7fb1d0009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:38.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/3484982073 shutdown_connections 2024-03-16T09:03:38.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 --2- 172.21.15.7:0/3484982073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e00fff30 0x7fb1e0102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 --2- 172.21.15.7:0/3484982073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e0066ef0 0x7fb1e00ff9f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/3484982073 >> 172.21.15.7:0/3484982073 conn(0x7fb1e00f7440 msgr2=0x7fb1e00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:38.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/3484982073 shutdown_connections 2024-03-16T09:03:38.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/3484982073 wait complete. 2024-03-16T09:03:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.688+0000 7fb1e5b2d700 1 Processor -- start 2024-03-16T09:03:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1e5b2d700 1 -- start start 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1e5b2d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e0066ef0 0x7fb1e010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1e5b2d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e00fff30 0x7fb1e010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1e5b2d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e010e4d0 con 0x7fb1e00fff30 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1e5b2d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e010e640 con 0x7fb1e0066ef0 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1deffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e00fff30 0x7fb1e010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1deffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e00fff30 0x7fb1e010ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34356/0 (socket says 172.21.15.7:34356) 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1deffd700 1 -- 172.21.15.7:0/1721039057 learned_addr learned my addr 172.21.15.7:0/1721039057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1df7fe700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e0066ef0 0x7fb1e010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1deffd700 1 -- 172.21.15.7:0/1721039057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e0066ef0 msgr2=0x7fb1e010d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1deffd700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e0066ef0 0x7fb1e010d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1deffd700 1 -- 172.21.15.7:0/1721039057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0005040 con 0x7fb1e00fff30 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.689+0000 7fb1df7fe700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e0066ef0 0x7fb1e010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.690+0000 7fb1deffd700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e00fff30 0x7fb1e010ded0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c830 tx=0x7fb1d4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:38.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.690+0000 7fb1c7fff700 1 -- 172.21.15.7:0/1721039057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d400a760 con 0x7fb1e00fff30 2024-03-16T09:03:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.690+0000 7fb1c7fff700 1 -- 172.21.15.7:0/1721039057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1d400ad90 con 0x7fb1e00fff30 2024-03-16T09:03:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.690+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e0113050 con 0x7fb1e00fff30 2024-03-16T09:03:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.690+0000 7fb1c7fff700 1 -- 172.21.15.7:0/1721039057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d40113a0 con 0x7fb1e00fff30 2024-03-16T09:03:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.690+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e0113590 con 0x7fb1e00fff30 2024-03-16T09:03:38.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.691+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e004c710 con 0x7fb1e00fff30 2024-03-16T09:03:38.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.691+0000 7fb1c7fff700 1 -- 172.21.15.7:0/1721039057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 32) v1 ==== 91655+0+0 (secure 0 0 0) 0x7fb1d4012400 con 0x7fb1e00fff30 2024-03-16T09:03:38.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.692+0000 7fb1c7fff700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fb1c806d910 0x7fb1c806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:03:38.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.692+0000 7fb1c7fff700 1 -- 172.21.15.7:0/1721039057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb1d408cf70 con 0x7fb1e00fff30 2024-03-16T09:03:38.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.692+0000 7fb1df7fe700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fb1c806d910 0x7fb1c806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:03:38.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.692+0000 7fb1df7fe700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fb1c806d910 0x7fb1c806fdc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b3b0 tx=0x7fb1d0009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:03:38.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.697+0000 7fb1c7fff700 1 -- 172.21.15.7:0/1721039057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fb1d40c1040 con 0x7fb1e00fff30 2024-03-16T09:03:38.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.923+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1e00f8fa0 con 0x7fb1c806d910 2024-03-16T09:03:38.926 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:38 smithi007 ceph-mon[29470]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:38.926 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:38 smithi007 ceph-mon[29470]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:38.926 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:38 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3448541700' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.925+0000 7fb1c7fff700 1 -- 172.21.15.7:0/1721039057 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fb1e00f8fa0 con 0x7fb1c806d910 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/2 daemons upgraded", 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:03:38.927 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.927+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fb1c806d910 msgr2=0x7fb1c806fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fb1c806d910 0x7fb1c806fdc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b3b0 tx=0x7fb1d0009e30 comp rx=0 tx=0).stop 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e00fff30 msgr2=0x7fb1e010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e00fff30 0x7fb1e010ded0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c830 tx=0x7fb1d4004470 comp rx=0 tx=0).stop 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 shutdown_connections 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fb1c806d910 0x7fb1c806fdc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e0066ef0 0x7fb1e010d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 --2- 172.21.15.7:0/1721039057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e00fff30 0x7fb1e010ded0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:03:38.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 >> 172.21.15.7:0/1721039057 conn(0x7fb1e00f7440 msgr2=0x7fb1e00f7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:03:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 shutdown_connections 2024-03-16T09:03:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:03:38.928+0000 7fb1e5b2d700 1 -- 172.21.15.7:0/1721039057 wait complete. 2024-03-16T09:03:38.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:38 smithi019 ceph-mon[36407]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:38.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:38 smithi019 ceph-mon[36407]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:38.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:38 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3448541700' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:39.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:39 smithi019 ceph-mon[36407]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:39.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:39 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:39.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:39 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:39.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:39 smithi019 ceph-mon[36407]: pgmap v37: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:39.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:39 smithi007 ceph-mon[29470]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:39.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:39 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:39.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:39 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:39.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:39 smithi007 ceph-mon[29470]: pgmap v37: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:40 smithi007 ceph-mon[29470]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:40 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:40 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:41.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:40 smithi019 ceph-mon[36407]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:03:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:40 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:40 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:42.063 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:41 smithi007 ceph-mon[29470]: pgmap v38: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:41 smithi019 ceph-mon[36407]: pgmap v38: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: pgmap v39: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:03:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:03:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:43 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: pgmap v39: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:44.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:44.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:44.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:44.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:43 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:03:45.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:44 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:03:45.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:44 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T09:03:46.114 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:45 smithi019 ceph-mon[36407]: pgmap v40: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:46.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:45 smithi007 ceph-mon[29470]: pgmap v40: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:46 smithi019 ceph-mon[36407]: Upgrade: Updating alertmanager.smithi007 2024-03-16T09:03:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:46 smithi019 ceph-mon[36407]: Deploying daemon alertmanager.smithi007 on smithi007 2024-03-16T09:03:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:46 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:03:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:46 smithi007 ceph-mon[29470]: Upgrade: Updating alertmanager.smithi007 2024-03-16T09:03:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:46 smithi007 ceph-mon[29470]: Deploying daemon alertmanager.smithi007 on smithi007 2024-03-16T09:03:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:46 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:03:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:47 smithi007 ceph-mon[29470]: pgmap v41: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:48.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:47 smithi019 ceph-mon[36407]: pgmap v41: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:49 smithi007 ceph-mon[29470]: pgmap v42: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:50.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:49 smithi019 ceph-mon[36407]: pgmap v42: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:51 smithi007 ceph-mon[29470]: pgmap v43: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:51 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:51 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:51 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:51 smithi019 ceph-mon[36407]: pgmap v43: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:51 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:51 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:51 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:03:53.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:52 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:53.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:52 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:53.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:52 smithi007 ceph-mon[29470]: pgmap v44: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:53.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:52 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:53.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:52 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:53.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:52 smithi019 ceph-mon[36407]: pgmap v44: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:55 smithi019 ceph-mon[36407]: pgmap v45: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:55 smithi007 ceph-mon[29470]: pgmap v45: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:57 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:57 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:57 smithi019 ceph-mon[36407]: pgmap v46: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:58.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:57 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:58.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:57 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:58.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:57 smithi007 ceph-mon[29470]: pgmap v46: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:59.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:58 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:59.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:58 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:59.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:03:58 smithi019 ceph-mon[36407]: pgmap v47: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:03:59.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:58 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:59.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:58 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:03:59.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:03:58 smithi007 ceph-mon[29470]: pgmap v47: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: pgmap v48: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: pgmap v48: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:02.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:04:02.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:02 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T09:04:03.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:02 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T09:04:04.117 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:03 smithi007 ceph-mon[29470]: pgmap v49: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:04.117 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:03 smithi007 ceph-mon[29470]: mgrmap e33: smithi007.uamwtp(active, since 92s), standbys: smithi019.iuqxhi 2024-03-16T09:04:04.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:03 smithi019 ceph-mon[36407]: pgmap v49: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:03 smithi019 ceph-mon[36407]: mgrmap e33: smithi007.uamwtp(active, since 92s), standbys: smithi019.iuqxhi 2024-03-16T09:04:04.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:04 smithi007 ceph-mon[29470]: Upgrade: Updating grafana.smithi007 2024-03-16T09:04:04.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:04 smithi007 ceph-mon[29470]: Deploying daemon grafana.smithi007 on smithi007 2024-03-16T09:04:05.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:04 smithi019 ceph-mon[36407]: Upgrade: Updating grafana.smithi007 2024-03-16T09:04:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:04 smithi019 ceph-mon[36407]: Deploying daemon grafana.smithi007 on smithi007 2024-03-16T09:04:06.138 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:05 smithi007 ceph-mon[29470]: pgmap v50: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:06.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:05 smithi019 ceph-mon[36407]: pgmap v50: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:07.716 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:07 smithi007 ceph-mon[29470]: pgmap v51: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:07.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:07 smithi019 ceph-mon[36407]: pgmap v51: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.057+0000 7f7ae703b700 1 -- 172.21.15.7:0/1513482530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0066b70 msgr2=0x7f7ae00ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.057+0000 7f7ae703b700 1 --2- 172.21.15.7:0/1513482530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0066b70 0x7f7ae00ffa50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4004940 tx=0x7f7ad4009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.057+0000 7f7ae703b700 1 -- 172.21.15.7:0/1513482530 shutdown_connections 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.057+0000 7f7ae703b700 1 --2- 172.21.15.7:0/1513482530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae00fff90 0x7f7ae01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.057+0000 7f7ae703b700 1 --2- 172.21.15.7:0/1513482530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae0066b70 0x7f7ae00ffa50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.057+0000 7f7ae703b700 1 -- 172.21.15.7:0/1513482530 >> 172.21.15.7:0/1513482530 conn(0x7f7ae00f74e0 msgr2=0x7f7ae00f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.057+0000 7f7ae703b700 1 -- 172.21.15.7:0/1513482530 shutdown_connections 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.057+0000 7f7ae703b700 1 -- 172.21.15.7:0/1513482530 wait complete. 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.058+0000 7f7ae703b700 1 Processor -- start 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.058+0000 7f7ae703b700 1 -- start start 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.058+0000 7f7ae703b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae00fff90 0x7f7ae010db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.058+0000 7f7ae703b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae010e0c0 0x7f7ae010ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.058+0000 7f7ae703b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae010e580 con 0x7f7ae00fff90 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.058+0000 7f7ae703b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae010e6c0 con 0x7f7ae010e0c0 2024-03-16T09:04:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.058+0000 7f7ae4dd7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae00fff90 0x7f7ae010db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7ae4dd7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae00fff90 0x7f7ae010db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54850/0 (socket says 172.21.15.7:54850) 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7ae4dd7700 1 -- 172.21.15.7:0/297919415 learned_addr learned my addr 172.21.15.7:0/297919415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7adffff700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae010e0c0 0x7f7ae010ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7ae4dd7700 1 -- 172.21.15.7:0/297919415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae010e0c0 msgr2=0x7f7ae010ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7ae4dd7700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae010e0c0 0x7f7ae010ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7ae4dd7700 1 -- 172.21.15.7:0/297919415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad4005040 con 0x7f7ae00fff90 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7ae4dd7700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae00fff90 0x7f7ae010db80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4000c00 tx=0x7f7ad4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7add7fa700 1 -- 172.21.15.7:0/297919415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ad400e040 con 0x7f7ae00fff90 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae010f510 con 0x7f7ae00fff90 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7add7fa700 1 -- 172.21.15.7:0/297919415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ad4004290 con 0x7f7ae00fff90 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7add7fa700 1 -- 172.21.15.7:0/297919415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ad401c3f0 con 0x7f7ae00fff90 2024-03-16T09:04:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.059+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae010faa0 con 0x7f7ae00fff90 2024-03-16T09:04:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.060+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac40052e0 con 0x7f7ae00fff90 2024-03-16T09:04:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.062+0000 7f7add7fa700 1 -- 172.21.15.7:0/297919415 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f7ad4013420 con 0x7f7ae00fff90 2024-03-16T09:04:09.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.062+0000 7f7add7fa700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7ad006d7e0 0x7f7ad006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.062+0000 7f7add7fa700 1 -- 172.21.15.7:0/297919415 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7ad408deb0 con 0x7f7ae00fff90 2024-03-16T09:04:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.062+0000 7f7adffff700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7ad006d7e0 0x7f7ad006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.063+0000 7f7adffff700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7ad006d7e0 0x7f7ad006fc90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009fd0 tx=0x7f7acc00b040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:09.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.067+0000 7f7add7fa700 1 -- 172.21.15.7:0/297919415 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f7ad405b770 con 0x7f7ae00fff90 2024-03-16T09:04:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.309+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ac4000bf0 con 0x7f7ad006d7e0 2024-03-16T09:04:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.310+0000 7f7add7fa700 1 -- 172.21.15.7:0/297919415 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f7ac4000bf0 con 0x7f7ad006d7e0 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7ad006d7e0 msgr2=0x7f7ad006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7ad006d7e0 0x7f7ad006fc90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009fd0 tx=0x7f7acc00b040 comp rx=0 tx=0).stop 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae00fff90 msgr2=0x7f7ae010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae00fff90 0x7f7ae010db80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4000c00 tx=0x7f7ad4009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 shutdown_connections 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7ad006d7e0 0x7f7ad006fc90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ae00fff90 0x7f7ae010db80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 --2- 172.21.15.7:0/297919415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae010e0c0 0x7f7ae010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 >> 172.21.15.7:0/297919415 conn(0x7f7ae00f74e0 msgr2=0x7f7ae00f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 shutdown_connections 2024-03-16T09:04:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.312+0000 7f7ae703b700 1 -- 172.21.15.7:0/297919415 wait complete. 2024-03-16T09:04:09.325 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:04:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.420+0000 7faae6177700 1 -- 172.21.15.7:0/2793900513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fe400 msgr2=0x7faae00fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.420+0000 7faae6177700 1 --2- 172.21.15.7:0/2793900513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fe400 0x7faae00fe810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faad0004940 tx=0x7faad0009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.420+0000 7faae6177700 1 -- 172.21.15.7:0/2793900513 shutdown_connections 2024-03-16T09:04:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.420+0000 7faae6177700 1 --2- 172.21.15.7:0/2793900513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00fed50 0x7faae00fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.420+0000 7faae6177700 1 --2- 172.21.15.7:0/2793900513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fe400 0x7faae00fe810 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.420+0000 7faae6177700 1 -- 172.21.15.7:0/2793900513 >> 172.21.15.7:0/2793900513 conn(0x7faae00f6ba0 msgr2=0x7faae00f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.420+0000 7faae6177700 1 -- 172.21.15.7:0/2793900513 shutdown_connections 2024-03-16T09:04:09.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.420+0000 7faae6177700 1 -- 172.21.15.7:0/2793900513 wait complete. 2024-03-16T09:04:09.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.421+0000 7faae6177700 1 Processor -- start 2024-03-16T09:04:09.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.421+0000 7faae6177700 1 -- start start 2024-03-16T09:04:09.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.421+0000 7faae6177700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00fe400 0x7faae00fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.421+0000 7faae6177700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fed50 0x7faae00fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.421+0000 7faae6177700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae00fc700 con 0x7faae00fed50 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.421+0000 7faae6177700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae00fc870 con 0x7faae00fe400 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.421+0000 7faadeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fed50 0x7faae00fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.421+0000 7faadeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fed50 0x7faae00fc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54860/0 (socket says 172.21.15.7:54860) 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faadeffd700 1 -- 172.21.15.7:0/1562850796 learned_addr learned my addr 172.21.15.7:0/1562850796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faadf7fe700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00fe400 0x7faae00fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faadeffd700 1 -- 172.21.15.7:0/1562850796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00fe400 msgr2=0x7faae00fdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faadeffd700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00fe400 0x7faae00fdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faadeffd700 1 -- 172.21.15.7:0/1562850796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad0005040 con 0x7faae00fed50 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faadf7fe700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00fe400 0x7faae00fdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faadeffd700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fed50 0x7faae00fc0c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faad400b790 tx=0x7faad4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faac7fff700 1 -- 172.21.15.7:0/1562850796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faad400f770 con 0x7faae00fed50 2024-03-16T09:04:09.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faac7fff700 1 -- 172.21.15.7:0/1562850796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faad400fda0 con 0x7faae00fed50 2024-03-16T09:04:09.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae00fcb50 con 0x7faae00fed50 2024-03-16T09:04:09.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faac7fff700 1 -- 172.21.15.7:0/1562850796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faad400d310 con 0x7faae00fed50 2024-03-16T09:04:09.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.422+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae0115ab0 con 0x7faae00fed50 2024-03-16T09:04:09.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.424+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae005f5f0 con 0x7faae00fed50 2024-03-16T09:04:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.424+0000 7faac7fff700 1 -- 172.21.15.7:0/1562850796 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7faad4010420 con 0x7faae00fed50 2024-03-16T09:04:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.425+0000 7faac7fff700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7faac806d7e0 0x7faac806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.425+0000 7faac7fff700 1 -- 172.21.15.7:0/1562850796 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7faad408ce00 con 0x7faae00fed50 2024-03-16T09:04:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.425+0000 7faadf7fe700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7faac806d7e0 0x7faac806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.425+0000 7faadf7fe700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7faac806d7e0 0x7faac806fc90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faad0004890 tx=0x7faad0009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.431+0000 7faac7fff700 1 -- 172.21.15.7:0/1562850796 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7faad4056b90 con 0x7faae00fed50 2024-03-16T09:04:09.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.664+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faae0102450 con 0x7faac806d7e0 2024-03-16T09:04:09.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.665+0000 7faac7fff700 1 -- 172.21.15.7:0/1562850796 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7faae0102450 con 0x7faac806d7e0 2024-03-16T09:04:09.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7faac806d7e0 msgr2=0x7faac806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7faac806d7e0 0x7faac806fc90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faad0004890 tx=0x7faad0009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:09.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fed50 msgr2=0x7faae00fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fed50 0x7faae00fc0c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faad400b790 tx=0x7faad4009250 comp rx=0 tx=0).stop 2024-03-16T09:04:09.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 shutdown_connections 2024-03-16T09:04:09.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7faac806d7e0 0x7faac806fc90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae00fe400 0x7faae00fdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 --2- 172.21.15.7:0/1562850796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faae00fed50 0x7faae00fc0c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 >> 172.21.15.7:0/1562850796 conn(0x7faae00f6ba0 msgr2=0x7faae00f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:09.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 shutdown_connections 2024-03-16T09:04:09.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.667+0000 7faae6177700 1 -- 172.21.15.7:0/1562850796 wait complete. 2024-03-16T09:04:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.770+0000 7f6104a8c700 1 -- 172.21.15.7:0/2736607751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 msgr2=0x7f61000fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.770+0000 7f6104a8c700 1 --2- 172.21.15.7:0/2736607751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 0x7f61000fd280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f60f0004940 tx=0x7f60f0009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.770+0000 7f6104a8c700 1 -- 172.21.15.7:0/2736607751 shutdown_connections 2024-03-16T09:04:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.770+0000 7f6104a8c700 1 --2- 172.21.15.7:0/2736607751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61000fdfe0 0x7f61000fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.770+0000 7f6104a8c700 1 --2- 172.21.15.7:0/2736607751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 0x7f61000fd280 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.770+0000 7f6104a8c700 1 -- 172.21.15.7:0/2736607751 >> 172.21.15.7:0/2736607751 conn(0x7f61000f8550 msgr2=0x7f61000fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f6104a8c700 1 -- 172.21.15.7:0/2736607751 shutdown_connections 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f6104a8c700 1 -- 172.21.15.7:0/2736607751 wait complete. 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f6104a8c700 1 Processor -- start 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f6104a8c700 1 -- start start 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f6104a8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 0x7f6100071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f6104a8c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61000fdfe0 0x7f6100070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f6104a8c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6100072260 con 0x7f61000fce70 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f6104a8c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6100111650 con 0x7f61000fdfe0 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f60fe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 0x7f6100071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f60fe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 0x7f6100071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54872/0 (socket says 172.21.15.7:54872) 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f60fe59c700 1 -- 172.21.15.7:0/1942548474 learned_addr learned my addr 172.21.15.7:0/1942548474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.771+0000 7f60fdd9b700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61000fdfe0 0x7f6100070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f60fe59c700 1 -- 172.21.15.7:0/1942548474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61000fdfe0 msgr2=0x7f6100070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f60fe59c700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61000fdfe0 0x7f6100070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f60fe59c700 1 -- 172.21.15.7:0/1942548474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0005040 con 0x7f61000fce70 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f60fdd9b700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61000fdfe0 0x7f6100070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f60fe59c700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 0x7f6100071c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b3b0 tx=0x7f60f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f60eeffd700 1 -- 172.21.15.7:0/1942548474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f000e040 con 0x7f61000fce70 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61001118a0 con 0x7f61000fce70 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f60eeffd700 1 -- 172.21.15.7:0/1942548474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60f000bcc0 con 0x7f61000fce70 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f60eeffd700 1 -- 172.21.15.7:0/1942548474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f001c630 con 0x7f61000fce70 2024-03-16T09:04:09.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.772+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6100070c80 con 0x7f61000fce70 2024-03-16T09:04:09.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.773+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610004c710 con 0x7f61000fce70 2024-03-16T09:04:09.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.774+0000 7f60eeffd700 1 -- 172.21.15.7:0/1942548474 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f60f0013980 con 0x7f61000fce70 2024-03-16T09:04:09.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.774+0000 7f60eeffd700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f60e806d7e0 0x7f60e806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:09.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.774+0000 7f60eeffd700 1 -- 172.21.15.7:0/1942548474 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f60f008d930 con 0x7f61000fce70 2024-03-16T09:04:09.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.774+0000 7f60fdd9b700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f60e806d7e0 0x7f60e806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.775+0000 7f60fdd9b700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f60e806d7e0 0x7f60e806fc90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f60f40041d0 tx=0x7f60f4009040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:09.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:09.779+0000 7f60eeffd700 1 -- 172.21.15.7:0/1942548474 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f60f005b170 con 0x7f61000fce70 2024-03-16T09:04:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:09 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:09 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:09 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:09 smithi019 ceph-mon[36407]: pgmap v52: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:09.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:09 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:09.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:09 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:09.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:09 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:09.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:09 smithi007 ceph-mon[29470]: pgmap v52: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.002+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f61001118a0 con 0x7f60e806d7e0 2024-03-16T09:04:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.012+0000 7f60eeffd700 1 -- 172.21.15.7:0/1942548474 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f61001118a0 con 0x7f60e806d7e0 2024-03-16T09:04:10.014 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:04:10.014 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (19s) 13s ago 11m 14.4M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:04:10.014 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (11m) 13s ago 11m 6669k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (10m) 51s ago 10m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 starting - - - - 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (5m) 13s ago 5m 12.7M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (5m) 13s ago 5m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (5m) 51s ago 5m 121M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (5m) 51s ago 5m 13.1M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (119s) 13s ago 12m 499M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (71s) 51s ago 10m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (12m) 13s ago 12m 52.7M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (10m) 51s ago 10m 43.7M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (59s) 13s ago 11m 7746k - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (55s) 51s ago 10m 3565k - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (9m) 13s ago 9m 205M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:04:10.015 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (8m) 13s ago 8m 234M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:04:10.016 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (8m) 13s ago 8m 181M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:04:10.016 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (8m) 51s ago 8m 262M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:04:10.016 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (7m) 51s ago 7m 244M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:04:10.016 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (7m) 51s ago 7m 221M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:04:10.016 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (37s) 13s ago 10m 48.9M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f60e806d7e0 msgr2=0x7f60e806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f60e806d7e0 0x7f60e806fc90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f60f40041d0 tx=0x7f60f4009040 comp rx=0 tx=0).stop 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 msgr2=0x7f6100071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 0x7f6100071c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b3b0 tx=0x7f60f0009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 shutdown_connections 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f60e806d7e0 0x7f60e806fc90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61000fce70 0x7f6100071c60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 --2- 172.21.15.7:0/1942548474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61000fdfe0 0x7f6100070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 >> 172.21.15.7:0/1942548474 conn(0x7f61000f8550 msgr2=0x7f61000f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 shutdown_connections 2024-03-16T09:04:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.014+0000 7f6104a8c700 1 -- 172.21.15.7:0/1942548474 wait complete. 2024-03-16T09:04:10.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 -- 172.21.15.7:0/3373313805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 msgr2=0x7fec040ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 --2- 172.21.15.7:0/3373313805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 0x7fec040ff9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7febf4009ce0 tx=0x7febf4004d10 comp rx=0 tx=0).stop 2024-03-16T09:04:10.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 -- 172.21.15.7:0/3373313805 shutdown_connections 2024-03-16T09:04:10.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 --2- 172.21.15.7:0/3373313805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec040fff00 0x7fec04102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 --2- 172.21.15.7:0/3373313805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 0x7fec040ff9c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 -- 172.21.15.7:0/3373313805 >> 172.21.15.7:0/3373313805 conn(0x7fec040f7430 msgr2=0x7fec040f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:10.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 -- 172.21.15.7:0/3373313805 shutdown_connections 2024-03-16T09:04:10.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 -- 172.21.15.7:0/3373313805 wait complete. 2024-03-16T09:04:10.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.117+0000 7fec09ca3700 1 Processor -- start 2024-03-16T09:04:10.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec09ca3700 1 -- start start 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec09ca3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 0x7fec0410fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec09ca3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec040fff00 0x7fec04110380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec09ca3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec041110d0 con 0x7fec040662e0 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec09ca3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec04111240 con 0x7fec040fff00 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 0x7fec0410fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 0x7fec0410fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54888/0 (socket says 172.21.15.7:54888) 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec037fe700 1 -- 172.21.15.7:0/3107700706 learned_addr learned my addr 172.21.15.7:0/3107700706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec037fe700 1 -- 172.21.15.7:0/3107700706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec040fff00 msgr2=0x7fec04110380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec037fe700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec040fff00 0x7fec04110380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec037fe700 1 -- 172.21.15.7:0/3107700706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4009940 con 0x7fec040662e0 2024-03-16T09:04:10.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7fec037fe700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 0x7fec0410fe40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7febf4009380 tx=0x7febf4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7febfbfff700 1 -- 172.21.15.7:0/3107700706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf400e040 con 0x7fec040662e0 2024-03-16T09:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.118+0000 7febfbfff700 1 -- 172.21.15.7:0/3107700706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febf40128e0 con 0x7fec040662e0 2024-03-16T09:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.119+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec041114c0 con 0x7fec040662e0 2024-03-16T09:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.119+0000 7febfbfff700 1 -- 172.21.15.7:0/3107700706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf401c9f0 con 0x7fec040662e0 2024-03-16T09:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.119+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec04111a80 con 0x7fec040662e0 2024-03-16T09:04:10.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.119+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec041060d0 con 0x7fec040662e0 2024-03-16T09:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.120+0000 7febfbfff700 1 -- 172.21.15.7:0/3107700706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7febf4012a50 con 0x7fec040662e0 2024-03-16T09:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.120+0000 7febfbfff700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7febf006d7e0 0x7febf006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.121+0000 7febfbfff700 1 -- 172.21.15.7:0/3107700706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7febf408ea70 con 0x7fec040662e0 2024-03-16T09:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.121+0000 7fec02ffd700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7febf006d7e0 0x7febf006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:10.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.121+0000 7fec02ffd700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7febf006d7e0 0x7febf006fc90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fec0410fbd0 tx=0x7febec007d40 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:10.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.125+0000 7febfbfff700 1 -- 172.21.15.7:0/3107700706 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7febf4058780 con 0x7fec040662e0 2024-03-16T09:04:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.421+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec04111d60 con 0x7fec040662e0 2024-03-16T09:04:10.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.422+0000 7febfbfff700 1 -- 172.21.15.7:0/3107700706 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7febf405be40 con 0x7fec040662e0 2024-03-16T09:04:10.424 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:04:10.424 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:04:10.424 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:04:10.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 12, 2024-03-16T09:04:10.425 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:04:10.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:04:10.426 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:04:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7febf006d7e0 msgr2=0x7febf006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7febf006d7e0 0x7febf006fc90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fec0410fbd0 tx=0x7febec007d40 comp rx=0 tx=0).stop 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 msgr2=0x7fec0410fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 0x7fec0410fe40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7febf4009380 tx=0x7febf4004d10 comp rx=0 tx=0).stop 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 shutdown_connections 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7febf006d7e0 0x7febf006fc90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040662e0 0x7fec0410fe40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 --2- 172.21.15.7:0/3107700706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec040fff00 0x7fec04110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 >> 172.21.15.7:0/3107700706 conn(0x7fec040f7430 msgr2=0x7fec040fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 shutdown_connections 2024-03-16T09:04:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.424+0000 7fec09ca3700 1 -- 172.21.15.7:0/3107700706 wait complete. 2024-03-16T09:04:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.528+0000 7f0c04a43700 1 -- 172.21.15.7:0/3244772170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 msgr2=0x7f0c000fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.528+0000 7f0c04a43700 1 --2- 172.21.15.7:0/3244772170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 0x7f0c000fd310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0004940 tx=0x7f0bf0009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.528+0000 7f0c04a43700 1 -- 172.21.15.7:0/3244772170 shutdown_connections 2024-03-16T09:04:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.528+0000 7f0c04a43700 1 --2- 172.21.15.7:0/3244772170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c000fe070 0x7f0c000fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.528+0000 7f0c04a43700 1 --2- 172.21.15.7:0/3244772170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 0x7f0c000fd310 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.528+0000 7f0c04a43700 1 -- 172.21.15.7:0/3244772170 >> 172.21.15.7:0/3244772170 conn(0x7f0c000f85a0 msgr2=0x7f0c000fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.528+0000 7f0c04a43700 1 -- 172.21.15.7:0/3244772170 shutdown_connections 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.528+0000 7f0c04a43700 1 -- 172.21.15.7:0/3244772170 wait complete. 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0c04a43700 1 Processor -- start 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0c04a43700 1 -- start start 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0c04a43700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 0x7f0c00071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0c04a43700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c000fe070 0x7f0c00070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0c04a43700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c000708a0 con 0x7f0c000fcf00 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0c04a43700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c000709e0 con 0x7f0c000fe070 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0bfe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 0x7f0c00071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0bfe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 0x7f0c00071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54910/0 (socket says 172.21.15.7:54910) 2024-03-16T09:04:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0bfe59c700 1 -- 172.21.15.7:0/110124992 learned_addr learned my addr 172.21.15.7:0/110124992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.529+0000 7f0bfdd9b700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c000fe070 0x7f0c00070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0bfe59c700 1 -- 172.21.15.7:0/110124992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c000fe070 msgr2=0x7f0c00070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0bfe59c700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c000fe070 0x7f0c00070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0bfe59c700 1 -- 172.21.15.7:0/110124992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0005040 con 0x7f0c000fcf00 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0bfe59c700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 0x7f0c00071df0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b3b0 tx=0x7f0bf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0beeffd700 1 -- 172.21.15.7:0/110124992 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf000e040 con 0x7f0c000fcf00 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0beeffd700 1 -- 172.21.15.7:0/110124992 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bf000bcc0 con 0x7f0c000fcf00 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c00070c60 con 0x7f0c000fcf00 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0beeffd700 1 -- 172.21.15.7:0/110124992 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf001c630 con 0x7f0c000fcf00 2024-03-16T09:04:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.530+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c00115ac0 con 0x7f0c000fcf00 2024-03-16T09:04:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.531+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0005f5f0 con 0x7f0c000fcf00 2024-03-16T09:04:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.532+0000 7f0beeffd700 1 -- 172.21.15.7:0/110124992 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f0bf0013980 con 0x7f0c000fcf00 2024-03-16T09:04:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.532+0000 7f0beeffd700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0be806d7e0 0x7f0be806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.532+0000 7f0beeffd700 1 -- 172.21.15.7:0/110124992 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0bf008daf0 con 0x7f0c000fcf00 2024-03-16T09:04:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.532+0000 7f0bfdd9b700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0be806d7e0 0x7f0be806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.532+0000 7f0bfdd9b700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0be806d7e0 0x7f0be806fc90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4007fd0 tx=0x7f0bf4009040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.537+0000 7f0beeffd700 1 -- 172.21.15.7:0/110124992 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f0bf0057800 con 0x7f0c000fcf00 2024-03-16T09:04:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.780+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c001023e0 con 0x7f0be806d7e0 2024-03-16T09:04:10.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.781+0000 7f0beeffd700 1 -- 172.21.15.7:0/110124992 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f0c001023e0 con 0x7f0be806d7e0 2024-03-16T09:04:10.817 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:04:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:04:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:04:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-16T09:04:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:04:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-03-16T09:04:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:04:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/2 daemons upgraded", 2024-03-16T09:04:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading grafana daemons", 2024-03-16T09:04:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:04:10.818 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:04:10.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0be806d7e0 msgr2=0x7f0be806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0be806d7e0 0x7f0be806fc90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4007fd0 tx=0x7f0bf4009040 comp rx=0 tx=0).stop 2024-03-16T09:04:10.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 msgr2=0x7f0c00071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:10.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 0x7f0c00071df0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b3b0 tx=0x7f0bf0009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:10.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 shutdown_connections 2024-03-16T09:04:10.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f0be806d7e0 0x7f0be806fc90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c000fcf00 0x7f0c00071df0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 --2- 172.21.15.7:0/110124992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c000fe070 0x7f0c00070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:10.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 >> 172.21.15.7:0/110124992 conn(0x7f0c000f85a0 msgr2=0x7f0c000f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:10.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 shutdown_connections 2024-03-16T09:04:10.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:10.783+0000 7f0c04a43700 1 -- 172.21.15.7:0/110124992 wait complete. 2024-03-16T09:04:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:10 smithi019 ceph-mon[36407]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:10 smithi019 ceph-mon[36407]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:10 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/3107700706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:10.975 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:10 smithi007 ceph-mon[29470]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:10.975 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:10 smithi007 ceph-mon[29470]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:10.976 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:10 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/3107700706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:11.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:11 smithi007 ceph-mon[29470]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:11.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:11 smithi007 ceph-mon[29470]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:11.836 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:11 smithi007 ceph-mon[29470]: pgmap v53: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:11 smithi019 ceph-mon[36407]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:11 smithi019 ceph-mon[36407]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:11 smithi019 ceph-mon[36407]: pgmap v53: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:14.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:13 smithi019 ceph-mon[36407]: pgmap v54: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:13 smithi007 ceph-mon[29470]: pgmap v54: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T09:04:15.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:15 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:15.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:15 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:15.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:15 smithi019 ceph-mon[36407]: pgmap v55: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:15 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:15 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:15 smithi007 ceph-mon[29470]: pgmap v55: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:04:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:04:17.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:17 smithi019 ceph-mon[36407]: pgmap v56: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:17 smithi007 ceph-mon[29470]: pgmap v56: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:20.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: pgmap v57: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:20.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:04:20.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-16T09:04:20.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:04:20.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:04:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: pgmap v57: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-16T09:04:20.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.251 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:04:20.252 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:04:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:20 smithi019 ceph-mon[36407]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T09:04:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:20 smithi019 ceph-mon[36407]: Upgrade: Finalizing container_image settings 2024-03-16T09:04:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:20 smithi019 ceph-mon[36407]: Upgrade: Complete! 2024-03-16T09:04:21.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:20 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:20 smithi007 ceph-mon[29470]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T09:04:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:20 smithi007 ceph-mon[29470]: Upgrade: Finalizing container_image settings 2024-03-16T09:04:21.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:20 smithi007 ceph-mon[29470]: Upgrade: Complete! 2024-03-16T09:04:21.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:20 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:21 smithi007 ceph-mon[29470]: pgmap v58: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:21 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:22.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:21 smithi019 ceph-mon[36407]: pgmap v58: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:22.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:21 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:04:24.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:23 smithi019 ceph-mon[36407]: pgmap v59: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:23 smithi007 ceph-mon[29470]: pgmap v59: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:25.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:24 smithi019 ceph-mon[36407]: pgmap v60: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:25.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:24 smithi007 ceph-mon[29470]: pgmap v60: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:27.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:27 smithi019 ceph-mon[36407]: pgmap v61: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:27.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:27 smithi007 ceph-mon[29470]: pgmap v61: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:30.034 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:29 smithi007 ceph-mon[29470]: pgmap v62: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:30.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:29 smithi019 ceph-mon[36407]: pgmap v62: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:31.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:31 smithi007 ceph-mon[29470]: pgmap v63: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:31.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:31 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:04:32.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:31 smithi019 ceph-mon[36407]: pgmap v63: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:32.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:31 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:04:34.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:33 smithi019 ceph-mon[36407]: pgmap v64: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:34.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:33 smithi007 ceph-mon[29470]: pgmap v64: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:36.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:35 smithi019 ceph-mon[36407]: pgmap v65: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:35 smithi007 ceph-mon[29470]: pgmap v65: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:37.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:37 smithi019 ceph-mon[36407]: pgmap v66: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:37 smithi007 ceph-mon[29470]: pgmap v66: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:39 smithi019 ceph-mon[36407]: pgmap v67: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:39 smithi007 ceph-mon[29470]: pgmap v67: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.901+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/3688520903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40fff00 msgr2=0x7f7df4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.901+0000 7f7dfb4a2700 1 --2- 172.21.15.7:0/3688520903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40fff00 0x7f7df4102320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7de4004940 tx=0x7f7de4009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.901+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/3688520903 shutdown_connections 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.901+0000 7f7dfb4a2700 1 --2- 172.21.15.7:0/3688520903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40fff00 0x7f7df4102320 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.901+0000 7f7dfb4a2700 1 --2- 172.21.15.7:0/3688520903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40662e0 0x7f7df40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.901+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/3688520903 >> 172.21.15.7:0/3688520903 conn(0x7f7df40f7430 msgr2=0x7f7df40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.901+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/3688520903 shutdown_connections 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.901+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/3688520903 wait complete. 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7dfb4a2700 1 Processor -- start 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7dfb4a2700 1 -- start start 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7dfb4a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40662e0 0x7f7df40fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7dfb4a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40fff00 0x7f7df40fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7dfb4a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40fbe70 con 0x7f7df40662e0 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7dfb4a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40fbfe0 con 0x7f7df40fff00 2024-03-16T09:04:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7df923e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40662e0 0x7f7df40fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7df923e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40662e0 0x7f7df40fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58216/0 (socket says 172.21.15.7:58216) 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7df923e700 1 -- 172.21.15.7:0/2933917715 learned_addr learned my addr 172.21.15.7:0/2933917715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.902+0000 7f7df8a3d700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40fff00 0x7f7df40fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7df923e700 1 -- 172.21.15.7:0/2933917715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40fff00 msgr2=0x7f7df40fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7df923e700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40fff00 0x7f7df40fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7df923e700 1 -- 172.21.15.7:0/2933917715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4005040 con 0x7f7df40662e0 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7df8a3d700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40fff00 0x7f7df40fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7df923e700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40662e0 0x7f7df40fd3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7df000c890 tx=0x7f7df00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7de9ffb700 1 -- 172.21.15.7:0/2933917715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df000a7d0 con 0x7f7df40662e0 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7de9ffb700 1 -- 172.21.15.7:0/2933917715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7df000ae00 con 0x7f7df40662e0 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df40fc2c0 con 0x7f7df40662e0 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7de9ffb700 1 -- 172.21.15.7:0/2933917715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df00113a0 con 0x7f7df40662e0 2024-03-16T09:04:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.903+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df4115a90 con 0x7f7df40662e0 2024-03-16T09:04:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.904+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df405f5f0 con 0x7f7df40662e0 2024-03-16T09:04:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.904+0000 7f7de9ffb700 1 -- 172.21.15.7:0/2933917715 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f7df0012400 con 0x7f7df40662e0 2024-03-16T09:04:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.905+0000 7f7de9ffb700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7de006d7e0 0x7f7de006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.905+0000 7f7de9ffb700 1 -- 172.21.15.7:0/2933917715 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7df008cdb0 con 0x7f7df40662e0 2024-03-16T09:04:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.905+0000 7f7df8a3d700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7de006d7e0 0x7f7de006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.906+0000 7f7df8a3d700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7de006d7e0 0x7f7de006fc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b3b0 tx=0x7f7de4009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:40.910+0000 7f7de9ffb700 1 -- 172.21.15.7:0/2933917715 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f7df0056be0 con 0x7f7df40662e0 2024-03-16T09:04:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.133+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7df40f8f70 con 0x7f7de006d7e0 2024-03-16T09:04:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.134+0000 7f7de9ffb700 1 -- 172.21.15.7:0/2933917715 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f7df40f8f70 con 0x7f7de006d7e0 2024-03-16T09:04:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7de006d7e0 msgr2=0x7f7de006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7de006d7e0 0x7f7de006fc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b3b0 tx=0x7f7de4009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40662e0 msgr2=0x7f7df40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40662e0 0x7f7df40fd3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7df000c890 tx=0x7f7df00044c0 comp rx=0 tx=0).stop 2024-03-16T09:04:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 shutdown_connections 2024-03-16T09:04:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f7de006d7e0 0x7f7de006fc90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7df40662e0 0x7f7df40fd3f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 --2- 172.21.15.7:0/2933917715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40fff00 0x7f7df40fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:41.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.136+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 >> 172.21.15.7:0/2933917715 conn(0x7f7df40f7430 msgr2=0x7f7df40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:41.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.137+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 shutdown_connections 2024-03-16T09:04:41.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:41.137+0000 7f7dfb4a2700 1 -- 172.21.15.7:0/2933917715 wait complete. 2024-03-16T09:04:41.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-16T09:04:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:04:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:41 smithi019 ceph-mon[36407]: pgmap v68: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:42.232 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:41 smithi007 ceph-mon[29470]: pgmap v68: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:43.212 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:42 smithi007 ceph-mon[29470]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:43.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:42 smithi019 ceph-mon[36407]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 -- 172.21.15.7:0/3837529131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fde40 msgr2=0x7f42580fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 --2- 172.21.15.7:0/3837529131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fde40 0x7f42580fe210 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4254004940 tx=0x7f4254009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 -- 172.21.15.7:0/3837529131 shutdown_connections 2024-03-16T09:04:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 --2- 172.21.15.7:0/3837529131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580fe7e0 0x7f4258106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 --2- 172.21.15.7:0/3837529131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fde40 0x7f42580fe210 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 -- 172.21.15.7:0/3837529131 >> 172.21.15.7:0/3837529131 conn(0x7f42580f7880 msgr2=0x7f42580f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 -- 172.21.15.7:0/3837529131 shutdown_connections 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 -- 172.21.15.7:0/3837529131 wait complete. 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 Processor -- start 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.757+0000 7f425f359700 1 -- start start 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425f359700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580fde40 0x7f42580fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425f359700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fe7e0 0x7f42580fc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425f359700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42580fc7d0 con 0x7f42580fe7e0 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425f359700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42580fc910 con 0x7f42580fde40 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425c8f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fe7e0 0x7f42580fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425c8f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fe7e0 0x7f42580fc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33552/0 (socket says 172.21.15.7:33552) 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425c8f4700 1 -- 172.21.15.7:0/419328200 learned_addr learned my addr 172.21.15.7:0/419328200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425d0f5700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580fde40 0x7f42580fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425c8f4700 1 -- 172.21.15.7:0/419328200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580fde40 msgr2=0x7f42580fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425c8f4700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580fde40 0x7f42580fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425c8f4700 1 -- 172.21.15.7:0/419328200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4254005040 con 0x7f42580fe7e0 2024-03-16T09:04:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.758+0000 7f425c8f4700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fe7e0 0x7f42580fc240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4248002a50 tx=0x7f42480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.759+0000 7f424dffb700 1 -- 172.21.15.7:0/419328200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4248007cc0 con 0x7f42580fe7e0 2024-03-16T09:04:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.759+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425811a200 con 0x7f42580fe7e0 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.759+0000 7f424dffb700 1 -- 172.21.15.7:0/419328200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f424800f040 con 0x7f42580fe7e0 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.759+0000 7f424dffb700 1 -- 172.21.15.7:0/419328200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f424800da30 con 0x7f42580fe7e0 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.759+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425811a600 con 0x7f42580fe7e0 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.760+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42580666b0 con 0x7f42580fe7e0 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.760+0000 7f424dffb700 1 -- 172.21.15.7:0/419328200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f424800db90 con 0x7f42580fe7e0 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.761+0000 7f424dffb700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f424406d7e0 0x7f424406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.761+0000 7f424dffb700 1 -- 172.21.15.7:0/419328200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f424808cc10 con 0x7f42580fe7e0 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.761+0000 7f425d0f5700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f424406d7e0 0x7f424406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.761+0000 7f425d0f5700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f424406d7e0 0x7f424406fc90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f425400b3b0 tx=0x7f4254009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:43.765+0000 7f424dffb700 1 -- 172.21.15.7:0/419328200 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f4248056880 con 0x7f42580fe7e0 2024-03-16T09:04:44.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.061+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f425805f5f0 con 0x7f42580fe7e0 2024-03-16T09:04:44.064 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:43 smithi007 ceph-mon[29470]: pgmap v69: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.061+0000 7f424dffb700 1 -- 172.21.15.7:0/419328200 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f4248059f40 con 0x7f42580fe7e0 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f424406d7e0 msgr2=0x7f424406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f424406d7e0 0x7f424406fc90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f425400b3b0 tx=0x7f4254009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fe7e0 msgr2=0x7f42580fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fe7e0 0x7f42580fc240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4248002a50 tx=0x7f42480046b0 comp rx=0 tx=0).stop 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 shutdown_connections 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f424406d7e0 0x7f424406fc90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42580fde40 0x7f42580fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 --2- 172.21.15.7:0/419328200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42580fe7e0 0x7f42580fc240 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 >> 172.21.15.7:0/419328200 conn(0x7f42580f7880 msgr2=0x7f42580f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 shutdown_connections 2024-03-16T09:04:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:44.064+0000 7f425f359700 1 -- 172.21.15.7:0/419328200 wait complete. 2024-03-16T09:04:44.076 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:04:44.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:43 smithi019 ceph-mon[36407]: pgmap v69: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:44.950 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:44 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/419328200' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:44.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-16T09:04:45.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:44 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/419328200' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:04:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:45 smithi007 ceph-mon[29470]: pgmap v70: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:46.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:45 smithi019 ceph-mon[36407]: pgmap v70: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.777+0000 7fbfc4768700 1 -- 172.21.15.7:0/2655978881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 msgr2=0x7fbfbc0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.777+0000 7fbfc4768700 1 --2- 172.21.15.7:0/2655978881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 0x7fbfbc0fbd30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8004940 tx=0x7fbfb8009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.777+0000 7fbfc4768700 1 -- 172.21.15.7:0/2655978881 shutdown_connections 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.777+0000 7fbfc4768700 1 --2- 172.21.15.7:0/2655978881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 0x7fbfbc0fbd30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.777+0000 7fbfc4768700 1 --2- 172.21.15.7:0/2655978881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc101d20 0x7fbfbc1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.777+0000 7fbfc4768700 1 -- 172.21.15.7:0/2655978881 >> 172.21.15.7:0/2655978881 conn(0x7fbfbc0f7870 msgr2=0x7fbfbc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.777+0000 7fbfc4768700 1 -- 172.21.15.7:0/2655978881 shutdown_connections 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.777+0000 7fbfc4768700 1 -- 172.21.15.7:0/2655978881 wait complete. 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc4768700 1 Processor -- start 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc4768700 1 -- start start 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc4768700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc101d20 0x7fbfbc10c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc4768700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 0x7fbfbc10ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc4768700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0671b0 con 0x7fbfbc1026c0 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc4768700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc10d3d0 con 0x7fbfbc101d20 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc1d03700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 0x7fbfbc10ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc1d03700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 0x7fbfbc10ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33566/0 (socket says 172.21.15.7:33566) 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc1d03700 1 -- 172.21.15.7:0/1213662394 learned_addr learned my addr 172.21.15.7:0/1213662394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc2504700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc101d20 0x7fbfbc10c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc1d03700 1 -- 172.21.15.7:0/1213662394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc101d20 msgr2=0x7fbfbc10c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc1d03700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc101d20 0x7fbfbc10c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc1d03700 1 -- 172.21.15.7:0/1213662394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8005040 con 0x7fbfbc1026c0 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.778+0000 7fbfc1d03700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 0x7fbfbc10ce90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009100 tx=0x7fbfb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.779+0000 7fbfb2ffd700 1 -- 172.21.15.7:0/1213662394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb800e040 con 0x7fbfbc1026c0 2024-03-16T09:04:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.779+0000 7fbfb2ffd700 1 -- 172.21.15.7:0/1213662394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfb8013420 con 0x7fbfbc1026c0 2024-03-16T09:04:46.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.779+0000 7fbfb2ffd700 1 -- 172.21.15.7:0/1213662394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb801c630 con 0x7fbfbc1026c0 2024-03-16T09:04:46.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.779+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc10d650 con 0x7fbfbc1026c0 2024-03-16T09:04:46.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.779+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc11a200 con 0x7fbfbc1026c0 2024-03-16T09:04:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.780+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc0fd470 con 0x7fbfbc1026c0 2024-03-16T09:04:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.780+0000 7fbfb2ffd700 1 -- 172.21.15.7:0/1213662394 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7fbfb8013590 con 0x7fbfbc1026c0 2024-03-16T09:04:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.780+0000 7fbfb2ffd700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfa806d7e0 0x7fbfa806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.781+0000 7fbfb2ffd700 1 -- 172.21.15.7:0/1213662394 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbfb808df00 con 0x7fbfbc1026c0 2024-03-16T09:04:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.781+0000 7fbfc2504700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfa806d7e0 0x7fbfa806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.781+0000 7fbfc2504700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfa806d7e0 0x7fbfa806fc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbfac006fd0 tx=0x7fbfac008040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:46.785+0000 7fbfb2ffd700 1 -- 172.21.15.7:0/1213662394 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fbfb805b860 con 0x7fbfbc1026c0 2024-03-16T09:04:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.076+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbfbc04c710 con 0x7fbfbc1026c0 2024-03-16T09:04:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.077+0000 7fbfb2ffd700 1 -- 172.21.15.7:0/1213662394 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fbfb805b3f0 con 0x7fbfbc1026c0 2024-03-16T09:04:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfa806d7e0 msgr2=0x7fbfa806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfa806d7e0 0x7fbfa806fc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbfac006fd0 tx=0x7fbfac008040 comp rx=0 tx=0).stop 2024-03-16T09:04:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 msgr2=0x7fbfbc10ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 0x7fbfbc10ce90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009100 tx=0x7fbfb8009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 shutdown_connections 2024-03-16T09:04:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfa806d7e0 0x7fbfa806fc90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc101d20 0x7fbfbc10c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 --2- 172.21.15.7:0/1213662394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc1026c0 0x7fbfbc10ce90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 >> 172.21.15.7:0/1213662394 conn(0x7fbfbc0f7870 msgr2=0x7fbfbc0f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 shutdown_connections 2024-03-16T09:04:47.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:47.079+0000 7fbfc4768700 1 -- 172.21.15.7:0/1213662394 wait complete. 2024-03-16T09:04:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)" 2024-03-16T09:04:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:46 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:04:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:46 smithi019 ceph-mon[36407]: pgmap v71: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:46 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:04:47.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:46 smithi007 ceph-mon[29470]: pgmap v71: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:47.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-16T09:04:47.977 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:04:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:47 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1213662394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:48.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:47 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1213662394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:49.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:48 smithi019 ceph-mon[36407]: pgmap v72: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:49.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:48 smithi007 ceph-mon[29470]: pgmap v72: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.771+0000 7f066f0fb700 1 -- 172.21.15.7:0/3948997985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668101ce0 msgr2=0x7f06681020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.771+0000 7f066f0fb700 1 --2- 172.21.15.7:0/3948997985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668101ce0 0x7f06681020b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f065c004940 tx=0x7f065c009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 -- 172.21.15.7:0/3948997985 shutdown_connections 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 --2- 172.21.15.7:0/3948997985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0668102680 0x7f06680fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 --2- 172.21.15.7:0/3948997985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668101ce0 0x7f06681020b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 -- 172.21.15.7:0/3948997985 >> 172.21.15.7:0/3948997985 conn(0x7f06680f7890 msgr2=0x7f06680f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 -- 172.21.15.7:0/3948997985 shutdown_connections 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 -- 172.21.15.7:0/3948997985 wait complete. 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 Processor -- start 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 -- start start 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0668101ce0 0x7f0668112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668102680 0x7f0668112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0668113420 con 0x7f0668102680 2024-03-16T09:04:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.772+0000 7f066f0fb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066810ca20 con 0x7f0668101ce0 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f0667fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668102680 0x7f0668112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f066ce97700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0668101ce0 0x7f0668112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f066ce97700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0668101ce0 0x7f0668112890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:52262/0 (socket says 172.21.15.7:52262) 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f0667fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668102680 0x7f0668112dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33576/0 (socket says 172.21.15.7:33576) 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f066ce97700 1 -- 172.21.15.7:0/1897524976 learned_addr learned my addr 172.21.15.7:0/1897524976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f0667fff700 1 -- 172.21.15.7:0/1897524976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0668101ce0 msgr2=0x7f0668112890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f0667fff700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0668101ce0 0x7f0668112890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f0667fff700 1 -- 172.21.15.7:0/1897524976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c005040 con 0x7f0668102680 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f066ce97700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0668101ce0 0x7f0668112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f0667fff700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668102680 0x7f0668112dd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0654004fd0 tx=0x7f0654004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f06657fa700 1 -- 172.21.15.7:0/1897524976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0654007cc0 con 0x7f0668102680 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066810cca0 con 0x7f0668102680 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f06657fa700 1 -- 172.21.15.7:0/1897524976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f065400f040 con 0x7f0668102680 2024-03-16T09:04:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f06657fa700 1 -- 172.21.15.7:0/1897524976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0654013860 con 0x7f0668102680 2024-03-16T09:04:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.773+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066810d260 con 0x7f0668102680 2024-03-16T09:04:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.774+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06680fd430 con 0x7f0668102680 2024-03-16T09:04:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.775+0000 7f06657fa700 1 -- 172.21.15.7:0/1897524976 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f065401c400 con 0x7f0668102680 2024-03-16T09:04:49.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.775+0000 7f06657fa700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f065806d7e0 0x7f065806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:49.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.775+0000 7f06657fa700 1 -- 172.21.15.7:0/1897524976 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f065408ca30 con 0x7f0668102680 2024-03-16T09:04:49.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.776+0000 7f066ce97700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f065806d7e0 0x7f065806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:49.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.776+0000 7f066ce97700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f065806d7e0 0x7f065806fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f065c009fd0 tx=0x7f065c009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:49.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:49.780+0000 7f06657fa700 1 -- 172.21.15.7:0/1897524976 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f065405a270 con 0x7f0668102680 2024-03-16T09:04:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.072+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0668108de0 con 0x7f0668102680 2024-03-16T09:04:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.072+0000 7f06657fa700 1 -- 172.21.15.7:0/1897524976 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f0654059e00 con 0x7f0668102680 2024-03-16T09:04:50.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f065806d7e0 msgr2=0x7f065806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f065806d7e0 0x7f065806fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f065c009fd0 tx=0x7f065c009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668102680 msgr2=0x7f0668112dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668102680 0x7f0668112dd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0654004fd0 tx=0x7f0654004570 comp rx=0 tx=0).stop 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 shutdown_connections 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f065806d7e0 0x7f065806fc90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0668101ce0 0x7f0668112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 --2- 172.21.15.7:0/1897524976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0668102680 0x7f0668112dd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 >> 172.21.15.7:0/1897524976 conn(0x7f06680f7890 msgr2=0x7f06680f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 shutdown_connections 2024-03-16T09:04:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:50.074+0000 7f066f0fb700 1 -- 172.21.15.7:0/1897524976 wait complete. 2024-03-16T09:04:50.087 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:04:50.365 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:50 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1897524976' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:50.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:50 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1897524976' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:04:50.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-16T09:04:51.134 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:04:51.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:51 smithi019 ceph-mon[36407]: pgmap v73: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:51.475 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:51 smithi007 ceph-mon[29470]: pgmap v73: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.798+0000 7f4f683c7700 1 -- 172.21.15.7:0/1958672415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f60101520 msgr2=0x7f4f601018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.798+0000 7f4f683c7700 1 --2- 172.21.15.7:0/1958672415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f60101520 0x7f4f601018f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c004a50 tx=0x7f4f5c00af40 comp rx=0 tx=0).stop 2024-03-16T09:04:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 -- 172.21.15.7:0/1958672415 shutdown_connections 2024-03-16T09:04:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 --2- 172.21.15.7:0/1958672415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f600fb4c0 0x7f4f600fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 --2- 172.21.15.7:0/1958672415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f60101520 0x7f4f601018f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 -- 172.21.15.7:0/1958672415 >> 172.21.15.7:0/1958672415 conn(0x7f4f60072310 msgr2=0x7f4f60072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 -- 172.21.15.7:0/1958672415 shutdown_connections 2024-03-16T09:04:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 -- 172.21.15.7:0/1958672415 wait complete. 2024-03-16T09:04:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 Processor -- start 2024-03-16T09:04:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 -- start start 2024-03-16T09:04:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f600fb4c0 0x7f4f60113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f60101520 0x7f4f60113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f601144f0 con 0x7f4f600fb4c0 2024-03-16T09:04:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f683c7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6010daf0 con 0x7f4f60101520 2024-03-16T09:04:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f66163700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f600fb4c0 0x7f4f60113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.799+0000 7f4f66163700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f600fb4c0 0x7f4f60113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38534/0 (socket says 172.21.15.7:38534) 2024-03-16T09:04:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f66163700 1 -- 172.21.15.7:0/2311187439 learned_addr learned my addr 172.21.15.7:0/2311187439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f65962700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f60101520 0x7f4f60113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f66163700 1 -- 172.21.15.7:0/2311187439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f60101520 msgr2=0x7f4f60113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f66163700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f60101520 0x7f4f60113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f66163700 1 -- 172.21.15.7:0/2311187439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c005040 con 0x7f4f600fb4c0 2024-03-16T09:04:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f66163700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f600fb4c0 0x7f4f60113960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00ada0 tx=0x7f4f5c00af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f56ffd700 1 -- 172.21.15.7:0/2311187439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5c00f070 con 0x7f4f600fb4c0 2024-03-16T09:04:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f56ffd700 1 -- 172.21.15.7:0/2311187439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f5c00a4c0 con 0x7f4f600fb4c0 2024-03-16T09:04:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6010dd70 con 0x7f4f600fb4c0 2024-03-16T09:04:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f56ffd700 1 -- 172.21.15.7:0/2311187439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5c014760 con 0x7f4f600fb4c0 2024-03-16T09:04:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.800+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6010e2d0 con 0x7f4f600fb4c0 2024-03-16T09:04:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.801+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f60105910 con 0x7f4f600fb4c0 2024-03-16T09:04:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.802+0000 7f4f56ffd700 1 -- 172.21.15.7:0/2311187439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f4f5c0149d0 con 0x7f4f600fb4c0 2024-03-16T09:04:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.802+0000 7f4f56ffd700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f4f4c06d7e0 0x7f4f4c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.802+0000 7f4f56ffd700 1 -- 172.21.15.7:0/2311187439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4f5c08e540 con 0x7f4f600fb4c0 2024-03-16T09:04:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.802+0000 7f4f65962700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f4f4c06d7e0 0x7f4f4c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.803+0000 7f4f65962700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f4f4c06d7e0 0x7f4f4c06fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f6010ecb0 tx=0x7f4f5000b040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:52.807+0000 7f4f56ffd700 1 -- 172.21.15.7:0/2311187439 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f4f5c05be00 con 0x7f4f600fb4c0 2024-03-16T09:04:53.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:53.031+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f6010efb0 con 0x7f4f4c06d7e0 2024-03-16T09:04:54.144 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:53 smithi007 ceph-mon[29470]: pgmap v74: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:54.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:53 smithi019 ceph-mon[36407]: pgmap v74: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:55.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:54 smithi019 ceph-mon[36407]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:55.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:54 smithi007 ceph-mon[29470]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:04:55.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.351+0000 7f4f56ffd700 1 -- 172.21.15.7:0/2311187439 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f4f6010efb0 con 0x7f4f4c06d7e0 2024-03-16T09:04:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f4f4c06d7e0 msgr2=0x7f4f4c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f4f4c06d7e0 0x7f4f4c06fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f6010ecb0 tx=0x7f4f5000b040 comp rx=0 tx=0).stop 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f600fb4c0 msgr2=0x7f4f60113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f600fb4c0 0x7f4f60113960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00ada0 tx=0x7f4f5c00af40 comp rx=0 tx=0).stop 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 shutdown_connections 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f4f4c06d7e0 0x7f4f4c06fc90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f600fb4c0 0x7f4f60113960 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 --2- 172.21.15.7:0/2311187439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f60101520 0x7f4f60113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 >> 172.21.15.7:0/2311187439 conn(0x7f4f60072310 msgr2=0x7f4f600fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 shutdown_connections 2024-03-16T09:04:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:55.354+0000 7f4f683c7700 1 -- 172.21.15.7:0/2311187439 wait complete. 2024-03-16T09:04:55.369 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:04:56.047 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:55 smithi007 ceph-mon[29470]: pgmap v75: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:56.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch ps' 2024-03-16T09:04:56.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:55 smithi019 ceph-mon[36407]: pgmap v75: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:56.346 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:04:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:56 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:04:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:56 smithi019 ceph-mon[36407]: pgmap v76: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:56 smithi007 ceph-mon[29470]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:04:57.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:56 smithi007 ceph-mon[29470]: pgmap v76: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:04:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1356735743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 msgr2=0x7f6e380fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 --2- 172.21.15.7:0/1356735743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 0x7f6e380fe1f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e34004940 tx=0x7f6e34009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1356735743 shutdown_connections 2024-03-16T09:04:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 --2- 172.21.15.7:0/1356735743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e380fe7c0 0x7f6e38106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:58.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 --2- 172.21.15.7:0/1356735743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 0x7f6e380fe1f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1356735743 >> 172.21.15.7:0/1356735743 conn(0x7f6e380f7870 msgr2=0x7f6e380f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1356735743 shutdown_connections 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1356735743 wait complete. 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.051+0000 7f6e3fadd700 1 Processor -- start 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3fadd700 1 -- start start 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3fadd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 0x7f6e380fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3fadd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e380fe7c0 0x7f6e380fc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3fadd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380fdc30 con 0x7f6e380fde20 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3fadd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380fc6d0 con 0x7f6e380fe7c0 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3d879700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 0x7f6e380fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3d879700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 0x7f6e380fbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38566/0 (socket says 172.21.15.7:38566) 2024-03-16T09:04:58.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3d879700 1 -- 172.21.15.7:0/1354895397 learned_addr learned my addr 172.21.15.7:0/1354895397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:04:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3d078700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e380fe7c0 0x7f6e380fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3d879700 1 -- 172.21.15.7:0/1354895397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e380fe7c0 msgr2=0x7f6e380fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3d879700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e380fe7c0 0x7f6e380fc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3d879700 1 -- 172.21.15.7:0/1354895397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e34005040 con 0x7f6e380fde20 2024-03-16T09:04:58.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.052+0000 7f6e3d879700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 0x7f6e380fbc50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e34004890 tx=0x7f6e34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.053+0000 7f6e2e7fc700 1 -- 172.21.15.7:0/1354895397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3400e040 con 0x7f6e380fde20 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.053+0000 7f6e2e7fc700 1 -- 172.21.15.7:0/1354895397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e34009af0 con 0x7f6e380fde20 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.053+0000 7f6e2e7fc700 1 -- 172.21.15.7:0/1354895397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3401c630 con 0x7f6e380fde20 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.053+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e380fc950 con 0x7f6e380fde20 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.053+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3811a0f0 con 0x7f6e380fde20 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.054+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e38066b50 con 0x7f6e380fde20 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.054+0000 7f6e2e7fc700 1 -- 172.21.15.7:0/1354895397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f6e3400bbc0 con 0x7f6e380fde20 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.055+0000 7f6e2e7fc700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6e2406d7e0 0x7f6e2406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.055+0000 7f6e2e7fc700 1 -- 172.21.15.7:0/1354895397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6e3408dfc0 con 0x7f6e380fde20 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.055+0000 7f6e3d078700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6e2406d7e0 0x7f6e2406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:04:58.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.055+0000 7f6e3d078700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6e2406d7e0 0x7f6e2406fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006fd0 tx=0x7f6e28008040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:04:58.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.059+0000 7f6e2e7fc700 1 -- 172.21.15.7:0/1354895397 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f6e34057df0 con 0x7f6e380fde20 2024-03-16T09:04:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.271+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e3805e400 con 0x7f6e2406d7e0 2024-03-16T09:04:58.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.281+0000 7f6e2e7fc700 1 -- 172.21.15.7:0/1354895397 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6e3805e400 con 0x7f6e2406d7e0 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (67s) 44s ago 11m 14.4M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (12m) 44s ago 12m 6669k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (11m) 100s ago 11m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (50s) 44s ago 11m 57.4M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (6m) 44s ago 6m 12.8M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (6m) 44s ago 6m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (6m) 100s ago 6m 121M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (6m) 100s ago 6m 13.1M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (2m) 44s ago 13m 500M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (2m) 100s ago 10m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:04:58.284 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (13m) 44s ago 13m 53.5M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (10m) 100s ago 10m 43.7M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (107s) 44s ago 11m 7860k - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (103s) 100s ago 10m 3565k - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 44s ago 10m 205M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (9m) 44s ago 9m 235M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (9m) 44s ago 9m 181M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (8m) 100s ago 8m 262M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (8m) 100s ago 8m 244M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (7m) 100s ago 7m 221M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:04:58.285 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (85s) 44s ago 11m 51.2M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:04:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6e2406d7e0 msgr2=0x7f6e2406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6e2406d7e0 0x7f6e2406fc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006fd0 tx=0x7f6e28008040 comp rx=0 tx=0).stop 2024-03-16T09:04:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 msgr2=0x7f6e380fbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:04:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 0x7f6e380fbc50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e34004890 tx=0x7f6e34009e30 comp rx=0 tx=0).stop 2024-03-16T09:04:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 shutdown_connections 2024-03-16T09:04:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6e2406d7e0 0x7f6e2406fc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e380fde20 0x7f6e380fbc50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 --2- 172.21.15.7:0/1354895397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e380fe7c0 0x7f6e380fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:04:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 >> 172.21.15.7:0/1354895397 conn(0x7f6e380f7870 msgr2=0x7f6e380ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:04:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 shutdown_connections 2024-03-16T09:04:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:04:58.284+0000 7f6e3fadd700 1 -- 172.21.15.7:0/1354895397 wait complete. 2024-03-16T09:04:58.907 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-16T09:04:58.908 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T09:04:58.908 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-16T09:04:59.177 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:05:00.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:59 smithi007 ceph-mon[29470]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:00.086 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:04:59 smithi007 ceph-mon[29470]: pgmap v77: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:00.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:59 smithi019 ceph-mon[36407]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:04:59 smithi019 ceph-mon[36407]: pgmap v77: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 -- 172.21.15.7:0/2946319609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc102710 msgr2=0x7f8fbc0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 --2- 172.21.15.7:0/2946319609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc102710 0x7f8fbc0fbd80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0004940 tx=0x7f8fb0009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 -- 172.21.15.7:0/2946319609 shutdown_connections 2024-03-16T09:05:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 --2- 172.21.15.7:0/2946319609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc102710 0x7f8fbc0fbd80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 --2- 172.21.15.7:0/2946319609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc101d70 0x7f8fbc102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 -- 172.21.15.7:0/2946319609 >> 172.21.15.7:0/2946319609 conn(0x7f8fbc072bc0 msgr2=0x7f8fbc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 -- 172.21.15.7:0/2946319609 shutdown_connections 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 -- 172.21.15.7:0/2946319609 wait complete. 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 Processor -- start 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.884+0000 7f8fc40ad700 1 -- start start 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc40ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc101d70 0x7f8fbc0fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc40ad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc102710 0x7f8fbc0fdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc40ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0fe4c0 con 0x7f8fbc101d70 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc40ad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0fe600 con 0x7f8fbc102710 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc1e49700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc101d70 0x7f8fbc0fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc1e49700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc101d70 0x7f8fbc0fd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38580/0 (socket says 172.21.15.7:38580) 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc1e49700 1 -- 172.21.15.7:0/817601691 learned_addr learned my addr 172.21.15.7:0/817601691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc1648700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc102710 0x7f8fbc0fdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc1e49700 1 -- 172.21.15.7:0/817601691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc102710 msgr2=0x7f8fbc0fdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc1e49700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc102710 0x7f8fbc0fdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc1e49700 1 -- 172.21.15.7:0/817601691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb0005040 con 0x7f8fbc101d70 2024-03-16T09:05:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fc1e49700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc101d70 0x7f8fbc0fd9f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800eb40 tx=0x7f8fb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.885+0000 7f8fae7fc700 1 -- 172.21.15.7:0/817601691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fb8007cc0 con 0x7f8fbc101d70 2024-03-16T09:05:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.886+0000 7f8fae7fc700 1 -- 172.21.15.7:0/817601691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fb800f040 con 0x7f8fbc101d70 2024-03-16T09:05:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.886+0000 7f8fae7fc700 1 -- 172.21.15.7:0/817601691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fb800ca10 con 0x7f8fbc101d70 2024-03-16T09:05:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.886+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc0fe850 con 0x7f8fbc101d70 2024-03-16T09:05:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.886+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc11a150 con 0x7f8fbc101d70 2024-03-16T09:05:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.887+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc0fd4c0 con 0x7f8fbc101d70 2024-03-16T09:05:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.888+0000 7f8fae7fc700 1 -- 172.21.15.7:0/817601691 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f8fb800cc80 con 0x7f8fbc101d70 2024-03-16T09:05:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.888+0000 7f8fae7fc700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f8fa806d7e0 0x7f8fa806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.888+0000 7f8fae7fc700 1 -- 172.21.15.7:0/817601691 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8fb808cb70 con 0x7f8fbc101d70 2024-03-16T09:05:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.888+0000 7f8fc1648700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f8fa806d7e0 0x7f8fa806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.889+0000 7f8fc1648700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f8fa806d7e0 0x7f8fa806fc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b0d0 tx=0x7f8fb0009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:00.892+0000 7f8fae7fc700 1 -- 172.21.15.7:0/817601691 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f8fb80569b0 con 0x7f8fbc101d70 2024-03-16T09:05:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.097+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f8fbc05f5f0 con 0x7f8fbc101d70 2024-03-16T09:05:01.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.101+0000 7f8fae7fc700 1 -- 172.21.15.7:0/817601691 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v47)=0 v47) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8fb805a070 con 0x7f8fbc101d70 2024-03-16T09:05:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f8fa806d7e0 msgr2=0x7f8fa806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f8fa806d7e0 0x7f8fa806fc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b0d0 tx=0x7f8fb0009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc101d70 msgr2=0x7f8fbc0fd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc101d70 0x7f8fbc0fd9f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800eb40 tx=0x7f8fb800b040 comp rx=0 tx=0).stop 2024-03-16T09:05:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 shutdown_connections 2024-03-16T09:05:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f8fa806d7e0 0x7f8fa806fc90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc101d70 0x7f8fbc0fd9f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 --2- 172.21.15.7:0/817601691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc102710 0x7f8fbc0fdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 >> 172.21.15.7:0/817601691 conn(0x7f8fbc072bc0 msgr2=0x7f8fbc0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 shutdown_connections 2024-03-16T09:05:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:01.104+0000 7f8fc40ad700 1 -- 172.21.15.7:0/817601691 wait complete. 2024-03-16T09:05:01.735 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-16T09:05:01.735 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T09:05:01.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-16T09:05:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:01 smithi007 ceph-mon[29470]: pgmap v78: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:05:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:01 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/817601691' entity='client.admin' 2024-03-16T09:05:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:05:01.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:01 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:05:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:01 smithi019 ceph-mon[36407]: pgmap v78: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:05:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:01 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/817601691' entity='client.admin' 2024-03-16T09:05:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:05:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:01 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.608+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/199017894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 msgr2=0x7f5cc40fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.608+0000 7f5cc8ff8700 1 --2- 172.21.15.7:0/199017894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 0x7f5cc40fc050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc004940 tx=0x7f5cbc009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.608+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/199017894 shutdown_connections 2024-03-16T09:05:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.608+0000 7f5cc8ff8700 1 --2- 172.21.15.7:0/199017894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc40fc590 0x7f5cc41092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.608+0000 7f5cc8ff8700 1 --2- 172.21.15.7:0/199017894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 0x7f5cc40fc050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.608+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/199017894 >> 172.21.15.7:0/199017894 conn(0x7f5cc4072bc0 msgr2=0x7f5cc4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.608+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/199017894 shutdown_connections 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.608+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/199017894 wait complete. 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc8ff8700 1 Processor -- start 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc8ff8700 1 -- start start 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc8ff8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 0x7f5cc4115e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc8ff8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc40fc590 0x7f5cc4116390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc8ff8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc41169e0 con 0x7f5cc40fbc80 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc8ff8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc410fdb0 con 0x7f5cc40fc590 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 0x7f5cc4115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:03.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 0x7f5cc4115e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51574/0 (socket says 172.21.15.7:51574) 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc259c700 1 -- 172.21.15.7:0/3546850586 learned_addr learned my addr 172.21.15.7:0/3546850586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc1d9b700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc40fc590 0x7f5cc4116390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc259c700 1 -- 172.21.15.7:0/3546850586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc40fc590 msgr2=0x7f5cc4116390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc259c700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc40fc590 0x7f5cc4116390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc259c700 1 -- 172.21.15.7:0/3546850586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc005040 con 0x7f5cc40fbc80 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cc259c700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 0x7f5cc4115e50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00b8b0 tx=0x7f5cbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.609+0000 7f5cbaffd700 1 -- 172.21.15.7:0/3546850586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cbc00e040 con 0x7f5cc40fbc80 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.610+0000 7f5cbaffd700 1 -- 172.21.15.7:0/3546850586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cbc004290 con 0x7f5cc40fbc80 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.610+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc4110030 con 0x7f5cc40fbc80 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.610+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc4110570 con 0x7f5cc40fbc80 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.610+0000 7f5cbaffd700 1 -- 172.21.15.7:0/3546850586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cbc01c3f0 con 0x7f5cc40fbc80 2024-03-16T09:05:03.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.610+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4103c90 con 0x7f5cc40fbc80 2024-03-16T09:05:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.611+0000 7f5cbaffd700 1 -- 172.21.15.7:0/3546850586 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f5cbc0137b0 con 0x7f5cc40fbc80 2024-03-16T09:05:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.611+0000 7f5cbaffd700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f5cb006d7e0 0x7f5cb006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.611+0000 7f5cbaffd700 1 -- 172.21.15.7:0/3546850586 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5cbc08e120 con 0x7f5cc40fbc80 2024-03-16T09:05:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.612+0000 7f5cc1d9b700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f5cb006d7e0 0x7f5cb006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.612+0000 7f5cc1d9b700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f5cb006d7e0 0x7f5cb006fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5cac007fd0 tx=0x7f5cac009040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.616+0000 7f5cbaffd700 1 -- 172.21.15.7:0/3546850586 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f5cbc05b9e0 con 0x7f5cc40fbc80 2024-03-16T09:05:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.819+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f5cc4065e30 con 0x7f5cc40fbc80 2024-03-16T09:05:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.819+0000 7f5cbaffd700 1 -- 172.21.15.7:0/3546850586 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v47)=0 v47) v1 ==== 155+0+0 (secure 0 0 0) 0x7f5cc4065e30 con 0x7f5cc40fbc80 2024-03-16T09:05:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f5cb006d7e0 msgr2=0x7f5cb006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f5cb006d7e0 0x7f5cb006fc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5cac007fd0 tx=0x7f5cac009040 comp rx=0 tx=0).stop 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 msgr2=0x7f5cc4115e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 0x7f5cc4115e50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00b8b0 tx=0x7f5cbc009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 shutdown_connections 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f5cb006d7e0 0x7f5cb006fc90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cc40fbc80 0x7f5cc4115e50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 --2- 172.21.15.7:0/3546850586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc40fc590 0x7f5cc4116390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 >> 172.21.15.7:0/3546850586 conn(0x7f5cc4072bc0 msgr2=0x7f5cc40fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 shutdown_connections 2024-03-16T09:05:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:03.821+0000 7f5cc8ff8700 1 -- 172.21.15.7:0/3546850586 wait complete. 2024-03-16T09:05:04.124 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:03 smithi007 ceph-mon[29470]: pgmap v79: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:04.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:03 smithi019 ceph-mon[36407]: pgmap v79: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:04.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-16T09:05:04.713 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:05:06.109 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:05 smithi007 ceph-mon[29470]: pgmap v80: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:06.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:05 smithi019 ceph-mon[36407]: pgmap v80: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:06.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.371+0000 7efc06c80700 1 -- 172.21.15.7:0/2308513239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 msgr2=0x7efc000fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.371+0000 7efc06c80700 1 --2- 172.21.15.7:0/2308513239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 0x7efc000fbd40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efbec004940 tx=0x7efbec009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.371+0000 7efc06c80700 1 -- 172.21.15.7:0/2308513239 shutdown_connections 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.371+0000 7efc06c80700 1 --2- 172.21.15.7:0/2308513239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 0x7efc000fbd40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.371+0000 7efc06c80700 1 --2- 172.21.15.7:0/2308513239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc00101d30 0x7efc00102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.371+0000 7efc06c80700 1 -- 172.21.15.7:0/2308513239 >> 172.21.15.7:0/2308513239 conn(0x7efc000f78c0 msgr2=0x7efc000f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.371+0000 7efc06c80700 1 -- 172.21.15.7:0/2308513239 shutdown_connections 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.371+0000 7efc06c80700 1 -- 172.21.15.7:0/2308513239 wait complete. 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efc06c80700 1 Processor -- start 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efc06c80700 1 -- start start 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efc06c80700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc00101d30 0x7efc00113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efc06c80700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 0x7efc00113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efc06c80700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc001145b0 con 0x7efc001026d0 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efc06c80700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0010db20 con 0x7efc00101d30 2024-03-16T09:05:06.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efbfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 0x7efc00113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efbfffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 0x7efc00113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51596/0 (socket says 172.21.15.7:51596) 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.372+0000 7efbfffff700 1 -- 172.21.15.7:0/359961721 learned_addr learned my addr 172.21.15.7:0/359961721 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efc04a1c700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc00101d30 0x7efc00113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efbfffff700 1 -- 172.21.15.7:0/359961721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc00101d30 msgr2=0x7efc00113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efbfffff700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc00101d30 0x7efc00113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efbfffff700 1 -- 172.21.15.7:0/359961721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec005040 con 0x7efc001026d0 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efbfffff700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 0x7efc00113ed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efbec009100 tx=0x7efbec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efbfd7fa700 1 -- 172.21.15.7:0/359961721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbec00e040 con 0x7efc001026d0 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efbfd7fa700 1 -- 172.21.15.7:0/359961721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbec009af0 con 0x7efc001026d0 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efbfd7fa700 1 -- 172.21.15.7:0/359961721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbec01c3f0 con 0x7efc001026d0 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0010dda0 con 0x7efc001026d0 2024-03-16T09:05:06.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.373+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0010e360 con 0x7efc001026d0 2024-03-16T09:05:06.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.374+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc000fd480 con 0x7efc001026d0 2024-03-16T09:05:06.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.375+0000 7efbfd7fa700 1 -- 172.21.15.7:0/359961721 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7efbec00baf0 con 0x7efc001026d0 2024-03-16T09:05:06.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.375+0000 7efbfd7fa700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7efbf006d7e0 0x7efbf006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:06.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.375+0000 7efbfd7fa700 1 -- 172.21.15.7:0/359961721 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7efbec08df40 con 0x7efc001026d0 2024-03-16T09:05:06.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.375+0000 7efc04a1c700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7efbf006d7e0 0x7efbf006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:06.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.376+0000 7efc04a1c700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7efbf006d7e0 0x7efbf006fc90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009b60 tx=0x7efbf40091a0 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:06.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.380+0000 7efbfd7fa700 1 -- 172.21.15.7:0/359961721 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7efbec0c2040 con 0x7efc001026d0 2024-03-16T09:05:06.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.585+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7efc0010df30 con 0x7efc001026d0 2024-03-16T09:05:06.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.585+0000 7efbfd7fa700 1 -- 172.21.15.7:0/359961721 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v47)=0 v47) v1 ==== 163+0+0 (secure 0 0 0) 0x7efc0010df30 con 0x7efc001026d0 2024-03-16T09:05:06.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7efbf006d7e0 msgr2=0x7efbf006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:06.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7efbf006d7e0 0x7efbf006fc90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009b60 tx=0x7efbf40091a0 comp rx=0 tx=0).stop 2024-03-16T09:05:06.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 msgr2=0x7efc00113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:06.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 0x7efc00113ed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efbec009100 tx=0x7efbec009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 shutdown_connections 2024-03-16T09:05:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7efbf006d7e0 0x7efbf006fc90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc00101d30 0x7efc00113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 --2- 172.21.15.7:0/359961721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc001026d0 0x7efc00113ed0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 >> 172.21.15.7:0/359961721 conn(0x7efc000f78c0 msgr2=0x7efc000f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 shutdown_connections 2024-03-16T09:05:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:06.587+0000 7efc06c80700 1 -- 172.21.15.7:0/359961721 wait complete. 2024-03-16T09:05:07.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-16T09:05:07.474 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:05:07.909 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:07 smithi007 ceph-mon[29470]: pgmap v81: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:07.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:07 smithi019 ceph-mon[36407]: pgmap v81: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.084+0000 7f0666194700 1 -- 172.21.15.7:0/363583340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 msgr2=0x7f0660106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.084+0000 7f0666194700 1 --2- 172.21.15.7:0/363583340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 0x7f0660106f00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0648004940 tx=0x7f0648009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 -- 172.21.15.7:0/363583340 shutdown_connections 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 --2- 172.21.15.7:0/363583340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 0x7f0660106f00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 --2- 172.21.15.7:0/363583340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06600fde90 0x7f06600fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 -- 172.21.15.7:0/363583340 >> 172.21.15.7:0/363583340 conn(0x7f0660072bc0 msgr2=0x7f0660072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 -- 172.21.15.7:0/363583340 shutdown_connections 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 -- 172.21.15.7:0/363583340 wait complete. 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 Processor -- start 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 -- start start 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06600fde90 0x7f0660113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 0x7f06601140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0660114730 con 0x7f06600fe830 2024-03-16T09:05:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.085+0000 7f0666194700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066010db20 con 0x7f06600fde90 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f065effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 0x7f06601140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f065effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 0x7f06601140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51616/0 (socket says 172.21.15.7:51616) 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f065effd700 1 -- 172.21.15.7:0/2724897927 learned_addr learned my addr 172.21.15.7:0/2724897927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f065f7fe700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06600fde90 0x7f0660113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f065effd700 1 -- 172.21.15.7:0/2724897927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06600fde90 msgr2=0x7f0660113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f065effd700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06600fde90 0x7f0660113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f065effd700 1 -- 172.21.15.7:0/2724897927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648005040 con 0x7f06600fe830 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f065effd700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 0x7f06601140e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f064800b3b0 tx=0x7f0648009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f0657fff700 1 -- 172.21.15.7:0/2724897927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064800e040 con 0x7f06600fe830 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f0657fff700 1 -- 172.21.15.7:0/2724897927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0648013420 con 0x7f06600fe830 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066010dda0 con 0x7f06600fe830 2024-03-16T09:05:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f0657fff700 1 -- 172.21.15.7:0/2724897927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064801c5f0 con 0x7f06600fe830 2024-03-16T09:05:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.086+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066010e2e0 con 0x7f06600fe830 2024-03-16T09:05:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.087+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0660065e30 con 0x7f06600fe830 2024-03-16T09:05:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.088+0000 7f0657fff700 1 -- 172.21.15.7:0/2724897927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f0648013590 con 0x7f06600fe830 2024-03-16T09:05:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.088+0000 7f0657fff700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f064c06d7e0 0x7f064c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.088+0000 7f0657fff700 1 -- 172.21.15.7:0/2724897927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f064808df80 con 0x7f06600fe830 2024-03-16T09:05:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.088+0000 7f065f7fe700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f064c06d7e0 0x7f064c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.089+0000 7f065f7fe700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f064c06d7e0 0x7f064c06fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0650007a40 tx=0x7f0650009da0 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.093+0000 7f0657fff700 1 -- 172.21.15.7:0/2724897927 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f064805b840 con 0x7f06600fe830 2024-03-16T09:05:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.296+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f0660109f10 con 0x7f06600fe830 2024-03-16T09:05:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.296+0000 7f0657fff700 1 -- 172.21.15.7:0/2724897927 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v47)=0 v47) v1 ==== 135+0+0 (secure 0 0 0) 0x7f064805b3d0 con 0x7f06600fe830 2024-03-16T09:05:09.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f064c06d7e0 msgr2=0x7f064c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f064c06d7e0 0x7f064c06fc90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0650007a40 tx=0x7f0650009da0 comp rx=0 tx=0).stop 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 msgr2=0x7f06601140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 0x7f06601140e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f064800b3b0 tx=0x7f0648009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 shutdown_connections 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f064c06d7e0 0x7f064c06fc90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06600fde90 0x7f0660113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 --2- 172.21.15.7:0/2724897927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06600fe830 0x7f06601140e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 >> 172.21.15.7:0/2724897927 conn(0x7f0660072bc0 msgr2=0x7f06601037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 shutdown_connections 2024-03-16T09:05:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:09.298+0000 7f0666194700 1 -- 172.21.15.7:0/2724897927 wait complete. 2024-03-16T09:05:09.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-16T09:05:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:05:10.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:09 smithi019 ceph-mon[36407]: pgmap v82: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:10.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:09 smithi007 ceph-mon[29470]: pgmap v82: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:11 smithi007 ceph-mon[29470]: pgmap v83: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 -- 172.21.15.7:0/333585294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fe7b0 msgr2=0x7fbbac106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 --2- 172.21.15.7:0/333585294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fe7b0 0x7fbbac106e80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbba0004940 tx=0x7fbba0009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 -- 172.21.15.7:0/333585294 shutdown_connections 2024-03-16T09:05:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 --2- 172.21.15.7:0/333585294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fe7b0 0x7fbbac106e80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 --2- 172.21.15.7:0/333585294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac0fde10 0x7fbbac0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 -- 172.21.15.7:0/333585294 >> 172.21.15.7:0/333585294 conn(0x7fbbac0f7870 msgr2=0x7fbbac0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 -- 172.21.15.7:0/333585294 shutdown_connections 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 -- 172.21.15.7:0/333585294 wait complete. 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 Processor -- start 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 -- start start 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fde10 0x7fbbac113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac0fe7b0 0x7fbbac113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac114520 con 0x7fbbac0fde10 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb4335700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac10db20 con 0x7fbbac0fe7b0 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb20d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fde10 0x7fbbac113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.024+0000 7fbbb20d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fde10 0x7fbbac113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51634/0 (socket says 172.21.15.7:51634) 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbb20d1700 1 -- 172.21.15.7:0/4238961735 learned_addr learned my addr 172.21.15.7:0/4238961735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbb18d0700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac0fe7b0 0x7fbbac113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbb20d1700 1 -- 172.21.15.7:0/4238961735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac0fe7b0 msgr2=0x7fbbac113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbb20d1700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac0fe7b0 0x7fbbac113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbb20d1700 1 -- 172.21.15.7:0/4238961735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0005040 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbb20d1700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fde10 0x7fbbac113990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c001d80 tx=0x7fbb9c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbaaffd700 1 -- 172.21.15.7:0/4238961735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb9c007d10 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbaaffd700 1 -- 172.21.15.7:0/4238961735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb9c007e70 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac10de00 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbaaffd700 1 -- 172.21.15.7:0/4238961735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb9c014640 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.025+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac10e340 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.026+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac066af0 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.027+0000 7fbbaaffd700 1 -- 172.21.15.7:0/4238961735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7fbb9c00e960 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.027+0000 7fbbaaffd700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbb9406d7e0 0x7fbb9406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.027+0000 7fbbb18d0700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbb9406d7e0 0x7fbb9406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.027+0000 7fbbaaffd700 1 -- 172.21.15.7:0/4238961735 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbb9c08cbf0 con 0x7fbbac0fde10 2024-03-16T09:05:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.028+0000 7fbbb18d0700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbb9406d7e0 0x7fbb9406fc90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbba000b3b0 tx=0x7fbba0009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.032+0000 7fbbaaffd700 1 -- 172.21.15.7:0/4238961735 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fbb9c05a4e0 con 0x7fbbac0fde10 2024-03-16T09:05:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:11 smithi019 ceph-mon[36407]: pgmap v83: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.256+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbac109ea0 con 0x7fbb9406d7e0 2024-03-16T09:05:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.261+0000 7fbbaaffd700 1 -- 172.21.15.7:0/4238961735 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fbbac109ea0 con 0x7fbb9406d7e0 2024-03-16T09:05:12.263 INFO:teuthology.orchestra.run.smithi007.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:05:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbb9406d7e0 msgr2=0x7fbb9406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbb9406d7e0 0x7fbb9406fc90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbba000b3b0 tx=0x7fbba0009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fde10 msgr2=0x7fbbac113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fde10 0x7fbbac113990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c001d80 tx=0x7fbb9c009ec0 comp rx=0 tx=0).stop 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 shutdown_connections 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbb9406d7e0 0x7fbb9406fc90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbac0fde10 0x7fbbac113990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 --2- 172.21.15.7:0/4238961735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbac0fe7b0 0x7fbbac113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 >> 172.21.15.7:0/4238961735 conn(0x7fbbac0f7870 msgr2=0x7fbbac0f8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 shutdown_connections 2024-03-16T09:05:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:12.263+0000 7fbbb4335700 1 -- 172.21.15.7:0/4238961735 wait complete. 2024-03-16T09:05:12.931 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-16T09:05:12.932 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T09:05:12.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-16T09:05:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:05:13.548 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:13.549 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:05:13.549 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:13.549 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:13.549 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:13.549 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:05:13.549 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:13.549 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:05:13.549 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:13 smithi007 ceph-mon[29470]: pgmap v84: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:05:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:13 smithi019 ceph-mon[36407]: pgmap v84: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:15.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 -- 172.21.15.7:0/2903595113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 msgr2=0x7f468c1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:15.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 --2- 172.21.15.7:0/2903595113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 0x7f468c1020f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4688004940 tx=0x7f4688009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:15.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 -- 172.21.15.7:0/2903595113 shutdown_connections 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 --2- 172.21.15.7:0/2903595113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468c1026c0 0x7f468c0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 --2- 172.21.15.7:0/2903595113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 0x7f468c1020f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 -- 172.21.15.7:0/2903595113 >> 172.21.15.7:0/2903595113 conn(0x7f468c0f7870 msgr2=0x7f468c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 -- 172.21.15.7:0/2903595113 shutdown_connections 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 -- 172.21.15.7:0/2903595113 wait complete. 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 Processor -- start 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.826+0000 7f4694524700 1 -- start start 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f4694524700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 0x7f468c1128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f4694524700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468c1026c0 0x7f468c112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f4694524700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c113460 con 0x7f468c101d20 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f4694524700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c10ca60 con 0x7f468c1026c0 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f46922c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 0x7f468c1128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f46922c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 0x7f468c1128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47162/0 (socket says 172.21.15.7:47162) 2024-03-16T09:05:15.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f46922c0700 1 -- 172.21.15.7:0/3693418240 learned_addr learned my addr 172.21.15.7:0/3693418240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:15.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f4691abf700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468c1026c0 0x7f468c112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:15.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f46922c0700 1 -- 172.21.15.7:0/3693418240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468c1026c0 msgr2=0x7f468c112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:15.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f46922c0700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468c1026c0 0x7f468c112e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:15.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f46922c0700 1 -- 172.21.15.7:0/3693418240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688005040 con 0x7f468c101d20 2024-03-16T09:05:15.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f46922c0700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 0x7f468c1128d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4688009180 tx=0x7f4688009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.827+0000 7f467effd700 1 -- 172.21.15.7:0/3693418240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f468800e040 con 0x7f468c101d20 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.828+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c10cce0 con 0x7f468c101d20 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.828+0000 7f467effd700 1 -- 172.21.15.7:0/3693418240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f468800bd00 con 0x7f468c101d20 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.828+0000 7f467effd700 1 -- 172.21.15.7:0/3693418240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4688013660 con 0x7f468c101d20 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.828+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c10d240 con 0x7f468c101d20 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.829+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468c0fd470 con 0x7f468c101d20 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.829+0000 7f467effd700 1 -- 172.21.15.7:0/3693418240 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f468801e9f0 con 0x7f468c101d20 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.830+0000 7f467effd700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f467806d7e0 0x7f467806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.830+0000 7f467effd700 1 -- 172.21.15.7:0/3693418240 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4688090b50 con 0x7f468c101d20 2024-03-16T09:05:15.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.830+0000 7f4691abf700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f467806d7e0 0x7f467806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:15.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.830+0000 7f4691abf700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f467806d7e0 0x7f467806fc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f468c10dc20 tx=0x7f468000b040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:15.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:15.834+0000 7f467effd700 1 -- 172.21.15.7:0/3693418240 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f468805e390 con 0x7f468c101d20 2024-03-16T09:05:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:15 smithi007 ceph-mon[29470]: pgmap v85: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.066+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f468c108e40 con 0x7f467806d7e0 2024-03-16T09:05:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.067+0000 7f467effd700 1 -- 172.21.15.7:0/3693418240 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f468c108e40 con 0x7f467806d7e0 2024-03-16T09:05:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f467806d7e0 msgr2=0x7f467806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f467806d7e0 0x7f467806fc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f468c10dc20 tx=0x7f468000b040 comp rx=0 tx=0).stop 2024-03-16T09:05:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 msgr2=0x7f468c1128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 0x7f468c1128d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4688009180 tx=0x7f4688009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 shutdown_connections 2024-03-16T09:05:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f467806d7e0 0x7f467806fc90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c101d20 0x7f468c1128d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 --2- 172.21.15.7:0/3693418240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468c1026c0 0x7f468c112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 >> 172.21.15.7:0/3693418240 conn(0x7f468c0f7870 msgr2=0x7f468c0f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:16.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 shutdown_connections 2024-03-16T09:05:16.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.069+0000 7f4694524700 1 -- 172.21.15.7:0/3693418240 wait complete. 2024-03-16T09:05:16.081 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:05:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.172+0000 7fbff51c5700 1 -- 172.21.15.7:0/1201563048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff0101d20 msgr2=0x7fbff01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.172+0000 7fbff51c5700 1 --2- 172.21.15.7:0/1201563048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff0101d20 0x7fbff01020f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0004940 tx=0x7fbfe0009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.172+0000 7fbff51c5700 1 -- 172.21.15.7:0/1201563048 shutdown_connections 2024-03-16T09:05:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.172+0000 7fbff51c5700 1 --2- 172.21.15.7:0/1201563048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff01026c0 0x7fbff00fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.172+0000 7fbff51c5700 1 --2- 172.21.15.7:0/1201563048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff0101d20 0x7fbff01020f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.172+0000 7fbff51c5700 1 -- 172.21.15.7:0/1201563048 >> 172.21.15.7:0/1201563048 conn(0x7fbff00f78b0 msgr2=0x7fbff00f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.172+0000 7fbff51c5700 1 -- 172.21.15.7:0/1201563048 shutdown_connections 2024-03-16T09:05:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.172+0000 7fbff51c5700 1 -- 172.21.15.7:0/1201563048 wait complete. 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbff51c5700 1 Processor -- start 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbff51c5700 1 -- start start 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbff51c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0101d20 0x7fbff0102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbff51c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff01026c0 0x7fbff01030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbff51c5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff0104ad0 con 0x7fbff01026c0 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbff51c5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff0103600 con 0x7fbff0101d20 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbfee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff01026c0 0x7fbff01030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbfee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff01026c0 0x7fbff01030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47178/0 (socket says 172.21.15.7:47178) 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.173+0000 7fbfee59c700 1 -- 172.21.15.7:0/3624107029 learned_addr learned my addr 172.21.15.7:0/3624107029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbfeed9d700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0101d20 0x7fbff0102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbfee59c700 1 -- 172.21.15.7:0/3624107029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0101d20 msgr2=0x7fbff0102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbfee59c700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0101d20 0x7fbff0102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbfee59c700 1 -- 172.21.15.7:0/3624107029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0005040 con 0x7fbff01026c0 2024-03-16T09:05:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbfee59c700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff01026c0 0x7fbff01030c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4002a50 tx=0x7fbfe4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbfd77fe700 1 -- 172.21.15.7:0/3624107029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe4007cc0 con 0x7fbff01026c0 2024-03-16T09:05:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbfd77fe700 1 -- 172.21.15.7:0/3624107029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfe400f040 con 0x7fbff01026c0 2024-03-16T09:05:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff01038e0 con 0x7fbff01026c0 2024-03-16T09:05:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbfd77fe700 1 -- 172.21.15.7:0/3624107029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfe400da40 con 0x7fbff01026c0 2024-03-16T09:05:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.174+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff011a150 con 0x7fbff01026c0 2024-03-16T09:05:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.175+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff00fd470 con 0x7fbff01026c0 2024-03-16T09:05:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.176+0000 7fbfd77fe700 1 -- 172.21.15.7:0/3624107029 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7fbfe40097c0 con 0x7fbff01026c0 2024-03-16T09:05:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.176+0000 7fbfd77fe700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfd806d7e0 0x7fbfd806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.176+0000 7fbfd77fe700 1 -- 172.21.15.7:0/3624107029 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbfe408caf0 con 0x7fbff01026c0 2024-03-16T09:05:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.176+0000 7fbfeed9d700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfd806d7e0 0x7fbfd806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.176+0000 7fbfeed9d700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfd806d7e0 0x7fbfd806fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0004f90 tx=0x7fbfe0009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.181+0000 7fbfd77fe700 1 -- 172.21.15.7:0/3624107029 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fbfe4056880 con 0x7fbff01026c0 2024-03-16T09:05:16.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:15 smithi019 ceph-mon[36407]: pgmap v85: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:16.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.410+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbff00f9620 con 0x7fbfd806d7e0 2024-03-16T09:05:16.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.412+0000 7fbfd77fe700 1 -- 172.21.15.7:0/3624107029 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbff00f9620 con 0x7fbfd806d7e0 2024-03-16T09:05:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfd806d7e0 msgr2=0x7fbfd806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfd806d7e0 0x7fbfd806fc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0004f90 tx=0x7fbfe0009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff01026c0 msgr2=0x7fbff01030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff01026c0 0x7fbff01030c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4002a50 tx=0x7fbfe4004600 comp rx=0 tx=0).stop 2024-03-16T09:05:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 shutdown_connections 2024-03-16T09:05:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fbfd806d7e0 0x7fbfd806fc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0101d20 0x7fbff0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 --2- 172.21.15.7:0/3624107029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbff01026c0 0x7fbff01030c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 >> 172.21.15.7:0/3624107029 conn(0x7fbff00f78b0 msgr2=0x7fbff00f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 shutdown_connections 2024-03-16T09:05:16.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.415+0000 7fbff51c5700 1 -- 172.21.15.7:0/3624107029 wait complete. 2024-03-16T09:05:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 -- 172.21.15.7:0/1309795127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 msgr2=0x7ff16c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 --2- 172.21.15.7:0/1309795127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 0x7ff16c0fe260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff160004940 tx=0x7ff160009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 -- 172.21.15.7:0/1309795127 shutdown_connections 2024-03-16T09:05:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 --2- 172.21.15.7:0/1309795127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff16c0fe830 0x7ff16c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 --2- 172.21.15.7:0/1309795127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 0x7ff16c0fe260 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 -- 172.21.15.7:0/1309795127 >> 172.21.15.7:0/1309795127 conn(0x7ff16c072bc0 msgr2=0x7ff16c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 -- 172.21.15.7:0/1309795127 shutdown_connections 2024-03-16T09:05:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 -- 172.21.15.7:0/1309795127 wait complete. 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.525+0000 7ff172384700 1 Processor -- start 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff172384700 1 -- start start 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff172384700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 0x7ff16c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff172384700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff16c0fe830 0x7ff16c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff172384700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c114730 con 0x7ff16c0fde90 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff172384700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c10db40 con 0x7ff16c0fe830 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff16bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 0x7ff16c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff16bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 0x7ff16c113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47200/0 (socket says 172.21.15.7:47200) 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff16bfff700 1 -- 172.21.15.7:0/789941024 learned_addr learned my addr 172.21.15.7:0/789941024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff16b7fe700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff16c0fe830 0x7ff16c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff16bfff700 1 -- 172.21.15.7:0/789941024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff16c0fe830 msgr2=0x7ff16c1140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff16bfff700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff16c0fe830 0x7ff16c1140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff16bfff700 1 -- 172.21.15.7:0/789941024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff160005040 con 0x7ff16c0fde90 2024-03-16T09:05:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.526+0000 7ff16bfff700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 0x7ff16c113ba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff16000b3b0 tx=0x7ff160009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.527+0000 7ff168ff9700 1 -- 172.21.15.7:0/789941024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff16000e040 con 0x7ff16c0fde90 2024-03-16T09:05:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.527+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c10ddc0 con 0x7ff16c0fde90 2024-03-16T09:05:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.527+0000 7ff168ff9700 1 -- 172.21.15.7:0/789941024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff160013420 con 0x7ff16c0fde90 2024-03-16T09:05:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.527+0000 7ff168ff9700 1 -- 172.21.15.7:0/789941024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff16001c5f0 con 0x7ff16c0fde90 2024-03-16T09:05:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.527+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c10e300 con 0x7ff16c0fde90 2024-03-16T09:05:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.528+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c065e30 con 0x7ff16c0fde90 2024-03-16T09:05:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.528+0000 7ff168ff9700 1 -- 172.21.15.7:0/789941024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7ff160013590 con 0x7ff16c0fde90 2024-03-16T09:05:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.529+0000 7ff168ff9700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff15406d7e0 0x7ff15406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.529+0000 7ff168ff9700 1 -- 172.21.15.7:0/789941024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff16008df80 con 0x7ff16c0fde90 2024-03-16T09:05:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.529+0000 7ff16b7fe700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff15406d7e0 0x7ff15406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.529+0000 7ff16b7fe700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff15406d7e0 0x7ff15406fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff16c10f440 tx=0x7ff15c0043c0 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.533+0000 7ff168ff9700 1 -- 172.21.15.7:0/789941024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7ff160057d10 con 0x7ff16c0fde90 2024-03-16T09:05:16.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.750+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff16c04bf40 con 0x7ff15406d7e0 2024-03-16T09:05:16.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.761+0000 7ff168ff9700 1 -- 172.21.15.7:0/789941024 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff16c04bf40 con 0x7ff15406d7e0 2024-03-16T09:05:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:05:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (86s) 62s ago 12m 14.4M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:05:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (12m) 62s ago 12m 6669k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:05:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (11m) 118s ago 11m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:05:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (68s) 62s ago 11m 57.4M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:05:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (6m) 62s ago 6m 12.8M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:05:16.763 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (6m) 62s ago 6m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (6m) 118s ago 6m 121M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (6m) 118s ago 6m 13.1M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (3m) 62s ago 13m 500M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (2m) 118s ago 11m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (13m) 62s ago 13m 53.5M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce aaa292e58c25 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (11m) 118s ago 11m 43.7M 2048M 17.2.7-648-g1df27247 0bdcfccf14ce bfd7571d5d35 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (2m) 62s ago 12m 7860k - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (2m) 118s ago 11m 3565k - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 62s ago 10m 205M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (9m) 62s ago 9m 235M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (9m) 62s ago 9m 181M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (9m) 118s ago 9m 262M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (8m) 118s ago 8m 244M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (8m) 118s ago 8m 221M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:05:16.764 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (103s) 62s ago 11m 51.2M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:05:16.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff15406d7e0 msgr2=0x7ff15406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff15406d7e0 0x7ff15406fc90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff16c10f440 tx=0x7ff15c0043c0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 msgr2=0x7ff16c113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 0x7ff16c113ba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff16000b3b0 tx=0x7ff160009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 shutdown_connections 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7ff15406d7e0 0x7ff15406fc90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c0fde90 0x7ff16c113ba0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 --2- 172.21.15.7:0/789941024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff16c0fe830 0x7ff16c1140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 >> 172.21.15.7:0/789941024 conn(0x7ff16c072bc0 msgr2=0x7ff16c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 shutdown_connections 2024-03-16T09:05:16.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.763+0000 7ff172384700 1 -- 172.21.15.7:0/789941024 wait complete. 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 -- 172.21.15.7:0/2752432845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 msgr2=0x7fdd0c1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 --2- 172.21.15.7:0/2752432845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 0x7fdd0c1020f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdd08004940 tx=0x7fdd08009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 -- 172.21.15.7:0/2752432845 shutdown_connections 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 --2- 172.21.15.7:0/2752432845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd0c1026c0 0x7fdd0c0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 --2- 172.21.15.7:0/2752432845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 0x7fdd0c1020f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 -- 172.21.15.7:0/2752432845 >> 172.21.15.7:0/2752432845 conn(0x7fdd0c0f7870 msgr2=0x7fdd0c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 -- 172.21.15.7:0/2752432845 shutdown_connections 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 -- 172.21.15.7:0/2752432845 wait complete. 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 Processor -- start 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.884+0000 7fdd13ca8700 1 -- start start 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd13ca8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 0x7fdd0c113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd13ca8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd0c1026c0 0x7fdd0c113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd13ca8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c1145f0 con 0x7fdd0c101d20 2024-03-16T09:05:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd13ca8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c10dbd0 con 0x7fdd0c1026c0 2024-03-16T09:05:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd11a44700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 0x7fdd0c113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd11a44700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 0x7fdd0c113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47206/0 (socket says 172.21.15.7:47206) 2024-03-16T09:05:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd11a44700 1 -- 172.21.15.7:0/1851342253 learned_addr learned my addr 172.21.15.7:0/1851342253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd11243700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd0c1026c0 0x7fdd0c113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd11a44700 1 -- 172.21.15.7:0/1851342253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd0c1026c0 msgr2=0x7fdd0c113fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd11a44700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd0c1026c0 0x7fdd0c113fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd11a44700 1 -- 172.21.15.7:0/1851342253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08005040 con 0x7fdd0c101d20 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd11a44700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 0x7fdd0c113a60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b3b0 tx=0x7fdd08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd027fc700 1 -- 172.21.15.7:0/1851342253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0800e040 con 0x7fdd0c101d20 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd027fc700 1 -- 172.21.15.7:0/1851342253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd08013420 con 0x7fdd0c101d20 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c10de50 con 0x7fdd0c101d20 2024-03-16T09:05:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.885+0000 7fdd027fc700 1 -- 172.21.15.7:0/1851342253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0801c630 con 0x7fdd0c101d20 2024-03-16T09:05:16.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.886+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c10e390 con 0x7fdd0c101d20 2024-03-16T09:05:16.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.886+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0c0fd470 con 0x7fdd0c101d20 2024-03-16T09:05:16.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.887+0000 7fdd027fc700 1 -- 172.21.15.7:0/1851342253 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7fdd08013590 con 0x7fdd0c101d20 2024-03-16T09:05:16.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.887+0000 7fdd027fc700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fdcf806d7e0 0x7fdcf806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:16.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.888+0000 7fdd11243700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fdcf806d7e0 0x7fdcf806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:16.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.888+0000 7fdd027fc700 1 -- 172.21.15.7:0/1851342253 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fdd0808df50 con 0x7fdd0c101d20 2024-03-16T09:05:16.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.888+0000 7fdd11243700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fdcf806d7e0 0x7fdcf806fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c10f300 tx=0x7fdcfc00b040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:16.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:16.892+0000 7fdd027fc700 1 -- 172.21.15.7:0/1851342253 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fdd0805b830 con 0x7fdd0c101d20 2024-03-16T09:05:17.116 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:05:17.116 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:17.116 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:17.116 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:05:17.116 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:17.116 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:16 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-16T09:05:17.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.195+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdd0c10f0b0 con 0x7fdd0c101d20 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.196+0000 7fdd027fc700 1 -- 172.21.15.7:0/1851342253 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fdd0805b3c0 con 0x7fdd0c101d20 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:05:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:05:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:05:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:05:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:05:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 12, 2024-03-16T09:05:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:05:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:05:17.199 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:05:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fdcf806d7e0 msgr2=0x7fdcf806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fdcf806d7e0 0x7fdcf806fc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c10f300 tx=0x7fdcfc00b040 comp rx=0 tx=0).stop 2024-03-16T09:05:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 msgr2=0x7fdd0c113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 0x7fdd0c113a60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b3b0 tx=0x7fdd08009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 shutdown_connections 2024-03-16T09:05:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fdcf806d7e0 0x7fdcf806fc90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd0c101d20 0x7fdd0c113a60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 --2- 172.21.15.7:0/1851342253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd0c1026c0 0x7fdd0c113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 >> 172.21.15.7:0/1851342253 conn(0x7fdd0c0f7870 msgr2=0x7fdd0c0f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 shutdown_connections 2024-03-16T09:05:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.198+0000 7fdd13ca8700 1 -- 172.21.15.7:0/1851342253 wait complete. 2024-03-16T09:05:17.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:05:17.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:17.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:17.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:05:17.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:17.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:16 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-16T09:05:17.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.302+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/2113811322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4102710 msgr2=0x7f6ec40fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.302+0000 7f6ec8fa5700 1 --2- 172.21.15.7:0/2113811322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4102710 0x7f6ec40fbd80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8004940 tx=0x7f6eb8009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.302+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/2113811322 shutdown_connections 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.302+0000 7f6ec8fa5700 1 --2- 172.21.15.7:0/2113811322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4102710 0x7f6ec40fbd80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.302+0000 7f6ec8fa5700 1 --2- 172.21.15.7:0/2113811322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec4101d70 0x7f6ec4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.302+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/2113811322 >> 172.21.15.7:0/2113811322 conn(0x7f6ec4072bc0 msgr2=0x7f6ec4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.302+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/2113811322 shutdown_connections 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.302+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/2113811322 wait complete. 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec8fa5700 1 Processor -- start 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec8fa5700 1 -- start start 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec8fa5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4101d70 0x7f6ec410c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec8fa5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec4102710 0x7f6ec410ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec8fa5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec410d430 con 0x7f6ec4101d70 2024-03-16T09:05:17.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec8fa5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec410d5a0 con 0x7f6ec4102710 2024-03-16T09:05:17.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4101d70 0x7f6ec410c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:17.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4101d70 0x7f6ec410c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47234/0 (socket says 172.21.15.7:47234) 2024-03-16T09:05:17.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec259c700 1 -- 172.21.15.7:0/994855972 learned_addr learned my addr 172.21.15.7:0/994855972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:17.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.303+0000 7f6ec1d9b700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec4102710 0x7f6ec410ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6ec259c700 1 -- 172.21.15.7:0/994855972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec4102710 msgr2=0x7f6ec410ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6ec259c700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec4102710 0x7f6ec410ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6ec259c700 1 -- 172.21.15.7:0/994855972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb8005040 con 0x7f6ec4101d70 2024-03-16T09:05:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6ec259c700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4101d70 0x7f6ec410c920 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400b790 tx=0x7f6eb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6eb2ffd700 1 -- 172.21.15.7:0/994855972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb400f6f0 con 0x7f6ec4101d70 2024-03-16T09:05:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6eb2ffd700 1 -- 172.21.15.7:0/994855972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eb400fd20 con 0x7f6ec4101d70 2024-03-16T09:05:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec4119f90 con 0x7f6ec4101d70 2024-03-16T09:05:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6eb2ffd700 1 -- 172.21.15.7:0/994855972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb40103a0 con 0x7f6ec4101d70 2024-03-16T09:05:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.304+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec411a360 con 0x7f6ec4101d70 2024-03-16T09:05:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.305+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec40ff080 con 0x7f6ec4101d70 2024-03-16T09:05:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.306+0000 7f6eb2ffd700 1 -- 172.21.15.7:0/994855972 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f6eb4010500 con 0x7f6ec4101d70 2024-03-16T09:05:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.306+0000 7f6eb2ffd700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6eac06d7f0 0x7f6eac06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.307+0000 7f6eb2ffd700 1 -- 172.21.15.7:0/994855972 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6eb408ca50 con 0x7f6ec4101d70 2024-03-16T09:05:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.307+0000 7f6ec1d9b700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6eac06d7f0 0x7f6eac06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.307+0000 7f6ec1d9b700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6eac06d7f0 0x7f6eac06fca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40660b0 tx=0x7f6eb8009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.311+0000 7f6eb2ffd700 1 -- 172.21.15.7:0/994855972 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f6eb405a490 con 0x7f6ec4101d70 2024-03-16T09:05:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.560+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6ec404bed0 con 0x7f6ec4101d70 2024-03-16T09:05:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.561+0000 7f6eb2ffd700 1 -- 172.21.15.7:0/994855972 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1827 (secure 0 0 0) 0x7f6eb405a020 con 0x7f6ec4101d70 2024-03-16T09:05:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:e19 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 19 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:04:55.865707+0000 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:05:17.564 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:05:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:17.566 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6eac06d7f0 msgr2=0x7f6eac06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6eac06d7f0 0x7f6eac06fca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40660b0 tx=0x7f6eb8009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4101d70 msgr2=0x7f6ec410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4101d70 0x7f6ec410c920 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400b790 tx=0x7f6eb4009250 comp rx=0 tx=0).stop 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 shutdown_connections 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f6eac06d7f0 0x7f6eac06fca0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ec4101d70 0x7f6ec410c920 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 --2- 172.21.15.7:0/994855972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ec4102710 0x7f6ec410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 >> 172.21.15.7:0/994855972 conn(0x7f6ec4072bc0 msgr2=0x7f6ec40faf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 shutdown_connections 2024-03-16T09:05:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.563+0000 7f6ec8fa5700 1 -- 172.21.15.7:0/994855972 wait complete. 2024-03-16T09:05:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-03-16T09:05:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 -- 172.21.15.7:0/3591720110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc103740 msgr2=0x7f79dc0fb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 --2- 172.21.15.7:0/3591720110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc103740 0x7f79dc0fb430 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f79d0004940 tx=0x7f79d0009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 -- 172.21.15.7:0/3591720110 shutdown_connections 2024-03-16T09:05:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 --2- 172.21.15.7:0/3591720110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc103740 0x7f79dc0fb430 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 --2- 172.21.15.7:0/3591720110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79dc102da0 0x7f79dc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 -- 172.21.15.7:0/3591720110 >> 172.21.15.7:0/3591720110 conn(0x7f79dc071450 msgr2=0x7f79dc071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 -- 172.21.15.7:0/3591720110 shutdown_connections 2024-03-16T09:05:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 -- 172.21.15.7:0/3591720110 wait complete. 2024-03-16T09:05:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 Processor -- start 2024-03-16T09:05:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.667+0000 7f79e1f2f700 1 -- start start 2024-03-16T09:05:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79e1f2f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc102da0 0x7f79dc0fc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79e1f2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79dc103740 0x7f79dc0fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79e1f2f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0fd170 con 0x7f79dc102da0 2024-03-16T09:05:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79e1f2f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0fd2b0 con 0x7f79dc103740 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc102da0 0x7f79dc0fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc102da0 0x7f79dc0fc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47262/0 (socket says 172.21.15.7:47262) 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79db7fe700 1 -- 172.21.15.7:0/1304190700 learned_addr learned my addr 172.21.15.7:0/1304190700 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79daffd700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79dc103740 0x7f79dc0fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79db7fe700 1 -- 172.21.15.7:0/1304190700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79dc103740 msgr2=0x7f79dc0fcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79db7fe700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79dc103740 0x7f79dc0fcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79db7fe700 1 -- 172.21.15.7:0/1304190700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0005040 con 0x7f79dc102da0 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79daffd700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79dc103740 0x7f79dc0fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79db7fe700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc102da0 0x7f79dc0fc610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00b790 tx=0x7f79cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.668+0000 7f79c3fff700 1 -- 172.21.15.7:0/1304190700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79cc00f770 con 0x7f79dc102da0 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.669+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc119ec0 con 0x7f79dc102da0 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.669+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc11a320 con 0x7f79dc102da0 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.669+0000 7f79c3fff700 1 -- 172.21.15.7:0/1304190700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79cc00fda0 con 0x7f79dc102da0 2024-03-16T09:05:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.669+0000 7f79c3fff700 1 -- 172.21.15.7:0/1304190700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79cc00d310 con 0x7f79dc102da0 2024-03-16T09:05:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.669+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c80052b0 con 0x7f79dc102da0 2024-03-16T09:05:17.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.672+0000 7f79c3fff700 1 -- 172.21.15.7:0/1304190700 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7f79cc010420 con 0x7f79dc102da0 2024-03-16T09:05:17.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.672+0000 7f79c3fff700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f79c406d7e0 0x7f79c406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:17.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.673+0000 7f79c3fff700 1 -- 172.21.15.7:0/1304190700 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f79cc08d4c0 con 0x7f79dc102da0 2024-03-16T09:05:17.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.673+0000 7f79daffd700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f79c406d7e0 0x7f79c406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:17.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.673+0000 7f79daffd700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f79c406d7e0 0x7f79c406fc90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b3b0 tx=0x7f79d0009e30 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:17.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.676+0000 7f79c3fff700 1 -- 172.21.15.7:0/1304190700 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7f79cc057230 con 0x7f79dc102da0 2024-03-16T09:05:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.905+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 --> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79c8000bc0 con 0x7f79c406d7e0 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.907+0000 7f79c3fff700 1 -- 172.21.15.7:0/1304190700 <== mgr.24475 v2:172.21.15.7:6800/3493394788 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f79c8000bc0 con 0x7f79c406d7e0 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "2/21 daemons upgraded", 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mon daemons", 2024-03-16T09:05:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:05:17.910 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:05:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f79c406d7e0 msgr2=0x7f79c406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f79c406d7e0 0x7f79c406fc90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b3b0 tx=0x7f79d0009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc102da0 msgr2=0x7f79dc0fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc102da0 0x7f79dc0fc610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00b790 tx=0x7f79cc009250 comp rx=0 tx=0).stop 2024-03-16T09:05:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 shutdown_connections 2024-03-16T09:05:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7f79c406d7e0 0x7f79c406fc90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc102da0 0x7f79dc0fc610 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 --2- 172.21.15.7:0/1304190700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79dc103740 0x7f79dc0fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 >> 172.21.15.7:0/1304190700 conn(0x7f79dc071450 msgr2=0x7f79dc0fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 shutdown_connections 2024-03-16T09:05:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:17.909+0000 7f79e1f2f700 1 -- 172.21.15.7:0/1304190700 wait complete. 2024-03-16T09:05:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.012+0000 7fe739528700 1 -- 172.21.15.7:0/695894697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 msgr2=0x7fe7340fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.012+0000 7fe739528700 1 --2- 172.21.15.7:0/695894697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 0x7fe7340fce10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe724004940 tx=0x7fe724009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.012+0000 7fe739528700 1 -- 172.21.15.7:0/695894697 shutdown_connections 2024-03-16T09:05:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.012+0000 7fe739528700 1 --2- 172.21.15.7:0/695894697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 0x7fe7340fce10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.012+0000 7fe739528700 1 --2- 172.21.15.7:0/695894697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe734102e00 0x7fe7341031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.012+0000 7fe739528700 1 -- 172.21.15.7:0/695894697 >> 172.21.15.7:0/695894697 conn(0x7fe7340f89b0 msgr2=0x7fe7340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.012+0000 7fe739528700 1 -- 172.21.15.7:0/695894697 shutdown_connections 2024-03-16T09:05:18.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.012+0000 7fe739528700 1 -- 172.21.15.7:0/695894697 wait complete. 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.013+0000 7fe739528700 1 Processor -- start 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.013+0000 7fe739528700 1 -- start start 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.013+0000 7fe739528700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe734102e00 0x7fe734113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.013+0000 7fe739528700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 0x7fe734113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.013+0000 7fe739528700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe734114510 con 0x7fe7341037a0 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe739528700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73410db30 con 0x7fe734102e00 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe7327fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 0x7fe734113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe7327fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 0x7fe734113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47276/0 (socket says 172.21.15.7:47276) 2024-03-16T09:05:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe7327fc700 1 -- 172.21.15.7:0/2591729496 learned_addr learned my addr 172.21.15.7:0/2591729496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe732ffd700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe734102e00 0x7fe734113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe7327fc700 1 -- 172.21.15.7:0/2591729496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe734102e00 msgr2=0x7fe734113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe7327fc700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe734102e00 0x7fe734113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe7327fc700 1 -- 172.21.15.7:0/2591729496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724005040 con 0x7fe7341037a0 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe732ffd700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe734102e00 0x7fe734113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe7327fc700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 0x7fe734113ec0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe724009100 tx=0x7fe724009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2591729496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe72400e040 con 0x7fe7341037a0 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2591729496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe724013420 con 0x7fe7341037a0 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2591729496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe72401c610 con 0x7fe7341037a0 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.014+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73410ddb0 con 0x7fe7341037a0 2024-03-16T09:05:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.015+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73410e320 con 0x7fe7341037a0 2024-03-16T09:05:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.015+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7341052d0 con 0x7fe7341037a0 2024-03-16T09:05:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.016+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2591729496 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 33) v1 ==== 91553+0+0 (secure 0 0 0) 0x7fe724013590 con 0x7fe7341037a0 2024-03-16T09:05:18.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.017+0000 7fe71b7fe700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe71c06d7e0 0x7fe71c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.017+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2591729496 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe72408e060 con 0x7fe7341037a0 2024-03-16T09:05:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.017+0000 7fe732ffd700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe71c06d7e0 0x7fe71c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:18.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.017+0000 7fe732ffd700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe71c06d7e0 0x7fe71c06fc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe728004d60 tx=0x7fe72800b040 comp rx=0 tx=0).ready entity=mgr.24475 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:18.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.022+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2591729496 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184357 (secure 0 0 0) 0x7fe724057df0 con 0x7fe7341037a0 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (squid) 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: Upgrade: Setting container_image for all mgr 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: pgmap v86: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/1851342253' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:05:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:17 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/994855972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:05:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (squid) 2024-03-16T09:05:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-16T09:05:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: Upgrade: Setting container_image for all mgr 2024-03-16T09:05:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: pgmap v86: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/1851342253' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:05:18.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:17 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/994855972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:05:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.330+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe73410efd0 con 0x7fe7341037a0 2024-03-16T09:05:18.333 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_OK 2024-03-16T09:05:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.330+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2591729496 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe72405b4b0 con 0x7fe7341037a0 2024-03-16T09:05:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.332+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe71c06d7e0 msgr2=0x7fe71c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.332+0000 7fe739528700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe71c06d7e0 0x7fe71c06fc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe728004d60 tx=0x7fe72800b040 comp rx=0 tx=0).stop 2024-03-16T09:05:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 msgr2=0x7fe734113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 0x7fe734113ec0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe724009100 tx=0x7fe724009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 shutdown_connections 2024-03-16T09:05:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:6800/3493394788,v1:172.21.15.7:6801/3493394788] conn(0x7fe71c06d7e0 0x7fe71c06fc90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe734102e00 0x7fe734113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 --2- 172.21.15.7:0/2591729496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7341037a0 0x7fe734113ec0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 >> 172.21.15.7:0/2591729496 conn(0x7fe7340f89b0 msgr2=0x7fe7341021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 shutdown_connections 2024-03-16T09:05:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:18.333+0000 7fe739528700 1 -- 172.21.15.7:0/2591729496 wait complete. 2024-03-16T09:05:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: Upgrade: Updating mon.smithi007 2024-03-16T09:05:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:05:19.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:05:19.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:19.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: Deploying daemon mon.smithi007 on smithi007 2024-03-16T09:05:19.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: from='client.? 172.21.15.7:0/2591729496' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:05:19.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 ceph-mon[29470]: pgmap v87: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:19.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: Upgrade: Updating mon.smithi007 2024-03-16T09:05:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:05:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:05:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: Deploying daemon mon.smithi007 on smithi007 2024-03-16T09:05:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: from='client.? 172.21.15.7:0/2591729496' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:05:19.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:19 smithi019 ceph-mon[36407]: pgmap v87: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:20.175 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:19 smithi007 systemd[1]: Stopping Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:05:20.175 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:20 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[29466]: 2024-03-16T09:05:20.151+0000 7f9f0383f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi007 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:05:20.175 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:20 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[29466]: 2024-03-16T09:05:20.151+0000 7f9f0383f700 -1 mon.smithi007@0(leader) e2 *** Got Signal Terminated *** 2024-03-16T09:05:20.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:20 smithi007 podman[113368]: 2024-03-16 09:05:20.2341569 +0000 UTC m=+0.175348301 container died aaa292e58c256d214655e6adedfb221c3fc3223fc8c21bbd025f68a28346176c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=quincy-1df2724, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T09:05:20.880 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:20 smithi007 podman[113368]: 2024-03-16 09:05:20.611067517 +0000 UTC m=+0.552258925 container cleanup aaa292e58c256d214655e6adedfb221c3fc3223fc8c21bbd025f68a28346176c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=quincy-1df2724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:05:20.880 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:20 smithi007 bash[113368]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007 2024-03-16T09:05:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:20 smithi007 podman[113393]: 2024-03-16 09:05:20.878919269 +0000 UTC m=+0.636685411 container remove aaa292e58c256d214655e6adedfb221c3fc3223fc8c21bbd025f68a28346176c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-1df2724) 2024-03-16T09:05:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007.service: Deactivated successfully. 2024-03-16T09:05:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 systemd[1]: Stopped Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:05:21.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007.service: Consumed 13.173s CPU time. 2024-03-16T09:05:21.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 systemd[1]: Starting Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:05:21.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 podman[113527]: 2024-03-16 09:05:21.580149082 +0000 UTC m=+0.110428462 container create bef77d9053ffa9cc11166b4943ae8a30326313b6d51c3f0fe37ebd7607609591 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T09:05:21.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 podman[113527]: 2024-03-16 09:05:21.489062821 +0000 UTC m=+0.019342221 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:05:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 podman[113527]: 2024-03-16 09:05:21.808193836 +0000 UTC m=+0.338473217 container init bef77d9053ffa9cc11166b4943ae8a30326313b6d51c3f0fe37ebd7607609591 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-16T09:05:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 podman[113527]: 2024-03-16 09:05:21.811467651 +0000 UTC m=+0.341747042 container start bef77d9053ffa9cc11166b4943ae8a30326313b6d51c3f0fe37ebd7607609591 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: pidfile_write: ignore empty --pid-file 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: load: jerasure load: lrc 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: RocksDB version: 7.9.2 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Git sha 0 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: DB SUMMARY 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: DB Session ID: BXUIIKZFB9HZGQ4AWR9F 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: CURRENT file: CURRENT 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1170 Bytes 2024-03-16T09:05:21.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi007/store.db dir, Total Num: 1, files: 000033.sst 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi007/store.db: 000031.log size: 3386803 ; 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.error_if_exists: 0 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.create_if_missing: 0 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.paranoid_checks: 1 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.env: 0x55d8c6657c60 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.info_log: 0x55d8c8633980 2024-03-16T09:05:22.000 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T09:05:22.001 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.statistics: (nil) 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.use_fsync: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_log_file_size: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.allow_fallocate: 1 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.use_direct_reads: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.db_log_dir: 2024-03-16T09:05:22.002 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.wal_dir: 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.write_buffer_manager: 0x55d8c86061e0 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T09:05:22.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.unordered_write: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.row_cache: None 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.wal_filter: None 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.two_write_queues: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.wal_compression: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.atomic_flush: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T09:05:22.004 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.log_readahead_size: 0 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_background_jobs: 2 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_background_compactions: -1 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_subcompactions: 1 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T09:05:22.005 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_open_files: -1 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_background_flushes: -1 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Compression algorithms supported: 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: kZSTD supported: 0 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: kXpressCompression supported: 0 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: kZlibCompression supported: 1 2024-03-16T09:05:22.006 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: kSnappyCompression supported: 1 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: kLZ4Compression supported: 1 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: kBZip2Compression supported: 0 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi007/store.db/MANIFEST-000015 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.merge_operator: 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_filter: None 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d8c8633a80) 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-03-16T09:05:22.007 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_type: 0 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_block_index_type: 0 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_shortening: 1 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: checksum: 4 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: no_block_cache: 0 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache: 0x55d8c8639090 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_options: 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: capacity : 536870912 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: num_shard_bits : 4 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: strict_capacity_limit : 0 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_cache_compressed: (nil) 2024-03-16T09:05:22.008 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: persistent_cache: (nil) 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_size: 4096 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_size_deviation: 10 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_restart_interval: 16 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: index_block_restart_interval: 1 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: metadata_block_size: 4096 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: partition_filters: 0 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: use_delta_encoding: 1 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: filter_policy: bloomfilter 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: whole_key_filtering: 1 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: verify_compression: 0 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: format_version: 5 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: enable_index_compression: 1 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: block_align: 0 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_auto_readahead_size: 262144 2024-03-16T09:05:22.009 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: prepopulate_block_cache: 0 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: initial_auto_readahead_size: 8192 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression: NoCompression 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.num_levels: 7 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T09:05:22.010 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T09:05:22.011 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T09:05:22.012 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T09:05:22.013 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.table_properties_collectors: 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.inplace_update_support: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.bloom_locality: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.max_successive_merges: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.ttl: 2592000 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T09:05:22.014 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.enable_blob_files: false 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.min_blob_size: 0 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi007/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12126, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a672d714-f643-4b7b-b813-2bc1dbcde421 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579921840531, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-03-16T09:05:22.015 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579921850894, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 2939485, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12127, "largest_seqno": 15086, "table_properties": {"data_size": 2928248, "index_size": 6824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3397, "raw_key_size": 30083, "raw_average_key_size": 22, "raw_value_size": 2901917, "raw_average_value_size": 2140, "num_data_blocks": 311, "num_entries": 1356, "num_filter_entries": 1356, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710579921, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a672d714-f643-4b7b-b813-2bc1dbcde421", "db_session_id": "BXUIIKZFB9HZGQ4AWR9F", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579921850998, "job": 1, "event": "recovery_finished"} 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi007/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d8c872c000 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: DB pointer 0x55d8c8716000 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** DB Stats ** 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:22.016 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: L0 1/0 2.80 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 584.6 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: L6 1/0 5.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Sum 2/0 8.68 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 584.6 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 584.6 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 584.6 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T09:05:22.017 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Cumulative compaction: 0.00 GB write, 219.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Interval compaction: 0.00 GB write, 219.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Block cache BinnedLRUCache@0x55d8c8639090#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: starting mon.smithi007 rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi007 fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: mon.smithi007@-1(???) e2 preinit fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: mon.smithi007@-1(???).mds e19 new map 2024-03-16T09:05:22.018 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: mon.smithi007@-1(???).mds e19 print_map 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: e19 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: legacy client fscid: 1 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Filesystem 'cephfs' (1) 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: fs_name cephfs 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: epoch 19 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: modified 2024-03-16T09:04:55.865707+0000 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: tableserver 0 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: root 0 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: session_timeout 60 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: session_autoclose 300 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_file_size 1099511627776 2024-03-16T09:05:22.019 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_xattr_size 65536 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: required_client_features {} 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: last_failure 0 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: last_failure_osd_epoch 46 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: max_mds 2 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: in 0,1 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: up {0=14482,1=24281} 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: failed 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: damaged 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: stopped 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: data_pools [3] 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: metadata_pool 2 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: inline_data disabled 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: balancer 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: bal_rank_mask -1 2024-03-16T09:05:22.020 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: standby_count_wanted 1 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: qdb_cluster leader: 0 members: 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: [mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: [mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: Standby daemons: 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: [mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout: [mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: mon.smithi007@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: mon.smithi007@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: mon.smithi007@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: mon.smithi007@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 ceph-mon[113546]: mon.smithi007@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 bash[113527]: bef77d9053ffa9cc11166b4943ae8a30326313b6d51c3f0fe37ebd7607609591 2024-03-16T09:05:22.021 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:21 smithi007 systemd[1]: Started Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: pgmap v88: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: pgmap v89: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: mon.smithi007 calling monitor election 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: mon.smithi007 is new leader, mons smithi007,smithi019 in quorum (ranks 0,1) 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: monmap epoch 2 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: last_changed 2024-03-16T08:54:07.017918+0000 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: created 2024-03-16T08:51:33.787188+0000 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: min_mon_release 17 (quincy) 2024-03-16T09:05:23.624 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: election_strategy: 1 2024-03-16T09:05:23.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-03-16T09:05:23.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: 1: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.smithi019 2024-03-16T09:05:23.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:05:23.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: osdmap e48: 6 total, 6 up, 6 in 2024-03-16T09:05:23.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: mgrmap e33: smithi007.uamwtp(active, since 2m), standbys: smithi019.iuqxhi 2024-03-16T09:05:23.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: overall HEALTH_OK 2024-03-16T09:05:23.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: from='mgr.24475 ' entity='' 2024-03-16T09:05:23.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:23 smithi007 ceph-mon[113546]: mgrmap e34: smithi007.uamwtp(active, since 2m), standbys: smithi019.iuqxhi 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: from='mgr.24475 172.21.15.7:0/3545347975' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: pgmap v88: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: pgmap v89: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: mon.smithi007 calling monitor election 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: mon.smithi007 is new leader, mons smithi007,smithi019 in quorum (ranks 0,1) 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: monmap epoch 2 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T09:05:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: last_changed 2024-03-16T08:54:07.017918+0000 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: created 2024-03-16T08:51:33.787188+0000 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: min_mon_release 17 (quincy) 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: election_strategy: 1 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: 1: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.smithi019 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: osdmap e48: 6 total, 6 up, 6 in 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: mgrmap e33: smithi007.uamwtp(active, since 2m), standbys: smithi019.iuqxhi 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: overall HEALTH_OK 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: from='mgr.24475 ' entity='' 2024-03-16T09:05:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:23 smithi019 ceph-mon[36407]: mgrmap e34: smithi007.uamwtp(active, since 2m), standbys: smithi019.iuqxhi 2024-03-16T09:05:27.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:27 smithi019 ceph-mon[36407]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:05:27.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:27 smithi007 ceph-mon[113546]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:05:30.643 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: Active manager daemon smithi007.uamwtp restarted 2024-03-16T09:05:30.643 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: Activating manager daemon smithi007.uamwtp 2024-03-16T09:05:30.643 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: osdmap e49: 6 total, 6 up, 6 in 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: mgrmap e35: smithi007.uamwtp(active, starting, since 0.00576908s), standbys: smithi019.iuqxhi 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.? 172.21.15.19:0/140704299' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.iuqxhi", "id": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: Standby manager daemon smithi019.iuqxhi restarted 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.? 172.21.15.19:0/140704299' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:05:30.644 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.? 172.21.15.19:0/140704299' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: from='mgr.? 172.21.15.19:0/140704299' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:05:30.645 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:30 smithi007 ceph-mon[113546]: Manager daemon smithi007.uamwtp is now available 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: Active manager daemon smithi007.uamwtp restarted 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: Activating manager daemon smithi007.uamwtp 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: osdmap e49: 6 total, 6 up, 6 in 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: mgrmap e35: smithi007.uamwtp(active, starting, since 0.00576908s), standbys: smithi019.iuqxhi 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:05:30.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi007.uamwtp", "id": "smithi007.uamwtp"}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/140704299' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/crt"}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr metadata", "who": "smithi019.iuqxhi", "id": "smithi019.iuqxhi"}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: Standby manager daemon smithi019.iuqxhi restarted 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: Standby manager daemon smithi019.iuqxhi started 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/140704299' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/140704299' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi019.iuqxhi/key"}]: dispatch 2024-03-16T09:05:30.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:05:30.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:05:30.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: from='mgr.? 172.21.15.19:0/140704299' entity='mgr.smithi019.iuqxhi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:05:30.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:30 smithi019 ceph-mon[36407]: Manager daemon smithi007.uamwtp is now available 2024-03-16T09:05:31.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: [16/Mar/2024:09:05:31] ENGINE Bus STARTING 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: [16/Mar/2024:09:05:31] ENGINE Serving on http://172.21.15.7:8765 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: [16/Mar/2024:09:05:31] ENGINE Serving on https://172.21.15.7:7150 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: [16/Mar/2024:09:05:31] ENGINE Bus STARTED 2024-03-16T09:05:31.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:31 smithi007 ceph-mon[113546]: mgrmap e36: smithi007.uamwtp(active, since 1.00988s), standbys: smithi019.iuqxhi 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi007.uamwtp/trash_purge_schedule"}]: dispatch 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: [16/Mar/2024:09:05:31] ENGINE Bus STARTING 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: [16/Mar/2024:09:05:31] ENGINE Serving on http://172.21.15.7:8765 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: [16/Mar/2024:09:05:31] ENGINE Serving on https://172.21.15.7:7150 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: [16/Mar/2024:09:05:31] ENGINE Bus STARTED 2024-03-16T09:05:31.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:31 smithi019 ceph-mon[36407]: mgrmap e36: smithi007.uamwtp(active, since 1.00988s), standbys: smithi019.iuqxhi 2024-03-16T09:05:32.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:32 smithi019 ceph-mon[36407]: pgmap v3: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:32.724 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:32 smithi007 ceph-mon[113546]: pgmap v3: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:33.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:33 smithi019 ceph-mon[36407]: pgmap v4: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:33.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:33 smithi019 ceph-mon[36407]: mgrmap e37: smithi007.uamwtp(active, since 3s), standbys: smithi019.iuqxhi 2024-03-16T09:05:33.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:33 smithi007 ceph-mon[113546]: pgmap v4: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:33.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:33 smithi007 ceph-mon[113546]: mgrmap e37: smithi007.uamwtp(active, since 3s), standbys: smithi019.iuqxhi 2024-03-16T09:05:35.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:34 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:35.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:34 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:35.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:35.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:36.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:35 smithi019 ceph-mon[36407]: pgmap v5: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:36.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:35 smithi019 ceph-mon[36407]: mgrmap e38: smithi007.uamwtp(active, since 4s), standbys: smithi019.iuqxhi 2024-03-16T09:05:36.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:35 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:36.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:35 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:35 smithi007 ceph-mon[113546]: pgmap v5: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:35 smithi007 ceph-mon[113546]: mgrmap e38: smithi007.uamwtp(active, since 4s), standbys: smithi019.iuqxhi 2024-03-16T09:05:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:35 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:35 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:37.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:36 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:37.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:36 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:37.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:36 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:37.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:36 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:38.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:38 smithi019 ceph-mon[36407]: pgmap v6: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:38.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:38 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:38.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:38 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:38.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:38 smithi007 ceph-mon[113546]: pgmap v6: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:38.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:38 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:38.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:38 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:40.222 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:40 smithi007 ceph-mon[113546]: pgmap v7: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-16T09:05:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:40 smithi019 ceph-mon[36407]: pgmap v7: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: pgmap v8: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Detected new or changed devices on smithi007 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:05:41.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:05:41.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:05:41.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:05:41.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:05:41.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:41 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: pgmap v8: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-16T09:05:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Detected new or changed devices on smithi007 2024-03-16T09:05:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:41.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Updating smithi019:/etc/ceph/ceph.conf 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Updating smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: Updating smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:41.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:42.883 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:42 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:42.883 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:42 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:05:42.883 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:42 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-16T09:05:42.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:42.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:05:42.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-16T09:05:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:44 smithi019 ceph-mon[36407]: pgmap v9: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-16T09:05:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:44 smithi019 ceph-mon[36407]: Upgrade: Updating mon.smithi019 2024-03-16T09:05:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:44 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:44 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:05:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:44 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:05:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:44 smithi019 ceph-mon[36407]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:44 smithi019 ceph-mon[36407]: Deploying daemon mon.smithi019 on smithi019 2024-03-16T09:05:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:44 smithi007 ceph-mon[113546]: pgmap v9: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-16T09:05:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:44 smithi007 ceph-mon[113546]: Upgrade: Updating mon.smithi019 2024-03-16T09:05:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:05:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:05:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:44.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:44 smithi007 ceph-mon[113546]: Deploying daemon mon.smithi019 on smithi019 2024-03-16T09:05:45.444 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:45 smithi019 systemd[1]: Stopping Ceph mon.smithi019 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:05:45.444 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:45 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019[36403]: 2024-03-16T09:05:45.417+0000 7f5230e74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi019 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:05:45.444 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:45 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019[36403]: 2024-03-16T09:05:45.417+0000 7f5230e74700 -1 mon.smithi019@1(peon) e2 *** Got Signal Terminated *** 2024-03-16T09:05:45.745 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:45 smithi019 podman[83202]: 2024-03-16 09:05:45.469663136 +0000 UTC m=+0.148218799 container died bfd7571d5d35dd4f94bf7366ec5c9fb2c78451736bb325bb68cb52449e011d2b (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, ceph=True, GIT_BRANCH=HEAD, RELEASE=quincy-1df2724, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:05:46.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:45 smithi019 podman[83202]: 2024-03-16 09:05:45.843352032 +0000 UTC m=+0.521907688 container cleanup bfd7571d5d35dd4f94bf7366ec5c9fb2c78451736bb325bb68cb52449e011d2b (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=quincy-1df2724, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T09:05:46.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:45 smithi019 bash[83202]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019 2024-03-16T09:05:46.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 podman[83216]: 2024-03-16 09:05:46.094780507 +0000 UTC m=+0.617782013 container remove bfd7571d5d35dd4f94bf7366ec5c9fb2c78451736bb325bb68cb52449e011d2b (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=quincy-1df2724, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T09:05:46.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi019.service: Deactivated successfully. 2024-03-16T09:05:46.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 systemd[1]: Stopped Ceph mon.smithi019 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:05:46.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi019.service: Consumed 7.244s CPU time. 2024-03-16T09:05:46.787 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 systemd[1]: Starting Ceph mon.smithi019 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:05:46.787 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 podman[83347]: 2024-03-16 09:05:46.588849026 +0000 UTC m=+0.017944653 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:05:46.787 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 podman[83347]: 2024-03-16 09:05:46.687360787 +0000 UTC m=+0.116456438 container create 1abff6ceff3e6a1a9c348636ce5782a4d68198ab269f8ea7c74b42416e0b06c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T09:05:46.921 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 podman[83347]: 2024-03-16 09:05:46.881332257 +0000 UTC m=+0.310427870 container init 1abff6ceff3e6a1a9c348636ce5782a4d68198ab269f8ea7c74b42416e0b06c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-16T09:05:46.921 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 podman[83347]: 2024-03-16 09:05:46.884285994 +0000 UTC m=+0.313381607 container start 1abff6ceff3e6a1a9c348636ce5782a4d68198ab269f8ea7c74b42416e0b06c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T09:05:46.921 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T09:05:46.921 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T09:05:46.921 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: pidfile_write: ignore empty --pid-file 2024-03-16T09:05:46.921 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: load: jerasure load: lrc 2024-03-16T09:05:46.921 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: RocksDB version: 7.9.2 2024-03-16T09:05:46.921 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Git sha 0 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: DB SUMMARY 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: DB Session ID: J57DR81TXU8HPQ1C19RA 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: CURRENT file: CURRENT 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1095 Bytes 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi019/store.db dir, Total Num: 1, files: 000027.sst 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi019/store.db: 000025.log size: 7675286 ; 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.error_if_exists: 0 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.create_if_missing: 0 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.paranoid_checks: 1 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T09:05:46.922 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T09:05:46.923 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T09:05:46.923 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.env: 0x5631885adc60 2024-03-16T09:05:46.923 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T09:05:46.923 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.info_log: 0x56318aa71980 2024-03-16T09:05:46.923 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T09:05:46.924 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.statistics: (nil) 2024-03-16T09:05:46.924 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.use_fsync: 0 2024-03-16T09:05:46.924 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_log_file_size: 0 2024-03-16T09:05:46.924 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T09:05:46.924 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.allow_fallocate: 1 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.use_direct_reads: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.db_log_dir: 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.wal_dir: 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T09:05:46.925 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.write_buffer_manager: 0x56318aa441e0 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T09:05:46.926 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.unordered_write: 0 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.row_cache: None 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.wal_filter: None 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.two_write_queues: 0 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.wal_compression: 0 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.atomic_flush: 0 2024-03-16T09:05:46.927 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.log_readahead_size: 0 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_background_jobs: 2 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_background_compactions: -1 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_subcompactions: 1 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T09:05:46.928 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_open_files: -1 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_background_flushes: -1 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Compression algorithms supported: 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: kZSTD supported: 0 2024-03-16T09:05:46.929 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: kXpressCompression supported: 0 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: kZlibCompression supported: 1 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: kSnappyCompression supported: 1 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: kLZ4Compression supported: 1 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: kBZip2Compression supported: 0 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi019/store.db/MANIFEST-000009 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.merge_operator: 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_filter: None 2024-03-16T09:05:46.930 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56318aa71a80) 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_type: 0 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_block_index_type: 0 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_shortening: 1 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: checksum: 4 2024-03-16T09:05:46.931 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: no_block_cache: 0 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache: 0x56318aa77090 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_options: 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: capacity : 536870912 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: num_shard_bits : 4 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: strict_capacity_limit : 0 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_cache_compressed: (nil) 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: persistent_cache: (nil) 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_size: 4096 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_size_deviation: 10 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_restart_interval: 16 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: index_block_restart_interval: 1 2024-03-16T09:05:46.932 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: metadata_block_size: 4096 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: partition_filters: 0 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: use_delta_encoding: 1 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: filter_policy: bloomfilter 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: whole_key_filtering: 1 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: verify_compression: 0 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: format_version: 5 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: enable_index_compression: 1 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: block_align: 0 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: max_auto_readahead_size: 262144 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: prepopulate_block_cache: 0 2024-03-16T09:05:46.933 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: initial_auto_readahead_size: 8192 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression: NoCompression 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.num_levels: 7 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T09:05:46.934 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T09:05:46.935 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T09:05:46.936 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T09:05:46.937 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.table_properties_collectors: 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.inplace_update_support: 0 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.bloom_locality: 0 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.max_successive_merges: 0 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T09:05:46.938 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.ttl: 2592000 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.enable_blob_files: false 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.min_blob_size: 0 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T09:05:46.939 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-03-16T09:05:46.940 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-16T09:05:46.940 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi019/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12256, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-16T09:05:46.940 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-16T09:05:46.940 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 19cd7237-d486-4995-b65b-43ae3fe1c1ba 2024-03-16T09:05:46.940 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579946915607, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-16T09:05:46.940 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-16T09:05:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579946935017, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4567861, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12261, "largest_seqno": 16034, "table_properties": {"data_size": 4554496, "index_size": 8376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3973, "raw_key_size": 35063, "raw_average_key_size": 22, "raw_value_size": 4523622, "raw_average_value_size": 2883, "num_data_blocks": 378, "num_entries": 1569, "num_filter_entries": 1569, "num_deletions": 260, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710579946, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "19cd7237-d486-4995-b65b-43ae3fe1c1ba", "db_session_id": "J57DR81TXU8HPQ1C19RA", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-16T09:05:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579946935164, "job": 1, "event": "recovery_finished"} 2024-03-16T09:05:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-16T09:05:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-16T09:05:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi019/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T09:05:47.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56318ab6a000 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: DB pointer 0x56318ab54000 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** DB Stats ** 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-03-16T09:05:47.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: L0 1/0 4.36 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 565.7 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: L6 1/0 5.87 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Sum 2/0 10.23 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 565.7 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 565.7 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** Compaction Stats [default] ** 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 565.7 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:05:47.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Cumulative compaction: 0.00 GB write, 193.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Interval compaction: 0.00 GB write, 193.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Block cache BinnedLRUCache@0x56318aa77090#2 capacity: 512.00 MB usage: 13.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: starting mon.smithi019 rank 1 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi019 fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: mon.smithi019@-1(???) e2 preinit fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: mon.smithi019@-1(???).mds e20 new map 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: mon.smithi019@-1(???).mds e20 print_map 2024-03-16T09:05:47.216 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: e20 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: legacy client fscid: 1 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Filesystem 'cephfs' (1) 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: fs_name cephfs 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: epoch 20 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: tableserver 0 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: root 0 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: session_timeout 60 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: session_autoclose 300 2024-03-16T09:05:47.217 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: max_file_size 1099511627776 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: max_xattr_size 65536 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: required_client_features {} 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: last_failure 0 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: last_failure_osd_epoch 46 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: max_mds 2 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: in 0,1 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: up {0=14482,1=24281} 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: failed 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: damaged 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: stopped 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: data_pools [3] 2024-03-16T09:05:47.218 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: metadata_pool 2 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: inline_data disabled 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: balancer 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: bal_rank_mask -1 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: standby_count_wanted 1 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: [mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: [mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: Standby daemons: 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: [mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout: [mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: mon.smithi019@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: mon.smithi019@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T09:05:47.219 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: mon.smithi019@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T09:05:47.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: mon.smithi019@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T09:05:47.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 ceph-mon[83366]: mon.smithi019@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-16T09:05:47.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 bash[83347]: 1abff6ceff3e6a1a9c348636ce5782a4d68198ab269f8ea7c74b42416e0b06c2 2024-03-16T09:05:47.220 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:46 smithi019 systemd[1]: Started Ceph mon.smithi019 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:05:47.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:47 smithi019 ceph-mon[83366]: pgmap v10: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-16T09:05:47.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:47 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:05:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:47 smithi007 ceph-mon[113546]: pgmap v10: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-16T09:05:47.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:47 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:05:48.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 -- 172.21.15.7:0/1165059886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 msgr2=0x7ff8bc1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 --2- 172.21.15.7:0/1165059886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 0x7ff8bc1031e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009ce0 tx=0x7ff8b4004d10 comp rx=0 tx=0).stop 2024-03-16T09:05:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 -- 172.21.15.7:0/1165059886 shutdown_connections 2024-03-16T09:05:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 --2- 172.21.15.7:0/1165059886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 0x7ff8bc0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 --2- 172.21.15.7:0/1165059886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 0x7ff8bc1031e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 -- 172.21.15.7:0/1165059886 >> 172.21.15.7:0/1165059886 conn(0x7ff8bc0f89a0 msgr2=0x7ff8bc0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 -- 172.21.15.7:0/1165059886 shutdown_connections 2024-03-16T09:05:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 -- 172.21.15.7:0/1165059886 wait complete. 2024-03-16T09:05:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.453+0000 7ff8c41da700 1 Processor -- start 2024-03-16T09:05:48.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.458+0000 7ff8c41da700 1 -- start start 2024-03-16T09:05:48.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.458+0000 7ff8c41da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 0x7ff8bc113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:48.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.458+0000 7ff8c41da700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 0x7ff8bc114220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:48.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.458+0000 7ff8c41da700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc114870 con 0x7ff8bc102e10 2024-03-16T09:05:48.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.458+0000 7ff8c41da700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc10dc40 con 0x7ff8bc1037b0 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1f76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 0x7ff8bc113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1775700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 0x7ff8bc114220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1775700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 0x7ff8bc114220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:39080/0 (socket says 172.21.15.7:39080) 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1f76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 0x7ff8bc113ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55876/0 (socket says 172.21.15.7:55876) 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1775700 1 -- 172.21.15.7:0/3060626959 learned_addr learned my addr 172.21.15.7:0/3060626959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1775700 1 -- 172.21.15.7:0/3060626959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 msgr2=0x7ff8bc114220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1775700 1 -- 172.21.15.7:0/3060626959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 msgr2=0x7ff8bc114220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1775700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 0x7ff8bc114220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1775700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 0x7ff8bc114220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1f76700 1 -- 172.21.15.7:0/3060626959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 msgr2=0x7ff8bc114220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1f76700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 0x7ff8bc114220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1f76700 1 -- 172.21.15.7:0/3060626959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4009940 con 0x7ff8bc102e10 2024-03-16T09:05:48.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8c1f76700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 0x7ff8bc113ce0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009ce0 tx=0x7ff8b4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.459+0000 7ff8ba7fc700 1 -- 172.21.15.7:0/3060626959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b400e040 con 0x7ff8bc102e10 2024-03-16T09:05:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.460+0000 7ff8ba7fc700 1 -- 172.21.15.7:0/3060626959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8b4004610 con 0x7ff8bc102e10 2024-03-16T09:05:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.460+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc10dec0 con 0x7ff8bc102e10 2024-03-16T09:05:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.460+0000 7ff8ba7fc700 1 -- 172.21.15.7:0/3060626959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b401c640 con 0x7ff8bc102e10 2024-03-16T09:05:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.460+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc10e480 con 0x7ff8bc102e10 2024-03-16T09:05:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.461+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc100120 con 0x7ff8bc102e10 2024-03-16T09:05:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.461+0000 7ff8ba7fc700 1 -- 172.21.15.7:0/3060626959 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7ff8b40129e0 con 0x7ff8bc102e10 2024-03-16T09:05:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.462+0000 7ff8ba7fc700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff8a406e000 0x7ff8a40704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.462+0000 7ff8ba7fc700 1 -- 172.21.15.7:0/3060626959 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff8b408f350 con 0x7ff8bc102e10 2024-03-16T09:05:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.462+0000 7ff8c1775700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff8a406e000 0x7ff8a40704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.462+0000 7ff8c1775700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff8a406e000 0x7ff8a40704b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009fd0 tx=0x7ff8b0007ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:48.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.467+0000 7ff8ba7fc700 1 -- 172.21.15.7:0/3060626959 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8b405c170 con 0x7ff8bc102e10 2024-03-16T09:05:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.694+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8bc10f330 con 0x7ff8a406e000 2024-03-16T09:05:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.695+0000 7ff8ba7fc700 1 -- 172.21.15.7:0/3060626959 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7ff8bc10f330 con 0x7ff8a406e000 2024-03-16T09:05:48.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff8a406e000 msgr2=0x7ff8a40704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff8a406e000 0x7ff8a40704b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009fd0 tx=0x7ff8b0007ab0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 msgr2=0x7ff8bc113ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 0x7ff8bc113ce0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009ce0 tx=0x7ff8b4004d10 comp rx=0 tx=0).stop 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 shutdown_connections 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff8a406e000 0x7ff8a40704b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc102e10 0x7ff8bc113ce0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 --2- 172.21.15.7:0/3060626959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8bc1037b0 0x7ff8bc114220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 >> 172.21.15.7:0/3060626959 conn(0x7ff8bc0f89a0 msgr2=0x7ff8bc106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 shutdown_connections 2024-03-16T09:05:48.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.698+0000 7ff8c41da700 1 -- 172.21.15.7:0/3060626959 wait complete. 2024-03-16T09:05:48.710 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:05:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.802+0000 7febeecbb700 1 -- 172.21.15.7:0/59490321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 msgr2=0x7febe80fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.802+0000 7febeecbb700 1 --2- 172.21.15.7:0/59490321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 0x7febe80fbd20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7febd4004940 tx=0x7febd4009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.802+0000 7febeecbb700 1 -- 172.21.15.7:0/59490321 shutdown_connections 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.802+0000 7febeecbb700 1 --2- 172.21.15.7:0/59490321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 0x7febe80fbd20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.802+0000 7febeecbb700 1 --2- 172.21.15.7:0/59490321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 0x7febe81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.802+0000 7febeecbb700 1 -- 172.21.15.7:0/59490321 >> 172.21.15.7:0/59490321 conn(0x7febe80f7880 msgr2=0x7febe80f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.803+0000 7febeecbb700 1 -- 172.21.15.7:0/59490321 shutdown_connections 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.803+0000 7febeecbb700 1 -- 172.21.15.7:0/59490321 wait complete. 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.803+0000 7febeecbb700 1 Processor -- start 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.803+0000 7febeecbb700 1 -- start start 2024-03-16T09:05:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.803+0000 7febeecbb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 0x7febe80fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.803+0000 7febeecbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 0x7febe80fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.803+0000 7febeecbb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80ff930 con 0x7febe81026b0 2024-03-16T09:05:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.803+0000 7febeecbb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80fe410 con 0x7febe8101d10 2024-03-16T09:05:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 0x7febe80fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febeca57700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 0x7febe80fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 0x7febe80fded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55896/0 (socket says 172.21.15.7:55896) 2024-03-16T09:05:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe7fff700 1 -- 172.21.15.7:0/2246180001 learned_addr learned my addr 172.21.15.7:0/2246180001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febeca57700 1 -- 172.21.15.7:0/2246180001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 msgr2=0x7febe80fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febeca57700 1 -- 172.21.15.7:0/2246180001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 msgr2=0x7febe80fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febeca57700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 0x7febe80fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febeca57700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 0x7febe80fd990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe7fff700 1 -- 172.21.15.7:0/2246180001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 msgr2=0x7febe80fd990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe7fff700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 0x7febe80fd990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe7fff700 1 -- 172.21.15.7:0/2246180001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4005040 con 0x7febe81026b0 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe7fff700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 0x7febe80fded0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7febd4009100 tx=0x7febd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe57fa700 1 -- 172.21.15.7:0/2246180001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd400e040 con 0x7febe81026b0 2024-03-16T09:05:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe57fa700 1 -- 172.21.15.7:0/2246180001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febd4013420 con 0x7febe81026b0 2024-03-16T09:05:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe80fe660 con 0x7febe81026b0 2024-03-16T09:05:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febe57fa700 1 -- 172.21.15.7:0/2246180001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd401c610 con 0x7febe81026b0 2024-03-16T09:05:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.804+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe811a0b0 con 0x7febe81026b0 2024-03-16T09:05:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.805+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe80fd460 con 0x7febe81026b0 2024-03-16T09:05:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.806+0000 7febe57fa700 1 -- 172.21.15.7:0/2246180001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7febd4013590 con 0x7febe81026b0 2024-03-16T09:05:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.806+0000 7febe57fa700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7febd806e060 0x7febd8070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.806+0000 7febe57fa700 1 -- 172.21.15.7:0/2246180001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7febd408ec60 con 0x7febe81026b0 2024-03-16T09:05:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.806+0000 7febeca57700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7febd806e060 0x7febd8070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.807+0000 7febeca57700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7febd806e060 0x7febd8070510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7febdc007fd0 tx=0x7febdc009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:48.811+0000 7febe57fa700 1 -- 172.21.15.7:0/2246180001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7febd4057f30 con 0x7febe81026b0 2024-03-16T09:05:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:05:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:05:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: mon.smithi007 calling monitor election 2024-03-16T09:05:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: mon.smithi019 calling monitor election 2024-03-16T09:05:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: mon.smithi007 is new leader, mons smithi007,smithi019 in quorum (ranks 0,1) 2024-03-16T09:05:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: monmap epoch 3 2024-03-16T09:05:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T09:05:48.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: last_changed 2024-03-16T09:05:47.694225+0000 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: created 2024-03-16T08:51:33.787188+0000 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: min_mon_release 19 (squid) 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: election_strategy: 1 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: 1: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.smithi019 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: osdmap e49: 6 total, 6 up, 6 in 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: mgrmap e38: smithi007.uamwtp(active, since 17s), standbys: smithi019.iuqxhi 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: overall HEALTH_OK 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:48.964 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:48.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi007"}]: dispatch 2024-03-16T09:05:48.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mon metadata", "id": "smithi019"}]: dispatch 2024-03-16T09:05:48.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: mon.smithi007 calling monitor election 2024-03-16T09:05:48.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: mon.smithi019 calling monitor election 2024-03-16T09:05:48.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: mon.smithi007 is new leader, mons smithi007,smithi019 in quorum (ranks 0,1) 2024-03-16T09:05:48.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: monmap epoch 3 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: fsid 4c46b77e-e372-11ee-95c9-87774f69a715 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: last_changed 2024-03-16T09:05:47.694225+0000 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: created 2024-03-16T08:51:33.787188+0000 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: min_mon_release 19 (squid) 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: election_strategy: 1 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.smithi007 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: 1: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.smithi019 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: fsmap cephfs:2 {0=cephfs.smithi007.znurst=up:active,1=cephfs.smithi019.bncsil=up:active} 2 up:standby 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: osdmap e49: 6 total, 6 up, 6 in 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: mgrmap e38: smithi007.uamwtp(active, since 17s), standbys: smithi019.iuqxhi 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: overall HEALTH_OK 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:48.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:05:49.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.037+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febe80669c0 con 0x7febd806e060 2024-03-16T09:05:49.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.038+0000 7febe57fa700 1 -- 172.21.15.7:0/2246180001 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7febe80669c0 con 0x7febd806e060 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7febd806e060 msgr2=0x7febd8070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7febd806e060 0x7febd8070510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7febdc007fd0 tx=0x7febdc009ee0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 msgr2=0x7febe80fded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 0x7febe80fded0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7febd4009100 tx=0x7febd4009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 shutdown_connections 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101d10 0x7febe80fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7febd806e060 0x7febd8070510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 --2- 172.21.15.7:0/2246180001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe81026b0 0x7febe80fded0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.040+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 >> 172.21.15.7:0/2246180001 conn(0x7febe80f7880 msgr2=0x7febe80662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:49.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.041+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 shutdown_connections 2024-03-16T09:05:49.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.041+0000 7febeecbb700 1 -- 172.21.15.7:0/2246180001 wait complete. 2024-03-16T09:05:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 -- 172.21.15.7:0/1914684920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910101d10 msgr2=0x7f09101020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 --2- 172.21.15.7:0/1914684920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910101d10 0x7f09101020e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0908009ce0 tx=0x7f0908004d10 comp rx=0 tx=0).stop 2024-03-16T09:05:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 -- 172.21.15.7:0/1914684920 shutdown_connections 2024-03-16T09:05:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 --2- 172.21.15.7:0/1914684920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09101026b0 0x7f09100fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 --2- 172.21.15.7:0/1914684920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0910101d10 0x7f09101020e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 -- 172.21.15.7:0/1914684920 >> 172.21.15.7:0/1914684920 conn(0x7f09100f7880 msgr2=0x7f09100f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 -- 172.21.15.7:0/1914684920 shutdown_connections 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 -- 172.21.15.7:0/1914684920 wait complete. 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 Processor -- start 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.143+0000 7f0915630700 1 -- start start 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f0915630700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 0x7f0910113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f0915630700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09101026b0 0x7f0910114530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f0915630700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0910114b80 con 0x7f09101026b0 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f0915630700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091010df50 con 0x7f0910101d10 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09101026b0 0x7f0910114530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09101026b0 0x7f0910114530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55910/0 (socket says 172.21.15.7:55910) 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090e7fc700 1 -- 172.21.15.7:0/788901257 learned_addr learned my addr 172.21.15.7:0/788901257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090effd700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 0x7f0910113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090effd700 1 -- 172.21.15.7:0/788901257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 msgr2=0x7f0910113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090effd700 1 -- 172.21.15.7:0/788901257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 msgr2=0x7f0910113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090effd700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 0x7f0910113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090effd700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 0x7f0910113ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090e7fc700 1 -- 172.21.15.7:0/788901257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 msgr2=0x7f0910113ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090e7fc700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 0x7f0910113ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090e7fc700 1 -- 172.21.15.7:0/788901257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908009940 con 0x7f09101026b0 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.144+0000 7f090e7fc700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09101026b0 0x7f0910114530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08f800c890 tx=0x7f08f80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.145+0000 7f09077fe700 1 -- 172.21.15.7:0/788901257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f800a7d0 con 0x7f09101026b0 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.145+0000 7f09077fe700 1 -- 172.21.15.7:0/788901257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08f800ae00 con 0x7f09101026b0 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.145+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091010e230 con 0x7f09101026b0 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.145+0000 7f09077fe700 1 -- 172.21.15.7:0/788901257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f80113a0 con 0x7f09101026b0 2024-03-16T09:05:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.145+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091010e7f0 con 0x7f09101026b0 2024-03-16T09:05:49.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.146+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09100fd460 con 0x7f09101026b0 2024-03-16T09:05:49.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.146+0000 7f09077fe700 1 -- 172.21.15.7:0/788901257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f08f8012400 con 0x7f09101026b0 2024-03-16T09:05:49.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.146+0000 7f09077fe700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f08fc06e000 0x7f08fc0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.147+0000 7f090effd700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f08fc06e000 0x7f08fc0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.147+0000 7f09077fe700 1 -- 172.21.15.7:0/788901257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f08f808d970 con 0x7f09101026b0 2024-03-16T09:05:49.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.147+0000 7f090effd700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f08fc06e000 0x7f08fc0704b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f090800b740 tx=0x7f0908004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:49.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.151+0000 7f09077fe700 1 -- 172.21.15.7:0/788901257 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08f805a920 con 0x7f09101026b0 2024-03-16T09:05:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.368+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0910108e80 con 0x7f08fc06e000 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.378+0000 7f09077fe700 1 -- 172.21.15.7:0/788901257 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0910108e80 con 0x7f08fc06e000 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (118s) 13s ago 12m 17.2M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (12m) 13s ago 12m 6669k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (11m) 15s ago 11m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (101s) 13s ago 12m 78.7M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (7m) 13s ago 7m 13.3M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (7m) 13s ago 7m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (7m) 15s ago 7m 121M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:05:49.381 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (7m) 15s ago 7m 13.8M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (3m) 13s ago 14m 473M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (2m) 15s ago 11m 447M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (27s) 13s ago 14m 31.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 starting - - - 2048M 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (2m) 13s ago 12m 15.6M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (2m) 15s ago 11m 15.3M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (10m) 13s ago 10m 206M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (10m) 13s ago 10m 235M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (10m) 13s ago 10m 182M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (9m) 15s ago 9m 264M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (9m) 15s ago 9m 246M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (8m) 15s ago 8m 222M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:05:49.382 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (2m) 13s ago 12m 63.5M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:05:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f08fc06e000 msgr2=0x7f08fc0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f08fc06e000 0x7f08fc0704b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f090800b740 tx=0x7f0908004d10 comp rx=0 tx=0).stop 2024-03-16T09:05:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09101026b0 msgr2=0x7f0910114530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09101026b0 0x7f0910114530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08f800c890 tx=0x7f08f80044c0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 shutdown_connections 2024-03-16T09:05:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0910101d10 0x7f0910113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f08fc06e000 0x7f08fc0704b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 --2- 172.21.15.7:0/788901257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09101026b0 0x7f0910114530 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 >> 172.21.15.7:0/788901257 conn(0x7f09100f7880 msgr2=0x7f09100f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 shutdown_connections 2024-03-16T09:05:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.381+0000 7f0915630700 1 -- 172.21.15.7:0/788901257 wait complete. 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.482+0000 7f0e1ac16700 1 -- 172.21.15.7:0/2703646663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e14101d20 msgr2=0x7f0e141020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.482+0000 7f0e1ac16700 1 --2- 172.21.15.7:0/2703646663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e14101d20 0x7f0e141020f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0e08004940 tx=0x7f0e08009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.482+0000 7f0e1ac16700 1 -- 172.21.15.7:0/2703646663 shutdown_connections 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.482+0000 7f0e1ac16700 1 --2- 172.21.15.7:0/2703646663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e141026c0 0x7f0e140fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.482+0000 7f0e1ac16700 1 --2- 172.21.15.7:0/2703646663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e14101d20 0x7f0e141020f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.482+0000 7f0e1ac16700 1 -- 172.21.15.7:0/2703646663 >> 172.21.15.7:0/2703646663 conn(0x7f0e140f7870 msgr2=0x7f0e140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.483+0000 7f0e1ac16700 1 -- 172.21.15.7:0/2703646663 shutdown_connections 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.483+0000 7f0e1ac16700 1 -- 172.21.15.7:0/2703646663 wait complete. 2024-03-16T09:05:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.483+0000 7f0e1ac16700 1 Processor -- start 2024-03-16T09:05:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.483+0000 7f0e1ac16700 1 -- start start 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.483+0000 7f0e1ac16700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 0x7f0e141139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.483+0000 7f0e1ac16700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e141026c0 0x7f0e14113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.483+0000 7f0e1ac16700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e141145c0 con 0x7f0e141026c0 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.483+0000 7f0e1ac16700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1410db30 con 0x7f0e14101d20 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e13fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e141026c0 0x7f0e14113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e13fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e141026c0 0x7f0e14113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55938/0 (socket says 172.21.15.7:55938) 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e13fff700 1 -- 172.21.15.7:0/1831371570 learned_addr learned my addr 172.21.15.7:0/1831371570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e189b2700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 0x7f0e141139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e189b2700 1 -- 172.21.15.7:0/1831371570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 msgr2=0x7f0e141139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T09:05:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e189b2700 1 -- 172.21.15.7:0/1831371570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 msgr2=0x7f0e141139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e189b2700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 0x7f0e141139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e13fff700 1 -- 172.21.15.7:0/1831371570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 msgr2=0x7f0e141139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e189b2700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 0x7f0e141139a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e13fff700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 0x7f0e141139a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e13fff700 1 -- 172.21.15.7:0/1831371570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08005040 con 0x7f0e141026c0 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e13fff700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e141026c0 0x7f0e14113ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000b790 tx=0x7f0e00009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e117fa700 1 -- 172.21.15.7:0/1831371570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0000f6f0 con 0x7f0e141026c0 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1410de10 con 0x7f0e141026c0 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e117fa700 1 -- 172.21.15.7:0/1831371570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e0000fd20 con 0x7f0e141026c0 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e117fa700 1 -- 172.21.15.7:0/1831371570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e000103a0 con 0x7f0e141026c0 2024-03-16T09:05:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.484+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1410e3d0 con 0x7f0e141026c0 2024-03-16T09:05:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.485+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e140fd470 con 0x7f0e141026c0 2024-03-16T09:05:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.486+0000 7f0e117fa700 1 -- 172.21.15.7:0/1831371570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f0e00011400 con 0x7f0e141026c0 2024-03-16T09:05:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.486+0000 7f0e117fa700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e0406e060 0x7f0e04070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.486+0000 7f0e189b2700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e0406e060 0x7f0e04070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.486+0000 7f0e117fa700 1 -- 172.21.15.7:0/1831371570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0e0008d7a0 con 0x7f0e141026c0 2024-03-16T09:05:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.487+0000 7f0e189b2700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e0406e060 0x7f0e04070510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e08004f90 tx=0x7f0e08009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.491+0000 7f0e117fa700 1 -- 172.21.15.7:0/1831371570 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e0005a630 con 0x7f0e141026c0 2024-03-16T09:05:49.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.784+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0e14108e90 con 0x7f0e141026c0 2024-03-16T09:05:49.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.784+0000 7f0e117fa700 1 -- 172.21.15.7:0/1831371570 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f0e0005a1c0 con 0x7f0e141026c0 2024-03-16T09:05:49.786 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 10, 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T09:05:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:05:49.788 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:05:49.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e0406e060 msgr2=0x7f0e04070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e0406e060 0x7f0e04070510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e08004f90 tx=0x7f0e08009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e141026c0 msgr2=0x7f0e14113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e141026c0 0x7f0e14113ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000b790 tx=0x7f0e00009220 comp rx=0 tx=0).stop 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 shutdown_connections 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e14101d20 0x7f0e141139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e0406e060 0x7f0e04070510 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 --2- 172.21.15.7:0/1831371570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e141026c0 0x7f0e14113ee0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 >> 172.21.15.7:0/1831371570 conn(0x7f0e140f7870 msgr2=0x7f0e140662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 shutdown_connections 2024-03-16T09:05:49.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.786+0000 7f0e1ac16700 1 -- 172.21.15.7:0/1831371570 wait complete. 2024-03-16T09:05:49.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 -- 172.21.15.7:0/2801051007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 msgr2=0x7f53580fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 --2- 172.21.15.7:0/2801051007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 0x7f53580fe250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5348004940 tx=0x7f5348009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 -- 172.21.15.7:0/2801051007 shutdown_connections 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 --2- 172.21.15.7:0/2801051007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 0x7f5358106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 --2- 172.21.15.7:0/2801051007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 0x7f53580fe250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 -- 172.21.15.7:0/2801051007 >> 172.21.15.7:0/2801051007 conn(0x7f53580f7870 msgr2=0x7f53580f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 -- 172.21.15.7:0/2801051007 shutdown_connections 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 -- 172.21.15.7:0/2801051007 wait complete. 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 Processor -- start 2024-03-16T09:05:49.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.889+0000 7f535cd50700 1 -- start start 2024-03-16T09:05:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535cd50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 0x7f5358113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535cd50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 0x7f5358113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535cd50700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358114490 con 0x7f53580fde80 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535cd50700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535810da90 con 0x7f53580fe820 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 0x7f5358113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 0x7f5358113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55964/0 (socket says 172.21.15.7:55964) 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535659c700 1 -- 172.21.15.7:0/2304043907 learned_addr learned my addr 172.21.15.7:0/2304043907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f5355d9b700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 0x7f5358113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f5355d9b700 1 -- 172.21.15.7:0/2304043907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 msgr2=0x7f5358113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f5355d9b700 1 -- 172.21.15.7:0/2304043907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 msgr2=0x7f5358113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f5355d9b700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 0x7f5358113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f5355d9b700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 0x7f5358113e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535659c700 1 -- 172.21.15.7:0/2304043907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 msgr2=0x7f5358113e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535659c700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 0x7f5358113e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:49.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535659c700 1 -- 172.21.15.7:0/2304043907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5348005040 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535659c700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 0x7f5358113900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5348009380 tx=0x7f5348009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f5346ffd700 1 -- 172.21.15.7:0/2304043907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f534800e040 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f5346ffd700 1 -- 172.21.15.7:0/2304043907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5348009af0 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535810dd70 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f5346ffd700 1 -- 172.21.15.7:0/2304043907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f534801c630 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.890+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535810e2b0 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.891+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5358066b10 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.892+0000 7f5346ffd700 1 -- 172.21.15.7:0/2304043907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f534800bbc0 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.892+0000 7f5346ffd700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f534006e060 0x7f5340070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.892+0000 7f5346ffd700 1 -- 172.21.15.7:0/2304043907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f534808ecf0 con 0x7f53580fde80 2024-03-16T09:05:49.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.892+0000 7f5355d9b700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f534006e060 0x7f5340070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:49.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.893+0000 7f5355d9b700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f534006e060 0x7f5340070510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f535810ec50 tx=0x7f534c009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:49.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:49.897+0000 7f5346ffd700 1 -- 172.21.15.7:0/2304043907 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f534805baf0 con 0x7f53580fde80 2024-03-16T09:05:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:49 smithi007 ceph-mon[113546]: pgmap v12: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-16T09:05:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:49 smithi007 ceph-mon[113546]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:49.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:49 smithi007 ceph-mon[113546]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:50.078 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:49 smithi019 ceph-mon[83366]: pgmap v12: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-16T09:05:50.078 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:49 smithi019 ceph-mon[83366]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:50.078 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:49 smithi019 ceph-mon[83366]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:50.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.150+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5358109e60 con 0x7f53580fde80 2024-03-16T09:05:50.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.151+0000 7f5346ffd700 1 -- 172.21.15.7:0/2304043907 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1895 (secure 0 0 0) 0x7f534805b680 con 0x7f53580fde80 2024-03-16T09:05:50.153 INFO:teuthology.orchestra.run.smithi007.stdout:e20 2024-03-16T09:05:50.153 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:05:50.153 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:05:50.153 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:05:50.154 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:05:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:50.156 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:05:50.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f534006e060 msgr2=0x7f5340070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:50.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f534006e060 0x7f5340070510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f535810ec50 tx=0x7f534c009f70 comp rx=0 tx=0).stop 2024-03-16T09:05:50.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 msgr2=0x7f5358113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:50.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 0x7f5358113900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5348009380 tx=0x7f5348009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:50.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 shutdown_connections 2024-03-16T09:05:50.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f534006e060 0x7f5340070510 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53580fde80 0x7f5358113900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 --2- 172.21.15.7:0/2304043907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580fe820 0x7f5358113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 >> 172.21.15.7:0/2304043907 conn(0x7f53580f7870 msgr2=0x7f53580f8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:50.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 shutdown_connections 2024-03-16T09:05:50.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.153+0000 7f535cd50700 1 -- 172.21.15.7:0/2304043907 wait complete. 2024-03-16T09:05:50.158 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 -- 172.21.15.7:0/554679486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438101d80 msgr2=0x7fd438102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 --2- 172.21.15.7:0/554679486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438101d80 0x7fd438102150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd430004940 tx=0x7fd430009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 -- 172.21.15.7:0/554679486 shutdown_connections 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 --2- 172.21.15.7:0/554679486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438102720 0x7fd4380fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 --2- 172.21.15.7:0/554679486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438101d80 0x7fd438102150 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 -- 172.21.15.7:0/554679486 >> 172.21.15.7:0/554679486 conn(0x7fd4380f78f0 msgr2=0x7fd4380f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 -- 172.21.15.7:0/554679486 shutdown_connections 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 -- 172.21.15.7:0/554679486 wait complete. 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 Processor -- start 2024-03-16T09:05:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.256+0000 7fd440778700 1 -- start start 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd440778700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 0x7fd438113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd440778700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438102720 0x7fd438113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd440778700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4381144b0 con 0x7fd438102720 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd440778700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43810dab0 con 0x7fd438101d80 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43dd13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438102720 0x7fd438113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43dd13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438102720 0x7fd438113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55986/0 (socket says 172.21.15.7:55986) 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43dd13700 1 -- 172.21.15.7:0/2481483743 learned_addr learned my addr 172.21.15.7:0/2481483743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43e514700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 0x7fd438113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43e514700 1 -- 172.21.15.7:0/2481483743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 msgr2=0x7fd438113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43e514700 1 -- 172.21.15.7:0/2481483743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 msgr2=0x7fd438113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43e514700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 0x7fd438113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-16T09:05:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43e514700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 0x7fd438113920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43dd13700 1 -- 172.21.15.7:0/2481483743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 msgr2=0x7fd438113920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43dd13700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 0x7fd438113920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43dd13700 1 -- 172.21.15.7:0/2481483743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd430005040 con 0x7fd438102720 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.257+0000 7fd43dd13700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438102720 0x7fd438113e60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00c800 tx=0x7fd42c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.258+0000 7fd436ffd700 1 -- 172.21.15.7:0/2481483743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd42c00a760 con 0x7fd438102720 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.258+0000 7fd436ffd700 1 -- 172.21.15.7:0/2481483743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd42c00ad90 con 0x7fd438102720 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.258+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43810dd90 con 0x7fd438102720 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.258+0000 7fd436ffd700 1 -- 172.21.15.7:0/2481483743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd42c0113a0 con 0x7fd438102720 2024-03-16T09:05:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.258+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43810e350 con 0x7fd438102720 2024-03-16T09:05:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.259+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4380fd4d0 con 0x7fd438102720 2024-03-16T09:05:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.259+0000 7fd436ffd700 1 -- 172.21.15.7:0/2481483743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fd42c012400 con 0x7fd438102720 2024-03-16T09:05:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.259+0000 7fd436ffd700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd42006e010 0x7fd4200704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.259+0000 7fd436ffd700 1 -- 172.21.15.7:0/2481483743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd42c08d7b0 con 0x7fd438102720 2024-03-16T09:05:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.260+0000 7fd43e514700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd42006e010 0x7fd4200704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.260+0000 7fd43e514700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd42006e010 0x7fd4200704c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd430004f90 tx=0x7fd430009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:50.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.264+0000 7fd436ffd700 1 -- 172.21.15.7:0/2481483743 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd42c05a780 con 0x7fd438102720 2024-03-16T09:05:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.488+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd438108e80 con 0x7fd42006e010 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.490+0000 7fd436ffd700 1 -- 172.21.15.7:0/2481483743 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fd438108e80 con 0x7fd42006e010 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr" 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "3/21 daemons upgraded", 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mon daemons", 2024-03-16T09:05:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:05:50.493 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:05:50.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd42006e010 msgr2=0x7fd4200704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:50.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd42006e010 0x7fd4200704c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd430004f90 tx=0x7fd430009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:50.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438102720 msgr2=0x7fd438113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:50.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438102720 0x7fd438113e60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00c800 tx=0x7fd42c004470 comp rx=0 tx=0).stop 2024-03-16T09:05:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 shutdown_connections 2024-03-16T09:05:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd438101d80 0x7fd438113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd42006e010 0x7fd4200704c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 --2- 172.21.15.7:0/2481483743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438102720 0x7fd438113e60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 >> 172.21.15.7:0/2481483743 conn(0x7fd4380f78f0 msgr2=0x7fd438066550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 shutdown_connections 2024-03-16T09:05:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.492+0000 7fd440778700 1 -- 172.21.15.7:0/2481483743 wait complete. 2024-03-16T09:05:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.595+0000 7f155947f700 1 -- 172.21.15.7:0/4140523970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 msgr2=0x7f15540fb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.595+0000 7f155947f700 1 --2- 172.21.15.7:0/4140523970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 0x7f15540fb820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f153c004940 tx=0x7f153c009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.595+0000 7f155947f700 1 -- 172.21.15.7:0/4140523970 shutdown_connections 2024-03-16T09:05:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.595+0000 7f155947f700 1 --2- 172.21.15.7:0/4140523970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 0x7f15540fb820 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.595+0000 7f155947f700 1 --2- 172.21.15.7:0/4140523970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 0x7f1554103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.595+0000 7f155947f700 1 -- 172.21.15.7:0/4140523970 >> 172.21.15.7:0/4140523970 conn(0x7f1554072310 msgr2=0x7f1554072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.595+0000 7f155947f700 1 -- 172.21.15.7:0/4140523970 shutdown_connections 2024-03-16T09:05:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.595+0000 7f155947f700 1 -- 172.21.15.7:0/4140523970 wait complete. 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f155947f700 1 Processor -- start 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f155947f700 1 -- start start 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f155947f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 0x7f1554113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f155947f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 0x7f1554113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f155947f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15541144d0 con 0x7f15540fb3b0 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f155947f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155410dad0 con 0x7f1554102f70 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f1552ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 0x7f1554113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f1552ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 0x7f1554113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56002/0 (socket says 172.21.15.7:56002) 2024-03-16T09:05:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f1552ffd700 1 -- 172.21.15.7:0/731957415 learned_addr learned my addr 172.21.15.7:0/731957415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f15527fc700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 0x7f1554113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f15527fc700 1 -- 172.21.15.7:0/731957415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 msgr2=0x7f1554113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f15527fc700 1 -- 172.21.15.7:0/731957415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 msgr2=0x7f1554113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f15527fc700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 0x7f1554113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f15527fc700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 0x7f1554113e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f1552ffd700 1 -- 172.21.15.7:0/731957415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 msgr2=0x7f1554113e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f1552ffd700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 0x7f1554113e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.596+0000 7f1552ffd700 1 -- 172.21.15.7:0/731957415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c005040 con 0x7f15540fb3b0 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.597+0000 7f1552ffd700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 0x7f1554113940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f154c0041d0 tx=0x7f154c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.597+0000 7f154b7fe700 1 -- 172.21.15.7:0/731957415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154c007cc0 con 0x7f15540fb3b0 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.597+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155410ddb0 con 0x7f15540fb3b0 2024-03-16T09:05:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.597+0000 7f154b7fe700 1 -- 172.21.15.7:0/731957415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f154c00f040 con 0x7f15540fb3b0 2024-03-16T09:05:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.597+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155410e370 con 0x7f15540fb3b0 2024-03-16T09:05:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.597+0000 7f154b7fe700 1 -- 172.21.15.7:0/731957415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154c00dbe0 con 0x7f15540fb3b0 2024-03-16T09:05:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.598+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15340052b0 con 0x7f15540fb3b0 2024-03-16T09:05:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.598+0000 7f154b7fe700 1 -- 172.21.15.7:0/731957415 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f154c00dd40 con 0x7f15540fb3b0 2024-03-16T09:05:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.599+0000 7f154b7fe700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f154006e010 0x7f15400704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.599+0000 7f154b7fe700 1 -- 172.21.15.7:0/731957415 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f154c08d070 con 0x7f15540fb3b0 2024-03-16T09:05:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.599+0000 7f15527fc700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f154006e010 0x7f15400704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.599+0000 7f15527fc700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f154006e010 0x7f15400704c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f155410ec90 tx=0x7f153c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.604+0000 7f154b7fe700 1 -- 172.21.15.7:0/731957415 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f154c059e90 con 0x7f15540fb3b0 2024-03-16T09:05:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.896+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1534005830 con 0x7f15540fb3b0 2024-03-16T09:05:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.897+0000 7f154b7fe700 1 -- 172.21.15.7:0/731957415 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f154c059a20 con 0x7f15540fb3b0 2024-03-16T09:05:50.900 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_OK 2024-03-16T09:05:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.898+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f154006e010 msgr2=0x7f15400704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f154006e010 0x7f15400704c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f155410ec90 tx=0x7f153c009e30 comp rx=0 tx=0).stop 2024-03-16T09:05:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 msgr2=0x7f1554113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 0x7f1554113940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f154c0041d0 tx=0x7f154c0046b0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 shutdown_connections 2024-03-16T09:05:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f154006e010 0x7f15400704c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15540fb3b0 0x7f1554113940 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 --2- 172.21.15.7:0/731957415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1554102f70 0x7f1554113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 >> 172.21.15.7:0/731957415 conn(0x7f1554072310 msgr2=0x7f15540fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 shutdown_connections 2024-03-16T09:05:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:05:50.899+0000 7f155947f700 1 -- 172.21.15.7:0/731957415 wait complete. 2024-03-16T09:05:50.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:50 smithi007 ceph-mon[113546]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:50.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:50 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1831371570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:05:50.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:50 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2304043907' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:05:50.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:50 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:50.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:50 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:51.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:50 smithi019 ceph-mon[83366]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:51.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:50 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1831371570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:05:51.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:50 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2304043907' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:05:51.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:50 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:51.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:50 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:51 smithi007 ceph-mon[113546]: pgmap v13: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:51 smithi007 ceph-mon[113546]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:51 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/731957415' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:05:51.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:51 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:51.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:51 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:52.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:51 smithi019 ceph-mon[83366]: pgmap v13: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:51 smithi019 ceph-mon[83366]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:05:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:51 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/731957415' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:05:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:51 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:51 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:54.062 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:53 smithi019 ceph-mon[83366]: pgmap v14: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:54.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:53 smithi007 ceph-mon[113546]: pgmap v14: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: pgmap v15: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:05:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:56.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:05:56.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:05:56.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: pgmap v15: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:05:56.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:56.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:05:56.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:05:56.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:57.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:56 smithi007 ceph-mon[113546]: Detected new or changed devices on smithi019 2024-03-16T09:05:57.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:56 smithi007 ceph-mon[113546]: Reconfiguring mon.smithi007 (monmap changed)... 2024-03-16T09:05:57.132 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:56 smithi007 ceph-mon[113546]: Reconfiguring daemon mon.smithi007 on smithi007 2024-03-16T09:05:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:56 smithi019 ceph-mon[83366]: Detected new or changed devices on smithi019 2024-03-16T09:05:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:56 smithi019 ceph-mon[83366]: Reconfiguring mon.smithi007 (monmap changed)... 2024-03-16T09:05:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:56 smithi019 ceph-mon[83366]: Reconfiguring daemon mon.smithi007 on smithi007 2024-03-16T09:05:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:58 smithi019 ceph-mon[83366]: pgmap v16: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:58 smithi019 ceph-mon[83366]: Reconfiguring mgr.smithi007.uamwtp (monmap changed)... 2024-03-16T09:05:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.uamwtp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:05:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:05:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:58 smithi019 ceph-mon[83366]: Reconfiguring daemon mgr.smithi007.uamwtp on smithi007 2024-03-16T09:05:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:58 smithi007 ceph-mon[113546]: pgmap v16: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:05:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:05:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:58 smithi007 ceph-mon[113546]: Reconfiguring mgr.smithi007.uamwtp (monmap changed)... 2024-03-16T09:05:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi007.uamwtp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:05:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:05:58.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:05:58.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:58 smithi007 ceph-mon[113546]: Reconfiguring daemon mgr.smithi007.uamwtp on smithi007 2024-03-16T09:06:00.171 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:59 smithi007 ceph-mon[113546]: pgmap v17: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:00.171 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:59 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:00.172 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:59 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:00.172 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:59 smithi007 ceph-mon[113546]: Reconfiguring crash.smithi007 (monmap changed)... 2024-03-16T09:06:00.172 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:59 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T09:06:00.172 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:59 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:00.172 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:05:59 smithi007 ceph-mon[113546]: Reconfiguring daemon crash.smithi007 on smithi007 2024-03-16T09:06:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:59 smithi019 ceph-mon[83366]: pgmap v17: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:59 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:59 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:59 smithi019 ceph-mon[83366]: Reconfiguring crash.smithi007 (monmap changed)... 2024-03-16T09:06:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:59 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T09:06:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:59 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:05:59 smithi019 ceph-mon[83366]: Reconfiguring daemon crash.smithi007 on smithi007 2024-03-16T09:06:01.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:01 smithi007 ceph-mon[113546]: pgmap v18: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:01.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:01.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:01.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:06:01.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:01 smithi007 ceph-mon[113546]: Reconfiguring osd.0 (monmap changed)... 2024-03-16T09:06:01.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:06:01.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:01.708 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:01 smithi007 ceph-mon[113546]: Reconfiguring daemon osd.0 on smithi007 2024-03-16T09:06:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:01 smithi019 ceph-mon[83366]: pgmap v18: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:06:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:01 smithi019 ceph-mon[83366]: Reconfiguring osd.0 (monmap changed)... 2024-03-16T09:06:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:06:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:01 smithi019 ceph-mon[83366]: Reconfiguring daemon osd.0 on smithi007 2024-03-16T09:06:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:03 smithi019 ceph-mon[83366]: pgmap v19: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:03 smithi019 ceph-mon[83366]: Reconfiguring osd.1 (monmap changed)... 2024-03-16T09:06:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T09:06:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:03.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:03 smithi019 ceph-mon[83366]: Reconfiguring daemon osd.1 on smithi007 2024-03-16T09:06:03.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:03 smithi007 ceph-mon[113546]: pgmap v19: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:03.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:03.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:03.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:03 smithi007 ceph-mon[113546]: Reconfiguring osd.1 (monmap changed)... 2024-03-16T09:06:03.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T09:06:03.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:03.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:03 smithi007 ceph-mon[113546]: Reconfiguring daemon osd.1 on smithi007 2024-03-16T09:06:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:05 smithi007 ceph-mon[113546]: Reconfiguring osd.2 (monmap changed)... 2024-03-16T09:06:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T09:06:05.445 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:05.446 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:05 smithi007 ceph-mon[113546]: Reconfiguring daemon osd.2 on smithi007 2024-03-16T09:06:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:05 smithi019 ceph-mon[83366]: Reconfiguring osd.2 (monmap changed)... 2024-03-16T09:06:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T09:06:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:05 smithi019 ceph-mon[83366]: Reconfiguring daemon osd.2 on smithi007 2024-03-16T09:06:06.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:06 smithi007 ceph-mon[113546]: pgmap v20: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:06.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:06.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:06.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.bsxctp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:06:06.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:06 smithi019 ceph-mon[83366]: pgmap v20: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.bsxctp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:06:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:07.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:07 smithi007 ceph-mon[113546]: Reconfiguring mds.cephfs.smithi007.bsxctp (monmap changed)... 2024-03-16T09:06:07.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:07 smithi007 ceph-mon[113546]: Reconfiguring daemon mds.cephfs.smithi007.bsxctp on smithi007 2024-03-16T09:06:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:07 smithi019 ceph-mon[83366]: Reconfiguring mds.cephfs.smithi007.bsxctp (monmap changed)... 2024-03-16T09:06:07.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:07 smithi019 ceph-mon[83366]: Reconfiguring daemon mds.cephfs.smithi007.bsxctp on smithi007 2024-03-16T09:06:08.406 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:08 smithi007 ceph-mon[113546]: pgmap v21: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:08.406 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:08 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:08.406 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:08 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:08.406 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:08 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.znurst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:06:08.406 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:08 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:08.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:08 smithi019 ceph-mon[83366]: pgmap v21: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:08.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:08 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:08.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:08 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:08.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:08 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.znurst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:06:08.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:08 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:09.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:09 smithi007 ceph-mon[113546]: Reconfiguring mds.cephfs.smithi007.znurst (monmap changed)... 2024-03-16T09:06:09.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:09 smithi007 ceph-mon[113546]: Reconfiguring daemon mds.cephfs.smithi007.znurst on smithi007 2024-03-16T09:06:09.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:09 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:09.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:09 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:09.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:09 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T09:06:09.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:09 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:09.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:09 smithi019 ceph-mon[83366]: Reconfiguring mds.cephfs.smithi007.znurst (monmap changed)... 2024-03-16T09:06:09.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:09 smithi019 ceph-mon[83366]: Reconfiguring daemon mds.cephfs.smithi007.znurst on smithi007 2024-03-16T09:06:09.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:09 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:09.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:09 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:09.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:09 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T09:06:09.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:09 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:10.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:10 smithi019 ceph-mon[83366]: pgmap v22: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:10.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:10 smithi019 ceph-mon[83366]: Reconfiguring crash.smithi019 (monmap changed)... 2024-03-16T09:06:10.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:10 smithi019 ceph-mon[83366]: Reconfiguring daemon crash.smithi019 on smithi019 2024-03-16T09:06:10.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:10 smithi007 ceph-mon[113546]: pgmap v22: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:10.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:10 smithi007 ceph-mon[113546]: Reconfiguring crash.smithi019 (monmap changed)... 2024-03-16T09:06:10.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:10 smithi007 ceph-mon[113546]: Reconfiguring daemon crash.smithi019 on smithi019 2024-03-16T09:06:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:11 smithi007 ceph-mon[113546]: pgmap v23: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:11 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:11 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:11 smithi007 ceph-mon[113546]: Reconfiguring mgr.smithi019.iuqxhi (monmap changed)... 2024-03-16T09:06:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:11 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:06:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:11 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:06:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:11 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:11 smithi007 ceph-mon[113546]: Reconfiguring daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T09:06:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:11 smithi019 ceph-mon[83366]: pgmap v23: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:11 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:11 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:11 smithi019 ceph-mon[83366]: Reconfiguring mgr.smithi019.iuqxhi (monmap changed)... 2024-03-16T09:06:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:11 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi019.iuqxhi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:06:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:11 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:06:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:11 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:11 smithi019 ceph-mon[83366]: Reconfiguring daemon mgr.smithi019.iuqxhi on smithi019 2024-03-16T09:06:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:13 smithi019 ceph-mon[83366]: pgmap v24: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:13 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:13 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:13 smithi019 ceph-mon[83366]: Reconfiguring mon.smithi019 (monmap changed)... 2024-03-16T09:06:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:13 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:06:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:13 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:06:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:13 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:13 smithi019 ceph-mon[83366]: Reconfiguring daemon mon.smithi019 on smithi019 2024-03-16T09:06:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:13 smithi007 ceph-mon[113546]: pgmap v24: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:13 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:13 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:13.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:13 smithi007 ceph-mon[113546]: Reconfiguring mon.smithi019 (monmap changed)... 2024-03-16T09:06:13.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:13 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:06:13.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:13 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:06:13.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:13 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:13.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:13 smithi007 ceph-mon[113546]: Reconfiguring daemon mon.smithi019 on smithi019 2024-03-16T09:06:15.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:15.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:15.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:15 smithi019 ceph-mon[83366]: Reconfiguring osd.3 (monmap changed)... 2024-03-16T09:06:15.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T09:06:15.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:15.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:15 smithi019 ceph-mon[83366]: Reconfiguring daemon osd.3 on smithi019 2024-03-16T09:06:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:15 smithi007 ceph-mon[113546]: Reconfiguring osd.3 (monmap changed)... 2024-03-16T09:06:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T09:06:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:15.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:15 smithi007 ceph-mon[113546]: Reconfiguring daemon osd.3 on smithi019 2024-03-16T09:06:16.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:16 smithi019 ceph-mon[83366]: pgmap v25: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:16 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:06:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:16 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:16 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:16 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T09:06:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:16 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:16 smithi007 ceph-mon[113546]: pgmap v25: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:16 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:06:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:16 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:16 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:16 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T09:06:16.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:16 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:17.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:17 smithi007 ceph-mon[113546]: Reconfiguring osd.4 (monmap changed)... 2024-03-16T09:06:17.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:17 smithi007 ceph-mon[113546]: Reconfiguring daemon osd.4 on smithi019 2024-03-16T09:06:17.541 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:17 smithi019 ceph-mon[83366]: Reconfiguring osd.4 (monmap changed)... 2024-03-16T09:06:17.541 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:17 smithi019 ceph-mon[83366]: Reconfiguring daemon osd.4 on smithi019 2024-03-16T09:06:18.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:18 smithi007 ceph-mon[113546]: pgmap v26: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:18.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:18 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:18.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:18 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:18.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:18 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T09:06:18.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:18 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:18.510 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:18 smithi019 ceph-mon[83366]: pgmap v26: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:18.510 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:18 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:18.510 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:18 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:18.510 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:18 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T09:06:18.510 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:18 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:19 smithi007 ceph-mon[113546]: Reconfiguring osd.5 (monmap changed)... 2024-03-16T09:06:19.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:19 smithi007 ceph-mon[113546]: Reconfiguring daemon osd.5 on smithi019 2024-03-16T09:06:19.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:19 smithi019 ceph-mon[83366]: Reconfiguring osd.5 (monmap changed)... 2024-03-16T09:06:19.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:19 smithi019 ceph-mon[83366]: Reconfiguring daemon osd.5 on smithi019 2024-03-16T09:06:20.275 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:20 smithi019 ceph-mon[83366]: pgmap v27: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:20.276 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:20 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:20.276 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:20 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:20.276 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:20 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.bncsil", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:06:20.276 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:20 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:20.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:20 smithi007 ceph-mon[113546]: pgmap v27: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:20.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:20 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:20.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:20 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:20.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:20 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.bncsil", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:06:20.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:20 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.017+0000 7f7eb22fb700 1 -- 172.21.15.7:0/1758113259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 msgr2=0x7f7eac1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.017+0000 7f7eb22fb700 1 --2- 172.21.15.7:0/1758113259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 0x7f7eac1031d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c004940 tx=0x7f7e9c009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.017+0000 7f7eb22fb700 1 -- 172.21.15.7:0/1758113259 shutdown_connections 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.017+0000 7f7eb22fb700 1 --2- 172.21.15.7:0/1758113259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac1037a0 0x7f7eac0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.017+0000 7f7eb22fb700 1 --2- 172.21.15.7:0/1758113259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 0x7f7eac1031d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.017+0000 7f7eb22fb700 1 -- 172.21.15.7:0/1758113259 >> 172.21.15.7:0/1758113259 conn(0x7f7eac0f89b0 msgr2=0x7f7eac0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.017+0000 7f7eb22fb700 1 -- 172.21.15.7:0/1758113259 shutdown_connections 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.017+0000 7f7eb22fb700 1 -- 172.21.15.7:0/1758113259 wait complete. 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.018+0000 7f7eb22fb700 1 Processor -- start 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.018+0000 7f7eb22fb700 1 -- start start 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.018+0000 7f7eb22fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 0x7f7eac0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.018+0000 7f7eb22fb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac1037a0 0x7f7eac070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.018+0000 7f7eb22fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0721c0 con 0x7f7eac102e00 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.018+0000 7f7eb22fb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac070d40 con 0x7f7eac1037a0 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.018+0000 7f7eabfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 0x7f7eac0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eabfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 0x7f7eac0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38622/0 (socket says 172.21.15.7:38622) 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eabfff700 1 -- 172.21.15.7:0/2599306384 learned_addr learned my addr 172.21.15.7:0/2599306384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eab7fe700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac1037a0 0x7f7eac070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eabfff700 1 -- 172.21.15.7:0/2599306384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac1037a0 msgr2=0x7f7eac070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eabfff700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac1037a0 0x7f7eac070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eabfff700 1 -- 172.21.15.7:0/2599306384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c005040 con 0x7f7eac102e00 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eabfff700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 0x7f7eac0702c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009b30 tx=0x7f7e9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7ea8ff9700 1 -- 172.21.15.7:0/2599306384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e9c021070 con 0x7f7eac102e00 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac070fc0 con 0x7f7eac102e00 2024-03-16T09:06:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7ea8ff9700 1 -- 172.21.15.7:0/2599306384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e9c013c00 con 0x7f7eac102e00 2024-03-16T09:06:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7ea8ff9700 1 -- 172.21.15.7:0/2599306384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e9c00e070 con 0x7f7eac102e00 2024-03-16T09:06:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.019+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac11a010 con 0x7f7eac102e00 2024-03-16T09:06:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.020+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eac100110 con 0x7f7eac102e00 2024-03-16T09:06:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.021+0000 7f7ea8ff9700 1 -- 172.21.15.7:0/2599306384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f7e9c01c400 con 0x7f7eac102e00 2024-03-16T09:06:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.021+0000 7f7ea8ff9700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7e9806e010 0x7f7e980704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.021+0000 7f7ea8ff9700 1 -- 172.21.15.7:0/2599306384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7e9c08ed80 con 0x7f7eac102e00 2024-03-16T09:06:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.021+0000 7f7eab7fe700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7e9806e010 0x7f7e980704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.022+0000 7f7eab7fe700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7e9806e010 0x7f7e980704c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0719a0 tx=0x7f7e9400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.026+0000 7f7ea8ff9700 1 -- 172.21.15.7:0/2599306384 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e9c0580f0 con 0x7f7eac102e00 2024-03-16T09:06:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.262+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7eac109fa0 con 0x7f7e9806e010 2024-03-16T09:06:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.264+0000 7f7ea8ff9700 1 -- 172.21.15.7:0/2599306384 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f7e9400f040 con 0x7f7e9806e010 2024-03-16T09:06:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7e9806e010 msgr2=0x7f7e980704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7e9806e010 0x7f7e980704c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0719a0 tx=0x7f7e9400b040 comp rx=0 tx=0).stop 2024-03-16T09:06:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 msgr2=0x7f7eac0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 0x7f7eac0702c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009b30 tx=0x7f7e9c009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 shutdown_connections 2024-03-16T09:06:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7e9806e010 0x7f7e980704c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eac102e00 0x7f7eac0702c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 --2- 172.21.15.7:0/2599306384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac1037a0 0x7f7eac070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 >> 172.21.15.7:0/2599306384 conn(0x7f7eac0f89b0 msgr2=0x7f7eac0fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 shutdown_connections 2024-03-16T09:06:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.266+0000 7f7eb22fb700 1 -- 172.21.15.7:0/2599306384 wait complete. 2024-03-16T09:06:21.279 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:06:21.279 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:21 smithi007 ceph-mon[113546]: Reconfiguring mds.cephfs.smithi019.bncsil (monmap changed)... 2024-03-16T09:06:21.279 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:21 smithi007 ceph-mon[113546]: Reconfiguring daemon mds.cephfs.smithi019.bncsil on smithi019 2024-03-16T09:06:21.279 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:21 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:21.279 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:21 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:21.280 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:21 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.gmhzgo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:06:21.280 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:21 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 -- 172.21.15.7:0/1966548195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 msgr2=0x7f5f38102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 --2- 172.21.15.7:0/1966548195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 0x7f5f38102140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f28004940 tx=0x7f5f28009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 -- 172.21.15.7:0/1966548195 shutdown_connections 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 --2- 172.21.15.7:0/1966548195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f38102710 0x7f5f380fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 --2- 172.21.15.7:0/1966548195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 0x7f5f38102140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 -- 172.21.15.7:0/1966548195 >> 172.21.15.7:0/1966548195 conn(0x7f5f38072bc0 msgr2=0x7f5f38072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 -- 172.21.15.7:0/1966548195 shutdown_connections 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 -- 172.21.15.7:0/1966548195 wait complete. 2024-03-16T09:06:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.370+0000 7f5f3f224700 1 Processor -- start 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3f224700 1 -- start start 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3f224700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 0x7f5f38102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3f224700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f38102710 0x7f5f38103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3f224700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381036a0 con 0x7f5f38101d70 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3f224700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381037e0 con 0x7f5f38102710 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3cfc0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 0x7f5f38102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3cfc0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 0x7f5f38102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38628/0 (socket says 172.21.15.7:38628) 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3cfc0700 1 -- 172.21.15.7:0/2112748940 learned_addr learned my addr 172.21.15.7:0/2112748940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f37fff700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f38102710 0x7f5f38103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f37fff700 1 -- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 msgr2=0x7f5f38102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f37fff700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 0x7f5f38102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f37fff700 1 -- 172.21.15.7:0/2112748940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28005040 con 0x7f5f38102710 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.371+0000 7f5f3cfc0700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 0x7f5f38102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.372+0000 7f5f37fff700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f38102710 0x7f5f38103110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f38070b60 tx=0x7f5f2c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.372+0000 7f5f357fa700 1 -- 172.21.15.7:0/2112748940 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2c004e70 con 0x7f5f38102710 2024-03-16T09:06:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.372+0000 7f5f357fa700 1 -- 172.21.15.7:0/2112748940 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f2c00f040 con 0x7f5f38102710 2024-03-16T09:06:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.372+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f38103a30 con 0x7f5f38102710 2024-03-16T09:06:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.372+0000 7f5f357fa700 1 -- 172.21.15.7:0/2112748940 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f2c00c950 con 0x7f5f38102710 2024-03-16T09:06:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.372+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3811a1c0 con 0x7f5f38102710 2024-03-16T09:06:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.373+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f380fd4c0 con 0x7f5f38102710 2024-03-16T09:06:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.373+0000 7f5f357fa700 1 -- 172.21.15.7:0/2112748940 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f5f2c0090d0 con 0x7f5f38102710 2024-03-16T09:06:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.374+0000 7f5f357fa700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5f2006e000 0x7f5f200704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.374+0000 7f5f3cfc0700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5f2006e000 0x7f5f200704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.374+0000 7f5f357fa700 1 -- 172.21.15.7:0/2112748940 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5f2c08d480 con 0x7f5f38102710 2024-03-16T09:06:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.374+0000 7f5f3cfc0700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5f2006e000 0x7f5f200704b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800b0d0 tx=0x7f5f28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.378+0000 7f5f357fa700 1 -- 172.21.15.7:0/2112748940 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5f2c056750 con 0x7f5f38102710 2024-03-16T09:06:21.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:21 smithi019 ceph-mon[83366]: Reconfiguring mds.cephfs.smithi019.bncsil (monmap changed)... 2024-03-16T09:06:21.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:21 smithi019 ceph-mon[83366]: Reconfiguring daemon mds.cephfs.smithi019.bncsil on smithi019 2024-03-16T09:06:21.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:21 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:21.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:21 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:21.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:21 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.gmhzgo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:06:21.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:21 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:21.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.608+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f38108f30 con 0x7f5f2006e000 2024-03-16T09:06:21.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.609+0000 7f5f357fa700 1 -- 172.21.15.7:0/2112748940 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f5f280093c0 con 0x7f5f2006e000 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.611+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5f2006e000 msgr2=0x7f5f200704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.611+0000 7f5f3f224700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5f2006e000 0x7f5f200704b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800b0d0 tx=0x7f5f28009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.611+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f38102710 msgr2=0x7f5f38103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.611+0000 7f5f3f224700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f38102710 0x7f5f38103110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f38070b60 tx=0x7f5f2c00b040 comp rx=0 tx=0).stop 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.611+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 shutdown_connections 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.611+0000 7f5f3f224700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5f2006e000 0x7f5f200704b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.611+0000 7f5f3f224700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f38101d70 0x7f5f38102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.612+0000 7f5f3f224700 1 --2- 172.21.15.7:0/2112748940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f38102710 0x7f5f38103110 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.612+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 >> 172.21.15.7:0/2112748940 conn(0x7f5f38072bc0 msgr2=0x7f5f380658d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.612+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 shutdown_connections 2024-03-16T09:06:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.612+0000 7f5f3f224700 1 -- 172.21.15.7:0/2112748940 wait complete. 2024-03-16T09:06:21.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.715+0000 7f1d3e900700 1 -- 172.21.15.7:0/2859053231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38102e10 msgr2=0x7f1d381031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.715+0000 7f1d3e900700 1 --2- 172.21.15.7:0/2859053231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38102e10 0x7f1d381031e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d28004940 tx=0x7f1d28009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:21.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.715+0000 7f1d3e900700 1 -- 172.21.15.7:0/2859053231 shutdown_connections 2024-03-16T09:06:21.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.715+0000 7f1d3e900700 1 --2- 172.21.15.7:0/2859053231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d381037b0 0x7f1d380fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.715+0000 7f1d3e900700 1 --2- 172.21.15.7:0/2859053231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38102e10 0x7f1d381031e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.715+0000 7f1d3e900700 1 -- 172.21.15.7:0/2859053231 >> 172.21.15.7:0/2859053231 conn(0x7f1d380f8980 msgr2=0x7f1d380fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:21.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.715+0000 7f1d3e900700 1 -- 172.21.15.7:0/2859053231 shutdown_connections 2024-03-16T09:06:21.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.716+0000 7f1d3e900700 1 -- 172.21.15.7:0/2859053231 wait complete. 2024-03-16T09:06:21.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.716+0000 7f1d3e900700 1 Processor -- start 2024-03-16T09:06:21.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.716+0000 7f1d3e900700 1 -- start start 2024-03-16T09:06:21.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.716+0000 7f1d3e900700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38102e10 0x7f1d380702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.716+0000 7f1d3e900700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d381037b0 0x7f1d38070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.716+0000 7f1d3e900700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d38070d90 con 0x7f1d38102e10 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.716+0000 7f1d3e900700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d38070ed0 con 0x7f1d381037b0 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d381037b0 0x7f1d38070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d377fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d381037b0 0x7f1d38070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:37828/0 (socket says 172.21.15.7:37828) 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d377fe700 1 -- 172.21.15.7:0/2730085479 learned_addr learned my addr 172.21.15.7:0/2730085479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d37fff700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38102e10 0x7f1d380702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d377fe700 1 -- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38102e10 msgr2=0x7f1d380702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d377fe700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38102e10 0x7f1d380702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d377fe700 1 -- 172.21.15.7:0/2730085479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28005040 con 0x7f1d381037b0 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d377fe700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d381037b0 0x7f1d38070800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00eb40 tx=0x7f1d2c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d34ff9700 1 -- 172.21.15.7:0/2730085479 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d2c007cc0 con 0x7f1d381037b0 2024-03-16T09:06:21.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d34ff9700 1 -- 172.21.15.7:0/2730085479 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d2c00f040 con 0x7f1d381037b0 2024-03-16T09:06:21.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d38071120 con 0x7f1d381037b0 2024-03-16T09:06:21.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d34ff9700 1 -- 172.21.15.7:0/2730085479 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d2c00cad0 con 0x7f1d381037b0 2024-03-16T09:06:21.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.717+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3811a1f0 con 0x7f1d381037b0 2024-03-16T09:06:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.719+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d38100120 con 0x7f1d381037b0 2024-03-16T09:06:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.719+0000 7f1d34ff9700 1 -- 172.21.15.7:0/2730085479 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f1d2c00cc30 con 0x7f1d381037b0 2024-03-16T09:06:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.719+0000 7f1d34ff9700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1d2006e000 0x7f1d200704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.719+0000 7f1d37fff700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1d2006e000 0x7f1d200704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.719+0000 7f1d34ff9700 1 -- 172.21.15.7:0/2730085479 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1d2c08d5a0 con 0x7f1d381037b0 2024-03-16T09:06:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.720+0000 7f1d37fff700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1d2006e000 0x7f1d200704b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800b0d0 tx=0x7f1d28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.724+0000 7f1d34ff9700 1 -- 172.21.15.7:0/2730085479 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d2c0567f0 con 0x7f1d381037b0 2024-03-16T09:06:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.940+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d38109f30 con 0x7f1d2006e000 2024-03-16T09:06:21.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.951+0000 7f1d34ff9700 1 -- 172.21.15.7:0/2730085479 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1d38109f30 con 0x7f1d2006e000 2024-03-16T09:06:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:06:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (2m) 45s ago 13m 17.2M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:06:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (13m) 45s ago 13m 6669k - 17.2.7-648-g1df27247 0bdcfccf14ce abb8634d810d 2024-03-16T09:06:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (12m) 31s ago 12m 6665k - 17.2.7-648-g1df27247 0bdcfccf14ce 9714b3112d34 2024-03-16T09:06:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (2m) 45s ago 12m 78.7M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (7m) 45s ago 7m 13.3M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (7m) 45s ago 7m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (7m) 31s ago 7m 120M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (7m) 31s ago 7m 14.0M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (4m) 45s ago 14m 473M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (3m) 31s ago 12m 447M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (60s) 45s ago 14m 31.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (35s) 31s ago 12m 24.9M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (3m) 45s ago 13m 15.6M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (3m) 31s ago 12m 15.4M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (11m) 45s ago 11m 206M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (11m) 45s ago 11m 235M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:06:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (10m) 45s ago 10m 182M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:06:21.955 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (10m) 31s ago 10m 264M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:06:21.955 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (9m) 31s ago 9m 246M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:06:21.955 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (9m) 31s ago 9m 223M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:06:21.955 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (2m) 45s ago 12m 63.5M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:06:21.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1d2006e000 msgr2=0x7f1d200704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1d2006e000 0x7f1d200704b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800b0d0 tx=0x7f1d28009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d381037b0 msgr2=0x7f1d38070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d381037b0 0x7f1d38070800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00eb40 tx=0x7f1d2c00b040 comp rx=0 tx=0).stop 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 shutdown_connections 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1d2006e000 0x7f1d200704b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38102e10 0x7f1d380702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 --2- 172.21.15.7:0/2730085479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d381037b0 0x7f1d38070800 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 >> 172.21.15.7:0/2730085479 conn(0x7f1d380f8980 msgr2=0x7f1d381073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 shutdown_connections 2024-03-16T09:06:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:21.953+0000 7f1d3e900700 1 -- 172.21.15.7:0/2730085479 wait complete. 2024-03-16T09:06:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 -- 172.21.15.7:0/1150630988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 msgr2=0x7f8d480fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 --2- 172.21.15.7:0/1150630988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 0x7f8d480fce10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c004940 tx=0x7f8d3c009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 -- 172.21.15.7:0/1150630988 shutdown_connections 2024-03-16T09:06:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 --2- 172.21.15.7:0/1150630988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 0x7f8d480fce10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 --2- 172.21.15.7:0/1150630988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d48102e00 0x7f8d481031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 -- 172.21.15.7:0/1150630988 >> 172.21.15.7:0/1150630988 conn(0x7f8d480f89b0 msgr2=0x7f8d480fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 -- 172.21.15.7:0/1150630988 shutdown_connections 2024-03-16T09:06:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 -- 172.21.15.7:0/1150630988 wait complete. 2024-03-16T09:06:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 Processor -- start 2024-03-16T09:06:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.055+0000 7f8d5068a700 1 -- start start 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d5068a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d48102e00 0x7f8d480702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d5068a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 0x7f8d48070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d5068a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48072210 con 0x7f8d481037a0 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d5068a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48070d40 con 0x7f8d48102e00 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d4dc25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 0x7f8d48070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d4dc25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 0x7f8d48070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38654/0 (socket says 172.21.15.7:38654) 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d4dc25700 1 -- 172.21.15.7:0/3944615876 learned_addr learned my addr 172.21.15.7:0/3944615876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d4e426700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d48102e00 0x7f8d480702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d4dc25700 1 -- 172.21.15.7:0/3944615876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d48102e00 msgr2=0x7f8d480702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d4dc25700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d48102e00 0x7f8d480702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d4dc25700 1 -- 172.21.15.7:0/3944615876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c005040 con 0x7f8d481037a0 2024-03-16T09:06:22.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d4dc25700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 0x7f8d48070800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009100 tx=0x7f8d3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d46ffd700 1 -- 172.21.15.7:0/3944615876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3c00e040 con 0x7f8d481037a0 2024-03-16T09:06:22.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d46ffd700 1 -- 172.21.15.7:0/3944615876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d3c004290 con 0x7f8d481037a0 2024-03-16T09:06:22.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.056+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d48070fc0 con 0x7f8d481037a0 2024-03-16T09:06:22.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.057+0000 7f8d46ffd700 1 -- 172.21.15.7:0/3944615876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3c01c3f0 con 0x7f8d481037a0 2024-03-16T09:06:22.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.057+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4811a0d0 con 0x7f8d481037a0 2024-03-16T09:06:22.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.057+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d481052d0 con 0x7f8d481037a0 2024-03-16T09:06:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.058+0000 7f8d46ffd700 1 -- 172.21.15.7:0/3944615876 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f8d3c013690 con 0x7f8d481037a0 2024-03-16T09:06:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.058+0000 7f8d46ffd700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8d3006e010 0x7f8d300704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.058+0000 7f8d46ffd700 1 -- 172.21.15.7:0/3944615876 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8d3c08eae0 con 0x7f8d481037a0 2024-03-16T09:06:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.058+0000 7f8d4e426700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8d3006e010 0x7f8d300704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.059+0000 7f8d4e426700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8d3006e010 0x7f8d300704c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d38007a40 tx=0x7f8d38009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.063+0000 7f8d46ffd700 1 -- 172.21.15.7:0/3944615876 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d3c057e50 con 0x7f8d481037a0 2024-03-16T09:06:22.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.358+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d48109ee0 con 0x7f8d481037a0 2024-03-16T09:06:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.359+0000 7f8d46ffd700 1 -- 172.21.15.7:0/3944615876 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f8d3c05b510 con 0x7f8d481037a0 2024-03-16T09:06:22.361 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:06:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:06:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:06:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 10, 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:06:22.362 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:06:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8d3006e010 msgr2=0x7f8d300704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8d3006e010 0x7f8d300704c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d38007a40 tx=0x7f8d38009da0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 msgr2=0x7f8d48070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 0x7f8d48070800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009100 tx=0x7f8d3c009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 shutdown_connections 2024-03-16T09:06:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d48102e00 0x7f8d480702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8d3006e010 0x7f8d300704c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 --2- 172.21.15.7:0/3944615876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d481037a0 0x7f8d48070800 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 >> 172.21.15.7:0/3944615876 conn(0x7f8d480f89b0 msgr2=0x7f8d481021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 shutdown_connections 2024-03-16T09:06:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.361+0000 7f8d5068a700 1 -- 172.21.15.7:0/3944615876 wait complete. 2024-03-16T09:06:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 -- 172.21.15.7:0/3178072286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 msgr2=0x7f4ae80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 --2- 172.21.15.7:0/3178072286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 0x7f4ae80fce10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4adc004940 tx=0x7f4adc009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 -- 172.21.15.7:0/3178072286 shutdown_connections 2024-03-16T09:06:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 --2- 172.21.15.7:0/3178072286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 0x7f4ae80fce10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 --2- 172.21.15.7:0/3178072286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae8102e00 0x7f4ae81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 -- 172.21.15.7:0/3178072286 >> 172.21.15.7:0/3178072286 conn(0x7f4ae80f89b0 msgr2=0x7f4ae80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 -- 172.21.15.7:0/3178072286 shutdown_connections 2024-03-16T09:06:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 -- 172.21.15.7:0/3178072286 wait complete. 2024-03-16T09:06:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 Processor -- start 2024-03-16T09:06:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.464+0000 7f4aee506700 1 -- start start 2024-03-16T09:06:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4aee506700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae8102e00 0x7f4ae8113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4aee506700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 0x7f4ae8113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4aee506700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae8114490 con 0x7f4ae81037a0 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4aee506700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae810da90 con 0x7f4ae8102e00 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 0x7f4ae8113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 0x7f4ae8113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46640/0 (socket says 172.21.15.7:46640) 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae77fe700 1 -- 172.21.15.7:0/2241245532 learned_addr learned my addr 172.21.15.7:0/2241245532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae7fff700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae8102e00 0x7f4ae8113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae77fe700 1 -- 172.21.15.7:0/2241245532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae8102e00 msgr2=0x7f4ae8113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae77fe700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae8102e00 0x7f4ae8113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae77fe700 1 -- 172.21.15.7:0/2241245532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc005040 con 0x7f4ae81037a0 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae77fe700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 0x7f4ae8113e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000c00 tx=0x7f4adc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae4ff9700 1 -- 172.21.15.7:0/2241245532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4adc021070 con 0x7f4ae81037a0 2024-03-16T09:06:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae810dd10 con 0x7f4ae81037a0 2024-03-16T09:06:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.465+0000 7f4ae4ff9700 1 -- 172.21.15.7:0/2241245532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4adc013c00 con 0x7f4ae81037a0 2024-03-16T09:06:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.466+0000 7f4ae4ff9700 1 -- 172.21.15.7:0/2241245532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4adc00e070 con 0x7f4ae81037a0 2024-03-16T09:06:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.466+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae810e1f0 con 0x7f4ae81037a0 2024-03-16T09:06:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.467+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae81052d0 con 0x7f4ae81037a0 2024-03-16T09:06:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.467+0000 7f4ae4ff9700 1 -- 172.21.15.7:0/2241245532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f4adc01ca60 con 0x7f4ae81037a0 2024-03-16T09:06:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.468+0000 7f4ae4ff9700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4ad006e010 0x7f4ad00704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.468+0000 7f4ae7fff700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4ad006e010 0x7f4ad00704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.468+0000 7f4ae4ff9700 1 -- 172.21.15.7:0/2241245532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4adc08ed80 con 0x7f4ae81037a0 2024-03-16T09:06:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.468+0000 7f4ae7fff700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4ad006e010 0x7f4ad00704c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009b80 tx=0x7f4ad800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.472+0000 7f4ae4ff9700 1 -- 172.21.15.7:0/2241245532 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4adc05bc20 con 0x7f4ae81037a0 2024-03-16T09:06:22.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:22 smithi007 ceph-mon[113546]: pgmap v28: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:22.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:22 smithi007 ceph-mon[113546]: Reconfiguring mds.cephfs.smithi019.gmhzgo (monmap changed)... 2024-03-16T09:06:22.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:22 smithi007 ceph-mon[113546]: Reconfiguring daemon mds.cephfs.smithi019.gmhzgo on smithi019 2024-03-16T09:06:22.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:22 smithi007 ceph-mon[113546]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:22.617 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:22 smithi019 ceph-mon[83366]: pgmap v28: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:22.617 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:22 smithi019 ceph-mon[83366]: Reconfiguring mds.cephfs.smithi019.gmhzgo (monmap changed)... 2024-03-16T09:06:22.617 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:22 smithi019 ceph-mon[83366]: Reconfiguring daemon mds.cephfs.smithi019.gmhzgo on smithi019 2024-03-16T09:06:22.617 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:22 smithi019 ceph-mon[83366]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.733+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ae80fe550 con 0x7f4ae81037a0 2024-03-16T09:06:22.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.734+0000 7f4ae4ff9700 1 -- 172.21.15.7:0/2241245532 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1895 (secure 0 0 0) 0x7f4adc004490 con 0x7f4ae81037a0 2024-03-16T09:06:22.736 INFO:teuthology.orchestra.run.smithi007.stdout:e20 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:06:22.737 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:06:22.738 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:06:22.739 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:06:22.740 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:06:22.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4ad006e010 msgr2=0x7f4ad00704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4ad006e010 0x7f4ad00704c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009b80 tx=0x7f4ad800b040 comp rx=0 tx=0).stop 2024-03-16T09:06:22.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 msgr2=0x7f4ae8113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 0x7f4ae8113e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000c00 tx=0x7f4adc009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 shutdown_connections 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ae8102e00 0x7f4ae8113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4ad006e010 0x7f4ad00704c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 --2- 172.21.15.7:0/2241245532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81037a0 0x7f4ae8113e40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 >> 172.21.15.7:0/2241245532 conn(0x7f4ae80f89b0 msgr2=0x7f4ae80fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 shutdown_connections 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.736+0000 7f4aee506700 1 -- 172.21.15.7:0/2241245532 wait complete. 2024-03-16T09:06:22.741 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:06:22.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.843+0000 7f25a083d700 1 -- 172.21.15.7:0/2021712958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c1037a0 msgr2=0x7f259c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.843+0000 7f25a083d700 1 --2- 172.21.15.7:0/2021712958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c1037a0 0x7f259c0fce10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2590004940 tx=0x7f2590009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:22.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.843+0000 7f25a083d700 1 -- 172.21.15.7:0/2021712958 shutdown_connections 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.843+0000 7f25a083d700 1 --2- 172.21.15.7:0/2021712958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c1037a0 0x7f259c0fce10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.843+0000 7f25a083d700 1 --2- 172.21.15.7:0/2021712958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c102e00 0x7f259c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.843+0000 7f25a083d700 1 -- 172.21.15.7:0/2021712958 >> 172.21.15.7:0/2021712958 conn(0x7f259c0f89b0 msgr2=0x7f259c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.843+0000 7f25a083d700 1 -- 172.21.15.7:0/2021712958 shutdown_connections 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.843+0000 7f25a083d700 1 -- 172.21.15.7:0/2021712958 wait complete. 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f25a083d700 1 Processor -- start 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f25a083d700 1 -- start start 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f25a083d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c102e00 0x7f259c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f25a083d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c1037a0 0x7f259c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f25a083d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c072210 con 0x7f259c102e00 2024-03-16T09:06:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f25a083d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c070d40 con 0x7f259c1037a0 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f259a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c102e00 0x7f259c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f259a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c102e00 0x7f259c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46648/0 (socket says 172.21.15.7:46648) 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f259a59c700 1 -- 172.21.15.7:0/1607708726 learned_addr learned my addr 172.21.15.7:0/1607708726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.844+0000 7f2599d9b700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c1037a0 0x7f259c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f259a59c700 1 -- 172.21.15.7:0/1607708726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c1037a0 msgr2=0x7f259c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f259a59c700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c1037a0 0x7f259c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f259a59c700 1 -- 172.21.15.7:0/1607708726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590005040 con 0x7f259c102e00 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f259a59c700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c102e00 0x7f259c0702c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f258c00eb40 tx=0x7f258c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f258affd700 1 -- 172.21.15.7:0/1607708726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f258c007cc0 con 0x7f259c102e00 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f258affd700 1 -- 172.21.15.7:0/1607708726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f258c00f040 con 0x7f259c102e00 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c070fc0 con 0x7f259c102e00 2024-03-16T09:06:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c119fb0 con 0x7f259c102e00 2024-03-16T09:06:22.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.845+0000 7f258affd700 1 -- 172.21.15.7:0/1607708726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f258c00ca10 con 0x7f259c102e00 2024-03-16T09:06:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.846+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c0052b0 con 0x7f259c102e00 2024-03-16T09:06:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.847+0000 7f258affd700 1 -- 172.21.15.7:0/1607708726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f258c00cb70 con 0x7f259c102e00 2024-03-16T09:06:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.847+0000 7f258affd700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f258406e010 0x7f25840704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.847+0000 7f258affd700 1 -- 172.21.15.7:0/1607708726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f258c08d430 con 0x7f259c102e00 2024-03-16T09:06:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.847+0000 7f2599d9b700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f258406e010 0x7f25840704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.848+0000 7f2599d9b700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f258406e010 0x7f25840704c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2590004890 tx=0x7f2590009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:22.852+0000 7f258affd700 1 -- 172.21.15.7:0/1607708726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f258c056850 con 0x7f259c102e00 2024-03-16T09:06:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.077+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f257c000bc0 con 0x7f258406e010 2024-03-16T09:06:23.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.078+0000 7f258affd700 1 -- 172.21.15.7:0/1607708726 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f257c000bc0 con 0x7f258406e010 2024-03-16T09:06:23.080 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:06:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:06:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:06:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:06:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:06:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:06:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:06:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:06:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "4/21 daemons upgraded", 2024-03-16T09:06:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading crash daemons", 2024-03-16T09:06:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:06:23.081 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:06:23.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f258406e010 msgr2=0x7f25840704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:23.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f258406e010 0x7f25840704c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2590004890 tx=0x7f2590009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:23.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c102e00 msgr2=0x7f259c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c102e00 0x7f259c0702c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f258c00eb40 tx=0x7f258c00b040 comp rx=0 tx=0).stop 2024-03-16T09:06:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 shutdown_connections 2024-03-16T09:06:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f258406e010 0x7f25840704c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f259c102e00 0x7f259c0702c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 --2- 172.21.15.7:0/1607708726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c1037a0 0x7f259c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 >> 172.21.15.7:0/1607708726 conn(0x7f259c0f89b0 msgr2=0x7f259c102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 shutdown_connections 2024-03-16T09:06:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.080+0000 7f25a083d700 1 -- 172.21.15.7:0/1607708726 wait complete. 2024-03-16T09:06:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.187+0000 7fe064686700 1 -- 172.21.15.7:0/3102898573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 msgr2=0x7fe05c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.187+0000 7fe064686700 1 --2- 172.21.15.7:0/3102898573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 0x7fe05c0fbd80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe050004940 tx=0x7fe050009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.187+0000 7fe064686700 1 -- 172.21.15.7:0/3102898573 shutdown_connections 2024-03-16T09:06:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.187+0000 7fe064686700 1 --2- 172.21.15.7:0/3102898573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 0x7fe05c0fbd80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.187+0000 7fe064686700 1 --2- 172.21.15.7:0/3102898573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe05c101d70 0x7fe05c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.187+0000 7fe064686700 1 -- 172.21.15.7:0/3102898573 >> 172.21.15.7:0/3102898573 conn(0x7fe05c072bc0 msgr2=0x7fe05c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.187+0000 7fe064686700 1 -- 172.21.15.7:0/3102898573 shutdown_connections 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.188+0000 7fe064686700 1 -- 172.21.15.7:0/3102898573 wait complete. 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.188+0000 7fe064686700 1 Processor -- start 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.188+0000 7fe064686700 1 -- start start 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.188+0000 7fe064686700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe05c101d70 0x7fe05c112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.188+0000 7fe064686700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 0x7fe05c112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.188+0000 7fe064686700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c113630 con 0x7fe05c102710 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.188+0000 7fe064686700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c10ca20 con 0x7fe05c101d70 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe061c21700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 0x7fe05c112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe061c21700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 0x7fe05c112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46672/0 (socket says 172.21.15.7:46672) 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe061c21700 1 -- 172.21.15.7:0/4177567052 learned_addr learned my addr 172.21.15.7:0/4177567052 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe061c21700 1 -- 172.21.15.7:0/4177567052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe05c101d70 msgr2=0x7fe05c112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe061c21700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe05c101d70 0x7fe05c112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe061c21700 1 -- 172.21.15.7:0/4177567052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050005040 con 0x7fe05c102710 2024-03-16T09:06:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe061c21700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 0x7fe05c112fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe050009100 tx=0x7fe050009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe04effd700 1 -- 172.21.15.7:0/4177567052 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe05000e040 con 0x7fe05c102710 2024-03-16T09:06:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05c10cca0 con 0x7fe05c102710 2024-03-16T09:06:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe04effd700 1 -- 172.21.15.7:0/4177567052 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe050004290 con 0x7fe05c102710 2024-03-16T09:06:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe04effd700 1 -- 172.21.15.7:0/4177567052 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe050013660 con 0x7fe05c102710 2024-03-16T09:06:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.189+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c10d200 con 0x7fe05c102710 2024-03-16T09:06:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.191+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05c0fd4c0 con 0x7fe05c102710 2024-03-16T09:06:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.191+0000 7fe04effd700 1 -- 172.21.15.7:0/4177567052 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fe05000bd00 con 0x7fe05c102710 2024-03-16T09:06:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.192+0000 7fe04effd700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe04806e000 0x7fe0480704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.192+0000 7fe04effd700 1 -- 172.21.15.7:0/4177567052 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe050091700 con 0x7fe05c102710 2024-03-16T09:06:23.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.192+0000 7fe062422700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe04806e000 0x7fe0480704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:23.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.193+0000 7fe062422700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe04806e000 0x7fe0480704b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe058004af0 tx=0x7fe058007c30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:23.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.197+0000 7fe04effd700 1 -- 172.21.15.7:0/4177567052 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe05005e5a0 con 0x7fe05c102710 2024-03-16T09:06:23.338 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='client.44109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3944615876' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi007"}]: dispatch 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi007"}]': finished 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi019"}]: dispatch 2024-03-16T09:06:23.339 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi019"}]': finished 2024-03-16T09:06:23.340 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:23 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2241245532' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:06:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.501+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe05c108e50 con 0x7fe05c102710 2024-03-16T09:06:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.501+0000 7fe04effd700 1 -- 172.21.15.7:0/4177567052 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe05005e130 con 0x7fe05c102710 2024-03-16T09:06:23.504 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_OK 2024-03-16T09:06:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe04806e000 msgr2=0x7fe0480704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe04806e000 0x7fe0480704b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe058004af0 tx=0x7fe058007c30 comp rx=0 tx=0).stop 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 msgr2=0x7fe05c112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 0x7fe05c112fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe050009100 tx=0x7fe050009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 shutdown_connections 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe05c101d70 0x7fe05c112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe04806e000 0x7fe0480704b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 --2- 172.21.15.7:0/4177567052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05c102710 0x7fe05c112fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 >> 172.21.15.7:0/4177567052 conn(0x7fe05c072bc0 msgr2=0x7fe05c0fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 shutdown_connections 2024-03-16T09:06:23.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:23.504+0000 7fe064686700 1 -- 172.21.15.7:0/4177567052 wait complete. 2024-03-16T09:06:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='client.44109 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3944615876' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi007"}]: dispatch 2024-03-16T09:06:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi007"}]': finished 2024-03-16T09:06:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi019"}]: dispatch 2024-03-16T09:06:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi019"}]': finished 2024-03-16T09:06:23.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:23 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2241245532' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:06:24.487 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:24 smithi007 ceph-mon[113546]: pgmap v29: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:24.487 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:24 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all mon 2024-03-16T09:06:24.487 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:24 smithi007 ceph-mon[113546]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:24.487 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:24 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/4177567052' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:06:24.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:24 smithi019 ceph-mon[83366]: pgmap v29: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:24.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:24 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all mon 2024-03-16T09:06:24.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:24 smithi019 ceph-mon[83366]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:24.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:24 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/4177567052' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:06:25.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:25 smithi007 ceph-mon[113546]: pgmap v30: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:25.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:25 smithi007 ceph-mon[113546]: Upgrade: Updating crash.smithi007 (1/2) 2024-03-16T09:06:25.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:25.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T09:06:25.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:25.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:25 smithi007 ceph-mon[113546]: Deploying daemon crash.smithi007 on smithi007 2024-03-16T09:06:25.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:25 smithi019 ceph-mon[83366]: pgmap v30: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:25.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:25 smithi019 ceph-mon[83366]: Upgrade: Updating crash.smithi007 (1/2) 2024-03-16T09:06:25.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:25.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi007", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T09:06:25.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:25.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:25 smithi019 ceph-mon[83366]: Deploying daemon crash.smithi007 on smithi007 2024-03-16T09:06:27.686 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:27 smithi007 ceph-mon[113546]: pgmap v31: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:27.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:27 smithi019 ceph-mon[83366]: pgmap v31: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:29.606 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:29 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:29.607 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:29 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:29.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:29 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:29.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:29 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:30.668 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:30 smithi019 ceph-mon[83366]: pgmap v32: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:30.669 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:30 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:30.669 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:30 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T09:06:30.669 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:30 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:30.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:30 smithi007 ceph-mon[113546]: pgmap v32: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:30.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:30.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi019", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T09:06:30.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:31.616 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:31 smithi019 ceph-mon[83366]: Upgrade: Updating crash.smithi019 (2/2) 2024-03-16T09:06:31.616 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:31 smithi019 ceph-mon[83366]: Deploying daemon crash.smithi019 on smithi019 2024-03-16T09:06:31.617 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:31 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:06:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:31 smithi007 ceph-mon[113546]: Upgrade: Updating crash.smithi019 (2/2) 2024-03-16T09:06:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:31 smithi007 ceph-mon[113546]: Deploying daemon crash.smithi019 on smithi019 2024-03-16T09:06:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:06:32.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:32 smithi019 ceph-mon[83366]: pgmap v33: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:32.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:32 smithi007 ceph-mon[113546]: pgmap v33: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:34.610 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:34 smithi007 ceph-mon[113546]: pgmap v34: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:34.610 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:34.610 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:34.610 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:34.640 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:34 smithi019 ceph-mon[83366]: pgmap v34: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:34.640 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:34.640 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:34.640 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:36.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:36 smithi019 ceph-mon[83366]: pgmap v35: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:36.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:36 smithi007 ceph-mon[113546]: pgmap v35: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:37.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:37 smithi007 ceph-mon[113546]: pgmap v36: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:37.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:37 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:37.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:37 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:37.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:37 smithi019 ceph-mon[83366]: pgmap v36: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:37 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:37 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:38.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:38 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:38.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:38 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:38.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:38 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:38.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:38 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:39 smithi019 ceph-mon[83366]: pgmap v37: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:39 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:39 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:39 smithi007 ceph-mon[113546]: pgmap v37: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:39 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:39 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:41.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:41.292 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:41 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:41.292 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:41 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:42.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:42 smithi019 ceph-mon[83366]: pgmap v38: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:42.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:42.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:42 smithi007 ceph-mon[113546]: pgmap v38: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.625 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: pgmap v39: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi007"}]: dispatch 2024-03-16T09:06:44.626 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi007"}]': finished 2024-03-16T09:06:44.627 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi019"}]: dispatch 2024-03-16T09:06:44.627 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi019"}]': finished 2024-03-16T09:06:44.627 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: pgmap v39: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:44.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:44.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:44.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi007"}]: dispatch 2024-03-16T09:06:44.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi007"}]': finished 2024-03-16T09:06:44.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi019"}]: dispatch 2024-03-16T09:06:44.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi019"}]': finished 2024-03-16T09:06:44.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-16T09:06:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:45 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all crash 2024-03-16T09:06:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:45 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-16T09:06:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:45 smithi019 ceph-mon[83366]: Upgrade: osd.0 is safe to restart 2024-03-16T09:06:45.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:45 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all crash 2024-03-16T09:06:45.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:45 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-16T09:06:45.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:45 smithi007 ceph-mon[113546]: Upgrade: osd.0 is safe to restart 2024-03-16T09:06:46.466 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:46 smithi007 ceph-mon[113546]: pgmap v40: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:46.466 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:06:46.466 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:46.466 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:06:46.466 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:46.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:46 smithi019 ceph-mon[83366]: pgmap v40: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:46.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:06:46.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:46.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:06:46.494 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:47.488 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:47 smithi007 ceph-mon[113546]: Upgrade: Updating osd.0 2024-03-16T09:06:47.488 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:47 smithi007 ceph-mon[113546]: Deploying daemon osd.0 on smithi007 2024-03-16T09:06:47.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:47 smithi019 ceph-mon[83366]: Upgrade: Updating osd.0 2024-03-16T09:06:47.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:47 smithi019 ceph-mon[83366]: Deploying daemon osd.0 on smithi007 2024-03-16T09:06:48.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:48 smithi019 ceph-mon[83366]: pgmap v41: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:48 smithi007 ceph-mon[113546]: pgmap v41: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:49.248 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:48 smithi007 systemd[1]: Stopping Ceph osd.0 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:06:49.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:49 smithi019 ceph-mon[83366]: pgmap v42: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:49.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:49 smithi019 ceph-mon[83366]: osd.0 marked itself down and dead 2024-03-16T09:06:49.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:49 smithi007 ceph-mon[113546]: pgmap v42: 65 pgs: 65 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:49.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:49 smithi007 ceph-mon[113546]: osd.0 marked itself down and dead 2024-03-16T09:06:49.749 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:49 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[48134]: 2024-03-16T09:06:49.249+0000 7feb13358700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:06:49.749 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:49 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[48134]: 2024-03-16T09:06:49.249+0000 7feb13358700 -1 osd.0 49 *** Got signal Terminated *** 2024-03-16T09:06:49.749 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:49 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[48134]: 2024-03-16T09:06:49.249+0000 7feb13358700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:06:50.160 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:49 smithi007 podman[124743]: 2024-03-16 09:06:49.893360906 +0000 UTC m=+0.754528546 container died 667f5bef6da7019180599846a2b5daf6a243b51f14f27ea79823c96b8e2ac7df (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=quincy-1df2724, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5) 2024-03-16T09:06:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:50 smithi007 ceph-mon[113546]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:06:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:50 smithi007 ceph-mon[113546]: osdmap e50: 6 total, 5 up, 6 in 2024-03-16T09:06:50.498 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:50 smithi007 podman[124743]: 2024-03-16 09:06:50.248772868 +0000 UTC m=+1.109940508 container cleanup 667f5bef6da7019180599846a2b5daf6a243b51f14f27ea79823c96b8e2ac7df (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, RELEASE=quincy-1df2724, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:06:50.498 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:50 smithi007 bash[124743]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0 2024-03-16T09:06:50.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:50 smithi019 ceph-mon[83366]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:06:50.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:50 smithi019 ceph-mon[83366]: osdmap e50: 6 total, 5 up, 6 in 2024-03-16T09:06:50.919 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:50 smithi007 podman[124760]: 2024-03-16 09:06:50.5590765 +0000 UTC m=+0.668012005 container remove 667f5bef6da7019180599846a2b5daf6a243b51f14f27ea79823c96b8e2ac7df (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=quincy-1df2724, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:06:51.181 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:50 smithi007 podman[124851]: 2024-03-16 09:06:50.821306731 +0000 UTC m=+0.017795466 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:06:51.181 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:50 smithi007 podman[124851]: 2024-03-16 09:06:50.960314022 +0000 UTC m=+0.156802749 container create 30c726a5b5ac7f0aff0bcc5d9c05925e80209d5fb08faf45d666991d3e57d6af (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:06:51.481 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:51 smithi007 ceph-mon[113546]: osdmap e51: 6 total, 5 up, 6 in 2024-03-16T09:06:51.481 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:51 smithi007 ceph-mon[113546]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:51.482 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:51 smithi007 podman[124851]: 2024-03-16 09:06:51.179876345 +0000 UTC m=+0.376365073 container init 30c726a5b5ac7f0aff0bcc5d9c05925e80209d5fb08faf45d666991d3e57d6af (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-16T09:06:51.482 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:51 smithi007 podman[124851]: 2024-03-16 09:06:51.183087848 +0000 UTC m=+0.379576580 container start 30c726a5b5ac7f0aff0bcc5d9c05925e80209d5fb08faf45d666991d3e57d6af (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-16T09:06:51.482 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:51 smithi007 podman[124851]: 2024-03-16 09:06:51.254582866 +0000 UTC m=+0.451071596 container attach 30c726a5b5ac7f0aff0bcc5d9c05925e80209d5fb08faf45d666991d3e57d6af (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:06:51.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:51 smithi019 ceph-mon[83366]: osdmap e51: 6 total, 5 up, 6 in 2024-03-16T09:06:51.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:51 smithi019 ceph-mon[83366]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:06:51.748 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:51 smithi007 podman[124851]: 2024-03-16 09:06:51.480071149 +0000 UTC m=+0.676559877 container died 30c726a5b5ac7f0aff0bcc5d9c05925e80209d5fb08faf45d666991d3e57d6af (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-16T09:06:52.281 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 podman[124876]: 2024-03-16 09:06:52.242137644 +0000 UTC m=+0.752283613 container remove 30c726a5b5ac7f0aff0bcc5d9c05925e80209d5fb08faf45d666991d3e57d6af (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T09:06:52.282 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-16T09:06:52.282 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 systemd[1]: Stopped Ceph osd.0 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:06:52.282 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.0.service: Consumed 54.869s CPU time. 2024-03-16T09:06:52.535 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 systemd[1]: Starting Ceph osd.0 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:06:52.881 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:52 smithi007 ceph-mon[113546]: Health check failed: Degraded data redundancy: 68/354 objects degraded (19.209%), 19 pgs degraded (PG_DEGRADED) 2024-03-16T09:06:52.882 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 podman[124984]: 2024-03-16 09:06:52.604010525 +0000 UTC m=+0.018192024 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:06:52.882 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 podman[124984]: 2024-03-16 09:06:52.743562204 +0000 UTC m=+0.157743701 container create b22908a72744d5c7559fa79128f4c1732da678ff9ac357f220b5065b8cd10d63 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-16T09:06:52.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:52 smithi019 ceph-mon[83366]: Health check failed: Degraded data redundancy: 68/354 objects degraded (19.209%), 19 pgs degraded (PG_DEGRADED) 2024-03-16T09:06:53.248 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 podman[124984]: 2024-03-16 09:06:52.963076966 +0000 UTC m=+0.377258476 container init b22908a72744d5c7559fa79128f4c1732da678ff9ac357f220b5065b8cd10d63 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:06:53.248 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:52 smithi007 podman[124984]: 2024-03-16 09:06:52.966099774 +0000 UTC m=+0.380281272 container start b22908a72744d5c7559fa79128f4c1732da678ff9ac357f220b5065b8cd10d63 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:06:53.248 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 podman[124984]: 2024-03-16 09:06:53.046344382 +0000 UTC m=+0.460525890 container attach b22908a72744d5c7559fa79128f4c1732da678ff9ac357f220b5065b8cd10d63 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-16T09:06:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 -- 172.21.15.7:0/2790487787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d81037a0 msgr2=0x7f65d80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 --2- 172.21.15.7:0/2790487787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d81037a0 0x7f65d80fce10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004940 tx=0x7f65cc009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 -- 172.21.15.7:0/2790487787 shutdown_connections 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 --2- 172.21.15.7:0/2790487787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d81037a0 0x7f65d80fce10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 --2- 172.21.15.7:0/2790487787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65d8102e00 0x7f65d81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 -- 172.21.15.7:0/2790487787 >> 172.21.15.7:0/2790487787 conn(0x7f65d80f89b0 msgr2=0x7f65d80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 -- 172.21.15.7:0/2790487787 shutdown_connections 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 -- 172.21.15.7:0/2790487787 wait complete. 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.613+0000 7f65de714700 1 Processor -- start 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65de714700 1 -- start start 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65de714700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d8102e00 0x7f65d8113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65de714700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65d81037a0 0x7f65d8113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65de714700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8114490 con 0x7f65d8102e00 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65de714700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d810da90 con 0x7f65d81037a0 2024-03-16T09:06:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d8102e00 0x7f65d8113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d8102e00 0x7f65d8113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39790/0 (socket says 172.21.15.7:39790) 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65d7fff700 1 -- 172.21.15.7:0/666098728 learned_addr learned my addr 172.21.15.7:0/666098728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65d77fe700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65d81037a0 0x7f65d8113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65d7fff700 1 -- 172.21.15.7:0/666098728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65d81037a0 msgr2=0x7f65d8113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65d7fff700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65d81037a0 0x7f65d8113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65d7fff700 1 -- 172.21.15.7:0/666098728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc005040 con 0x7f65d8102e00 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.614+0000 7f65d7fff700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d8102e00 0x7f65d8113900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65c800eb70 tx=0x7f65c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.615+0000 7f65d4ff9700 1 -- 172.21.15.7:0/666098728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c8007cc0 con 0x7f65d8102e00 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.615+0000 7f65d4ff9700 1 -- 172.21.15.7:0/666098728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65c800f040 con 0x7f65d8102e00 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.615+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d810dd10 con 0x7f65d8102e00 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.615+0000 7f65d4ff9700 1 -- 172.21.15.7:0/666098728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c800ca90 con 0x7f65d8102e00 2024-03-16T09:06:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.615+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d810e220 con 0x7f65d8102e00 2024-03-16T09:06:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.616+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d81052d0 con 0x7f65d8102e00 2024-03-16T09:06:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.616+0000 7f65d4ff9700 1 -- 172.21.15.7:0/666098728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f65c800cbf0 con 0x7f65d8102e00 2024-03-16T09:06:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.617+0000 7f65d4ff9700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f65c006e010 0x7f65c00704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.617+0000 7f65d4ff9700 1 -- 172.21.15.7:0/666098728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f65c808d430 con 0x7f65d8102e00 2024-03-16T09:06:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.617+0000 7f65d77fe700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f65c006e010 0x7f65c00704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.617+0000 7f65d77fe700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f65c006e010 0x7f65c00704c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004890 tx=0x7f65cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.622+0000 7f65d4ff9700 1 -- 172.21.15.7:0/666098728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f65c805a2d0 con 0x7f65d8102e00 2024-03-16T09:06:53.726 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:53 smithi007 ceph-mon[113546]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:06:53.726 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate[124998]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T09:06:53.726 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate[124998]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 bash[124984]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 bash[124984]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 bash[124984]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 bash[124984]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 bash[124984]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 bash[124984]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 bash[124984]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate[124998]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate[124998]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate[124998]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate[124998]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate[124998]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-16T09:06:53.727 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:53 smithi007 podman[124984]: 2024-03-16 09:06:53.439377645 +0000 UTC m=+0.853559132 container died b22908a72744d5c7559fa79128f4c1732da678ff9ac357f220b5065b8cd10d63 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-03-16T09:06:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.852+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65d810eff0 con 0x7f65c006e010 2024-03-16T09:06:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.853+0000 7f65d4ff9700 1 -- 172.21.15.7:0/666098728 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f65d810eff0 con 0x7f65c006e010 2024-03-16T09:06:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f65c006e010 msgr2=0x7f65c00704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f65c006e010 0x7f65c00704c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004890 tx=0x7f65cc009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d8102e00 msgr2=0x7f65d8113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d8102e00 0x7f65d8113900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65c800eb70 tx=0x7f65c800b040 comp rx=0 tx=0).stop 2024-03-16T09:06:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 shutdown_connections 2024-03-16T09:06:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f65c006e010 0x7f65c00704c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65d8102e00 0x7f65d8113900 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 --2- 172.21.15.7:0/666098728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65d81037a0 0x7f65d8113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.855+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 >> 172.21.15.7:0/666098728 conn(0x7f65d80f89b0 msgr2=0x7f65d80fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.856+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 shutdown_connections 2024-03-16T09:06:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.856+0000 7f65de714700 1 -- 172.21.15.7:0/666098728 wait complete. 2024-03-16T09:06:53.868 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:06:53.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.959+0000 7fe2e8163700 1 -- 172.21.15.7:0/1970519438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 msgr2=0x7fe2e01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:53.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.959+0000 7fe2e8163700 1 --2- 172.21.15.7:0/1970519438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 0x7fe2e01031e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc004940 tx=0x7fe2dc009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:53.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.959+0000 7fe2e8163700 1 -- 172.21.15.7:0/1970519438 shutdown_connections 2024-03-16T09:06:53.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.959+0000 7fe2e8163700 1 --2- 172.21.15.7:0/1970519438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2e01037b0 0x7fe2e00fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.959+0000 7fe2e8163700 1 --2- 172.21.15.7:0/1970519438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 0x7fe2e01031e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.959+0000 7fe2e8163700 1 -- 172.21.15.7:0/1970519438 >> 172.21.15.7:0/1970519438 conn(0x7fe2e00f89a0 msgr2=0x7fe2e00fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:53.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e8163700 1 -- 172.21.15.7:0/1970519438 shutdown_connections 2024-03-16T09:06:53.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e8163700 1 -- 172.21.15.7:0/1970519438 wait complete. 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e8163700 1 Processor -- start 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e8163700 1 -- start start 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e8163700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 0x7fe2e00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e8163700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2e01037b0 0x7fe2e0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e8163700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0070d40 con 0x7fe2e0102e10 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e8163700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0070e80 con 0x7fe2e01037b0 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e5eff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 0x7fe2e00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e5eff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 0x7fe2e00702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39804/0 (socket says 172.21.15.7:39804) 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.960+0000 7fe2e5eff700 1 -- 172.21.15.7:0/1118558973 learned_addr learned my addr 172.21.15.7:0/1118558973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2e56fe700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2e01037b0 0x7fe2e0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2e5eff700 1 -- 172.21.15.7:0/1118558973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2e01037b0 msgr2=0x7fe2e0070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2e5eff700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2e01037b0 0x7fe2e0070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2e5eff700 1 -- 172.21.15.7:0/1118558973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc005040 con 0x7fe2e0102e10 2024-03-16T09:06:53.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2e5eff700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 0x7fe2e00702c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009180 tx=0x7fe2dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:53.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:53 smithi019 ceph-mon[83366]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:06:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2d67fc700 1 -- 172.21.15.7:0/1118558973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2dc00e040 con 0x7fe2e0102e10 2024-03-16T09:06:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2d67fc700 1 -- 172.21.15.7:0/1118558973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2dc013420 con 0x7fe2e0102e10 2024-03-16T09:06:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00710d0 con 0x7fe2e0102e10 2024-03-16T09:06:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2d67fc700 1 -- 172.21.15.7:0/1118558973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2dc01c630 con 0x7fe2e0102e10 2024-03-16T09:06:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.961+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e011a2f0 con 0x7fe2e0102e10 2024-03-16T09:06:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.962+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e0100120 con 0x7fe2e0102e10 2024-03-16T09:06:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.963+0000 7fe2d67fc700 1 -- 172.21.15.7:0/1118558973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fe2dc013590 con 0x7fe2e0102e10 2024-03-16T09:06:53.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.963+0000 7fe2d67fc700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe2cc06e000 0x7fe2cc0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.963+0000 7fe2d67fc700 1 -- 172.21.15.7:0/1118558973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe2dc08ead0 con 0x7fe2e0102e10 2024-03-16T09:06:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.963+0000 7fe2e56fe700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe2cc06e000 0x7fe2cc0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:53.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.964+0000 7fe2e56fe700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe2cc06e000 0x7fe2cc0704b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0071bb0 tx=0x7fe2d0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:53.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:53.968+0000 7fe2d67fc700 1 -- 172.21.15.7:0/1118558973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2dc057e40 con 0x7fe2e0102e10 2024-03-16T09:06:54.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.199+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2e004bed0 con 0x7fe2cc06e000 2024-03-16T09:06:54.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.200+0000 7fe2d67fc700 1 -- 172.21.15.7:0/1118558973 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe2e004bed0 con 0x7fe2cc06e000 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.202+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe2cc06e000 msgr2=0x7fe2cc0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.202+0000 7fe2e8163700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe2cc06e000 0x7fe2cc0704b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0071bb0 tx=0x7fe2d0009040 comp rx=0 tx=0).stop 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.202+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 msgr2=0x7fe2e00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.202+0000 7fe2e8163700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 0x7fe2e00702c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009180 tx=0x7fe2dc009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.202+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 shutdown_connections 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.202+0000 7fe2e8163700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe2cc06e000 0x7fe2cc0704b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.202+0000 7fe2e8163700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e0102e10 0x7fe2e00702c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.203+0000 7fe2e8163700 1 --2- 172.21.15.7:0/1118558973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2e01037b0 0x7fe2e0070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.203+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 >> 172.21.15.7:0/1118558973 conn(0x7fe2e00f89a0 msgr2=0x7fe2e00facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.203+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 shutdown_connections 2024-03-16T09:06:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.203+0000 7fe2e8163700 1 -- 172.21.15.7:0/1118558973 wait complete. 2024-03-16T09:06:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.313+0000 7fbdfeabe700 1 -- 172.21.15.7:0/3618879895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fde90 msgr2=0x7fbdf80fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.313+0000 7fbdfeabe700 1 --2- 172.21.15.7:0/3618879895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fde90 0x7fbdf80fe260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdec004940 tx=0x7fbdec009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.313+0000 7fbdfeabe700 1 -- 172.21.15.7:0/3618879895 shutdown_connections 2024-03-16T09:06:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.313+0000 7fbdfeabe700 1 --2- 172.21.15.7:0/3618879895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf80fe830 0x7fbdf8106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.313+0000 7fbdfeabe700 1 --2- 172.21.15.7:0/3618879895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fde90 0x7fbdf80fe260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.313+0000 7fbdfeabe700 1 -- 172.21.15.7:0/3618879895 >> 172.21.15.7:0/3618879895 conn(0x7fbdf8072bc0 msgr2=0x7fbdf8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.313+0000 7fbdfeabe700 1 -- 172.21.15.7:0/3618879895 shutdown_connections 2024-03-16T09:06:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.313+0000 7fbdfeabe700 1 -- 172.21.15.7:0/3618879895 wait complete. 2024-03-16T09:06:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdfeabe700 1 Processor -- start 2024-03-16T09:06:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdfeabe700 1 -- start start 2024-03-16T09:06:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdfeabe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf80fde90 0x7fbdf8113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdfeabe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fe830 0x7fbdf81140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdfeabe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8114720 con 0x7fbdf80fe830 2024-03-16T09:06:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdfeabe700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf810db30 con 0x7fbdf80fde90 2024-03-16T09:06:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fe830 0x7fbdf81140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fe830 0x7fbdf81140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39830/0 (socket says 172.21.15.7:39830) 2024-03-16T09:06:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.314+0000 7fbdf7fff700 1 -- 172.21.15.7:0/2199413518 learned_addr learned my addr 172.21.15.7:0/2199413518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdfc85a700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf80fde90 0x7fbdf8113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdf7fff700 1 -- 172.21.15.7:0/2199413518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf80fde90 msgr2=0x7fbdf8113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdf7fff700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf80fde90 0x7fbdf8113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdf7fff700 1 -- 172.21.15.7:0/2199413518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec005040 con 0x7fbdf80fe830 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdf7fff700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fe830 0x7fbdf81140d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbde4002a50 tx=0x7fbde40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdf57fa700 1 -- 172.21.15.7:0/2199413518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde4007cc0 con 0x7fbdf80fe830 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdf57fa700 1 -- 172.21.15.7:0/2199413518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbde400f040 con 0x7fbdf80fe830 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf810de10 con 0x7fbdf80fe830 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.315+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf810e350 con 0x7fbdf80fe830 2024-03-16T09:06:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.316+0000 7fbdf57fa700 1 -- 172.21.15.7:0/2199413518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde4009640 con 0x7fbdf80fe830 2024-03-16T09:06:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.316+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf8101980 con 0x7fbdf80fe830 2024-03-16T09:06:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.316+0000 7fbdf57fa700 1 -- 172.21.15.7:0/2199413518 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fbde4009860 con 0x7fbdf80fe830 2024-03-16T09:06:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.317+0000 7fbdf57fa700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbde806de00 0x7fbde80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.317+0000 7fbdf57fa700 1 -- 172.21.15.7:0/2199413518 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbde408d720 con 0x7fbdf80fe830 2024-03-16T09:06:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.317+0000 7fbdfc85a700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbde806de00 0x7fbde80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.317+0000 7fbdfc85a700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbde806de00 0x7fbde80702b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00b3b0 tx=0x7fbdec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.322+0000 7fbdf57fa700 1 -- 172.21.15.7:0/2199413518 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbde4056a10 con 0x7fbdf80fe830 2024-03-16T09:06:54.491 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:54 smithi007 podman[125137]: 2024-03-16 09:06:54.168638242 +0000 UTC m=+0.724322552 container remove b22908a72744d5c7559fa79128f4c1732da678ff9ac357f220b5065b8cd10d63 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:06:54.491 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:54 smithi007 podman[125253]: 2024-03-16 09:06:54.286016268 +0000 UTC m=+0.019391030 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:06:54.491 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:54 smithi007 podman[125253]: 2024-03-16 09:06:54.394834403 +0000 UTC m=+0.128209146 container create 1f775f30a8555e232b7e6e11b2cf81032f7f9229e38266a098b2ed8532715b65 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-16T09:06:54.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.539+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdf805f5f0 con 0x7fbde806de00 2024-03-16T09:06:54.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.550+0000 7fbdf57fa700 1 -- 172.21.15.7:0/2199413518 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbdf805f5f0 con 0x7fbde806de00 2024-03-16T09:06:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:06:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (3m) 15s ago 13m 29.0M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:06:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (26s) 15s ago 14m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:06:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (21s) 18s ago 13m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:06:54.552 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (2m) 15s ago 13m 77.7M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (8m) 15s ago 8m 14.0M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (8m) 15s ago 8m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (8m) 18s ago 8m 120M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (8m) 18s ago 8m 14.1M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (4m) 15s ago 15m 495M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (3m) 18s ago 12m 449M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (93s) 15s ago 15m 44.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (67s) 18s ago 12m 34.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (3m) 15s ago 13m 14.9M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (3m) 18s ago 12m 15.2M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (12m) 15s ago 12m 207M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 667f5bef6da7 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (11m) 15s ago 11m 236M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:06:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (11m) 15s ago 11m 182M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:06:54.554 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (10m) 18s ago 10m 264M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:06:54.554 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (10m) 18s ago 10m 247M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:06:54.554 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (9m) 18s ago 9m 223M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:06:54.554 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (3m) 15s ago 13m 61.4M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:06:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbde806de00 msgr2=0x7fbde80702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbde806de00 0x7fbde80702b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00b3b0 tx=0x7fbdec009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fe830 msgr2=0x7fbdf81140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fe830 0x7fbdf81140d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbde4002a50 tx=0x7fbde40046b0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 shutdown_connections 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf80fde90 0x7fbdf8113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbde806de00 0x7fbde80702b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 --2- 172.21.15.7:0/2199413518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf80fe830 0x7fbdf81140d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 >> 172.21.15.7:0/2199413518 conn(0x7fbdf8072bc0 msgr2=0x7fbdf80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 shutdown_connections 2024-03-16T09:06:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.552+0000 7fbdfeabe700 1 -- 172.21.15.7:0/2199413518 wait complete. 2024-03-16T09:06:54.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.657+0000 7f833b846700 1 -- 172.21.15.7:0/786092720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 msgr2=0x7f83340fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.657+0000 7f833b846700 1 --2- 172.21.15.7:0/786092720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 0x7f83340fce10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8328004940 tx=0x7f8328009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.657+0000 7f833b846700 1 -- 172.21.15.7:0/786092720 shutdown_connections 2024-03-16T09:06:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.657+0000 7f833b846700 1 --2- 172.21.15.7:0/786092720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 0x7f83340fce10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.657+0000 7f833b846700 1 --2- 172.21.15.7:0/786092720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334102e00 0x7f83341031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.657+0000 7f833b846700 1 -- 172.21.15.7:0/786092720 >> 172.21.15.7:0/786092720 conn(0x7f83340f89b0 msgr2=0x7f83340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.657+0000 7f833b846700 1 -- 172.21.15.7:0/786092720 shutdown_connections 2024-03-16T09:06:54.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.657+0000 7f833b846700 1 -- 172.21.15.7:0/786092720 wait complete. 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f833b846700 1 Processor -- start 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f833b846700 1 -- start start 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f833b846700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334102e00 0x7f8334113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f833b846700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 0x7f8334113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f833b846700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8334114510 con 0x7f83341037a0 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f833b846700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833410db10 con 0x7f8334102e00 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f8338de1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 0x7f8334113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f8338de1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 0x7f8334113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39850/0 (socket says 172.21.15.7:39850) 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f8338de1700 1 -- 172.21.15.7:0/3134379937 learned_addr learned my addr 172.21.15.7:0/3134379937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:54.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f83395e2700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334102e00 0x7f8334113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f8338de1700 1 -- 172.21.15.7:0/3134379937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334102e00 msgr2=0x7f8334113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f8338de1700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334102e00 0x7f8334113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f8338de1700 1 -- 172.21.15.7:0/3134379937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328005040 con 0x7f83341037a0 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.658+0000 7f83395e2700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334102e00 0x7f8334113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.659+0000 7f8338de1700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 0x7f8334113ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8328009100 tx=0x7f8328009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.659+0000 7f8331ffb700 1 -- 172.21.15.7:0/3134379937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832800e040 con 0x7f83341037a0 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.659+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833410dd90 con 0x7f83341037a0 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.659+0000 7f8331ffb700 1 -- 172.21.15.7:0/3134379937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8328013420 con 0x7f83341037a0 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.659+0000 7f8331ffb700 1 -- 172.21.15.7:0/3134379937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832801c610 con 0x7f83341037a0 2024-03-16T09:06:54.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.659+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833410e350 con 0x7f83341037a0 2024-03-16T09:06:54.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.660+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83341052d0 con 0x7f83341037a0 2024-03-16T09:06:54.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.661+0000 7f8331ffb700 1 -- 172.21.15.7:0/3134379937 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f8328013590 con 0x7f83341037a0 2024-03-16T09:06:54.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.661+0000 7f8331ffb700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f831c06e010 0x7f831c0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:54.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.661+0000 7f8331ffb700 1 -- 172.21.15.7:0/3134379937 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f832808eac0 con 0x7f83341037a0 2024-03-16T09:06:54.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.661+0000 7f83395e2700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f831c06e010 0x7f831c0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:54.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.661+0000 7f83395e2700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f831c06e010 0x7f831c0704c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f832c004f50 tx=0x7f832c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.666+0000 7f8331ffb700 1 -- 172.21.15.7:0/3134379937 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f832805ba00 con 0x7f83341037a0 2024-03-16T09:06:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:54 smithi007 ceph-mon[113546]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:54 smithi007 ceph-mon[113546]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:54.748 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:54 smithi007 podman[125253]: 2024-03-16 09:06:54.606143144 +0000 UTC m=+0.339517900 container init 1f775f30a8555e232b7e6e11b2cf81032f7f9229e38266a098b2ed8532715b65 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True) 2024-03-16T09:06:54.749 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:54 smithi007 podman[125253]: 2024-03-16 09:06:54.609525806 +0000 UTC m=+0.342900563 container start 1f775f30a8555e232b7e6e11b2cf81032f7f9229e38266a098b2ed8532715b65 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-16T09:06:54.749 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:54 smithi007 ceph-osd[125294]: -- 172.21.15.7:0/3786988284 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598c6553680 con 0x5598c651cc00 2024-03-16T09:06:54.749 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:54 smithi007 bash[125253]: 1f775f30a8555e232b7e6e11b2cf81032f7f9229e38266a098b2ed8532715b65 2024-03-16T09:06:54.749 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:54 smithi007 systemd[1]: Started Ceph osd.0 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:06:54.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:54 smithi019 ceph-mon[83366]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:54.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:54 smithi019 ceph-mon[83366]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.963+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f833404c740 con 0x7f83341037a0 2024-03-16T09:06:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.964+0000 7f8331ffb700 1 -- 172.21.15.7:0/3134379937 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f832805b590 con 0x7f83341037a0 2024-03-16T09:06:54.966 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:06:54.966 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 5 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 9, 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:06:54.967 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:06:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f831c06e010 msgr2=0x7f831c0704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f831c06e010 0x7f831c0704c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f832c004f50 tx=0x7f832c00b040 comp rx=0 tx=0).stop 2024-03-16T09:06:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 msgr2=0x7f8334113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 0x7f8334113ec0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8328009100 tx=0x7f8328009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 shutdown_connections 2024-03-16T09:06:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8334102e00 0x7f8334113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f831c06e010 0x7f831c0704c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 --2- 172.21.15.7:0/3134379937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341037a0 0x7f8334113ec0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 >> 172.21.15.7:0/3134379937 conn(0x7f83340f89b0 msgr2=0x7f83341021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 shutdown_connections 2024-03-16T09:06:54.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:54.966+0000 7f833b846700 1 -- 172.21.15.7:0/3134379937 wait complete. 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.071+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2534541369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 msgr2=0x7f4d04106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.071+0000 7f4d0a2a2700 1 --2- 172.21.15.7:0/2534541369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 0x7f4d04106f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8004940 tx=0x7f4cf8009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.071+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2534541369 shutdown_connections 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.071+0000 7f4d0a2a2700 1 --2- 172.21.15.7:0/2534541369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 0x7f4d04106f00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.071+0000 7f4d0a2a2700 1 --2- 172.21.15.7:0/2534541369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d040fde90 0x7f4d040fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.071+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2534541369 >> 172.21.15.7:0/2534541369 conn(0x7f4d04072bc0 msgr2=0x7f4d04072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.071+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2534541369 shutdown_connections 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.071+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2534541369 wait complete. 2024-03-16T09:06:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d0a2a2700 1 Processor -- start 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d0a2a2700 1 -- start start 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d0a2a2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d040fde90 0x7f4d04113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d0a2a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 0x7f4d041140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d0a2a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04114730 con 0x7f4d040fe830 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d0a2a2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0410db20 con 0x7f4d040fde90 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 0x7f4d041140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 0x7f4d041140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39874/0 (socket says 172.21.15.7:39874) 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d037fe700 1 -- 172.21.15.7:0/2590410027 learned_addr learned my addr 172.21.15.7:0/2590410027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.072+0000 7f4d03fff700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d040fde90 0x7f4d04113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d037fe700 1 -- 172.21.15.7:0/2590410027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d040fde90 msgr2=0x7f4d04113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d037fe700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d040fde90 0x7f4d04113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d037fe700 1 -- 172.21.15.7:0/2590410027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf8005040 con 0x7f4d040fe830 2024-03-16T09:06:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d037fe700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 0x7f4d041140e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80093b0 tx=0x7f4cf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d00ff9700 1 -- 172.21.15.7:0/2590410027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf800e040 con 0x7f4d040fe830 2024-03-16T09:06:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0410dda0 con 0x7f4d040fe830 2024-03-16T09:06:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d00ff9700 1 -- 172.21.15.7:0/2590410027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cf8004290 con 0x7f4d040fe830 2024-03-16T09:06:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d00ff9700 1 -- 172.21.15.7:0/2590410027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf801c3f0 con 0x7f4d040fe830 2024-03-16T09:06:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.073+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0410e2e0 con 0x7f4d040fe830 2024-03-16T09:06:55.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.074+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d04065e30 con 0x7f4d040fe830 2024-03-16T09:06:55.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.075+0000 7f4d00ff9700 1 -- 172.21.15.7:0/2590410027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f4cf800ba90 con 0x7f4d040fe830 2024-03-16T09:06:55.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.075+0000 7f4d00ff9700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4cec06e000 0x7f4cec0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.075+0000 7f4d00ff9700 1 -- 172.21.15.7:0/2590410027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4cf808eac0 con 0x7f4d040fe830 2024-03-16T09:06:55.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.075+0000 7f4d03fff700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4cec06e000 0x7f4cec0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.076+0000 7f4d03fff700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4cec06e000 0x7f4cec0704b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4007fd0 tx=0x7f4cf4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:55.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.080+0000 7f4d00ff9700 1 -- 172.21.15.7:0/2590410027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4cf805ba00 con 0x7f4d040fe830 2024-03-16T09:06:55.248 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:55 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[125287]: 2024-03-16T09:06:55.146+0000 7f270a0b77c0 -1 Falling back to public interface 2024-03-16T09:06:55.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.349+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d0410f1f0 con 0x7f4d040fe830 2024-03-16T09:06:55.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.349+0000 7f4d00ff9700 1 -- 172.21.15.7:0/2590410027 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1895 (secure 0 0 0) 0x7f4cf805b590 con 0x7f4d040fe830 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:e20 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:06:55.352 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:06:55.353 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:06:55.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:06:55.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:06:55.355 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:06:55.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.351+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4cec06e000 msgr2=0x7f4cec0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.351+0000 7f4d0a2a2700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4cec06e000 0x7f4cec0704b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4007fd0 tx=0x7f4cf4009040 comp rx=0 tx=0).stop 2024-03-16T09:06:55.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.351+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 msgr2=0x7f4d041140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.351+0000 7f4d0a2a2700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 0x7f4d041140e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80093b0 tx=0x7f4cf8009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.351+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 shutdown_connections 2024-03-16T09:06:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.351+0000 7f4d0a2a2700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d040fde90 0x7f4d04113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.351+0000 7f4d0a2a2700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4cec06e000 0x7f4cec0704b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.351+0000 7f4d0a2a2700 1 --2- 172.21.15.7:0/2590410027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d040fe830 0x7f4d041140e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.352+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 >> 172.21.15.7:0/2590410027 conn(0x7f4d04072bc0 msgr2=0x7f4d041037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.352+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 shutdown_connections 2024-03-16T09:06:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.352+0000 7f4d0a2a2700 1 -- 172.21.15.7:0/2590410027 wait complete. 2024-03-16T09:06:55.357 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:06:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 -- 172.21.15.7:0/109773747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 msgr2=0x7f0af80fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 --2- 172.21.15.7:0/109773747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 0x7f0af80fbd30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0aec004940 tx=0x7f0aec009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 -- 172.21.15.7:0/109773747 shutdown_connections 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 --2- 172.21.15.7:0/109773747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 0x7f0af80fbd30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 --2- 172.21.15.7:0/109773747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8101d20 0x7f0af81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 -- 172.21.15.7:0/109773747 >> 172.21.15.7:0/109773747 conn(0x7f0af80f7870 msgr2=0x7f0af80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 -- 172.21.15.7:0/109773747 shutdown_connections 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 -- 172.21.15.7:0/109773747 wait complete. 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 Processor -- start 2024-03-16T09:06:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.465+0000 7f0afe8ed700 1 -- start start 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0afe8ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8101d20 0x7f0af81128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0afe8ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 0x7f0af8112df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0afe8ed700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af8113440 con 0x7f0af81026c0 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0afe8ed700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af810ca40 con 0x7f0af8101d20 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 0x7f0af8112df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 0x7f0af8112df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39904/0 (socket says 172.21.15.7:39904) 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af77fe700 1 -- 172.21.15.7:0/4207522661 learned_addr learned my addr 172.21.15.7:0/4207522661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af7fff700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8101d20 0x7f0af81128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af77fe700 1 -- 172.21.15.7:0/4207522661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8101d20 msgr2=0x7f0af81128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af77fe700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8101d20 0x7f0af81128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af77fe700 1 -- 172.21.15.7:0/4207522661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec005040 con 0x7f0af81026c0 2024-03-16T09:06:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af77fe700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 0x7f0af8112df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0aec000c00 tx=0x7f0aec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af4ff9700 1 -- 172.21.15.7:0/4207522661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aec00e040 con 0x7f0af81026c0 2024-03-16T09:06:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.466+0000 7f0af4ff9700 1 -- 172.21.15.7:0/4207522661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0aec00bd00 con 0x7f0af81026c0 2024-03-16T09:06:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.467+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af810ccc0 con 0x7f0af81026c0 2024-03-16T09:06:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.467+0000 7f0af4ff9700 1 -- 172.21.15.7:0/4207522661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aec013660 con 0x7f0af81026c0 2024-03-16T09:06:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.467+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af810d280 con 0x7f0af81026c0 2024-03-16T09:06:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.467+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af80fd470 con 0x7f0af81026c0 2024-03-16T09:06:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.468+0000 7f0af4ff9700 1 -- 172.21.15.7:0/4207522661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f0aec01e9f0 con 0x7f0af81026c0 2024-03-16T09:06:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.468+0000 7f0af4ff9700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ae006e0e0 0x7f0ae0070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.468+0000 7f0af4ff9700 1 -- 172.21.15.7:0/4207522661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0aec0913c0 con 0x7f0af81026c0 2024-03-16T09:06:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.469+0000 7f0af7fff700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ae006e0e0 0x7f0ae0070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.469+0000 7f0af7fff700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ae006e0e0 0x7f0ae0070590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8004f40 tx=0x7f0ae800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.473+0000 7f0af4ff9700 1 -- 172.21.15.7:0/4207522661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0aec05e1c0 con 0x7f0af81026c0 2024-03-16T09:06:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.701+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0af8108e00 con 0x7f0ae006e0e0 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.703+0000 7f0af4ff9700 1 -- 172.21.15.7:0/4207522661 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0af8108e00 con 0x7f0ae006e0e0 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:06:55.705 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:06:55.706 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "6/21 daemons upgraded", 2024-03-16T09:06:55.706 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T09:06:55.706 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:06:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:06:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ae006e0e0 msgr2=0x7f0ae0070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ae006e0e0 0x7f0ae0070590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8004f40 tx=0x7f0ae800b040 comp rx=0 tx=0).stop 2024-03-16T09:06:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 msgr2=0x7f0af8112df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 0x7f0af8112df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0aec000c00 tx=0x7f0aec009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 shutdown_connections 2024-03-16T09:06:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8101d20 0x7f0af81128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ae006e0e0 0x7f0ae0070590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 --2- 172.21.15.7:0/4207522661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0af81026c0 0x7f0af8112df0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 >> 172.21.15.7:0/4207522661 conn(0x7f0af80f7870 msgr2=0x7f0af80f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 shutdown_connections 2024-03-16T09:06:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.705+0000 7f0afe8ed700 1 -- 172.21.15.7:0/4207522661 wait complete. 2024-03-16T09:06:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.817+0000 7fa53ba7d700 1 -- 172.21.15.7:0/3769277953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534101d70 msgr2=0x7fa534102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.817+0000 7fa53ba7d700 1 --2- 172.21.15.7:0/3769277953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534101d70 0x7fa534102140 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa530004940 tx=0x7fa530009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:55.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.817+0000 7fa53ba7d700 1 -- 172.21.15.7:0/3769277953 shutdown_connections 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.817+0000 7fa53ba7d700 1 --2- 172.21.15.7:0/3769277953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa534102710 0x7fa5340fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.817+0000 7fa53ba7d700 1 --2- 172.21.15.7:0/3769277953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534101d70 0x7fa534102140 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.817+0000 7fa53ba7d700 1 -- 172.21.15.7:0/3769277953 >> 172.21.15.7:0/3769277953 conn(0x7fa534072bc0 msgr2=0x7fa534072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.817+0000 7fa53ba7d700 1 -- 172.21.15.7:0/3769277953 shutdown_connections 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.817+0000 7fa53ba7d700 1 -- 172.21.15.7:0/3769277953 wait complete. 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa53ba7d700 1 Processor -- start 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa53ba7d700 1 -- start start 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa53ba7d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa534101d70 0x7fa5341032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa53ba7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534102710 0x7fa534103810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa53ba7d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534103da0 con 0x7fa534102710 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa53ba7d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534103ee0 con 0x7fa534101d70 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa539018700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534102710 0x7fa534103810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa539018700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534102710 0x7fa534103810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa539018700 1 -- 172.21.15.7:0/1305647869 learned_addr learned my addr 172.21.15.7:0/1305647869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa539819700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa534101d70 0x7fa5341032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa539018700 1 -- 172.21.15.7:0/1305647869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa534101d70 msgr2=0x7fa5341032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa539018700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa534101d70 0x7fa5341032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa539018700 1 -- 172.21.15.7:0/1305647869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa530005040 con 0x7fa534102710 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.818+0000 7fa539018700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534102710 0x7fa534103810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa524002a50 tx=0x7fa524004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.819+0000 7fa52a7fc700 1 -- 172.21.15.7:0/1305647869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa524007cc0 con 0x7fa534102710 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.819+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa534119e20 con 0x7fa534102710 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.819+0000 7fa52a7fc700 1 -- 172.21.15.7:0/1305647869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa52400f040 con 0x7fa534102710 2024-03-16T09:06:55.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.819+0000 7fa52a7fc700 1 -- 172.21.15.7:0/1305647869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa52400d9c0 con 0x7fa534102710 2024-03-16T09:06:55.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.819+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53411a2a0 con 0x7fa534102710 2024-03-16T09:06:55.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.820+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5340fd4c0 con 0x7fa534102710 2024-03-16T09:06:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.820+0000 7fa52a7fc700 1 -- 172.21.15.7:0/1305647869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 38) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fa524009730 con 0x7fa534102710 2024-03-16T09:06:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.821+0000 7fa52a7fc700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa52006e000 0x7fa5200704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.821+0000 7fa52a7fc700 1 -- 172.21.15.7:0/1305647869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa52408d6a0 con 0x7fa534102710 2024-03-16T09:06:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.821+0000 7fa539819700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa52006e000 0x7fa5200704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.821+0000 7fa539819700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa52006e000 0x7fa5200704b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa53000b3b0 tx=0x7fa530009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:55.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:55.825+0000 7fa52a7fc700 1 -- 172.21.15.7:0/1305647869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa524056990 con 0x7fa534102710 2024-03-16T09:06:55.920 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:55 smithi007 ceph-mon[113546]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:06:55.921 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:55 smithi007 ceph-mon[113546]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:55.921 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:55.921 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:55.921 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:55.921 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:55 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3134379937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:55.921 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:55 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2590410027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:06:56.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.115+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa53405f640 con 0x7fa534102710 2024-03-16T09:06:56.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.116+0000 7fa52a7fc700 1 -- 172.21.15.7:0/1305647869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1328 (secure 0 0 0) 0x7fa52405a050 con 0x7fa534102710 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 68/354 objects degraded (19.209%), 19 pgs degraded 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: osd.0 (root=default,host=smithi007) is down 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 68/354 objects degraded (19.209%), 19 pgs degraded 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-16T09:06:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-16T09:06:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa52006e000 msgr2=0x7fa5200704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa52006e000 0x7fa5200704b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa53000b3b0 tx=0x7fa530009e30 comp rx=0 tx=0).stop 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534102710 msgr2=0x7fa534103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534102710 0x7fa534103810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa524002a50 tx=0x7fa524004600 comp rx=0 tx=0).stop 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 shutdown_connections 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa534101d70 0x7fa5341032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa52006e000 0x7fa5200704b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 --2- 172.21.15.7:0/1305647869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa534102710 0x7fa534103810 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 >> 172.21.15.7:0/1305647869 conn(0x7fa534072bc0 msgr2=0x7fa534105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 shutdown_connections 2024-03-16T09:06:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:06:56.118+0000 7fa53ba7d700 1 -- 172.21.15.7:0/1305647869 wait complete. 2024-03-16T09:06:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:55 smithi019 ceph-mon[83366]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:06:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:55 smithi019 ceph-mon[83366]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:06:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:55 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3134379937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:06:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:55 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2590410027' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:06:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:56 smithi007 ceph-mon[113546]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:56.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:56 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1305647869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:06:57.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:56 smithi019 ceph-mon[83366]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:56 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1305647869' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:06:57.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:57 smithi007 ceph-mon[113546]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:06:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:57 smithi019 ceph-mon[83366]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:06:59.248 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:59 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[125287]: 2024-03-16T09:06:59.061+0000 7f270a0b77c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-16T09:06:59.780 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:06:59 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[125287]: 2024-03-16T09:06:59.515+0000 7f270a0b77c0 -1 osd.0 49 log_to_monitors true 2024-03-16T09:07:00.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:59 smithi019 ceph-mon[83366]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:07:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:06:59 smithi019 ceph-mon[83366]: from='osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T09:07:00.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:59 smithi007 ceph-mon[113546]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:07:00.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:06:59 smithi007 ceph-mon[113546]: from='osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T09:07:00.998 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[125287]: 2024-03-16T09:07:00.563+0000 7f26f7507700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:07:00.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-mon[113546]: from='osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T09:07:00.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-mon[113546]: osdmap e52: 6 total, 5 up, 6 in 2024-03-16T09:07:00.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-mon[113546]: from='osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T09:07:00.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:00.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:00.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:00.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:07:00.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:00 smithi007 ceph-mon[113546]: from='osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313]' entity='osd.0' 2024-03-16T09:07:01.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:00 smithi019 ceph-mon[83366]: from='osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T09:07:01.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:00 smithi019 ceph-mon[83366]: osdmap e52: 6 total, 5 up, 6 in 2024-03-16T09:07:01.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:00 smithi019 ceph-mon[83366]: from='osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T09:07:01.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:01.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:01.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:01.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:07:01.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:00 smithi019 ceph-mon[83366]: from='osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313]' entity='osd.0' 2024-03-16T09:07:02.438 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:02 smithi007 ceph-mon[113546]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:07:02.438 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:02 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:02.438 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:02 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:02.438 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:02 smithi007 ceph-mon[113546]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:07:02.438 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:02 smithi007 ceph-mon[113546]: osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313] boot 2024-03-16T09:07:02.438 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:02 smithi007 ceph-mon[113546]: osdmap e53: 6 total, 6 up, 6 in 2024-03-16T09:07:02.438 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:02 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:07:02.439 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:02 smithi007 ceph-mon[113546]: mgrmap e39: smithi007.uamwtp(active, since 91s), standbys: smithi019.iuqxhi 2024-03-16T09:07:02.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:02 smithi019 ceph-mon[83366]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 68/354 objects degraded (19.209%) 2024-03-16T09:07:02.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:02 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:02.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:02 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:02.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:02 smithi019 ceph-mon[83366]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:07:02.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:02 smithi019 ceph-mon[83366]: osd.0 [v2:172.21.15.7:6802/2558509313,v1:172.21.15.7:6803/2558509313] boot 2024-03-16T09:07:02.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:02 smithi019 ceph-mon[83366]: osdmap e53: 6 total, 6 up, 6 in 2024-03-16T09:07:02.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:02 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:07:02.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:02 smithi019 ceph-mon[83366]: mgrmap e39: smithi007.uamwtp(active, since 91s), standbys: smithi019.iuqxhi 2024-03-16T09:07:03.871 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:03 smithi007 ceph-mon[113546]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 68/354 objects degraded (19.209%) 2024-03-16T09:07:03.872 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:03 smithi007 ceph-mon[113546]: osdmap e54: 6 total, 6 up, 6 in 2024-03-16T09:07:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:03 smithi019 ceph-mon[83366]: pgmap v53: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 68/354 objects degraded (19.209%) 2024-03-16T09:07:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:03 smithi019 ceph-mon[83366]: osdmap e54: 6 total, 6 up, 6 in 2024-03-16T09:07:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:05 smithi007 ceph-mon[113546]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 682 B/s wr, 0 op/s; 68/354 objects degraded (19.209%) 2024-03-16T09:07:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:05.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:05.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:05 smithi019 ceph-mon[83366]: pgmap v55: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 682 B/s wr, 0 op/s; 68/354 objects degraded (19.209%) 2024-03-16T09:07:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:06.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:06.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:06.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:06.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T09:07:06.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:06 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T09:07:06.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:06 smithi019 ceph-mon[83366]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-16T09:07:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T09:07:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:06 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T09:07:06.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:06 smithi007 ceph-mon[113546]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-16T09:07:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:07 smithi019 ceph-mon[83366]: pgmap v56: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 615 B/s wr, 0 op/s; 21 B/s, 0 objects/s recovering 2024-03-16T09:07:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:07 smithi019 ceph-mon[83366]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/354 objects degraded (19.209%), 19 pgs degraded) 2024-03-16T09:07:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:07 smithi019 ceph-mon[83366]: Cluster is now healthy 2024-03-16T09:07:07.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:07 smithi007 ceph-mon[113546]: pgmap v56: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 615 B/s wr, 0 op/s; 21 B/s, 0 objects/s recovering 2024-03-16T09:07:07.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:07 smithi007 ceph-mon[113546]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 68/354 objects degraded (19.209%), 19 pgs degraded) 2024-03-16T09:07:07.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:07 smithi007 ceph-mon[113546]: Cluster is now healthy 2024-03-16T09:07:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:09 smithi019 ceph-mon[83366]: pgmap v57: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 17 B/s, 0 objects/s recovering 2024-03-16T09:07:09.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:09 smithi007 ceph-mon[113546]: pgmap v57: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 17 B/s, 0 objects/s recovering 2024-03-16T09:07:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:11 smithi019 ceph-mon[83366]: pgmap v58: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 15 B/s, 0 objects/s recovering 2024-03-16T09:07:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:11 smithi007 ceph-mon[113546]: pgmap v58: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 15 B/s, 0 objects/s recovering 2024-03-16T09:07:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:13 smithi019 ceph-mon[83366]: pgmap v59: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 14 B/s, 0 objects/s recovering 2024-03-16T09:07:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:13 smithi007 ceph-mon[113546]: pgmap v59: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 14 B/s, 0 objects/s recovering 2024-03-16T09:07:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:15 smithi019 ceph-mon[83366]: pgmap v60: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 B/s, 0 objects/s recovering 2024-03-16T09:07:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:07:15.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:15 smithi007 ceph-mon[113546]: pgmap v60: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 B/s, 0 objects/s recovering 2024-03-16T09:07:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:15.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:07:15.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:17 smithi019 ceph-mon[83366]: pgmap v61: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 B/s, 0 objects/s recovering 2024-03-16T09:07:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:17 smithi007 ceph-mon[113546]: pgmap v61: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 B/s, 0 objects/s recovering 2024-03-16T09:07:19.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:19 smithi019 ceph-mon[83366]: pgmap v62: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:19.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:19 smithi007 ceph-mon[113546]: pgmap v62: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:20.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:20 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T09:07:20.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:20 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T09:07:21.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:21 smithi019 ceph-mon[83366]: pgmap v63: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:21.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:21 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T09:07:21.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:21 smithi019 ceph-mon[83366]: Upgrade: osd.1 is safe to restart 2024-03-16T09:07:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:21 smithi007 ceph-mon[113546]: pgmap v63: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:21 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T09:07:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:21 smithi007 ceph-mon[113546]: Upgrade: osd.1 is safe to restart 2024-03-16T09:07:23.467 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:23 smithi007 ceph-mon[113546]: Upgrade: Updating osd.1 2024-03-16T09:07:23.467 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:23.467 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T09:07:23.467 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:23 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:23.467 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:23 smithi007 ceph-mon[113546]: Deploying daemon osd.1 on smithi007 2024-03-16T09:07:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:23 smithi019 ceph-mon[83366]: Upgrade: Updating osd.1 2024-03-16T09:07:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T09:07:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:23 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:23.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:23 smithi019 ceph-mon[83366]: Deploying daemon osd.1 on smithi007 2024-03-16T09:07:24.371 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:24 smithi007 ceph-mon[113546]: pgmap v64: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:24.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:24 smithi019 ceph-mon[83366]: pgmap v64: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:25.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:25 smithi007 systemd[1]: Stopping Ceph osd.1 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:07:25.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:25 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[55550]: 2024-03-16T09:07:25.845+0000 7f2cf36f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:07:25.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:25 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[55550]: 2024-03-16T09:07:25.845+0000 7f2cf36f1700 -1 osd.1 54 *** Got signal Terminated *** 2024-03-16T09:07:25.999 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:25 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[55550]: 2024-03-16T09:07:25.845+0000 7f2cf36f1700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:07:26.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 -- 172.21.15.7:0/3437100070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf181037b0 msgr2=0x7fdf180fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 --2- 172.21.15.7:0/3437100070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf181037b0 0x7fdf180fce20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c004940 tx=0x7fdf0c009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 -- 172.21.15.7:0/3437100070 shutdown_connections 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 --2- 172.21.15.7:0/3437100070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf181037b0 0x7fdf180fce20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 --2- 172.21.15.7:0/3437100070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf18102e10 0x7fdf181031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 -- 172.21.15.7:0/3437100070 >> 172.21.15.7:0/3437100070 conn(0x7fdf180f89a0 msgr2=0x7fdf180fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 -- 172.21.15.7:0/3437100070 shutdown_connections 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 -- 172.21.15.7:0/3437100070 wait complete. 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.229+0000 7fdf1e82d700 1 Processor -- start 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf1e82d700 1 -- start start 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf1e82d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf18102e10 0x7fdf18113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf1e82d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf181037b0 0x7fdf181140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf1e82d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18114730 con 0x7fdf18102e10 2024-03-16T09:07:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf1e82d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1810db20 con 0x7fdf181037b0 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf18102e10 0x7fdf18113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf18102e10 0x7fdf18113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52748/0 (socket says 172.21.15.7:52748) 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf17fff700 1 -- 172.21.15.7:0/1592697132 learned_addr learned my addr 172.21.15.7:0/1592697132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.230+0000 7fdf177fe700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf181037b0 0x7fdf181140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf17fff700 1 -- 172.21.15.7:0/1592697132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf181037b0 msgr2=0x7fdf181140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf17fff700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf181037b0 0x7fdf181140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf17fff700 1 -- 172.21.15.7:0/1592697132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c005040 con 0x7fdf18102e10 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf17fff700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf18102e10 0x7fdf18113ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c8c0 tx=0x7fdf080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf14ff9700 1 -- 172.21.15.7:0/1592697132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0800a760 con 0x7fdf18102e10 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf14ff9700 1 -- 172.21.15.7:0/1592697132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf0800ad90 con 0x7fdf18102e10 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1810de00 con 0x7fdf18102e10 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf14ff9700 1 -- 172.21.15.7:0/1592697132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf080113a0 con 0x7fdf18102e10 2024-03-16T09:07:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.231+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1810e390 con 0x7fdf18102e10 2024-03-16T09:07:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.232+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf181052e0 con 0x7fdf18102e10 2024-03-16T09:07:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.233+0000 7fdf14ff9700 1 -- 172.21.15.7:0/1592697132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdf08012400 con 0x7fdf18102e10 2024-03-16T09:07:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.233+0000 7fdf14ff9700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf0006def0 0x7fdf000703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.233+0000 7fdf14ff9700 1 -- 172.21.15.7:0/1592697132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdf0808d6d0 con 0x7fdf18102e10 2024-03-16T09:07:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.233+0000 7fdf177fe700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf0006def0 0x7fdf000703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.234+0000 7fdf177fe700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf0006def0 0x7fdf000703a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf1810ee00 tx=0x7fdf0c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:26.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.238+0000 7fdf14ff9700 1 -- 172.21.15.7:0/1592697132 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf0805a790 con 0x7fdf18102e10 2024-03-16T09:07:26.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.470+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf1810f260 con 0x7fdf0006def0 2024-03-16T09:07:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.471+0000 7fdf14ff9700 1 -- 172.21.15.7:0/1592697132 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdf1810f260 con 0x7fdf0006def0 2024-03-16T09:07:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf0006def0 msgr2=0x7fdf000703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf0006def0 0x7fdf000703a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf1810ee00 tx=0x7fdf0c009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf18102e10 msgr2=0x7fdf18113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf18102e10 0x7fdf18113ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c8c0 tx=0x7fdf080044c0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 shutdown_connections 2024-03-16T09:07:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf0006def0 0x7fdf000703a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf18102e10 0x7fdf18113ba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 --2- 172.21.15.7:0/1592697132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf181037b0 0x7fdf181140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 >> 172.21.15.7:0/1592697132 conn(0x7fdf180f89a0 msgr2=0x7fdf18102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 shutdown_connections 2024-03-16T09:07:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.474+0000 7fdf1e82d700 1 -- 172.21.15.7:0/1592697132 wait complete. 2024-03-16T09:07:26.487 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:07:26.487 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:26 smithi007 ceph-mon[113546]: pgmap v65: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:26.487 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:26 smithi007 ceph-mon[113546]: osd.1 marked itself down and dead 2024-03-16T09:07:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.581+0000 7fbdf5460700 1 -- 172.21.15.7:0/1502235933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf0102e00 msgr2=0x7fbdf01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.581+0000 7fbdf5460700 1 --2- 172.21.15.7:0/1502235933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf0102e00 0x7fbdf01031d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbde0004940 tx=0x7fbde0009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.581+0000 7fbdf5460700 1 -- 172.21.15.7:0/1502235933 shutdown_connections 2024-03-16T09:07:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.581+0000 7fbdf5460700 1 --2- 172.21.15.7:0/1502235933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf01037a0 0x7fbdf00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.581+0000 7fbdf5460700 1 --2- 172.21.15.7:0/1502235933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf0102e00 0x7fbdf01031d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.581+0000 7fbdf5460700 1 -- 172.21.15.7:0/1502235933 >> 172.21.15.7:0/1502235933 conn(0x7fbdf00f89b0 msgr2=0x7fbdf00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.581+0000 7fbdf5460700 1 -- 172.21.15.7:0/1502235933 shutdown_connections 2024-03-16T09:07:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.581+0000 7fbdf5460700 1 -- 172.21.15.7:0/1502235933 wait complete. 2024-03-16T09:07:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdf5460700 1 Processor -- start 2024-03-16T09:07:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdf5460700 1 -- start start 2024-03-16T09:07:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdf5460700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0102e00 0x7fbdf0113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdf5460700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf01037a0 0x7fbdf01140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdf5460700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf01146f0 con 0x7fbdf01037a0 2024-03-16T09:07:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdf5460700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf010db30 con 0x7fbdf0102e00 2024-03-16T09:07:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdeeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0102e00 0x7fbdf0113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdee7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf01037a0 0x7fbdf01140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdeeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0102e00 0x7fbdf0113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:59166/0 (socket says 172.21.15.7:59166) 2024-03-16T09:07:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.582+0000 7fbdeeffd700 1 -- 172.21.15.7:0/1789587282 learned_addr learned my addr 172.21.15.7:0/1789587282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdeeffd700 1 -- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf01037a0 msgr2=0x7fbdf01140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdeeffd700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf01037a0 0x7fbdf01140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdeeffd700 1 -- 172.21.15.7:0/1789587282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0005040 con 0x7fbdf0102e00 2024-03-16T09:07:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdeeffd700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0102e00 0x7fbdf0113b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbde000b8b0 tx=0x7fbde0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdd77fe700 1 -- 172.21.15.7:0/1789587282 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde000e040 con 0x7fbdf0102e00 2024-03-16T09:07:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf010ddb0 con 0x7fbdf0102e00 2024-03-16T09:07:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdd77fe700 1 -- 172.21.15.7:0/1789587282 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbde0004290 con 0x7fbdf0102e00 2024-03-16T09:07:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdd77fe700 1 -- 172.21.15.7:0/1789587282 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde001c3f0 con 0x7fbdf0102e00 2024-03-16T09:07:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.583+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf010e370 con 0x7fbdf0102e00 2024-03-16T09:07:26.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.584+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf0100110 con 0x7fbdf0102e00 2024-03-16T09:07:26.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.585+0000 7fbdd77fe700 1 -- 172.21.15.7:0/1789587282 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fbde0013420 con 0x7fbdf0102e00 2024-03-16T09:07:26.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.586+0000 7fbdd77fe700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbdd806dee0 0x7fbdd8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.586+0000 7fbdd77fe700 1 -- 172.21.15.7:0/1789587282 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbde008e880 con 0x7fbdf0102e00 2024-03-16T09:07:26.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.586+0000 7fbdee7fc700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbdd806dee0 0x7fbdd8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.586+0000 7fbdee7fc700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbdd806dee0 0x7fbdd8070390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdf010eca0 tx=0x7fbde400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.591+0000 7fbdd77fe700 1 -- 172.21.15.7:0/1789587282 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbde005b960 con 0x7fbdf0102e00 2024-03-16T09:07:26.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:26 smithi019 ceph-mon[83366]: pgmap v65: 65 pgs: 65 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:26.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:26 smithi019 ceph-mon[83366]: osd.1 marked itself down and dead 2024-03-16T09:07:26.748 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:26 smithi007 podman[130548]: 2024-03-16 09:07:26.601928296 +0000 UTC m=+0.856184584 container died beacd6539c646b8526316b0be936576de710fc8351307ededdd94b3ff8500b1e (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=quincy-1df2724, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-16T09:07:26.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.821+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdf010f220 con 0x7fbdd806dee0 2024-03-16T09:07:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.822+0000 7fbdd77fe700 1 -- 172.21.15.7:0/1789587282 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbdf010f220 con 0x7fbdd806dee0 2024-03-16T09:07:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.824+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbdd806dee0 msgr2=0x7fbdd8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.824+0000 7fbdf5460700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbdd806dee0 0x7fbdd8070390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdf010eca0 tx=0x7fbde400b040 comp rx=0 tx=0).stop 2024-03-16T09:07:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.824+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0102e00 msgr2=0x7fbdf0113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.824+0000 7fbdf5460700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0102e00 0x7fbdf0113b60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbde000b8b0 tx=0x7fbde0009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.825+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 shutdown_connections 2024-03-16T09:07:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.825+0000 7fbdf5460700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdf0102e00 0x7fbdf0113b60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.825+0000 7fbdf5460700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbdd806dee0 0x7fbdd8070390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.825+0000 7fbdf5460700 1 --2- 172.21.15.7:0/1789587282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdf01037a0 0x7fbdf01140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.825+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 >> 172.21.15.7:0/1789587282 conn(0x7fbdf00f89b0 msgr2=0x7fbdf0107370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.825+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 shutdown_connections 2024-03-16T09:07:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.825+0000 7fbdf5460700 1 -- 172.21.15.7:0/1789587282 wait complete. 2024-03-16T09:07:26.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.930+0000 7fad7b3a5700 1 -- 172.21.15.7:0/2252676608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 msgr2=0x7fad740fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.930+0000 7fad7b3a5700 1 --2- 172.21.15.7:0/2252676608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 0x7fad740fce10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad68004940 tx=0x7fad68009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:26.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.930+0000 7fad7b3a5700 1 -- 172.21.15.7:0/2252676608 shutdown_connections 2024-03-16T09:07:26.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.930+0000 7fad7b3a5700 1 --2- 172.21.15.7:0/2252676608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 0x7fad740fce10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.930+0000 7fad7b3a5700 1 --2- 172.21.15.7:0/2252676608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad74102e00 0x7fad741031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.930+0000 7fad7b3a5700 1 -- 172.21.15.7:0/2252676608 >> 172.21.15.7:0/2252676608 conn(0x7fad740f89b0 msgr2=0x7fad740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.930+0000 7fad7b3a5700 1 -- 172.21.15.7:0/2252676608 shutdown_connections 2024-03-16T09:07:26.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.930+0000 7fad7b3a5700 1 -- 172.21.15.7:0/2252676608 wait complete. 2024-03-16T09:07:26.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad7b3a5700 1 Processor -- start 2024-03-16T09:07:26.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad7b3a5700 1 -- start start 2024-03-16T09:07:26.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad7b3a5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad74102e00 0x7fad740702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad7b3a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 0x7fad74070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad7b3a5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad74072210 con 0x7fad741037a0 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad7b3a5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad74070d40 con 0x7fad74102e00 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad78940700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 0x7fad74070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad78940700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 0x7fad74070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52782/0 (socket says 172.21.15.7:52782) 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad78940700 1 -- 172.21.15.7:0/3506843682 learned_addr learned my addr 172.21.15.7:0/3506843682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad79141700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad74102e00 0x7fad740702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad78940700 1 -- 172.21.15.7:0/3506843682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad74102e00 msgr2=0x7fad740702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad78940700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad74102e00 0x7fad740702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad78940700 1 -- 172.21.15.7:0/3506843682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68005040 con 0x7fad741037a0 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad79141700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad74102e00 0x7fad740702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.931+0000 7fad78940700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 0x7fad74070800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad68009100 tx=0x7fad68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.932+0000 7fad71ffb700 1 -- 172.21.15.7:0/3506843682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad6800e040 con 0x7fad741037a0 2024-03-16T09:07:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.932+0000 7fad71ffb700 1 -- 172.21.15.7:0/3506843682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad68004290 con 0x7fad741037a0 2024-03-16T09:07:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.932+0000 7fad71ffb700 1 -- 172.21.15.7:0/3506843682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad6801c3f0 con 0x7fad741037a0 2024-03-16T09:07:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.932+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad74070fc0 con 0x7fad741037a0 2024-03-16T09:07:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.932+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7411a0d0 con 0x7fad741037a0 2024-03-16T09:07:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.933+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad741052d0 con 0x7fad741037a0 2024-03-16T09:07:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.933+0000 7fad71ffb700 1 -- 172.21.15.7:0/3506843682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fad68013790 con 0x7fad741037a0 2024-03-16T09:07:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.934+0000 7fad71ffb700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fad5c06dee0 0x7fad5c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.934+0000 7fad71ffb700 1 -- 172.21.15.7:0/3506843682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fad6808e8c0 con 0x7fad741037a0 2024-03-16T09:07:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.934+0000 7fad79141700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fad5c06dee0 0x7fad5c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:26.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.934+0000 7fad79141700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fad5c06dee0 0x7fad5c070390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad6c004550 tx=0x7fad6c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:26.939+0000 7fad71ffb700 1 -- 172.21.15.7:0/3506843682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fad68057ef0 con 0x7fad741037a0 2024-03-16T09:07:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.159+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fad74109ee0 con 0x7fad5c06dee0 2024-03-16T09:07:27.162 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:26 smithi007 podman[130548]: 2024-03-16 09:07:26.979891219 +0000 UTC m=+1.234147520 container cleanup beacd6539c646b8526316b0be936576de710fc8351307ededdd94b3ff8500b1e (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=quincy-1df2724) 2024-03-16T09:07:27.162 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:26 smithi007 bash[130548]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.171+0000 7fad71ffb700 1 -- 172.21.15.7:0/3506843682 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fad74109ee0 con 0x7fad5c06dee0 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (3m) 26s ago 14m 29.0M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (59s) 26s ago 14m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (54s) 50s ago 13m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (3m) 26s ago 14m 78.4M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (8m) 26s ago 8m 14.0M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (8m) 26s ago 8m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (8m) 50s ago 8m 120M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:07:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (8m) 50s ago 8m 14.1M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (5m) 26s ago 15m 501M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (4m) 50s ago 13m 449M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (2m) 26s ago 15m 48.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (100s) 50s ago 13m 34.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (4m) 26s ago 14m 15.6M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (4m) 50s ago 13m 15.2M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (32s) 26s ago 12m 12.6M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (12m) 26s ago 12m 236M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce beacd6539c64 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (11m) 26s ago 11m 181M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (11m) 50s ago 11m 264M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (10m) 50s ago 10m 247M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (10m) 50s ago 10m 223M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:07:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (3m) 26s ago 13m 61.5M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:07:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fad5c06dee0 msgr2=0x7fad5c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fad5c06dee0 0x7fad5c070390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad6c004550 tx=0x7fad6c009040 comp rx=0 tx=0).stop 2024-03-16T09:07:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 msgr2=0x7fad74070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 0x7fad74070800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad68009100 tx=0x7fad68009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 shutdown_connections 2024-03-16T09:07:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad74102e00 0x7fad740702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fad5c06dee0 0x7fad5c070390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 --2- 172.21.15.7:0/3506843682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad741037a0 0x7fad74070800 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 >> 172.21.15.7:0/3506843682 conn(0x7fad740f89b0 msgr2=0x7fad741021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 shutdown_connections 2024-03-16T09:07:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.173+0000 7fad7b3a5700 1 -- 172.21.15.7:0/3506843682 wait complete. 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/147834693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb4102e70 msgr2=0x7fdfb4103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 --2- 172.21.15.7:0/147834693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb4102e70 0x7fdfb4103240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4004940 tx=0x7fdfa4009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/147834693 shutdown_connections 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 --2- 172.21.15.7:0/147834693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb40fb4a0 0x7fdfb40fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 --2- 172.21.15.7:0/147834693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb4102e70 0x7fdfb4103240 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/147834693 >> 172.21.15.7:0/147834693 conn(0x7fdfb40714c0 msgr2=0x7fdfb40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/147834693 shutdown_connections 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/147834693 wait complete. 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.288+0000 7fdfb9ccc700 1 Processor -- start 2024-03-16T09:07:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb9ccc700 1 -- start start 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb9ccc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb40fb4a0 0x7fdfb40fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb9ccc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb4102e70 0x7fdfb40fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb9ccc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb40fd150 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb9ccc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb40fd290 con 0x7fdfb4102e70 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb40fb4a0 0x7fdfb40fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb40fb4a0 0x7fdfb40fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52802/0 (socket says 172.21.15.7:52802) 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb37fe700 1 -- 172.21.15.7:0/27916504 learned_addr learned my addr 172.21.15.7:0/27916504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb2ffd700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb4102e70 0x7fdfb40fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb37fe700 1 -- 172.21.15.7:0/27916504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb4102e70 msgr2=0x7fdfb40fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb37fe700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb4102e70 0x7fdfb40fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb37fe700 1 -- 172.21.15.7:0/27916504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4005040 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.289+0000 7fdfb2ffd700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb4102e70 0x7fdfb40fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:07:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.290+0000 7fdfb37fe700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb40fb4a0 0x7fdfb40fc680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009b30 tx=0x7fdfa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.290+0000 7fdf9bfff700 1 -- 172.21.15.7:0/27916504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa4021070 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.290+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb40fd4e0 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.290+0000 7fdf9bfff700 1 -- 172.21.15.7:0/27916504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfa4013c00 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.290+0000 7fdf9bfff700 1 -- 172.21.15.7:0/27916504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa400e070 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.290+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb411a120 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.291+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb4100110 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.291+0000 7fdf9bfff700 1 -- 172.21.15.7:0/27916504 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdfa401cad0 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.292+0000 7fdf9bfff700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf9c06dee0 0x7fdf9c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:27.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.292+0000 7fdf9bfff700 1 -- 172.21.15.7:0/27916504 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdfa408eae0 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.292+0000 7fdfb2ffd700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf9c06dee0 0x7fdf9c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.292+0000 7fdfb2ffd700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf9c06dee0 0x7fdf9c070390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40fdfc0 tx=0x7fdfa800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:27.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.297+0000 7fdf9bfff700 1 -- 172.21.15.7:0/27916504 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdfa4058070 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:27 smithi007 ceph-mon[113546]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:07:27.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:27 smithi007 ceph-mon[113546]: osdmap e55: 6 total, 5 up, 6 in 2024-03-16T09:07:27.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:27 smithi007 ceph-mon[113546]: osdmap e56: 6 total, 5 up, 6 in 2024-03-16T09:07:27.498 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:27 smithi007 podman[130614]: 2024-03-16 09:07:27.383000206 +0000 UTC m=+0.773833708 container remove beacd6539c646b8526316b0be936576de710fc8351307ededdd94b3ff8500b1e (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=quincy-1df2724, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T09:07:27.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.630+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdfb404bed0 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.631+0000 7fdf9bfff700 1 -- 172.21.15.7:0/27916504 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fdfa4004490 con 0x7fdfb40fb4a0 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:07:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4, 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 8, 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 5 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:07:27.634 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:07:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf9c06dee0 msgr2=0x7fdf9c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf9c06dee0 0x7fdf9c070390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40fdfc0 tx=0x7fdfa800b040 comp rx=0 tx=0).stop 2024-03-16T09:07:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb40fb4a0 msgr2=0x7fdfb40fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb40fb4a0 0x7fdfb40fc680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009b30 tx=0x7fdfa4009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 shutdown_connections 2024-03-16T09:07:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdf9c06dee0 0x7fdf9c070390 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb40fb4a0 0x7fdfb40fc680 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 --2- 172.21.15.7:0/27916504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb4102e70 0x7fdfb40fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 >> 172.21.15.7:0/27916504 conn(0x7fdfb40714c0 msgr2=0x7fdfb40fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 shutdown_connections 2024-03-16T09:07:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.633+0000 7fdfb9ccc700 1 -- 172.21.15.7:0/27916504 wait complete. 2024-03-16T09:07:27.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:27 smithi019 ceph-mon[83366]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:07:27.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:27 smithi019 ceph-mon[83366]: osdmap e55: 6 total, 5 up, 6 in 2024-03-16T09:07:27.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:27 smithi019 ceph-mon[83366]: osdmap e56: 6 total, 5 up, 6 in 2024-03-16T09:07:27.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.736+0000 7f31aa813700 1 -- 172.21.15.7:0/3337454498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 msgr2=0x7f31a40fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.736+0000 7f31aa813700 1 --2- 172.21.15.7:0/3337454498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 0x7f31a40fbd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3198004940 tx=0x7f3198009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.736+0000 7f31aa813700 1 -- 172.21.15.7:0/3337454498 shutdown_connections 2024-03-16T09:07:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.736+0000 7f31aa813700 1 --2- 172.21.15.7:0/3337454498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 0x7f31a40fbd30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.736+0000 7f31aa813700 1 --2- 172.21.15.7:0/3337454498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a4101d20 0x7f31a41020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.736+0000 7f31aa813700 1 -- 172.21.15.7:0/3337454498 >> 172.21.15.7:0/3337454498 conn(0x7f31a40f7870 msgr2=0x7f31a40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.736+0000 7f31aa813700 1 -- 172.21.15.7:0/3337454498 shutdown_connections 2024-03-16T09:07:27.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.736+0000 7f31aa813700 1 -- 172.21.15.7:0/3337454498 wait complete. 2024-03-16T09:07:27.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31aa813700 1 Processor -- start 2024-03-16T09:07:27.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31aa813700 1 -- start start 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31aa813700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a4101d20 0x7f31a4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31aa813700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 0x7f31a4113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31aa813700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4114520 con 0x7f31a41026c0 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31aa813700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a410db20 con 0x7f31a4101d20 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 0x7f31a4113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 0x7f31a4113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52810/0 (socket says 172.21.15.7:52810) 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.737+0000 7f31a37fe700 1 -- 172.21.15.7:0/537153505 learned_addr learned my addr 172.21.15.7:0/537153505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31a3fff700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a4101d20 0x7f31a4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31a37fe700 1 -- 172.21.15.7:0/537153505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a4101d20 msgr2=0x7f31a4113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31a37fe700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a4101d20 0x7f31a4113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31a37fe700 1 -- 172.21.15.7:0/537153505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3198005040 con 0x7f31a41026c0 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31a37fe700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 0x7f31a4113ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3198009100 tx=0x7f3198009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31a0ff9700 1 -- 172.21.15.7:0/537153505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319800e040 con 0x7f31a41026c0 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31a0ff9700 1 -- 172.21.15.7:0/537153505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3198004290 con 0x7f31a41026c0 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a410dda0 con 0x7f31a41026c0 2024-03-16T09:07:27.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31a0ff9700 1 -- 172.21.15.7:0/537153505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319801c3f0 con 0x7f31a41026c0 2024-03-16T09:07:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.738+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a410e360 con 0x7f31a41026c0 2024-03-16T09:07:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.739+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a40fd470 con 0x7f31a41026c0 2024-03-16T09:07:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.740+0000 7f31a0ff9700 1 -- 172.21.15.7:0/537153505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3198013790 con 0x7f31a41026c0 2024-03-16T09:07:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.740+0000 7f31a0ff9700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f318c06dee0 0x7f318c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.740+0000 7f31a0ff9700 1 -- 172.21.15.7:0/537153505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f319808e970 con 0x7f31a41026c0 2024-03-16T09:07:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.740+0000 7f31a3fff700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f318c06dee0 0x7f318c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.741+0000 7f31a3fff700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f318c06dee0 0x7f318c070390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3194004990 tx=0x7f3194009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:27.745+0000 7f31a0ff9700 1 -- 172.21.15.7:0/537153505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f319805ba30 con 0x7f31a41026c0 2024-03-16T09:07:27.947 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:27 smithi007 podman[130747]: 2024-03-16 09:07:27.6108051 +0000 UTC m=+0.018273912 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:07:27.947 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:27 smithi007 podman[130747]: 2024-03-16 09:07:27.725942056 +0000 UTC m=+0.133410847 container create e6b0b724e71415829dd9be8aa76baedca89b60335ca5389fc673ecb4f1c057c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-16T09:07:27.947 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:27 smithi007 podman[130747]: 2024-03-16 09:07:27.945390891 +0000 UTC m=+0.352859684 container init e6b0b724e71415829dd9be8aa76baedca89b60335ca5389fc673ecb4f1c057c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-03-16T09:07:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.022+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f31a410efe0 con 0x7f31a41026c0 2024-03-16T09:07:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.023+0000 7f31a0ff9700 1 -- 172.21.15.7:0/537153505 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1895 (secure 0 0 0) 0x7f3198013420 con 0x7f31a41026c0 2024-03-16T09:07:28.025 INFO:teuthology.orchestra.run.smithi007.stdout:e20 2024-03-16T09:07:28.025 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:07:28.025 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:07:28.025 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:07:28.026 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:07:28.027 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:07:28.028 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:07:28.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:07:28.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:07:28.028 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:07:28.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:07:28.028 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:07:28.028 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f318c06dee0 msgr2=0x7f318c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f318c06dee0 0x7f318c070390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3194004990 tx=0x7f3194009ee0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 msgr2=0x7f31a4113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 0x7f31a4113ed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3198009100 tx=0x7f3198009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 shutdown_connections 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31a4101d20 0x7f31a4113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f318c06dee0 0x7f318c070390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 --2- 172.21.15.7:0/537153505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a41026c0 0x7f31a4113ed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 >> 172.21.15.7:0/537153505 conn(0x7f31a40f7870 msgr2=0x7f31a40f8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 shutdown_connections 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.025+0000 7f31aa813700 1 -- 172.21.15.7:0/537153505 wait complete. 2024-03-16T09:07:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:07:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.130+0000 7f7aa3156700 1 -- 172.21.15.7:0/2448641782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c101d70 msgr2=0x7f7a9c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.130+0000 7f7aa3156700 1 --2- 172.21.15.7:0/2448641782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c101d70 0x7f7a9c102140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a90004940 tx=0x7f7a90009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 -- 172.21.15.7:0/2448641782 shutdown_connections 2024-03-16T09:07:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 --2- 172.21.15.7:0/2448641782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a9c102710 0x7f7a9c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 --2- 172.21.15.7:0/2448641782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c101d70 0x7f7a9c102140 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 -- 172.21.15.7:0/2448641782 >> 172.21.15.7:0/2448641782 conn(0x7f7a9c072bc0 msgr2=0x7f7a9c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 -- 172.21.15.7:0/2448641782 shutdown_connections 2024-03-16T09:07:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 -- 172.21.15.7:0/2448641782 wait complete. 2024-03-16T09:07:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 Processor -- start 2024-03-16T09:07:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 -- start start 2024-03-16T09:07:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a9c101d70 0x7f7a9c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c102710 0x7f7a9c103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c1036a0 con 0x7f7a9c102710 2024-03-16T09:07:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.131+0000 7f7aa3156700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c1037e0 con 0x7f7a9c101d70 2024-03-16T09:07:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c102710 0x7f7a9c103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c102710 0x7f7a9c103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52822/0 (socket says 172.21.15.7:52822) 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a9bfff700 1 -- 172.21.15.7:0/2827531098 learned_addr learned my addr 172.21.15.7:0/2827531098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7aa0ef2700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a9c101d70 0x7f7a9c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a9bfff700 1 -- 172.21.15.7:0/2827531098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a9c101d70 msgr2=0x7f7a9c102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a9bfff700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a9c101d70 0x7f7a9c102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a9bfff700 1 -- 172.21.15.7:0/2827531098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90005040 con 0x7f7a9c102710 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7aa0ef2700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a9c101d70 0x7f7a9c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a9bfff700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c102710 0x7f7a9c103110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800eb40 tx=0x7f7a8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a997fa700 1 -- 172.21.15.7:0/2827531098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a88007cc0 con 0x7f7a9c102710 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7a997fa700 1 -- 172.21.15.7:0/2827531098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a8800f040 con 0x7f7a9c102710 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.132+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c103a30 con 0x7f7a9c102710 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.133+0000 7f7a997fa700 1 -- 172.21.15.7:0/2827531098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a8800cad0 con 0x7f7a9c102710 2024-03-16T09:07:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.133+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c11a1c0 con 0x7f7a9c102710 2024-03-16T09:07:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.134+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c0fd4c0 con 0x7f7a9c102710 2024-03-16T09:07:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.134+0000 7f7a997fa700 1 -- 172.21.15.7:0/2827531098 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f7a8800cc30 con 0x7f7a9c102710 2024-03-16T09:07:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.134+0000 7f7a997fa700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7a8c06dee0 0x7f7a8c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.135+0000 7f7a997fa700 1 -- 172.21.15.7:0/2827531098 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7a8808d1a0 con 0x7f7a9c102710 2024-03-16T09:07:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.135+0000 7f7aa0ef2700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7a8c06dee0 0x7f7a8c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.135+0000 7f7aa0ef2700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7a8c06dee0 0x7f7a8c070390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009b30 tx=0x7f7a90009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.139+0000 7f7a997fa700 1 -- 172.21.15.7:0/2827531098 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a88056730 con 0x7f7a9c102710 2024-03-16T09:07:28.244 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:27 smithi007 podman[130747]: 2024-03-16 09:07:27.949276518 +0000 UTC m=+0.356745311 container start e6b0b724e71415829dd9be8aa76baedca89b60335ca5389fc673ecb4f1c057c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-16T09:07:28.244 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:28 smithi007 podman[130747]: 2024-03-16 09:07:28.079272783 +0000 UTC m=+0.486741579 container attach e6b0b724e71415829dd9be8aa76baedca89b60335ca5389fc673ecb4f1c057c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-16T09:07:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.372+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a9c108f30 con 0x7f7a8c06dee0 2024-03-16T09:07:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.374+0000 7f7a997fa700 1 -- 172.21.15.7:0/2827531098 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7a90009360 con 0x7f7a8c06dee0 2024-03-16T09:07:28.376 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:07:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:07:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:07:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:07:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:07:28.376 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-16T09:07:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:07:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:07:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:07:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "7/21 daemons upgraded", 2024-03-16T09:07:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T09:07:28.377 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:07:28.377 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:07:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7a8c06dee0 msgr2=0x7f7a8c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7a8c06dee0 0x7f7a8c070390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009b30 tx=0x7f7a90009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c102710 msgr2=0x7f7a9c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c102710 0x7f7a9c103110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800eb40 tx=0x7f7a8800b040 comp rx=0 tx=0).stop 2024-03-16T09:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 shutdown_connections 2024-03-16T09:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a9c101d70 0x7f7a9c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7a8c06dee0 0x7f7a8c070390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 --2- 172.21.15.7:0/2827531098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a9c102710 0x7f7a9c103110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 >> 172.21.15.7:0/2827531098 conn(0x7f7a9c072bc0 msgr2=0x7f7a9c0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 shutdown_connections 2024-03-16T09:07:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.376+0000 7f7aa3156700 1 -- 172.21.15.7:0/2827531098 wait complete. 2024-03-16T09:07:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 -- 172.21.15.7:0/2462876278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 msgr2=0x7f923c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 --2- 172.21.15.7:0/2462876278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 0x7f923c1031b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9234009ce0 tx=0x7f9234004d10 comp rx=0 tx=0).stop 2024-03-16T09:07:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 -- 172.21.15.7:0/2462876278 shutdown_connections 2024-03-16T09:07:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 --2- 172.21.15.7:0/2462876278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f923c103780 0x7f923c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 --2- 172.21.15.7:0/2462876278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 0x7f923c1031b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 -- 172.21.15.7:0/2462876278 >> 172.21.15.7:0/2462876278 conn(0x7f923c0f8950 msgr2=0x7f923c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 -- 172.21.15.7:0/2462876278 shutdown_connections 2024-03-16T09:07:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 -- 172.21.15.7:0/2462876278 wait complete. 2024-03-16T09:07:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 Processor -- start 2024-03-16T09:07:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.479+0000 7f92443d4700 1 -- start start 2024-03-16T09:07:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f92443d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 0x7f923c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f92443d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f923c103780 0x7f923c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f92443d4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c072210 con 0x7f923c102de0 2024-03-16T09:07:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f92443d4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c070d40 con 0x7f923c103780 2024-03-16T09:07:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9242170700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 0x7f923c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9242170700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 0x7f923c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52834/0 (socket says 172.21.15.7:52834) 2024-03-16T09:07:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9242170700 1 -- 172.21.15.7:0/257105269 learned_addr learned my addr 172.21.15.7:0/257105269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f924196f700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f923c103780 0x7f923c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9242170700 1 -- 172.21.15.7:0/257105269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f923c103780 msgr2=0x7f923c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9242170700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f923c103780 0x7f923c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9242170700 1 -- 172.21.15.7:0/257105269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234009940 con 0x7f923c102de0 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f924196f700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f923c103780 0x7f923c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9242170700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 0x7f923c0702c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f923400be90 tx=0x7f9234004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9232ffd700 1 -- 172.21.15.7:0/257105269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923400e040 con 0x7f923c102de0 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9232ffd700 1 -- 172.21.15.7:0/257105269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9234008ea0 con 0x7f923c102de0 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c070fc0 con 0x7f923c102de0 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f9232ffd700 1 -- 172.21.15.7:0/257105269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923401c810 con 0x7f923c102de0 2024-03-16T09:07:28.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.480+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c11a230 con 0x7f923c102de0 2024-03-16T09:07:28.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.481+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c1000f0 con 0x7f923c102de0 2024-03-16T09:07:28.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.482+0000 7f9232ffd700 1 -- 172.21.15.7:0/257105269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f9234012c00 con 0x7f923c102de0 2024-03-16T09:07:28.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.482+0000 7f9232ffd700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f922806dee0 0x7f9228070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:28.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.482+0000 7f9232ffd700 1 -- 172.21.15.7:0/257105269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f923408f210 con 0x7f923c102de0 2024-03-16T09:07:28.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.482+0000 7f924196f700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f922806dee0 0x7f9228070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:28.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.483+0000 7f924196f700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f922806dee0 0x7f9228070390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f923c0719f0 tx=0x7f922c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.487+0000 7f9232ffd700 1 -- 172.21.15.7:0/257105269 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f92340587a0 con 0x7f923c102de0 2024-03-16T09:07:28.498 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:28 smithi007 podman[130747]: 2024-03-16 09:07:28.243233761 +0000 UTC m=+0.650702566 container died e6b0b724e71415829dd9be8aa76baedca89b60335ca5389fc673ecb4f1c057c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T09:07:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:28 smithi007 ceph-mon[113546]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:28 smithi007 ceph-mon[113546]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:28 smithi007 ceph-mon[113546]: from='client.44153 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:28 smithi007 ceph-mon[113546]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:28 smithi007 ceph-mon[113546]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-16T09:07:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:28 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/27916504' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:28 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/537153505' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:07:28.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:28 smithi019 ceph-mon[83366]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:28.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:28 smithi019 ceph-mon[83366]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:28.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:28 smithi019 ceph-mon[83366]: from='client.44153 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:28.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:28 smithi019 ceph-mon[83366]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:28.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:28 smithi019 ceph-mon[83366]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-16T09:07:28.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:28 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/27916504' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:28.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:28 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/537153505' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:07:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.785+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f923c071860 con 0x7f923c102de0 2024-03-16T09:07:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.786+0000 7f9232ffd700 1 -- 172.21.15.7:0/257105269 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+901 (secure 0 0 0) 0x7f9234004e10 con 0x7f923c102de0 2024-03-16T09:07:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 9 pgs peering 2024-03-16T09:07:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-16T09:07:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: osd.1 (root=default,host=smithi007) is down 2024-03-16T09:07:28.788 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 9 pgs peering 2024-03-16T09:07:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.6 is stuck peering for 4m, current state peering, last acting [3,4] 2024-03-16T09:07:28.788 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.9 is stuck peering for 4m, current state peering, last acting [4,0] 2024-03-16T09:07:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.a is stuck peering for 4m, current state peering, last acting [4,3] 2024-03-16T09:07:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.d is stuck peering for 5m, current state peering, last acting [3,2] 2024-03-16T09:07:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: pg 2.15 is stuck peering for 4m, current state peering, last acting [3,0] 2024-03-16T09:07:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: pg 3.0 is stuck peering for 4m, current state peering, last acting [2,4] 2024-03-16T09:07:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: pg 3.2 is stuck peering for 4m, current state peering, last acting [5,3] 2024-03-16T09:07:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: pg 3.4 is stuck peering for 4m, current state peering, last acting [2,3] 2024-03-16T09:07:28.789 INFO:teuthology.orchestra.run.smithi007.stdout: pg 3.19 is stuck peering for 4m, current state peering, last acting [4,3] 2024-03-16T09:07:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f922806dee0 msgr2=0x7f9228070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f922806dee0 0x7f9228070390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f923c0719f0 tx=0x7f922c009040 comp rx=0 tx=0).stop 2024-03-16T09:07:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 msgr2=0x7f923c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 0x7f923c0702c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f923400be90 tx=0x7f9234004d10 comp rx=0 tx=0).stop 2024-03-16T09:07:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 shutdown_connections 2024-03-16T09:07:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f922806dee0 0x7f9228070390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f923c102de0 0x7f923c0702c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 --2- 172.21.15.7:0/257105269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f923c103780 0x7f923c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 >> 172.21.15.7:0/257105269 conn(0x7f923c0f8950 msgr2=0x7f923c107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 shutdown_connections 2024-03-16T09:07:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:28.788+0000 7f92443d4700 1 -- 172.21.15.7:0/257105269 wait complete. 2024-03-16T09:07:29.248 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:28 smithi007 podman[130819]: 2024-03-16 09:07:28.918972478 +0000 UTC m=+0.670039792 container remove e6b0b724e71415829dd9be8aa76baedca89b60335ca5389fc673ecb4f1c057c9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:07:29.248 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:28 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-16T09:07:29.248 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:28 smithi007 systemd[1]: Stopped Ceph osd.1 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:07:29.249 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:28 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.1.service: Consumed 1min 1.352s CPU time. 2024-03-16T09:07:29.249 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:29 smithi007 systemd[1]: Starting Ceph osd.1 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:07:29.565 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:29 smithi007 ceph-mon[113546]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:29.565 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:29 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/257105269' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:07:29.565 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:29 smithi007 podman[130948]: 2024-03-16 09:07:29.352398259 +0000 UTC m=+0.112025715 container create e3b29335cbd3cfd439620a290edc9083009f766b3e3653599669446143217c14 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-16T09:07:29.565 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:29 smithi007 podman[130948]: 2024-03-16 09:07:29.258657783 +0000 UTC m=+0.018285258 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:07:29.565 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:29 smithi007 podman[130948]: 2024-03-16 09:07:29.563672394 +0000 UTC m=+0.323299870 container init e3b29335cbd3cfd439620a290edc9083009f766b3e3653599669446143217c14 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T09:07:29.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:29 smithi019 ceph-mon[83366]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:29.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:29 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/257105269' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:07:29.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:29 smithi007 podman[130948]: 2024-03-16 09:07:29.567319551 +0000 UTC m=+0.326947018 container start e3b29335cbd3cfd439620a290edc9083009f766b3e3653599669446143217c14 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:07:29.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:29 smithi007 podman[130948]: 2024-03-16 09:07:29.663429066 +0000 UTC m=+0.423056523 container attach e3b29335cbd3cfd439620a290edc9083009f766b3e3653599669446143217c14 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:07:30.275 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate[130962]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T09:07:30.275 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate[130962]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-16T09:07:30.275 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 bash[130948]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T09:07:30.275 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 bash[130948]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 bash[130948]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 bash[130948]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 bash[130948]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 bash[130948]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 bash[130948]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate[130962]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate[130962]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate[130962]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate[130962]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate[130962]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-16T09:07:30.276 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 podman[130948]: 2024-03-16 09:07:30.043532313 +0000 UTC m=+0.803159770 container died e3b29335cbd3cfd439620a290edc9083009f766b3e3653599669446143217c14 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T09:07:30.705 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:30 smithi007 ceph-mon[113546]: pgmap v69: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:30.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:30 smithi019 ceph-mon[83366]: pgmap v69: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:30.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 podman[131101]: 2024-03-16 09:07:30.703671535 +0000 UTC m=+0.655473767 container remove e3b29335cbd3cfd439620a290edc9083009f766b3e3653599669446143217c14 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T09:07:30.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 podman[131167]: 2024-03-16 09:07:30.819797782 +0000 UTC m=+0.018267054 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:07:30.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:30 smithi007 podman[131167]: 2024-03-16 09:07:30.937985556 +0000 UTC m=+0.136454822 container create 4aa10e800f7285b7813be7848ff2e0c0c41e50e276022899d5b20fa59fd11cec (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-16T09:07:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:31 smithi007 ceph-mon[113546]: pgmap v70: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 7/354 objects degraded (1.977%) 2024-03-16T09:07:31.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:31.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:07:31.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:31 smithi007 ceph-mon[113546]: Health check failed: Degraded data redundancy: 7/354 objects degraded (1.977%), 2 pgs degraded (PG_DEGRADED) 2024-03-16T09:07:31.707 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:31 smithi007 podman[131167]: 2024-03-16 09:07:31.366080309 +0000 UTC m=+0.564549589 container init 4aa10e800f7285b7813be7848ff2e0c0c41e50e276022899d5b20fa59fd11cec (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T09:07:31.707 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:31 smithi007 podman[131167]: 2024-03-16 09:07:31.369606129 +0000 UTC m=+0.568075398 container start 4aa10e800f7285b7813be7848ff2e0c0c41e50e276022899d5b20fa59fd11cec (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True) 2024-03-16T09:07:31.707 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:31 smithi007 ceph-osd[131187]: -- 172.21.15.7:0/1720752226 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a3d0cd680 con 0x558a3d096c00 2024-03-16T09:07:31.707 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:31 smithi007 bash[131167]: 4aa10e800f7285b7813be7848ff2e0c0c41e50e276022899d5b20fa59fd11cec 2024-03-16T09:07:31.707 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:31 smithi007 systemd[1]: Started Ceph osd.1 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:07:31.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:31 smithi019 ceph-mon[83366]: pgmap v70: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 7/354 objects degraded (1.977%) 2024-03-16T09:07:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:31 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:31 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:07:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:31 smithi019 ceph-mon[83366]: Health check failed: Degraded data redundancy: 7/354 objects degraded (1.977%), 2 pgs degraded (PG_DEGRADED) 2024-03-16T09:07:31.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:31 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[131183]: 2024-03-16T09:07:31.913+0000 7f901f8f57c0 -1 Falling back to public interface 2024-03-16T09:07:32.568 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:32 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:32.568 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:32 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:32.568 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:32 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:32 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:32 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:32.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:32 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:33.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:33 smithi007 ceph-mon[113546]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:33.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:33 smithi007 ceph-mon[113546]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-16T09:07:33.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:33 smithi019 ceph-mon[83366]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:33 smithi019 ceph-mon[83366]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-16T09:07:35.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:35 smithi007 ceph-mon[113546]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:35.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:35 smithi019 ceph-mon[83366]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:36.748 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:36 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[131183]: 2024-03-16T09:07:36.459+0000 7f901f8f57c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-16T09:07:37.498 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:37 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[131183]: 2024-03-16T09:07:37.097+0000 7f901f8f57c0 -1 osd.1 54 log_to_monitors true 2024-03-16T09:07:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:37 smithi019 ceph-mon[83366]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:37 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 56/354 objects degraded (15.819%), 16 pgs degraded (PG_DEGRADED) 2024-03-16T09:07:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:37 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:37 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:37 smithi019 ceph-mon[83366]: from='osd.1 [v2:172.21.15.7:6810/3984255151,v1:172.21.15.7:6811/3984255151]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T09:07:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:37 smithi007 ceph-mon[113546]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:37 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 56/354 objects degraded (15.819%), 16 pgs degraded (PG_DEGRADED) 2024-03-16T09:07:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:37 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:37 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:37 smithi007 ceph-mon[113546]: from='osd.1 [v2:172.21.15.7:6810/3984255151,v1:172.21.15.7:6811/3984255151]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T09:07:38.498 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:07:38 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[131183]: 2024-03-16T09:07:38.032+0000 7f9010d4d700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:07:39.063 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:39 smithi007 ceph-mon[113546]: from='osd.1 [v2:172.21.15.7:6810/3984255151,v1:172.21.15.7:6811/3984255151]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T09:07:39.063 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:39 smithi007 ceph-mon[113546]: osdmap e57: 6 total, 5 up, 6 in 2024-03-16T09:07:39.063 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:39 smithi007 ceph-mon[113546]: from='osd.1 [v2:172.21.15.7:6810/3984255151,v1:172.21.15.7:6811/3984255151]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T09:07:39.063 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:39 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:39.063 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:39 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:39.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:39 smithi019 ceph-mon[83366]: from='osd.1 [v2:172.21.15.7:6810/3984255151,v1:172.21.15.7:6811/3984255151]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T09:07:39.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:39 smithi019 ceph-mon[83366]: osdmap e57: 6 total, 5 up, 6 in 2024-03-16T09:07:39.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:39 smithi019 ceph-mon[83366]: from='osd.1 [v2:172.21.15.7:6810/3984255151,v1:172.21.15.7:6811/3984255151]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T09:07:39.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:39 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:39.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:39 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:40.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:40 smithi007 ceph-mon[113546]: pgmap v75: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:40.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:40 smithi007 ceph-mon[113546]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:07:40.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:40 smithi007 ceph-mon[113546]: osd.1 [v2:172.21.15.7:6810/3984255151,v1:172.21.15.7:6811/3984255151] boot 2024-03-16T09:07:40.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:40 smithi007 ceph-mon[113546]: osdmap e58: 6 total, 6 up, 6 in 2024-03-16T09:07:40.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:40 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:07:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:40 smithi019 ceph-mon[83366]: pgmap v75: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:40 smithi019 ceph-mon[83366]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:07:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:40 smithi019 ceph-mon[83366]: osd.1 [v2:172.21.15.7:6810/3984255151,v1:172.21.15.7:6811/3984255151] boot 2024-03-16T09:07:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:40 smithi019 ceph-mon[83366]: osdmap e58: 6 total, 6 up, 6 in 2024-03-16T09:07:40.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:40 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:07:41.185 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:41 smithi007 ceph-mon[113546]: osdmap e59: 6 total, 6 up, 6 in 2024-03-16T09:07:41.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:41 smithi019 ceph-mon[83366]: osdmap e59: 6 total, 6 up, 6 in 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: pgmap v78: 65 pgs: 2 peering, 16 active+undersized+degraded, 16 active+undersized, 31 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:42.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T09:07:42.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: pgmap v78: 65 pgs: 2 peering, 16 active+undersized+degraded, 16 active+undersized, 31 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 56/354 objects degraded (15.819%) 2024-03-16T09:07:42.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:42.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:42.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:42.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:42.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:42.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:42.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:42.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:42.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:07:42.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T09:07:43.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:43 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T09:07:43.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:43 smithi019 ceph-mon[83366]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-16T09:07:43.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:43 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T09:07:43.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:43 smithi007 ceph-mon[113546]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-16T09:07:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:44 smithi019 ceph-mon[83366]: pgmap v79: 65 pgs: 2 peering, 7 active+undersized+degraded, 5 active+undersized, 51 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/354 objects degraded (6.497%) 2024-03-16T09:07:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:44 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 23/354 objects degraded (6.497%), 7 pgs degraded (PG_DEGRADED) 2024-03-16T09:07:44.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:44 smithi007 ceph-mon[113546]: pgmap v79: 65 pgs: 2 peering, 7 active+undersized+degraded, 5 active+undersized, 51 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/354 objects degraded (6.497%) 2024-03-16T09:07:44.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:44 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 23/354 objects degraded (6.497%), 7 pgs degraded (PG_DEGRADED) 2024-03-16T09:07:45.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:45 smithi019 ceph-mon[83366]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/354 objects degraded (6.497%), 7 pgs degraded) 2024-03-16T09:07:45.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:45 smithi019 ceph-mon[83366]: Cluster is now healthy 2024-03-16T09:07:45.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:45 smithi007 ceph-mon[113546]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/354 objects degraded (6.497%), 7 pgs degraded) 2024-03-16T09:07:45.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:45 smithi007 ceph-mon[113546]: Cluster is now healthy 2024-03-16T09:07:46.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:46 smithi019 ceph-mon[83366]: pgmap v80: 65 pgs: 2 peering, 63 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:46.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:46.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:07:46.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:46 smithi007 ceph-mon[113546]: pgmap v80: 65 pgs: 2 peering, 63 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:46.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:46.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:07:48.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:48 smithi019 ceph-mon[83366]: pgmap v81: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:48.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:48 smithi007 ceph-mon[113546]: pgmap v81: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:50.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:50 smithi019 ceph-mon[83366]: pgmap v82: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:50.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:50 smithi007 ceph-mon[113546]: pgmap v82: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:52.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:52 smithi019 ceph-mon[83366]: pgmap v83: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:52.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:52 smithi007 ceph-mon[113546]: pgmap v83: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:54.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:54 smithi019 ceph-mon[83366]: pgmap v84: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:54.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:54 smithi007 ceph-mon[113546]: pgmap v84: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:56.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:56 smithi019 ceph-mon[83366]: pgmap v85: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:56.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:56 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:56.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:56 smithi007 ceph-mon[113546]: pgmap v85: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:56.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:56 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:57.403 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:57 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T09:07:57.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:57 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T09:07:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:58 smithi019 ceph-mon[83366]: pgmap v86: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:58 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T09:07:58.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:58 smithi019 ceph-mon[83366]: Upgrade: osd.2 is safe to restart 2024-03-16T09:07:58.484 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:58 smithi007 ceph-mon[113546]: pgmap v86: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:58.484 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:58 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T09:07:58.484 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:58 smithi007 ceph-mon[113546]: Upgrade: osd.2 is safe to restart 2024-03-16T09:07:58.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/3532379421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 msgr2=0x7f1a480fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 --2- 172.21.15.7:0/3532379421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 0x7f1a480fe260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1a38004940 tx=0x7f1a38009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/3532379421 shutdown_connections 2024-03-16T09:07:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 --2- 172.21.15.7:0/3532379421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a480fe830 0x7f1a48106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 --2- 172.21.15.7:0/3532379421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 0x7f1a480fe260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/3532379421 >> 172.21.15.7:0/3532379421 conn(0x7f1a48072bc0 msgr2=0x7f1a48072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/3532379421 shutdown_connections 2024-03-16T09:07:58.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/3532379421 wait complete. 2024-03-16T09:07:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.896+0000 7f1a4dcf6700 1 Processor -- start 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a4dcf6700 1 -- start start 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a4dcf6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 0x7f1a480fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a4dcf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a480fe830 0x7f1a480fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a4dcf6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480fc820 con 0x7f1a480fde90 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a4dcf6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480fc960 con 0x7f1a480fe830 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 0x7f1a480fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 0x7f1a480fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51416/0 (socket says 172.21.15.7:51416) 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a477fe700 1 -- 172.21.15.7:0/543300388 learned_addr learned my addr 172.21.15.7:0/543300388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a46ffd700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a480fe830 0x7f1a480fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a477fe700 1 -- 172.21.15.7:0/543300388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a480fe830 msgr2=0x7f1a480fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a477fe700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a480fe830 0x7f1a480fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.897+0000 7f1a477fe700 1 -- 172.21.15.7:0/543300388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a38005040 con 0x7f1a480fde90 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.898+0000 7f1a477fe700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 0x7f1a480fbd50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009360 tx=0x7f1a38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.898+0000 7f1a3ffff700 1 -- 172.21.15.7:0/543300388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3800e040 con 0x7f1a480fde90 2024-03-16T09:07:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.898+0000 7f1a3ffff700 1 -- 172.21.15.7:0/543300388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a38013420 con 0x7f1a480fde90 2024-03-16T09:07:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.898+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a480fcbb0 con 0x7f1a480fde90 2024-03-16T09:07:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.898+0000 7f1a3ffff700 1 -- 172.21.15.7:0/543300388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3801c630 con 0x7f1a480fde90 2024-03-16T09:07:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.898+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4811a1e0 con 0x7f1a480fde90 2024-03-16T09:07:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.899+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a48065e30 con 0x7f1a480fde90 2024-03-16T09:07:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.900+0000 7f1a3ffff700 1 -- 172.21.15.7:0/543300388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f1a38013a90 con 0x7f1a480fde90 2024-03-16T09:07:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.900+0000 7f1a3ffff700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1a3406dee0 0x7f1a34070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.900+0000 7f1a3ffff700 1 -- 172.21.15.7:0/543300388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1a3808e920 con 0x7f1a480fde90 2024-03-16T09:07:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.900+0000 7f1a46ffd700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1a3406dee0 0x7f1a34070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.901+0000 7f1a46ffd700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1a3406dee0 0x7f1a34070390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009fd0 tx=0x7f1a30009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:58.905+0000 7f1a3ffff700 1 -- 172.21.15.7:0/543300388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a38057eb0 con 0x7f1a480fde90 2024-03-16T09:07:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.133+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a4804bf40 con 0x7f1a3406dee0 2024-03-16T09:07:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.135+0000 7f1a3ffff700 1 -- 172.21.15.7:0/543300388 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1a4804bf40 con 0x7f1a3406dee0 2024-03-16T09:07:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1a3406dee0 msgr2=0x7f1a34070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1a3406dee0 0x7f1a34070390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009fd0 tx=0x7f1a30009f60 comp rx=0 tx=0).stop 2024-03-16T09:07:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 msgr2=0x7f1a480fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 0x7f1a480fbd50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009360 tx=0x7f1a38009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 shutdown_connections 2024-03-16T09:07:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1a3406dee0 0x7f1a34070390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a480fde90 0x7f1a480fbd50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 --2- 172.21.15.7:0/543300388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a480fe830 0x7f1a480fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 >> 172.21.15.7:0/543300388 conn(0x7f1a48072bc0 msgr2=0x7f1a481037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 shutdown_connections 2024-03-16T09:07:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.137+0000 7f1a4dcf6700 1 -- 172.21.15.7:0/543300388 wait complete. 2024-03-16T09:07:59.150 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:07:59.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.241+0000 7f0ae1de9700 1 -- 172.21.15.7:0/128631271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fe830 msgr2=0x7f0adc106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.241+0000 7f0ae1de9700 1 --2- 172.21.15.7:0/128631271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fe830 0x7f0adc106e70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4004940 tx=0x7f0ac4009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:59.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.241+0000 7f0ae1de9700 1 -- 172.21.15.7:0/128631271 shutdown_connections 2024-03-16T09:07:59.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.241+0000 7f0ae1de9700 1 --2- 172.21.15.7:0/128631271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fe830 0x7f0adc106e70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.241+0000 7f0ae1de9700 1 --2- 172.21.15.7:0/128631271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0adc0fde90 0x7f0adc0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.241+0000 7f0ae1de9700 1 -- 172.21.15.7:0/128631271 >> 172.21.15.7:0/128631271 conn(0x7f0adc0f78e0 msgr2=0x7f0adc0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:59.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.242+0000 7f0ae1de9700 1 -- 172.21.15.7:0/128631271 shutdown_connections 2024-03-16T09:07:59.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.242+0000 7f0ae1de9700 1 -- 172.21.15.7:0/128631271 wait complete. 2024-03-16T09:07:59.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.242+0000 7f0ae1de9700 1 Processor -- start 2024-03-16T09:07:59.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.242+0000 7f0ae1de9700 1 -- start start 2024-03-16T09:07:59.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.242+0000 7f0ae1de9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fde90 0x7f0adc113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.242+0000 7f0ae1de9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0adc0fe830 0x7f0adc113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.242+0000 7f0ae1de9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc114530 con 0x7f0adc0fde90 2024-03-16T09:07:59.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.242+0000 7f0ae1de9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc10daa0 con 0x7f0adc0fe830 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0adb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fde90 0x7f0adc113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0adb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fde90 0x7f0adc113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51432/0 (socket says 172.21.15.7:51432) 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0adb7fe700 1 -- 172.21.15.7:0/885318200 learned_addr learned my addr 172.21.15.7:0/885318200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0ad2ffd700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0adc0fe830 0x7f0adc113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0adb7fe700 1 -- 172.21.15.7:0/885318200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0adc0fe830 msgr2=0x7f0adc113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0adb7fe700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0adc0fe830 0x7f0adc113e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0adb7fe700 1 -- 172.21.15.7:0/885318200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac4005040 con 0x7f0adc0fde90 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0adb7fe700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fde90 0x7f0adc113910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00b790 tx=0x7f0acc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0ad8ff9700 1 -- 172.21.15.7:0/885318200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc00f770 con 0x7f0adc0fde90 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0ad8ff9700 1 -- 172.21.15.7:0/885318200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0acc00fda0 con 0x7f0adc0fde90 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc10dd80 con 0x7f0adc0fde90 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0ad8ff9700 1 -- 172.21.15.7:0/885318200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc00d310 con 0x7f0adc0fde90 2024-03-16T09:07:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.243+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc10e2c0 con 0x7f0adc0fde90 2024-03-16T09:07:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.244+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc066af0 con 0x7f0adc0fde90 2024-03-16T09:07:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.245+0000 7f0ad8ff9700 1 -- 172.21.15.7:0/885318200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0acc010420 con 0x7f0adc0fde90 2024-03-16T09:07:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.245+0000 7f0ad8ff9700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ac806dee0 0x7f0ac8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.245+0000 7f0ad8ff9700 1 -- 172.21.15.7:0/885318200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0acc08d740 con 0x7f0adc0fde90 2024-03-16T09:07:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.245+0000 7f0ad2ffd700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ac806dee0 0x7f0ac8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.246+0000 7f0ad2ffd700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ac806dee0 0x7f0ac8070390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0adc10ec60 tx=0x7f0ac4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.250+0000 7f0ad8ff9700 1 -- 172.21.15.7:0/885318200 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0acc05a800 con 0x7f0adc0fde90 2024-03-16T09:07:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.487+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0adc109ed0 con 0x7f0ac806dee0 2024-03-16T09:07:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.488+0000 7f0ad8ff9700 1 -- 172.21.15.7:0/885318200 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0adc109ed0 con 0x7f0ac806dee0 2024-03-16T09:07:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ac806dee0 msgr2=0x7f0ac8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ac806dee0 0x7f0ac8070390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0adc10ec60 tx=0x7f0ac4009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fde90 msgr2=0x7f0adc113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fde90 0x7f0adc113910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00b790 tx=0x7f0acc009250 comp rx=0 tx=0).stop 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 shutdown_connections 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0ac806dee0 0x7f0ac8070390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0adc0fde90 0x7f0adc113910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 --2- 172.21.15.7:0/885318200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0adc0fe830 0x7f0adc113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 >> 172.21.15.7:0/885318200 conn(0x7f0adc0f78e0 msgr2=0x7f0adc101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 shutdown_connections 2024-03-16T09:07:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.490+0000 7f0ae1de9700 1 -- 172.21.15.7:0/885318200 wait complete. 2024-03-16T09:07:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.601+0000 7fb524cfb700 1 -- 172.21.15.7:0/2219426300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520102e10 msgr2=0x7fb5201031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.601+0000 7fb524cfb700 1 --2- 172.21.15.7:0/2219426300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520102e10 0x7fb5201031e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb518009ce0 tx=0x7fb518004d10 comp rx=0 tx=0).stop 2024-03-16T09:07:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.601+0000 7fb524cfb700 1 -- 172.21.15.7:0/2219426300 shutdown_connections 2024-03-16T09:07:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.601+0000 7fb524cfb700 1 --2- 172.21.15.7:0/2219426300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5201037b0 0x7fb5200fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.601+0000 7fb524cfb700 1 --2- 172.21.15.7:0/2219426300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520102e10 0x7fb5201031e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.601+0000 7fb524cfb700 1 -- 172.21.15.7:0/2219426300 >> 172.21.15.7:0/2219426300 conn(0x7fb5200f8980 msgr2=0x7fb5200fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.601+0000 7fb524cfb700 1 -- 172.21.15.7:0/2219426300 shutdown_connections 2024-03-16T09:07:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.601+0000 7fb524cfb700 1 -- 172.21.15.7:0/2219426300 wait complete. 2024-03-16T09:07:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb524cfb700 1 Processor -- start 2024-03-16T09:07:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb524cfb700 1 -- start start 2024-03-16T09:07:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb524cfb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb520102e10 0x7fb520115f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb524cfb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5201037b0 0x7fb5201164a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb524cfb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb520116af0 con 0x7fb5201037b0 2024-03-16T09:07:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb524cfb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52010fec0 con 0x7fb520102e10 2024-03-16T09:07:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb51dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5201037b0 0x7fb5201164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb51dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5201037b0 0x7fb5201164a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51448/0 (socket says 172.21.15.7:51448) 2024-03-16T09:07:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb51dd9b700 1 -- 172.21.15.7:0/3953973364 learned_addr learned my addr 172.21.15.7:0/3953973364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb51e59c700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb520102e10 0x7fb520115f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb51dd9b700 1 -- 172.21.15.7:0/3953973364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb520102e10 msgr2=0x7fb520115f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.602+0000 7fb51dd9b700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb520102e10 0x7fb520115f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.603+0000 7fb51dd9b700 1 -- 172.21.15.7:0/3953973364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518009940 con 0x7fb5201037b0 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.603+0000 7fb51dd9b700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5201037b0 0x7fb5201164a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c800 tx=0x7fb508004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.603+0000 7fb516ffd700 1 -- 172.21.15.7:0/3953973364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5080106f0 con 0x7fb5201037b0 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.603+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5201101a0 con 0x7fb5201037b0 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.603+0000 7fb516ffd700 1 -- 172.21.15.7:0/3953973364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb508010d20 con 0x7fb5201037b0 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.603+0000 7fb516ffd700 1 -- 172.21.15.7:0/3953973364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5080113a0 con 0x7fb5201037b0 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.603+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb520110760 con 0x7fb5201037b0 2024-03-16T09:07:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.604+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb520107310 con 0x7fb5201037b0 2024-03-16T09:07:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.605+0000 7fb516ffd700 1 -- 172.21.15.7:0/3953973364 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fb508012400 con 0x7fb5201037b0 2024-03-16T09:07:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.605+0000 7fb516ffd700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb50c06def0 0x7fb50c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.605+0000 7fb516ffd700 1 -- 172.21.15.7:0/3953973364 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb50808d6b0 con 0x7fb5201037b0 2024-03-16T09:07:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.605+0000 7fb51e59c700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb50c06def0 0x7fb50c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.606+0000 7fb51e59c700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb50c06def0 0x7fb50c0703a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b740 tx=0x7fb518004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:59.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.614+0000 7fb516ffd700 1 -- 172.21.15.7:0/3953973364 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb50805a6d0 con 0x7fb5201037b0 2024-03-16T09:07:59.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:59 smithi007 ceph-mon[113546]: pgmap v87: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:59.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:59 smithi007 ceph-mon[113546]: Upgrade: Updating osd.2 2024-03-16T09:07:59.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:59 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:59.715 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:59 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T09:07:59.716 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:59 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:59.716 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:59 smithi007 ceph-mon[113546]: Deploying daemon osd.2 on smithi007 2024-03-16T09:07:59.716 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:07:59 smithi007 ceph-mon[113546]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:59.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.836+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb520111620 con 0x7fb50c06def0 2024-03-16T09:07:59.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.846+0000 7fb516ffd700 1 -- 172.21.15.7:0/3953973364 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb520111620 con 0x7fb50c06def0 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (4m) 22s ago 14m 29.0M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (92s) 22s ago 15m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (86s) 83s ago 14m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (3m) 22s ago 14m 78.5M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (9m) 22s ago 9m 14.2M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (9m) 22s ago 9m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (9m) 83s ago 9m 120M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (9m) 83s ago 9m 14.1M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (5m) 22s ago 16m 502M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (5m) 83s ago 13m 449M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (2m) 22s ago 16m 49.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (2m) 83s ago 13m 34.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (4m) 22s ago 15m 15.2M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:07:59.849 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (4m) 83s ago 14m 15.2M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:07:59.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (65s) 22s ago 13m 91.7M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:07:59.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (29s) 22s ago 12m 11.9M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4aa10e800f72 2024-03-16T09:07:59.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (12m) 22s ago 12m 179M 1559M 17.2.7-648-g1df27247 0bdcfccf14ce 4885be1ec696 2024-03-16T09:07:59.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (11m) 83s ago 11m 264M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:07:59.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (11m) 83s ago 11m 247M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:07:59.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (10m) 83s ago 10m 223M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:07:59.850 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (4m) 22s ago 14m 62.6M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:07:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb50c06def0 msgr2=0x7fb50c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb50c06def0 0x7fb50c0703a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b740 tx=0x7fb518004d10 comp rx=0 tx=0).stop 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5201037b0 msgr2=0x7fb5201164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5201037b0 0x7fb5201164a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c800 tx=0x7fb508004470 comp rx=0 tx=0).stop 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 shutdown_connections 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb520102e10 0x7fb520115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb50c06def0 0x7fb50c0703a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 --2- 172.21.15.7:0/3953973364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5201037b0 0x7fb5201164a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 >> 172.21.15.7:0/3953973364 conn(0x7fb5200f8980 msgr2=0x7fb520070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 shutdown_connections 2024-03-16T09:07:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.849+0000 7fb524cfb700 1 -- 172.21.15.7:0/3953973364 wait complete. 2024-03-16T09:07:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.958+0000 7f341e97c700 1 -- 172.21.15.7:0/689177831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34180702e0 msgr2=0x7f34180706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.958+0000 7f341e97c700 1 --2- 172.21.15.7:0/689177831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34180702e0 0x7f34180706b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3400004940 tx=0x7f3400009e30 comp rx=0 tx=0).stop 2024-03-16T09:07:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.958+0000 7f341e97c700 1 -- 172.21.15.7:0/689177831 shutdown_connections 2024-03-16T09:07:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.958+0000 7f341e97c700 1 --2- 172.21.15.7:0/689177831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3418070bf0 0x7f34181091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.958+0000 7f341e97c700 1 --2- 172.21.15.7:0/689177831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34180702e0 0x7f34180706b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.958+0000 7f341e97c700 1 -- 172.21.15.7:0/689177831 >> 172.21.15.7:0/689177831 conn(0x7f34180f8980 msgr2=0x7f34180fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.958+0000 7f341e97c700 1 -- 172.21.15.7:0/689177831 shutdown_connections 2024-03-16T09:07:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.958+0000 7f341e97c700 1 -- 172.21.15.7:0/689177831 wait complete. 2024-03-16T09:07:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.959+0000 7f341e97c700 1 Processor -- start 2024-03-16T09:07:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.959+0000 7f341e97c700 1 -- start start 2024-03-16T09:07:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.959+0000 7f341e97c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34180702e0 0x7f3418115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.959+0000 7f341e97c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3418070bf0 0x7f3418116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.959+0000 7f341e97c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34181168d0 con 0x7f3418070bf0 2024-03-16T09:07:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.959+0000 7f341e97c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341810fca0 con 0x7f34180702e0 2024-03-16T09:07:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f34177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3418070bf0 0x7f3418116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f34177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3418070bf0 0x7f3418116280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51466/0 (socket says 172.21.15.7:51466) 2024-03-16T09:07:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f34177fe700 1 -- 172.21.15.7:0/2715283798 learned_addr learned my addr 172.21.15.7:0/2715283798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:07:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f3417fff700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34180702e0 0x7f3418115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f34177fe700 1 -- 172.21.15.7:0/2715283798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34180702e0 msgr2=0x7f3418115d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f34177fe700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34180702e0 0x7f3418115d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f34177fe700 1 -- 172.21.15.7:0/2715283798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3400005040 con 0x7f3418070bf0 2024-03-16T09:07:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f34177fe700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3418070bf0 0x7f3418116280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3408002a80 tx=0x7f34080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f3414ff9700 1 -- 172.21.15.7:0/2715283798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3408007cc0 con 0x7f3418070bf0 2024-03-16T09:07:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341810ff80 con 0x7f3418070bf0 2024-03-16T09:07:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f3414ff9700 1 -- 172.21.15.7:0/2715283798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f340800f040 con 0x7f3418070bf0 2024-03-16T09:07:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f3414ff9700 1 -- 172.21.15.7:0/2715283798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f340800dac0 con 0x7f3418070bf0 2024-03-16T09:07:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.960+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34181104c0 con 0x7f3418070bf0 2024-03-16T09:07:59.968 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:59 smithi019 ceph-mon[83366]: pgmap v87: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:07:59.968 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:59 smithi019 ceph-mon[83366]: Upgrade: Updating osd.2 2024-03-16T09:07:59.968 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:59 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:07:59.969 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:59 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T09:07:59.969 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:59 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:59.969 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:59 smithi019 ceph-mon[83366]: Deploying daemon osd.2 on smithi007 2024-03-16T09:07:59.969 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:07:59 smithi019 ceph-mon[83366]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.961+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3418105fd0 con 0x7f3418070bf0 2024-03-16T09:07:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.962+0000 7f3414ff9700 1 -- 172.21.15.7:0/2715283798 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f340800dc20 con 0x7f3418070bf0 2024-03-16T09:07:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.962+0000 7f3414ff9700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f340406def0 0x7f34040703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.963+0000 7f3414ff9700 1 -- 172.21.15.7:0/2715283798 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f340808d330 con 0x7f3418070bf0 2024-03-16T09:07:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.963+0000 7f3417fff700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f340406def0 0x7f34040703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.963+0000 7f3417fff700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f340406def0 0x7f34040703a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3400009fd0 tx=0x7f3400009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:07:59.967+0000 7f3414ff9700 1 -- 172.21.15.7:0/2715283798 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f340805a370 con 0x7f3418070bf0 2024-03-16T09:08:00.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.264+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3418071a20 con 0x7f3418070bf0 2024-03-16T09:08:00.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.265+0000 7f3414ff9700 1 -- 172.21.15.7:0/2715283798 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f3408059f00 con 0x7f3418070bf0 2024-03-16T09:08:00.267 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:08:00.267 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:08:00.267 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:08:00.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:08:00.267 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4, 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 8, 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:08:00.268 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f340406def0 msgr2=0x7f34040703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f340406def0 0x7f34040703a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3400009fd0 tx=0x7f3400009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3418070bf0 msgr2=0x7f3418116280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3418070bf0 0x7f3418116280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3408002a80 tx=0x7f34080046b0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 shutdown_connections 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34180702e0 0x7f3418115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f340406def0 0x7f34040703a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 --2- 172.21.15.7:0/2715283798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3418070bf0 0x7f3418116280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 >> 172.21.15.7:0/2715283798 conn(0x7f34180f8980 msgr2=0x7f3418103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 shutdown_connections 2024-03-16T09:08:00.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.268+0000 7f341e97c700 1 -- 172.21.15.7:0/2715283798 wait complete. 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.374+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3572404685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fc500 msgr2=0x7fddb8109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.374+0000 7fddbdb5b700 1 --2- 172.21.15.7:0/3572404685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fc500 0x7fddb8109260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fddac004940 tx=0x7fddac009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.374+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3572404685 shutdown_connections 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.374+0000 7fddbdb5b700 1 --2- 172.21.15.7:0/3572404685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fc500 0x7fddb8109260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.374+0000 7fddbdb5b700 1 --2- 172.21.15.7:0/3572404685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb80fbbf0 0x7fddb80fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.374+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3572404685 >> 172.21.15.7:0/3572404685 conn(0x7fddb80f7880 msgr2=0x7fddb80f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.374+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3572404685 shutdown_connections 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.374+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3572404685 wait complete. 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddbdb5b700 1 Processor -- start 2024-03-16T09:08:00.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddbdb5b700 1 -- start start 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddbdb5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fbbf0 0x7fddb8115ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddbdb5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb80fc500 0x7fddb8116530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddbdb5b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb8116b80 con 0x7fddb80fbbf0 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddbdb5b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb8110000 con 0x7fddb80fc500 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddb77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fbbf0 0x7fddb8115ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddb77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fbbf0 0x7fddb8115ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51472/0 (socket says 172.21.15.7:51472) 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddb77fe700 1 -- 172.21.15.7:0/3669647211 learned_addr learned my addr 172.21.15.7:0/3669647211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddb6ffd700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb80fc500 0x7fddb8116530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddb77fe700 1 -- 172.21.15.7:0/3669647211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb80fc500 msgr2=0x7fddb8116530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddb77fe700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb80fc500 0x7fddb8116530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.375+0000 7fddb77fe700 1 -- 172.21.15.7:0/3669647211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac005040 con 0x7fddb80fbbf0 2024-03-16T09:08:00.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.376+0000 7fddb77fe700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fbbf0 0x7fddb8115ff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdda8002a50 tx=0x7fdda80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:00.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.376+0000 7fdd9ffff700 1 -- 172.21.15.7:0/3669647211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdda8007cc0 con 0x7fddb80fbbf0 2024-03-16T09:08:00.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.376+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb81102e0 con 0x7fddb80fbbf0 2024-03-16T09:08:00.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.376+0000 7fdd9ffff700 1 -- 172.21.15.7:0/3669647211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdda800f040 con 0x7fddb80fbbf0 2024-03-16T09:08:00.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.376+0000 7fdd9ffff700 1 -- 172.21.15.7:0/3669647211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdda800da30 con 0x7fddb80fbbf0 2024-03-16T09:08:00.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.376+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb81108a0 con 0x7fddb80fbbf0 2024-03-16T09:08:00.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.377+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb8106990 con 0x7fddb80fbbf0 2024-03-16T09:08:00.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.377+0000 7fdd9ffff700 1 -- 172.21.15.7:0/3669647211 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdda800db90 con 0x7fddb80fbbf0 2024-03-16T09:08:00.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.378+0000 7fdd9ffff700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdda006dee0 0x7fdda0070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:00.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.378+0000 7fdd9ffff700 1 -- 172.21.15.7:0/3669647211 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdda808d480 con 0x7fddb80fbbf0 2024-03-16T09:08:00.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.378+0000 7fddb6ffd700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdda006dee0 0x7fdda0070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:00.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.378+0000 7fddb6ffd700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdda006dee0 0x7fdda0070390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fddb80f9d40 tx=0x7fddac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:00.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.383+0000 7fdd9ffff700 1 -- 172.21.15.7:0/3669647211 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdda8056990 con 0x7fddb80fbbf0 2024-03-16T09:08:00.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.659+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fddb810c0b0 con 0x7fddb80fbbf0 2024-03-16T09:08:00.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.660+0000 7fdd9ffff700 1 -- 172.21.15.7:0/3669647211 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1895 (secure 0 0 0) 0x7fdda805a050 con 0x7fddb80fbbf0 2024-03-16T09:08:00.661 INFO:teuthology.orchestra.run.smithi007.stdout:e20 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:08:00.662 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:08:00.663 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:08:00.664 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:08:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdda006dee0 msgr2=0x7fdda0070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdda006dee0 0x7fdda0070390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fddb80f9d40 tx=0x7fddac009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fbbf0 msgr2=0x7fddb8115ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fbbf0 0x7fddb8115ff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdda8002a50 tx=0x7fdda80046b0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 shutdown_connections 2024-03-16T09:08:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdda006dee0 0x7fdda0070390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb80fbbf0 0x7fddb8115ff0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 --2- 172.21.15.7:0/3669647211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb80fc500 0x7fddb8116530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 >> 172.21.15.7:0/3669647211 conn(0x7fddb80f7880 msgr2=0x7fddb80f7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 shutdown_connections 2024-03-16T09:08:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.663+0000 7fddbdb5b700 1 -- 172.21.15.7:0/3669647211 wait complete. 2024-03-16T09:08:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:08:00.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 -- 172.21.15.7:0/1952741129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 msgr2=0x7fbed00fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 --2- 172.21.15.7:0/1952741129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 0x7fbed00fce20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbec4004940 tx=0x7fbec4009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 -- 172.21.15.7:0/1952741129 shutdown_connections 2024-03-16T09:08:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 --2- 172.21.15.7:0/1952741129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 0x7fbed00fce20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 --2- 172.21.15.7:0/1952741129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbed0102e10 0x7fbed01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 -- 172.21.15.7:0/1952741129 >> 172.21.15.7:0/1952741129 conn(0x7fbed00f89a0 msgr2=0x7fbed00fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 -- 172.21.15.7:0/1952741129 shutdown_connections 2024-03-16T09:08:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 -- 172.21.15.7:0/1952741129 wait complete. 2024-03-16T09:08:00.788 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:00 smithi007 ceph-mon[113546]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:00.788 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:00 smithi007 ceph-mon[113546]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:00.789 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:00 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2715283798' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:00.789 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:08:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.786+0000 7fbed669c700 1 Processor -- start 2024-03-16T09:08:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbed669c700 1 -- start start 2024-03-16T09:08:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbed669c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbed0102e10 0x7fbed0115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbed669c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 0x7fbed01162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbed669c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed0116940 con 0x7fbed01037b0 2024-03-16T09:08:00.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbed669c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed010fd10 con 0x7fbed0102e10 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbecf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 0x7fbed01162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbecf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 0x7fbed01162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51488/0 (socket says 172.21.15.7:51488) 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbecf7fe700 1 -- 172.21.15.7:0/2927156444 learned_addr learned my addr 172.21.15.7:0/2927156444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbecffff700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbed0102e10 0x7fbed0115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbecf7fe700 1 -- 172.21.15.7:0/2927156444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbed0102e10 msgr2=0x7fbed0115db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbecf7fe700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbed0102e10 0x7fbed0115db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbecf7fe700 1 -- 172.21.15.7:0/2927156444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec4005040 con 0x7fbed01037b0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.787+0000 7fbecffff700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbed0102e10 0x7fbed0115db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.788+0000 7fbecf7fe700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 0x7fbed01162f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009180 tx=0x7fbec4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.788+0000 7fbeccff9700 1 -- 172.21.15.7:0/2927156444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbec400e040 con 0x7fbed01037b0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.788+0000 7fbeccff9700 1 -- 172.21.15.7:0/2927156444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbec400bbc0 con 0x7fbed01037b0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.788+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed010ff90 con 0x7fbed01037b0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.788+0000 7fbeccff9700 1 -- 172.21.15.7:0/2927156444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbec401c3f0 con 0x7fbed01037b0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.788+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed0110550 con 0x7fbed01037b0 2024-03-16T09:08:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.789+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed01019c0 con 0x7fbed01037b0 2024-03-16T09:08:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.789+0000 7fbeccff9700 1 -- 172.21.15.7:0/2927156444 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fbec400bd30 con 0x7fbed01037b0 2024-03-16T09:08:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.790+0000 7fbeccff9700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbeb806dee0 0x7fbeb8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.790+0000 7fbeccff9700 1 -- 172.21.15.7:0/2927156444 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbec408ea20 con 0x7fbed01037b0 2024-03-16T09:08:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.790+0000 7fbecffff700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbeb806dee0 0x7fbeb8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.790+0000 7fbecffff700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbeb806dee0 0x7fbeb8070390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009fd0 tx=0x7fbec000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:00.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:00.795+0000 7fbeccff9700 1 -- 172.21.15.7:0/2927156444 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbec405bae0 con 0x7fbed01037b0 2024-03-16T09:08:00.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:00 smithi019 ceph-mon[83366]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:00.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:00 smithi019 ceph-mon[83366]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:00.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:00 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2715283798' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:00.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:08:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.020+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbed0111400 con 0x7fbeb806dee0 2024-03-16T09:08:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.021+0000 7fbeccff9700 1 -- 172.21.15.7:0/2927156444 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbed0111400 con 0x7fbeb806dee0 2024-03-16T09:08:01.022 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "8/21 daemons upgraded", 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:08:01.023 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:08:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbeb806dee0 msgr2=0x7fbeb8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbeb806dee0 0x7fbeb8070390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009fd0 tx=0x7fbec000b040 comp rx=0 tx=0).stop 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 msgr2=0x7fbed01162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 0x7fbed01162f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009180 tx=0x7fbec4009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 shutdown_connections 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbed0102e10 0x7fbed0115db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbeb806dee0 0x7fbeb8070390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 --2- 172.21.15.7:0/2927156444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbed01037b0 0x7fbed01162f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 >> 172.21.15.7:0/2927156444 conn(0x7fbed00f89a0 msgr2=0x7fbed00face0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 shutdown_connections 2024-03-16T09:08:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.025+0000 7fbed669c700 1 -- 172.21.15.7:0/2927156444 wait complete. 2024-03-16T09:08:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.137+0000 7fdb8d31d700 1 -- 172.21.15.7:0/4165956868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb881037a0 msgr2=0x7fdb880fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.137+0000 7fdb8d31d700 1 --2- 172.21.15.7:0/4165956868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb881037a0 0x7fdb880fce10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c004940 tx=0x7fdb7c009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.137+0000 7fdb8d31d700 1 -- 172.21.15.7:0/4165956868 shutdown_connections 2024-03-16T09:08:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.137+0000 7fdb8d31d700 1 --2- 172.21.15.7:0/4165956868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb881037a0 0x7fdb880fce10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.137+0000 7fdb8d31d700 1 --2- 172.21.15.7:0/4165956868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb88102e00 0x7fdb881031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.137+0000 7fdb8d31d700 1 -- 172.21.15.7:0/4165956868 >> 172.21.15.7:0/4165956868 conn(0x7fdb880f89b0 msgr2=0x7fdb880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.137+0000 7fdb8d31d700 1 -- 172.21.15.7:0/4165956868 shutdown_connections 2024-03-16T09:08:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.137+0000 7fdb8d31d700 1 -- 172.21.15.7:0/4165956868 wait complete. 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb8d31d700 1 Processor -- start 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb8d31d700 1 -- start start 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb8d31d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb88102e00 0x7fdb880702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb8d31d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb881037a0 0x7fdb88070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb8d31d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88072210 con 0x7fdb88102e00 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb8d31d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88070d40 con 0x7fdb881037a0 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb86ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb88102e00 0x7fdb880702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb86ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb88102e00 0x7fdb880702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51510/0 (socket says 172.21.15.7:51510) 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb86ffd700 1 -- 172.21.15.7:0/920170213 learned_addr learned my addr 172.21.15.7:0/920170213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.138+0000 7fdb867fc700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb881037a0 0x7fdb88070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb86ffd700 1 -- 172.21.15.7:0/920170213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb881037a0 msgr2=0x7fdb88070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb86ffd700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb881037a0 0x7fdb88070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb86ffd700 1 -- 172.21.15.7:0/920170213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c005040 con 0x7fdb88102e00 2024-03-16T09:08:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb86ffd700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb88102e00 0x7fdb880702c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdb78002a50 tx=0x7fdb780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb6f7fe700 1 -- 172.21.15.7:0/920170213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb78007cc0 con 0x7fdb88102e00 2024-03-16T09:08:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb6f7fe700 1 -- 172.21.15.7:0/920170213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb7800f040 con 0x7fdb88102e00 2024-03-16T09:08:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb6f7fe700 1 -- 172.21.15.7:0/920170213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7800da30 con 0x7fdb88102e00 2024-03-16T09:08:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb88071020 con 0x7fdb88102e00 2024-03-16T09:08:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.139+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8811a0d0 con 0x7fdb88102e00 2024-03-16T09:08:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.141+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb881052d0 con 0x7fdb88102e00 2024-03-16T09:08:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.141+0000 7fdb6f7fe700 1 -- 172.21.15.7:0/920170213 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdb7800dbb0 con 0x7fdb88102e00 2024-03-16T09:08:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.141+0000 7fdb6f7fe700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb7006dee0 0x7fdb70070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.141+0000 7fdb6f7fe700 1 -- 172.21.15.7:0/920170213 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdb7808d150 con 0x7fdb88102e00 2024-03-16T09:08:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.141+0000 7fdb867fc700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb7006dee0 0x7fdb70070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.142+0000 7fdb867fc700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb7006dee0 0x7fdb70070390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c004890 tx=0x7fdb7c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.146+0000 7fdb6f7fe700 1 -- 172.21.15.7:0/920170213 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb780566e0 con 0x7fdb88102e00 2024-03-16T09:08:01.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.451+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdb880717a0 con 0x7fdb88102e00 2024-03-16T09:08:01.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.451+0000 7fdb6f7fe700 1 -- 172.21.15.7:0/920170213 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fdb78059da0 con 0x7fdb88102e00 2024-03-16T09:08:01.453 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_OK 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb7006dee0 msgr2=0x7fdb70070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb7006dee0 0x7fdb70070390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c004890 tx=0x7fdb7c009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb88102e00 msgr2=0x7fdb880702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb88102e00 0x7fdb880702c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdb78002a50 tx=0x7fdb780046b0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 shutdown_connections 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb7006dee0 0x7fdb70070390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb88102e00 0x7fdb880702c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 --2- 172.21.15.7:0/920170213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb881037a0 0x7fdb88070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.453+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 >> 172.21.15.7:0/920170213 conn(0x7fdb880f89b0 msgr2=0x7fdb881021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.454+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 shutdown_connections 2024-03-16T09:08:01.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:01.454+0000 7fdb8d31d700 1 -- 172.21.15.7:0/920170213 wait complete. 2024-03-16T09:08:01.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:01 smithi007 ceph-mon[113546]: pgmap v88: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:08:01.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:01 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3669647211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:08:01.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:01 smithi007 ceph-mon[113546]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:01.834 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:01 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/920170213' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:08:01.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:01 smithi019 ceph-mon[83366]: pgmap v88: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:08:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:01 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3669647211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:08:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:01 smithi019 ceph-mon[83366]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:01 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/920170213' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:08:02.498 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:02 smithi007 systemd[1]: Stopping Ceph osd.2 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:08:02.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:02 smithi019 ceph-mon[83366]: osd.2 marked itself down and dead 2024-03-16T09:08:02.967 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:02 smithi007 ceph-mon[113546]: osd.2 marked itself down and dead 2024-03-16T09:08:02.968 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:02 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[62905]: 2024-03-16T09:08:02.546+0000 7f90022a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:08:02.968 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:02 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[62905]: 2024-03-16T09:08:02.546+0000 7f90022a2700 -1 osd.2 59 *** Got signal Terminated *** 2024-03-16T09:08:02.968 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:02 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[62905]: 2024-03-16T09:08:02.546+0000 7f90022a2700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:08:03.248 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:02 smithi007 podman[136522]: 2024-03-16 09:08:02.966257159 +0000 UTC m=+0.524777164 container died 4885be1ec69673b74e38affd34cdfccab9229b411ac4797e0e64e2619891342a (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=quincy-1df2724, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:08:03.558 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:03 smithi007 podman[136522]: 2024-03-16 09:08:03.356001534 +0000 UTC m=+0.914521538 container cleanup 4885be1ec69673b74e38affd34cdfccab9229b411ac4797e0e64e2619891342a (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, GIT_CLEAN=True, RELEASE=quincy-1df2724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-16T09:08:03.558 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:03 smithi007 bash[136522]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2 2024-03-16T09:08:03.910 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:03 smithi007 ceph-mon[113546]: pgmap v89: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:08:03.910 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:03 smithi007 ceph-mon[113546]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:08:03.910 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:03 smithi007 ceph-mon[113546]: osdmap e60: 6 total, 5 up, 6 in 2024-03-16T09:08:03.910 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:03 smithi007 podman[136539]: 2024-03-16 09:08:03.582517715 +0000 UTC m=+0.619864232 container remove 4885be1ec69673b74e38affd34cdfccab9229b411ac4797e0e64e2619891342a (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=quincy-1df2724, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-16T09:08:03.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:03 smithi019 ceph-mon[83366]: pgmap v89: 65 pgs: 65 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:08:03.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:03 smithi019 ceph-mon[83366]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:08:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:03 smithi019 ceph-mon[83366]: osdmap e60: 6 total, 5 up, 6 in 2024-03-16T09:08:04.238 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:03 smithi007 podman[136631]: 2024-03-16 09:08:03.908659499 +0000 UTC m=+0.105019087 container create cef750cbcb544ced7c133279aaff6f9cda7d5f9d8514a3286b88a703da6a7c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-16T09:08:04.238 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:03 smithi007 podman[136631]: 2024-03-16 09:08:03.821530159 +0000 UTC m=+0.017889710 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:08:04.238 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:04 smithi007 podman[136631]: 2024-03-16 09:08:04.153578 +0000 UTC m=+0.349937532 container init cef750cbcb544ced7c133279aaff6f9cda7d5f9d8514a3286b88a703da6a7c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-16T09:08:04.238 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:04 smithi007 podman[136631]: 2024-03-16 09:08:04.15725755 +0000 UTC m=+0.353617082 container start cef750cbcb544ced7c133279aaff6f9cda7d5f9d8514a3286b88a703da6a7c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-16T09:08:04.238 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:04 smithi007 podman[136631]: 2024-03-16 09:08:04.236315966 +0000 UTC m=+0.432675508 container attach cef750cbcb544ced7c133279aaff6f9cda7d5f9d8514a3286b88a703da6a7c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-16T09:08:04.498 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:04 smithi007 podman[136631]: 2024-03-16 09:08:04.450712796 +0000 UTC m=+0.647072337 container died cef750cbcb544ced7c133279aaff6f9cda7d5f9d8514a3286b88a703da6a7c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-16T09:08:04.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:04 smithi019 ceph-mon[83366]: osdmap e61: 6 total, 5 up, 6 in 2024-03-16T09:08:04.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:04 smithi007 ceph-mon[113546]: osdmap e61: 6 total, 5 up, 6 in 2024-03-16T09:08:05.463 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 podman[136657]: 2024-03-16 09:08:05.007314539 +0000 UTC m=+0.551364211 container remove cef750cbcb544ced7c133279aaff6f9cda7d5f9d8514a3286b88a703da6a7c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:08:05.463 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-16T09:08:05.463 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 systemd[1]: Stopped Ceph osd.2 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:08:05.463 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.2.service: Consumed 47.871s CPU time. 2024-03-16T09:08:05.463 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 systemd[1]: Starting Ceph osd.2 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:08:05.729 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:05 smithi007 ceph-mon[113546]: pgmap v92: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:08:05.730 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:05 smithi007 ceph-mon[113546]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-16T09:08:05.730 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 podman[136762]: 2024-03-16 09:08:05.364874711 +0000 UTC m=+0.018061642 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:08:05.730 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 podman[136762]: 2024-03-16 09:08:05.517108675 +0000 UTC m=+0.170295594 container create ae60326e534a095219518671b53e8c1121504b659f4ca38c3cdc211487723889 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T09:08:05.730 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 podman[136762]: 2024-03-16 09:08:05.728087191 +0000 UTC m=+0.381274110 container init ae60326e534a095219518671b53e8c1121504b659f4ca38c3cdc211487723889 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-16T09:08:05.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:05 smithi019 ceph-mon[83366]: pgmap v92: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-03-16T09:08:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:05 smithi019 ceph-mon[83366]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-16T09:08:05.998 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 podman[136762]: 2024-03-16 09:08:05.731256584 +0000 UTC m=+0.384443515 container start ae60326e534a095219518671b53e8c1121504b659f4ca38c3cdc211487723889 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-03-16T09:08:05.998 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:05 smithi007 podman[136762]: 2024-03-16 09:08:05.836731675 +0000 UTC m=+0.489918608 container attach ae60326e534a095219518671b53e8c1121504b659f4ca38c3cdc211487723889 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-16T09:08:06.498 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate[136776]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate[136776]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 bash[136762]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 bash[136762]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 bash[136762]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 bash[136762]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 bash[136762]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 bash[136762]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 bash[136762]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate[136776]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate[136776]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate[136776]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate[136776]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate[136776]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-16T09:08:06.499 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 podman[136762]: 2024-03-16 09:08:06.203501886 +0000 UTC m=+0.856688808 container died ae60326e534a095219518671b53e8c1121504b659f4ca38c3cdc211487723889 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-03-16T09:08:06.809 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:06 smithi007 ceph-mon[113546]: Health check failed: Degraded data redundancy: 30/354 objects degraded (8.475%), 9 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:06.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:06 smithi019 ceph-mon[83366]: Health check failed: Degraded data redundancy: 30/354 objects degraded (8.475%), 9 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:07.102 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:06 smithi007 podman[136899]: 2024-03-16 09:08:06.808120678 +0000 UTC m=+0.599430541 container remove ae60326e534a095219518671b53e8c1121504b659f4ca38c3cdc211487723889 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-03-16T09:08:07.102 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:07 smithi007 podman[136965]: 2024-03-16 09:08:06.927589197 +0000 UTC m=+0.018897486 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:08:07.103 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:07 smithi007 podman[136965]: 2024-03-16 09:08:07.101186813 +0000 UTC m=+0.192495129 container create 5449f945991b5364849d62d1e363a170e718059e472d93786b334cab793c264a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:08:07.414 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:07 smithi007 podman[136965]: 2024-03-16 09:08:07.320436591 +0000 UTC m=+0.411744862 container init 5449f945991b5364849d62d1e363a170e718059e472d93786b334cab793c264a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:08:07.414 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:07 smithi007 podman[136965]: 2024-03-16 09:08:07.324097521 +0000 UTC m=+0.415405801 container start 5449f945991b5364849d62d1e363a170e718059e472d93786b334cab793c264a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T09:08:07.414 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:07 smithi007 ceph-osd[136984]: -- 172.21.15.7:0/3009370774 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556120485680 con 0x55612044ec00 2024-03-16T09:08:07.748 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:07 smithi007 bash[136965]: 5449f945991b5364849d62d1e363a170e718059e472d93786b334cab793c264a 2024-03-16T09:08:07.748 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:07 smithi007 systemd[1]: Started Ceph osd.2 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:08:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:07 smithi007 ceph-mon[113546]: pgmap v93: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 30/354 objects degraded (8.475%) 2024-03-16T09:08:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:07 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:07 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:07.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:07 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:07.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:07 smithi019 ceph-mon[83366]: pgmap v93: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 30/354 objects degraded (8.475%) 2024-03-16T09:08:07.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:07 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:07 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:07 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:08.248 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:07 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[136980]: 2024-03-16T09:08:07.861+0000 7f31780897c0 -1 Falling back to public interface 2024-03-16T09:08:08.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:08 smithi019 ceph-mon[83366]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-16T09:08:08.986 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:08 smithi007 ceph-mon[113546]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-16T09:08:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:09 smithi019 ceph-mon[83366]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/354 objects degraded (12.147%) 2024-03-16T09:08:09.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:09 smithi007 ceph-mon[113546]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/354 objects degraded (12.147%) 2024-03-16T09:08:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:11 smithi019 ceph-mon[83366]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/354 objects degraded (12.147%) 2024-03-16T09:08:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:11 smithi007 ceph-mon[113546]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/354 objects degraded (12.147%) 2024-03-16T09:08:11.998 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:11 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[136980]: 2024-03-16T09:08:11.796+0000 7f31780897c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-16T09:08:12.498 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:12 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[136980]: 2024-03-16T09:08:12.192+0000 7f31780897c0 -1 osd.2 59 log_to_monitors true 2024-03-16T09:08:12.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:12 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 43/354 objects degraded (12.147%), 12 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:12.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:12 smithi019 ceph-mon[83366]: from='osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T09:08:12.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:12 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:12.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:12 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:12 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 43/354 objects degraded (12.147%), 12 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:12 smithi007 ceph-mon[113546]: from='osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T09:08:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:12 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:12 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:13.680 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:08:13 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[136980]: 2024-03-16T09:08:13.405+0000 7f31654d9700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:08:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:13 smithi019 ceph-mon[83366]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/354 objects degraded (12.147%) 2024-03-16T09:08:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:13 smithi019 ceph-mon[83366]: from='osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T09:08:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:13 smithi019 ceph-mon[83366]: osdmap e62: 6 total, 5 up, 6 in 2024-03-16T09:08:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:13 smithi019 ceph-mon[83366]: from='osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T09:08:13.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:13 smithi019 ceph-mon[83366]: from='osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509]' entity='osd.2' 2024-03-16T09:08:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:13 smithi007 ceph-mon[113546]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 300 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 43/354 objects degraded (12.147%) 2024-03-16T09:08:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:13 smithi007 ceph-mon[113546]: from='osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T09:08:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:13 smithi007 ceph-mon[113546]: osdmap e62: 6 total, 5 up, 6 in 2024-03-16T09:08:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:13 smithi007 ceph-mon[113546]: from='osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-16T09:08:13.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:13 smithi007 ceph-mon[113546]: from='osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509]' entity='osd.2' 2024-03-16T09:08:15.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:14 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:15.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:14 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:15.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:14 smithi019 ceph-mon[83366]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:08:15.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:14 smithi019 ceph-mon[83366]: osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509] boot 2024-03-16T09:08:15.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:14 smithi019 ceph-mon[83366]: osdmap e63: 6 total, 6 up, 6 in 2024-03-16T09:08:15.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:14 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:15.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:14 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:15.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:14 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:15.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:14 smithi007 ceph-mon[113546]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:08:15.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:14 smithi007 ceph-mon[113546]: osd.2 [v2:172.21.15.7:6818/7192509,v1:172.21.15.7:6819/7192509] boot 2024-03-16T09:08:15.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:14 smithi007 ceph-mon[113546]: osdmap e63: 6 total, 6 up, 6 in 2024-03-16T09:08:15.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:14 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:16.126 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:15 smithi007 ceph-mon[113546]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 43/354 objects degraded (12.147%) 2024-03-16T09:08:16.126 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:15 smithi007 ceph-mon[113546]: osdmap e64: 6 total, 6 up, 6 in 2024-03-16T09:08:16.126 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:16.126 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:08:16.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:15 smithi019 ceph-mon[83366]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 43/354 objects degraded (12.147%) 2024-03-16T09:08:16.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:15 smithi019 ceph-mon[83366]: osdmap e64: 6 total, 6 up, 6 in 2024-03-16T09:08:16.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:16.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:08:17.138 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:16 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 18/354 objects degraded (5.085%), 5 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:17.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:16 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 18/354 objects degraded (5.085%), 5 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: pgmap v101: 65 pgs: 5 peering, 5 active+undersized+degraded, 7 active+undersized, 48 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 18/354 objects degraded (5.085%) 2024-03-16T09:08:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:18.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:18.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:18.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:18.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:18.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:18.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:18.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:18.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:18.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T09:08:18.364 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: pgmap v101: 65 pgs: 5 peering, 5 active+undersized+degraded, 7 active+undersized, 48 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 18/354 objects degraded (5.085%) 2024-03-16T09:08:18.364 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:18.365 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:18.365 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:18.365 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:18.365 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:18.365 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:18.365 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:18.365 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:18.366 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:18.366 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T09:08:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:18 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T09:08:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:18 smithi007 ceph-mon[113546]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-16T09:08:19.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:18 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T09:08:19.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:18 smithi019 ceph-mon[83366]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-16T09:08:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:19 smithi007 ceph-mon[113546]: pgmap v102: 65 pgs: 5 peering, 60 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:19 smithi007 ceph-mon[113546]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/354 objects degraded (5.085%), 5 pgs degraded) 2024-03-16T09:08:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:19 smithi007 ceph-mon[113546]: Cluster is now healthy 2024-03-16T09:08:20.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:19 smithi019 ceph-mon[83366]: pgmap v102: 65 pgs: 5 peering, 60 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:20.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:19 smithi019 ceph-mon[83366]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/354 objects degraded (5.085%), 5 pgs degraded) 2024-03-16T09:08:20.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:19 smithi019 ceph-mon[83366]: Cluster is now healthy 2024-03-16T09:08:22.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:21 smithi007 ceph-mon[113546]: pgmap v103: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:22.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:21 smithi019 ceph-mon[83366]: pgmap v103: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:23 smithi007 ceph-mon[113546]: pgmap v104: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:24.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:23 smithi019 ceph-mon[83366]: pgmap v104: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:25 smithi007 ceph-mon[113546]: pgmap v105: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:26.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:25 smithi019 ceph-mon[83366]: pgmap v105: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:26.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:28.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:27 smithi007 ceph-mon[113546]: pgmap v106: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:28.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:27 smithi019 ceph-mon[83366]: pgmap v106: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:30.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:29 smithi007 ceph-mon[113546]: pgmap v107: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:30.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:29 smithi019 ceph-mon[83366]: pgmap v107: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:31.563 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:31 smithi007 ceph-mon[113546]: pgmap v108: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:31.563 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:31.563 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:08:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.560+0000 7fdbb13f4700 1 -- 172.21.15.7:0/911080090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 msgr2=0x7fdbac1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.560+0000 7fdbb13f4700 1 --2- 172.21.15.7:0/911080090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 0x7fdbac1020f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009ce0 tx=0x7fdb9c004d10 comp rx=0 tx=0).stop 2024-03-16T09:08:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.560+0000 7fdbb13f4700 1 -- 172.21.15.7:0/911080090 shutdown_connections 2024-03-16T09:08:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.560+0000 7fdbb13f4700 1 --2- 172.21.15.7:0/911080090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbac1026c0 0x7fdbac0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.560+0000 7fdbb13f4700 1 --2- 172.21.15.7:0/911080090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 0x7fdbac1020f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.560+0000 7fdbb13f4700 1 -- 172.21.15.7:0/911080090 >> 172.21.15.7:0/911080090 conn(0x7fdbac0f7870 msgr2=0x7fdbac0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.560+0000 7fdbb13f4700 1 -- 172.21.15.7:0/911080090 shutdown_connections 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.560+0000 7fdbb13f4700 1 -- 172.21.15.7:0/911080090 wait complete. 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.561+0000 7fdbb13f4700 1 Processor -- start 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.561+0000 7fdbb13f4700 1 -- start start 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbb13f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 0x7fdbac112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbb13f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbac1026c0 0x7fdbac112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbb13f4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac113610 con 0x7fdbac101d20 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbb13f4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac10cbf0 con 0x7fdbac1026c0 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbaaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 0x7fdbac112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbaaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 0x7fdbac112a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50208/0 (socket says 172.21.15.7:50208) 2024-03-16T09:08:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbaaffd700 1 -- 172.21.15.7:0/2473448429 learned_addr learned my addr 172.21.15.7:0/2473448429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbaa7fc700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbac1026c0 0x7fdbac112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbaaffd700 1 -- 172.21.15.7:0/2473448429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbac1026c0 msgr2=0x7fdbac112fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbaaffd700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbac1026c0 0x7fdbac112fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbaaffd700 1 -- 172.21.15.7:0/2473448429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c009940 con 0x7fdbac101d20 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbaaffd700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 0x7fdbac112a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005010 tx=0x7fdb9c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdba37fe700 1 -- 172.21.15.7:0/2473448429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb9c00e040 con 0x7fdbac101d20 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdba37fe700 1 -- 172.21.15.7:0/2473448429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb9c008da0 con 0x7fdbac101d20 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac10ce70 con 0x7fdbac101d20 2024-03-16T09:08:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdba37fe700 1 -- 172.21.15.7:0/2473448429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb9c01c620 con 0x7fdbac101d20 2024-03-16T09:08:31.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.562+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac10d430 con 0x7fdbac101d20 2024-03-16T09:08:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.563+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac0fd470 con 0x7fdbac101d20 2024-03-16T09:08:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.564+0000 7fdba37fe700 1 -- 172.21.15.7:0/2473448429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdb9c012a70 con 0x7fdbac101d20 2024-03-16T09:08:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.564+0000 7fdba37fe700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb9806dee0 0x7fdb98070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.564+0000 7fdba37fe700 1 -- 172.21.15.7:0/2473448429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdb9c08f1e0 con 0x7fdbac101d20 2024-03-16T09:08:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.564+0000 7fdbaa7fc700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb9806dee0 0x7fdb98070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.565+0000 7fdbaa7fc700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb9806dee0 0x7fdb98070390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbac10e320 tx=0x7fdb94009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.569+0000 7fdba37fe700 1 -- 172.21.15.7:0/2473448429 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb9c05c2a0 con 0x7fdbac101d20 2024-03-16T09:08:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.794+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbac108e90 con 0x7fdb9806dee0 2024-03-16T09:08:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.795+0000 7fdba37fe700 1 -- 172.21.15.7:0/2473448429 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdbac108e90 con 0x7fdb9806dee0 2024-03-16T09:08:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb9806dee0 msgr2=0x7fdb98070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb9806dee0 0x7fdb98070390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbac10e320 tx=0x7fdb94009040 comp rx=0 tx=0).stop 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 msgr2=0x7fdbac112a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 0x7fdbac112a80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005010 tx=0x7fdb9c004d10 comp rx=0 tx=0).stop 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 shutdown_connections 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb9806dee0 0x7fdb98070390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac101d20 0x7fdbac112a80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 --2- 172.21.15.7:0/2473448429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbac1026c0 0x7fdbac112fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 >> 172.21.15.7:0/2473448429 conn(0x7fdbac0f7870 msgr2=0x7fdbac0f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 shutdown_connections 2024-03-16T09:08:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.797+0000 7fdbb13f4700 1 -- 172.21.15.7:0/2473448429 wait complete. 2024-03-16T09:08:31.809 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:08:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.900+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1411903067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e41037a0 msgr2=0x7f38e40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.900+0000 7f38ebbd2700 1 --2- 172.21.15.7:0/1411903067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e41037a0 0x7f38e40fce10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f38e0004940 tx=0x7f38e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.900+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1411903067 shutdown_connections 2024-03-16T09:08:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.900+0000 7f38ebbd2700 1 --2- 172.21.15.7:0/1411903067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e41037a0 0x7f38e40fce10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.900+0000 7f38ebbd2700 1 --2- 172.21.15.7:0/1411903067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38e4102e00 0x7f38e41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.900+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1411903067 >> 172.21.15.7:0/1411903067 conn(0x7f38e40f89b0 msgr2=0x7f38e40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1411903067 shutdown_connections 2024-03-16T09:08:31.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1411903067 wait complete. 2024-03-16T09:08:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38ebbd2700 1 Processor -- start 2024-03-16T09:08:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38ebbd2700 1 -- start start 2024-03-16T09:08:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38ebbd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4102e00 0x7f38e4113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38ebbd2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38e41037a0 0x7f38e4113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38ebbd2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4114510 con 0x7f38e4102e00 2024-03-16T09:08:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38ebbd2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e410db10 con 0x7f38e41037a0 2024-03-16T09:08:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38e996e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4102e00 0x7f38e4113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38e996e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4102e00 0x7f38e4113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50224/0 (socket says 172.21.15.7:50224) 2024-03-16T09:08:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.901+0000 7f38e996e700 1 -- 172.21.15.7:0/1105118181 learned_addr learned my addr 172.21.15.7:0/1105118181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38e916d700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38e41037a0 0x7f38e4113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38e996e700 1 -- 172.21.15.7:0/1105118181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38e41037a0 msgr2=0x7f38e4113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38e996e700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38e41037a0 0x7f38e4113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38e996e700 1 -- 172.21.15.7:0/1105118181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0005040 con 0x7f38e4102e00 2024-03-16T09:08:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38e996e700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4102e00 0x7f38e4113980 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c830 tx=0x7f38d4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38da7fc700 1 -- 172.21.15.7:0/1105118181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d40106f0 con 0x7f38e4102e00 2024-03-16T09:08:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38da7fc700 1 -- 172.21.15.7:0/1105118181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d4010d20 con 0x7f38e4102e00 2024-03-16T09:08:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e410ddf0 con 0x7f38e4102e00 2024-03-16T09:08:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38da7fc700 1 -- 172.21.15.7:0/1105118181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d40113a0 con 0x7f38e4102e00 2024-03-16T09:08:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.902+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e410e380 con 0x7f38e4102e00 2024-03-16T09:08:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.903+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e41052d0 con 0x7f38e4102e00 2024-03-16T09:08:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.904+0000 7f38da7fc700 1 -- 172.21.15.7:0/1105118181 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f38d4012400 con 0x7f38e4102e00 2024-03-16T09:08:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.904+0000 7f38da7fc700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f38d006dee0 0x7f38d0070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:31.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.904+0000 7f38da7fc700 1 -- 172.21.15.7:0/1105118181 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f38d408e160 con 0x7f38e4102e00 2024-03-16T09:08:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.905+0000 7f38e916d700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f38d006dee0 0x7f38d0070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.905+0000 7f38e916d700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f38d006dee0 0x7f38d0070390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f38e0004890 tx=0x7f38e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:31.909+0000 7f38da7fc700 1 -- 172.21.15.7:0/1105118181 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38d405b2c0 con 0x7f38e4102e00 2024-03-16T09:08:31.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:31 smithi019 ceph-mon[83366]: pgmap v108: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:31 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:31 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:08:32.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.138+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38e4109ee0 con 0x7f38d006dee0 2024-03-16T09:08:32.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.139+0000 7f38da7fc700 1 -- 172.21.15.7:0/1105118181 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f38e4109ee0 con 0x7f38d006dee0 2024-03-16T09:08:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f38d006dee0 msgr2=0x7f38d0070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f38d006dee0 0x7f38d0070390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f38e0004890 tx=0x7f38e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4102e00 msgr2=0x7f38e4113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4102e00 0x7f38e4113980 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c830 tx=0x7f38d4004470 comp rx=0 tx=0).stop 2024-03-16T09:08:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 shutdown_connections 2024-03-16T09:08:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f38d006dee0 0x7f38d0070390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e4102e00 0x7f38e4113980 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 --2- 172.21.15.7:0/1105118181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38e41037a0 0x7f38e4113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.141+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 >> 172.21.15.7:0/1105118181 conn(0x7f38e40f89b0 msgr2=0x7f38e41021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.142+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 shutdown_connections 2024-03-16T09:08:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.142+0000 7f38ebbd2700 1 -- 172.21.15.7:0/1105118181 wait complete. 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.244+0000 7fb5a9d96700 1 -- 172.21.15.7:0/4289893160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102710 msgr2=0x7fb5a40fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.244+0000 7fb5a9d96700 1 --2- 172.21.15.7:0/4289893160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102710 0x7fb5a40fbd80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb58c004940 tx=0x7fb58c009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.244+0000 7fb5a9d96700 1 -- 172.21.15.7:0/4289893160 shutdown_connections 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.244+0000 7fb5a9d96700 1 --2- 172.21.15.7:0/4289893160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4102710 0x7fb5a40fbd80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.244+0000 7fb5a9d96700 1 --2- 172.21.15.7:0/4289893160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5a4101d70 0x7fb5a4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.244+0000 7fb5a9d96700 1 -- 172.21.15.7:0/4289893160 >> 172.21.15.7:0/4289893160 conn(0x7fb5a4072bc0 msgr2=0x7fb5a4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.244+0000 7fb5a9d96700 1 -- 172.21.15.7:0/4289893160 shutdown_connections 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.244+0000 7fb5a9d96700 1 -- 172.21.15.7:0/4289893160 wait complete. 2024-03-16T09:08:32.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a9d96700 1 Processor -- start 2024-03-16T09:08:32.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a9d96700 1 -- start start 2024-03-16T09:08:32.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a9d96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4101d70 0x7fb5a410eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:32.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a9d96700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5a4102710 0x7fb5a410f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a9d96700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a410f670 con 0x7fb5a4101d70 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a9d96700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a410f7b0 con 0x7fb5a4102710 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4101d70 0x7fb5a410eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4101d70 0x7fb5a410eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50238/0 (socket says 172.21.15.7:50238) 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a37fe700 1 -- 172.21.15.7:0/2168935723 learned_addr learned my addr 172.21.15.7:0/2168935723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.245+0000 7fb5a2ffd700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5a4102710 0x7fb5a410f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb5a37fe700 1 -- 172.21.15.7:0/2168935723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5a4102710 msgr2=0x7fb5a410f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb5a37fe700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5a4102710 0x7fb5a410f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb5a37fe700 1 -- 172.21.15.7:0/2168935723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c005040 con 0x7fb5a4101d70 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb5a37fe700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4101d70 0x7fb5a410eae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb59400ec30 tx=0x7fb59400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb59bfff700 1 -- 172.21.15.7:0/2168935723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb594007b30 con 0x7fb5a4101d70 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb59bfff700 1 -- 172.21.15.7:0/2168935723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb59400f040 con 0x7fb5a4101d70 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a4117b50 con 0x7fb5a4101d70 2024-03-16T09:08:32.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb59bfff700 1 -- 172.21.15.7:0/2168935723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb59400cc90 con 0x7fb5a4101d70 2024-03-16T09:08:32.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.246+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a4118000 con 0x7fb5a4101d70 2024-03-16T09:08:32.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.247+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a4104240 con 0x7fb5a4101d70 2024-03-16T09:08:32.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.248+0000 7fb59bfff700 1 -- 172.21.15.7:0/2168935723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fb594007c90 con 0x7fb5a4101d70 2024-03-16T09:08:32.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.248+0000 7fb59bfff700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb59006dee0 0x7fb590070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:32.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.248+0000 7fb59bfff700 1 -- 172.21.15.7:0/2168935723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb59408dad0 con 0x7fb5a4101d70 2024-03-16T09:08:32.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.248+0000 7fb5a2ffd700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb59006dee0 0x7fb590070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:32.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.249+0000 7fb5a2ffd700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb59006dee0 0x7fb590070390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb58c004890 tx=0x7fb58c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:32.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.253+0000 7fb59bfff700 1 -- 172.21.15.7:0/2168935723 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb594057060 con 0x7fb5a4101d70 2024-03-16T09:08:32.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.470+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5a404bed0 con 0x7fb59006dee0 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.480+0000 7fb59bfff700 1 -- 172.21.15.7:0/2168935723 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fb5a404bed0 con 0x7fb59006dee0 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (4m) 19s ago 15m 28.5M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (2m) 19s ago 15m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (119s) 115s ago 14m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (4m) 19s ago 15m 84.6M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (9m) 19s ago 9m 14.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (9m) 19s ago 9m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (9m) 115s ago 9m 120M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (9m) 115s ago 9m 14.1M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:08:32.483 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (6m) 19s ago 16m 503M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (5m) 115s ago 14m 449M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (3m) 19s ago 16m 51.1M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (2m) 115s ago 14m 34.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (5m) 19s ago 15m 14.7M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (5m) 115s ago 14m 15.2M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (98s) 19s ago 13m 92.2M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (61s) 19s ago 13m 91.9M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4aa10e800f72 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (25s) 19s ago 12m 12.2M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5449f945991b 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (12m) 115s ago 12m 264M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 3660ab3e59a4 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (11m) 115s ago 11m 247M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (11m) 115s ago 11m 223M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:08:32.484 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (4m) 19s ago 14m 63.4M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:08:32.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.482+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb59006dee0 msgr2=0x7fb590070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.482+0000 7fb5a9d96700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb59006dee0 0x7fb590070390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb58c004890 tx=0x7fb58c009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:32.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.482+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4101d70 msgr2=0x7fb5a410eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.482+0000 7fb5a9d96700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4101d70 0x7fb5a410eae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb59400ec30 tx=0x7fb59400b040 comp rx=0 tx=0).stop 2024-03-16T09:08:32.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.483+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 shutdown_connections 2024-03-16T09:08:32.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.483+0000 7fb5a9d96700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fb59006dee0 0x7fb590070390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.483+0000 7fb5a9d96700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a4101d70 0x7fb5a410eae0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.483+0000 7fb5a9d96700 1 --2- 172.21.15.7:0/2168935723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5a4102710 0x7fb5a410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.483+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 >> 172.21.15.7:0/2168935723 conn(0x7fb5a4072bc0 msgr2=0x7fb5a40faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:32.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.483+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 shutdown_connections 2024-03-16T09:08:32.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.483+0000 7fb5a9d96700 1 -- 172.21.15.7:0/2168935723 wait complete. 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.585+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3266505112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0102710 msgr2=0x7fc0f00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.585+0000 7fc0f69a1700 1 --2- 172.21.15.7:0/3266505112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0102710 0x7fc0f00fbd80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004940 tx=0x7fc0e4009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.585+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3266505112 shutdown_connections 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.585+0000 7fc0f69a1700 1 --2- 172.21.15.7:0/3266505112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0102710 0x7fc0f00fbd80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.585+0000 7fc0f69a1700 1 --2- 172.21.15.7:0/3266505112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0f0101d70 0x7fc0f0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.585+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3266505112 >> 172.21.15.7:0/3266505112 conn(0x7fc0f0072bc0 msgr2=0x7fc0f0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.585+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3266505112 shutdown_connections 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.585+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3266505112 wait complete. 2024-03-16T09:08:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0f69a1700 1 Processor -- start 2024-03-16T09:08:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0f69a1700 1 -- start start 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0f69a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0101d70 0x7fc0f010ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0f69a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0f0102710 0x7fc0f010efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0f69a1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f010f610 con 0x7fc0f0101d70 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0f69a1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f010f750 con 0x7fc0f0102710 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0101d70 0x7fc0f010ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0101d70 0x7fc0f010ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54608/0 (socket says 172.21.15.7:54608) 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0effff700 1 -- 172.21.15.7:0/3253003813 learned_addr learned my addr 172.21.15.7:0/3253003813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0ef7fe700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0f0102710 0x7fc0f010efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0effff700 1 -- 172.21.15.7:0/3253003813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0f0102710 msgr2=0x7fc0f010efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0effff700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0f0102710 0x7fc0f010efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.586+0000 7fc0effff700 1 -- 172.21.15.7:0/3253003813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4005040 con 0x7fc0f0101d70 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.587+0000 7fc0effff700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0101d70 0x7fc0f010ea80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000eb40 tx=0x7fc0e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.587+0000 7fc0ecff9700 1 -- 172.21.15.7:0/3253003813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e0007cc0 con 0x7fc0f0101d70 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.587+0000 7fc0ecff9700 1 -- 172.21.15.7:0/3253003813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0e000f040 con 0x7fc0f0101d70 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.587+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f0117af0 con 0x7fc0f0101d70 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.587+0000 7fc0ecff9700 1 -- 172.21.15.7:0/3253003813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e000ca10 con 0x7fc0f0101d70 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.587+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f0117f50 con 0x7fc0f0101d70 2024-03-16T09:08:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.588+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f0104240 con 0x7fc0f0101d70 2024-03-16T09:08:32.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.588+0000 7fc0ecff9700 1 -- 172.21.15.7:0/3253003813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc0e000cc80 con 0x7fc0f0101d70 2024-03-16T09:08:32.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.589+0000 7fc0ecff9700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc0d806dee0 0x7fc0d8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:32.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.589+0000 7fc0ecff9700 1 -- 172.21.15.7:0/3253003813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc0e008d270 con 0x7fc0f0101d70 2024-03-16T09:08:32.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.589+0000 7fc0ef7fe700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc0d806dee0 0x7fc0d8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:32.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.589+0000 7fc0ef7fe700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc0d806dee0 0x7fc0d8070390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004890 tx=0x7fc0e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.593+0000 7fc0ecff9700 1 -- 172.21.15.7:0/3253003813 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc0e0056830 con 0x7fc0f0101d70 2024-03-16T09:08:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.897+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc0f004bed0 con 0x7fc0f0101d70 2024-03-16T09:08:32.900 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:32 smithi007 ceph-mon[113546]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:32.900 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:32 smithi007 ceph-mon[113546]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:32.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.897+0000 7fc0ecff9700 1 -- 172.21.15.7:0/3253003813 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fc0e0059ef0 con 0x7fc0f0101d70 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 3, 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 3 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:08:32.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:08:32.902 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:08:32.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:08:32.902 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:08:32.902 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 7, 2024-03-16T09:08:32.902 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 7 2024-03-16T09:08:32.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:08:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.899+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc0d806dee0 msgr2=0x7fc0d8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.899+0000 7fc0f69a1700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc0d806dee0 0x7fc0d8070390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4004890 tx=0x7fc0e4009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.899+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0101d70 msgr2=0x7fc0f010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.899+0000 7fc0f69a1700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0101d70 0x7fc0f010ea80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000eb40 tx=0x7fc0e000b040 comp rx=0 tx=0).stop 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.900+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 shutdown_connections 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.900+0000 7fc0f69a1700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc0d806dee0 0x7fc0d8070390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.900+0000 7fc0f69a1700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0f0101d70 0x7fc0f010ea80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.900+0000 7fc0f69a1700 1 --2- 172.21.15.7:0/3253003813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0f0102710 0x7fc0f010efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.900+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 >> 172.21.15.7:0/3253003813 conn(0x7fc0f0072bc0 msgr2=0x7fc0f00faf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:32.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.900+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 shutdown_connections 2024-03-16T09:08:32.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:32.900+0000 7fc0f69a1700 1 -- 172.21.15.7:0/3253003813 wait complete. 2024-03-16T09:08:32.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:32 smithi019 ceph-mon[83366]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:32.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:32 smithi019 ceph-mon[83366]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 -- 172.21.15.7:0/1220134507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928101d70 msgr2=0x7f8928102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 --2- 172.21.15.7:0/1220134507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928101d70 0x7f8928102140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8920004940 tx=0x7f8920009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 -- 172.21.15.7:0/1220134507 shutdown_connections 2024-03-16T09:08:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 --2- 172.21.15.7:0/1220134507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8928102710 0x7f89280fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 --2- 172.21.15.7:0/1220134507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928101d70 0x7f8928102140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 -- 172.21.15.7:0/1220134507 >> 172.21.15.7:0/1220134507 conn(0x7f8928072bc0 msgr2=0x7f8928072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 -- 172.21.15.7:0/1220134507 shutdown_connections 2024-03-16T09:08:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 -- 172.21.15.7:0/1220134507 wait complete. 2024-03-16T09:08:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.002+0000 7f892fc2e700 1 Processor -- start 2024-03-16T09:08:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892fc2e700 1 -- start start 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892fc2e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8928101d70 0x7f892810eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892fc2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928102710 0x7f892810f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892fc2e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892810f650 con 0x7f8928102710 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892fc2e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892810f790 con 0x7f8928101d70 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892d1c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928102710 0x7f892810f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892d9ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8928101d70 0x7f892810eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892d1c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928102710 0x7f892810f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54626/0 (socket says 172.21.15.7:54626) 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892d1c9700 1 -- 172.21.15.7:0/4233451516 learned_addr learned my addr 172.21.15.7:0/4233451516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892d1c9700 1 -- 172.21.15.7:0/4233451516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8928101d70 msgr2=0x7f892810eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892d1c9700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8928101d70 0x7f892810eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.003+0000 7f892d1c9700 1 -- 172.21.15.7:0/4233451516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8920005040 con 0x7f8928102710 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.004+0000 7f892d1c9700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928102710 0x7f892810f000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f891c00eb40 tx=0x7f891c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.004+0000 7f89267fc700 1 -- 172.21.15.7:0/4233451516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f891c007cc0 con 0x7f8928102710 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.004+0000 7f89267fc700 1 -- 172.21.15.7:0/4233451516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f891c00f040 con 0x7f8928102710 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.004+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8928117b30 con 0x7f8928102710 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.004+0000 7f89267fc700 1 -- 172.21.15.7:0/4233451516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f891c00c9d0 con 0x7f8928102710 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.004+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8928117fc0 con 0x7f8928102710 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.005+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89280ff080 con 0x7f8928102710 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.005+0000 7f89267fc700 1 -- 172.21.15.7:0/4233451516 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f891c0090d0 con 0x7f8928102710 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.006+0000 7f89267fc700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f891006dee0 0x7f8910070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.006+0000 7f89267fc700 1 -- 172.21.15.7:0/4233451516 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f891c08d050 con 0x7f8928102710 2024-03-16T09:08:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.006+0000 7f892d9ca700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f891006dee0 0x7f8910070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.006+0000 7f892d9ca700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f891006dee0 0x7f8910070390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8920009b30 tx=0x7f8920009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:33.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.010+0000 7f89267fc700 1 -- 172.21.15.7:0/4233451516 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f891c056690 con 0x7f8928102710 2024-03-16T09:08:33.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.265+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f89280fd4c0 con 0x7f8928102710 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.265+0000 7f89267fc700 1 -- 172.21.15.7:0/4233451516 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1895 (secure 0 0 0) 0x7f891c059d50 con 0x7f8928102710 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:e20 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-03-16T09:08:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:08:33.269 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:08:33.270 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:08:33.270 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:08:33.270 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:08:33.270 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:08:33.270 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:08:33.271 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:08:33.271 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:08:33.271 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:08:33.271 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:08:33.271 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:08:33.271 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:08:33.271 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:08:33.272 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:08:33.272 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:08:33.272 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:08:33.272 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:08:33.272 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:08:33.272 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:08:33.272 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:08:33.273 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:08:33.273 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:08:33.273 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.267+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f891006dee0 msgr2=0x7f8910070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.267+0000 7f892fc2e700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f891006dee0 0x7f8910070390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8920009b30 tx=0x7f8920009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.267+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928102710 msgr2=0x7f892810f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.267+0000 7f892fc2e700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928102710 0x7f892810f000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f891c00eb40 tx=0x7f891c00b040 comp rx=0 tx=0).stop 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.267+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 shutdown_connections 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.267+0000 7f892fc2e700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8928101d70 0x7f892810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.267+0000 7f892fc2e700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f891006dee0 0x7f8910070390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.267+0000 7f892fc2e700 1 --2- 172.21.15.7:0/4233451516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8928102710 0x7f892810f000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.268+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 >> 172.21.15.7:0/4233451516 conn(0x7f8928072bc0 msgr2=0x7f89280faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.268+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 shutdown_connections 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.268+0000 7f892fc2e700 1 -- 172.21.15.7:0/4233451516 wait complete. 2024-03-16T09:08:33.274 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:08:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.371+0000 7f4a0acde700 1 -- 172.21.15.7:0/1658183437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 msgr2=0x7f4a041031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.371+0000 7f4a0acde700 1 --2- 172.21.15.7:0/1658183437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 0x7f4a041031d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f49f8004940 tx=0x7f49f8009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 -- 172.21.15.7:0/1658183437 shutdown_connections 2024-03-16T09:08:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 --2- 172.21.15.7:0/1658183437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a041037a0 0x7f4a040fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 --2- 172.21.15.7:0/1658183437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 0x7f4a041031d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 -- 172.21.15.7:0/1658183437 >> 172.21.15.7:0/1658183437 conn(0x7f4a040f89b0 msgr2=0x7f4a040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 -- 172.21.15.7:0/1658183437 shutdown_connections 2024-03-16T09:08:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 -- 172.21.15.7:0/1658183437 wait complete. 2024-03-16T09:08:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 Processor -- start 2024-03-16T09:08:33.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 -- start start 2024-03-16T09:08:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 0x7f4a041138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.372+0000 7f4a0acde700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a041037a0 0x7f4a04113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a0acde700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04114460 con 0x7f4a04102e00 2024-03-16T09:08:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a0acde700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0410dab0 con 0x7f4a041037a0 2024-03-16T09:08:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a08a7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 0x7f4a041138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a08a7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 0x7f4a041138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54646/0 (socket says 172.21.15.7:54646) 2024-03-16T09:08:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a03fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a041037a0 0x7f4a04113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a08a7a700 1 -- 172.21.15.7:0/3838780071 learned_addr learned my addr 172.21.15.7:0/3838780071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a03fff700 1 -- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 msgr2=0x7f4a041138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a03fff700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 0x7f4a041138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a03fff700 1 -- 172.21.15.7:0/3838780071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8005040 con 0x7f4a041037a0 2024-03-16T09:08:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a08a7a700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 0x7f4a041138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:08:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a03fff700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a041037a0 0x7f4a04113e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a04067330 tx=0x7f49f000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a017fa700 1 -- 172.21.15.7:0/3838780071 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f0004e90 con 0x7f4a041037a0 2024-03-16T09:08:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a017fa700 1 -- 172.21.15.7:0/3838780071 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49f000f040 con 0x7f4a041037a0 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0410dcd0 con 0x7f4a041037a0 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a017fa700 1 -- 172.21.15.7:0/3838780071 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f000ca20 con 0x7f4a041037a0 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.373+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0410e1e0 con 0x7f4a041037a0 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.374+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04100110 con 0x7f4a041037a0 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.375+0000 7f4a017fa700 1 -- 172.21.15.7:0/3838780071 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f49f0007cc0 con 0x7f4a041037a0 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.375+0000 7f4a017fa700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f49f406dee0 0x7f49f4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.375+0000 7f4a017fa700 1 -- 172.21.15.7:0/3838780071 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f49f008d120 con 0x7f4a041037a0 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.375+0000 7f4a08a7a700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f49f406dee0 0x7f49f4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.376+0000 7f4a08a7a700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f49f406dee0 0x7f49f4070390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49f800b0d0 tx=0x7f49f8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.380+0000 7f4a017fa700 1 -- 172.21.15.7:0/3838780071 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49f005a170 con 0x7f4a041037a0 2024-03-16T09:08:33.610 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:33 smithi019 ceph-mon[83366]: pgmap v109: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:33.610 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:33 smithi019 ceph-mon[83366]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:33.610 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:33 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T09:08:33.610 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:33 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T09:08:33.610 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:33 smithi019 ceph-mon[83366]: Upgrade: osd.3 is safe to restart 2024-03-16T09:08:33.610 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:33 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3253003813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:33.610 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:33 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/4233451516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:08:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.617+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a0410ef90 con 0x7f49f406dee0 2024-03-16T09:08:33.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:33 smithi007 ceph-mon[113546]: pgmap v109: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:33.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:33 smithi007 ceph-mon[113546]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:33.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:33 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T09:08:33.619 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:33 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T09:08:33.620 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:33 smithi007 ceph-mon[113546]: Upgrade: osd.3 is safe to restart 2024-03-16T09:08:33.620 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:33 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3253003813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:33.620 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:33 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/4233451516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:08:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.618+0000 7f4a017fa700 1 -- 172.21.15.7:0/3838780071 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4a0410ef90 con 0x7f49f406dee0 2024-03-16T09:08:33.620 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:08:33.620 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "9/21 daemons upgraded", 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:08:33.621 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:08:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f49f406dee0 msgr2=0x7f49f4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f49f406dee0 0x7f49f4070390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49f800b0d0 tx=0x7f49f8009e30 comp rx=0 tx=0).stop 2024-03-16T09:08:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a041037a0 msgr2=0x7f4a04113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a041037a0 0x7f4a04113e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4a04067330 tx=0x7f49f000b040 comp rx=0 tx=0).stop 2024-03-16T09:08:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 shutdown_connections 2024-03-16T09:08:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f49f406dee0 0x7f49f4070390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a04102e00 0x7f4a041138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 --2- 172.21.15.7:0/3838780071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a041037a0 0x7f4a04113e10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 >> 172.21.15.7:0/3838780071 conn(0x7f4a040f89b0 msgr2=0x7f4a040fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 shutdown_connections 2024-03-16T09:08:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.621+0000 7f4a0acde700 1 -- 172.21.15.7:0/3838780071 wait complete. 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 -- 172.21.15.7:0/1668625986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fddf0 msgr2=0x7fa8280fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 --2- 172.21.15.7:0/1668625986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fddf0 0x7fa8280fe1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa818009ce0 tx=0x7fa818004d10 comp rx=0 tx=0).stop 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 -- 172.21.15.7:0/1668625986 shutdown_connections 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 --2- 172.21.15.7:0/1668625986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8280fe790 0x7fa828106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 --2- 172.21.15.7:0/1668625986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fddf0 0x7fa8280fe1c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 -- 172.21.15.7:0/1668625986 >> 172.21.15.7:0/1668625986 conn(0x7fa8280f7860 msgr2=0x7fa8280f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 -- 172.21.15.7:0/1668625986 shutdown_connections 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 -- 172.21.15.7:0/1668625986 wait complete. 2024-03-16T09:08:33.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 Processor -- start 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.724+0000 7fa82d2cd700 1 -- start start 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa82d2cd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8280fddf0 0x7fa828113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa82d2cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fe790 0x7fa8281140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa82d2cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa828114700 con 0x7fa8280fe790 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa82d2cd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82810daf0 con 0x7fa8280fddf0 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa826ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8280fddf0 0x7fa828113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa8267fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fe790 0x7fa8281140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa826ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8280fddf0 0x7fa828113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.7:54324/0 (socket says 172.21.15.7:54324) 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa826ffd700 1 -- 172.21.15.7:0/3189652640 learned_addr learned my addr 172.21.15.7:0/3189652640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa826ffd700 1 -- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fe790 msgr2=0x7fa8281140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa826ffd700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fe790 0x7fa8281140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa826ffd700 1 -- 172.21.15.7:0/3189652640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa818009940 con 0x7fa8280fddf0 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa8267fc700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fe790 0x7fa8281140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:08:33.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa826ffd700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8280fddf0 0x7fa828113b70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa818004f80 tx=0x7fa818004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa81f7fe700 1 -- 172.21.15.7:0/3189652640 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa81800e040 con 0x7fa8280fddf0 2024-03-16T09:08:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82810dd70 con 0x7fa8280fddf0 2024-03-16T09:08:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa81f7fe700 1 -- 172.21.15.7:0/3189652640 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa81801cb20 con 0x7fa8280fddf0 2024-03-16T09:08:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.725+0000 7fa81f7fe700 1 -- 172.21.15.7:0/3189652640 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8180129e0 con 0x7fa8280fddf0 2024-03-16T09:08:33.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.726+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82810e2b0 con 0x7fa8280fddf0 2024-03-16T09:08:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.726+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa828066770 con 0x7fa8280fddf0 2024-03-16T09:08:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.727+0000 7fa81f7fe700 1 -- 172.21.15.7:0/3189652640 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa81801cc90 con 0x7fa8280fddf0 2024-03-16T09:08:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.727+0000 7fa81f7fe700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa81406dee0 0x7fa814070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.727+0000 7fa81f7fe700 1 -- 172.21.15.7:0/3189652640 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa81808f8e0 con 0x7fa8280fddf0 2024-03-16T09:08:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.727+0000 7fa8267fc700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa81406dee0 0x7fa814070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.728+0000 7fa8267fc700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa81406dee0 0x7fa814070390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa82810ecb0 tx=0x7fa81000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:33.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:33.732+0000 7fa81f7fe700 1 -- 172.21.15.7:0/3189652640 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa81805c880 con 0x7fa8280fddf0 2024-03-16T09:08:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.026+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa828109ed0 con 0x7fa8280fddf0 2024-03-16T09:08:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.026+0000 7fa81f7fe700 1 -- 172.21.15.7:0/3189652640 <== mon.1 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa81805c410 con 0x7fa8280fddf0 2024-03-16T09:08:34.029 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_OK 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa81406dee0 msgr2=0x7fa814070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa81406dee0 0x7fa814070390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa82810ecb0 tx=0x7fa81000b040 comp rx=0 tx=0).stop 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8280fddf0 msgr2=0x7fa828113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8280fddf0 0x7fa828113b70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa818004f80 tx=0x7fa818004d10 comp rx=0 tx=0).stop 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 shutdown_connections 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8280fddf0 0x7fa828113b70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa81406dee0 0x7fa814070390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 --2- 172.21.15.7:0/3189652640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8280fe790 0x7fa8281140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.028+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 >> 172.21.15.7:0/3189652640 conn(0x7fa8280f7860 msgr2=0x7fa8280f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.029+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 shutdown_connections 2024-03-16T09:08:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:08:34.029+0000 7fa82d2cd700 1 -- 172.21.15.7:0/3189652640 wait complete. 2024-03-16T09:08:34.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:34 smithi019 ceph-mon[83366]: from='client.44215 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:34.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:34 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3189652640' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:08:34.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:34 smithi019 ceph-mon[83366]: Upgrade: Updating osd.3 2024-03-16T09:08:34.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:34.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T09:08:34.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:34.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:34 smithi019 ceph-mon[83366]: Deploying daemon osd.3 on smithi019 2024-03-16T09:08:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:34 smithi007 ceph-mon[113546]: from='client.44215 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:34 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3189652640' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:08:34.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:34 smithi007 ceph-mon[113546]: Upgrade: Updating osd.3 2024-03-16T09:08:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T09:08:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:34.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:34 smithi007 ceph-mon[113546]: Deploying daemon osd.3 on smithi019 2024-03-16T09:08:36.514 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:36 smithi019 ceph-mon[83366]: pgmap v110: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:36.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:36 smithi007 ceph-mon[113546]: pgmap v110: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:37.951 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:37 smithi019 systemd[1]: Stopping Ceph osd.3 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:08:38.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:37 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[41983]: 2024-03-16T09:08:37.948+0000 7f081ec3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:08:38.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:37 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[41983]: 2024-03-16T09:08:37.948+0000 7f081ec3b700 -1 osd.3 64 *** Got signal Terminated *** 2024-03-16T09:08:38.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:37 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[41983]: 2024-03-16T09:08:37.948+0000 7f081ec3b700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:08:38.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:38 smithi019 ceph-mon[83366]: pgmap v111: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:38.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:38 smithi019 ceph-mon[83366]: osd.3 marked itself down and dead 2024-03-16T09:08:38.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:38 smithi007 ceph-mon[113546]: pgmap v111: 65 pgs: 65 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:38.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:38 smithi007 ceph-mon[113546]: osd.3 marked itself down and dead 2024-03-16T09:08:39.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:38 smithi019 podman[92589]: 2024-03-16 09:08:38.975871134 +0000 UTC m=+1.121082145 container died 3660ab3e59a49ba14479a226d3bcacd4e6b3cbf98b80dc9d95054359e46ad17f (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-1df2724, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-16T09:08:39.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:39 smithi019 ceph-mon[83366]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:08:39.493 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:39 smithi019 ceph-mon[83366]: osdmap e65: 6 total, 5 up, 6 in 2024-03-16T09:08:39.493 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:39 smithi019 podman[92589]: 2024-03-16 09:08:39.298702717 +0000 UTC m=+1.443913729 container cleanup 3660ab3e59a49ba14479a226d3bcacd4e6b3cbf98b80dc9d95054359e46ad17f (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=quincy-1df2724, GIT_BRANCH=HEAD, ceph=True) 2024-03-16T09:08:39.493 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:39 smithi019 bash[92589]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3 2024-03-16T09:08:39.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:39 smithi007 ceph-mon[113546]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:08:39.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:39 smithi007 ceph-mon[113546]: osdmap e65: 6 total, 5 up, 6 in 2024-03-16T09:08:39.902 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:39 smithi019 podman[92617]: 2024-03-16 09:08:39.491824212 +0000 UTC m=+0.519797693 container remove 3660ab3e59a49ba14479a226d3bcacd4e6b3cbf98b80dc9d95054359e46ad17f (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=quincy-1df2724, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:08:40.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:39 smithi019 podman[92707]: 2024-03-16 09:08:39.901215376 +0000 UTC m=+0.110921610 container create 90366427abe3b4d0962fa64f1219e043e118ac8564cf3cf61e307049d23e8c84 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-16T09:08:40.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:39 smithi019 podman[92707]: 2024-03-16 09:08:39.80889288 +0000 UTC m=+0.018599088 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:08:40.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:40 smithi019 podman[92707]: 2024-03-16 09:08:40.112062122 +0000 UTC m=+0.321768319 container init 90366427abe3b4d0962fa64f1219e043e118ac8564cf3cf61e307049d23e8c84 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-03-16T09:08:40.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:40 smithi019 podman[92707]: 2024-03-16 09:08:40.115065971 +0000 UTC m=+0.324772161 container start 90366427abe3b4d0962fa64f1219e043e118ac8564cf3cf61e307049d23e8c84 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:08:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:40 smithi019 ceph-mon[83366]: pgmap v113: 65 pgs: 16 stale+active+clean, 49 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:40 smithi019 ceph-mon[83366]: osdmap e66: 6 total, 5 up, 6 in 2024-03-16T09:08:40.713 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:40 smithi019 podman[92707]: 2024-03-16 09:08:40.212136726 +0000 UTC m=+0.421842925 container attach 90366427abe3b4d0962fa64f1219e043e118ac8564cf3cf61e307049d23e8c84 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-16T09:08:40.713 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:40 smithi019 podman[92707]: 2024-03-16 09:08:40.416082748 +0000 UTC m=+0.625788936 container died 90366427abe3b4d0962fa64f1219e043e118ac8564cf3cf61e307049d23e8c84 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-16T09:08:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:40 smithi007 ceph-mon[113546]: pgmap v113: 65 pgs: 16 stale+active+clean, 49 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:40 smithi007 ceph-mon[113546]: osdmap e66: 6 total, 5 up, 6 in 2024-03-16T09:08:41.012 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:40 smithi019 podman[92732]: 2024-03-16 09:08:40.974079981 +0000 UTC m=+0.552973353 container remove 90366427abe3b4d0962fa64f1219e043e118ac8564cf3cf61e307049d23e8c84 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:08:41.012 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:40 smithi019 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-16T09:08:41.012 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:40 smithi019 systemd[1]: Stopped Ceph osd.3 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:08:41.012 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:40 smithi019 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.3.service: Consumed 1min 8.675s CPU time. 2024-03-16T09:08:41.439 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:41 smithi019 systemd[1]: Starting Ceph osd.3 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:08:41.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:41 smithi007 ceph-mon[113546]: pgmap v115: 65 pgs: 3 active+undersized+degraded, 13 stale+active+clean, 4 active+undersized, 45 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 8/354 objects degraded (2.260%) 2024-03-16T09:08:41.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:41 smithi007 ceph-mon[113546]: Health check failed: Degraded data redundancy: 8/354 objects degraded (2.260%), 3 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:41.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:41 smithi019 ceph-mon[83366]: pgmap v115: 65 pgs: 3 active+undersized+degraded, 13 stale+active+clean, 4 active+undersized, 45 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 8/354 objects degraded (2.260%) 2024-03-16T09:08:41.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:41 smithi019 ceph-mon[83366]: Health check failed: Degraded data redundancy: 8/354 objects degraded (2.260%), 3 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:41.713 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:41 smithi019 podman[92838]: 2024-03-16 09:08:41.340526275 +0000 UTC m=+0.017749969 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:08:41.713 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:41 smithi019 podman[92838]: 2024-03-16 09:08:41.44175308 +0000 UTC m=+0.118976757 container create d368a14ecc4c0e1d72206b613010bc2800cf2bff1efe11891730038f6c9c6364 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:08:41.713 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:41 smithi019 podman[92838]: 2024-03-16 09:08:41.677914787 +0000 UTC m=+0.355138463 container init d368a14ecc4c0e1d72206b613010bc2800cf2bff1efe11891730038f6c9c6364 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-16T09:08:41.713 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:41 smithi019 podman[92838]: 2024-03-16 09:08:41.680970338 +0000 UTC m=+0.358194023 container start d368a14ecc4c0e1d72206b613010bc2800cf2bff1efe11891730038f6c9c6364 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-16T09:08:42.134 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:41 smithi019 podman[92838]: 2024-03-16 09:08:41.786226398 +0000 UTC m=+0.463450082 container attach d368a14ecc4c0e1d72206b613010bc2800cf2bff1efe11891730038f6c9c6364 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate[92853]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate[92853]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 bash[92838]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 bash[92838]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 bash[92838]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 bash[92838]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 bash[92838]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 bash[92838]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 bash[92838]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate[92853]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate[92853]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-16T09:08:42.450 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate[92853]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-16T09:08:42.451 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate[92853]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T09:08:42.451 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate[92853]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-16T09:08:42.451 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 podman[92838]: 2024-03-16 09:08:42.15721473 +0000 UTC m=+0.834438413 container died d368a14ecc4c0e1d72206b613010bc2800cf2bff1efe11891730038f6c9c6364 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-16T09:08:43.075 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 podman[92988]: 2024-03-16 09:08:42.731877559 +0000 UTC m=+0.569673329 container remove d368a14ecc4c0e1d72206b613010bc2800cf2bff1efe11891730038f6c9c6364 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T09:08:43.075 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 podman[93052]: 2024-03-16 09:08:42.949327919 +0000 UTC m=+0.114900279 container create bb64247744fe74069e60e14781ae07a9abb1e6270c76d69c239172c12d960fbe (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-16T09:08:43.075 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:42 smithi019 podman[93052]: 2024-03-16 09:08:42.852480764 +0000 UTC m=+0.018053141 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:08:43.463 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:43 smithi019 podman[93052]: 2024-03-16 09:08:43.159947544 +0000 UTC m=+0.325519917 container init bb64247744fe74069e60e14781ae07a9abb1e6270c76d69c239172c12d960fbe (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-16T09:08:43.463 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:43 smithi019 podman[93052]: 2024-03-16 09:08:43.163265478 +0000 UTC m=+0.328837844 container start bb64247744fe74069e60e14781ae07a9abb1e6270c76d69c239172c12d960fbe (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-16T09:08:43.463 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:43 smithi019 ceph-osd[93072]: -- 172.21.15.19:0/2154048760 <== mon.1 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c62dab680 con 0x561c62d74c00 2024-03-16T09:08:43.463 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:43 smithi019 bash[93052]: bb64247744fe74069e60e14781ae07a9abb1e6270c76d69c239172c12d960fbe 2024-03-16T09:08:43.463 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:43 smithi019 systemd[1]: Started Ceph osd.3 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:08:43.963 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:43 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[93068]: 2024-03-16T09:08:43.702+0000 7fa105e257c0 -1 Falling back to public interface 2024-03-16T09:08:44.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:44 smithi019 ceph-mon[83366]: pgmap v116: 65 pgs: 12 active+undersized+degraded, 3 stale+active+clean, 13 active+undersized, 37 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 50/354 objects degraded (14.124%) 2024-03-16T09:08:44.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:44.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:44.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:44 smithi007 ceph-mon[113546]: pgmap v116: 65 pgs: 12 active+undersized+degraded, 3 stale+active+clean, 13 active+undersized, 37 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 50/354 objects degraded (14.124%) 2024-03-16T09:08:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:46.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:46 smithi019 ceph-mon[83366]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 73/354 objects degraded (20.621%) 2024-03-16T09:08:46.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:46.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:08:46.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:46.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:46 smithi007 ceph-mon[113546]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 73/354 objects degraded (20.621%) 2024-03-16T09:08:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:08:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:47.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:47 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 73/354 objects degraded (20.621%), 19 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:47.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:47 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:47.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:47 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:47.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:47 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 73/354 objects degraded (20.621%), 19 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:47.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:47 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:47.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:47 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:48.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:48 smithi019 ceph-mon[83366]: pgmap v118: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 73/354 objects degraded (20.621%) 2024-03-16T09:08:48.713 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[93068]: 2024-03-16T09:08:48.272+0000 7fa105e257c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-16T09:08:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:48 smithi007 ceph-mon[113546]: pgmap v118: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 73/354 objects degraded (20.621%) 2024-03-16T09:08:49.202 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[93068]: 2024-03-16T09:08:48.856+0000 7fa105e257c0 -1 osd.3 64 log_to_monitors true 2024-03-16T09:08:49.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:49 smithi019 ceph-mon[83366]: from='osd.3 [v2:172.21.15.19:6800/2696958381,v1:172.21.15.19:6801/2696958381]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T09:08:49.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:49 smithi019 ceph-mon[83366]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T09:08:49.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:49 smithi007 ceph-mon[113546]: from='osd.3 [v2:172.21.15.19:6800/2696958381,v1:172.21.15.19:6801/2696958381]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T09:08:49.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:49 smithi007 ceph-mon[113546]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T09:08:50.579 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:08:50 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[93068]: 2024-03-16T09:08:50.262+0000 7fa0f3275700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:08:50.579 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:50 smithi019 ceph-mon[83366]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 73/354 objects degraded (20.621%) 2024-03-16T09:08:50.579 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:50 smithi019 ceph-mon[83366]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T09:08:50.579 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:50 smithi019 ceph-mon[83366]: from='osd.3 [v2:172.21.15.19:6800/2696958381,v1:172.21.15.19:6801/2696958381]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:08:50.579 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:50 smithi019 ceph-mon[83366]: osdmap e67: 6 total, 5 up, 6 in 2024-03-16T09:08:50.579 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:50 smithi019 ceph-mon[83366]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:08:50.579 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:50 smithi019 ceph-mon[83366]: from='osd.3 ' entity='osd.3' 2024-03-16T09:08:50.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:50 smithi007 ceph-mon[113546]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 300 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 73/354 objects degraded (20.621%) 2024-03-16T09:08:50.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:50 smithi007 ceph-mon[113546]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T09:08:50.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:50 smithi007 ceph-mon[113546]: from='osd.3 [v2:172.21.15.19:6800/2696958381,v1:172.21.15.19:6801/2696958381]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:08:50.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:50 smithi007 ceph-mon[113546]: osdmap e67: 6 total, 5 up, 6 in 2024-03-16T09:08:50.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:50 smithi007 ceph-mon[113546]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:08:50.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:50 smithi007 ceph-mon[113546]: from='osd.3 ' entity='osd.3' 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: pgmap v121: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/354 objects degraded (20.621%) 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:52.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:52.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T09:08:52.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T09:08:52.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-16T09:08:52.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:08:52.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: osd.3 [v2:172.21.15.19:6800/2696958381,v1:172.21.15.19:6801/2696958381] boot 2024-03-16T09:08:52.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: osdmap e68: 6 total, 6 up, 6 in 2024-03-16T09:08:52.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:52 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:08:52.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: pgmap v121: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/354 objects degraded (20.621%) 2024-03-16T09:08:52.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:52.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: osd.3 [v2:172.21.15.19:6800/2696958381,v1:172.21.15.19:6801/2696958381] boot 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: osdmap e68: 6 total, 6 up, 6 in 2024-03-16T09:08:52.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:52 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:08:53.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:53 smithi019 ceph-mon[83366]: osdmap e69: 6 total, 6 up, 6 in 2024-03-16T09:08:53.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:53 smithi007 ceph-mon[113546]: osdmap e69: 6 total, 6 up, 6 in 2024-03-16T09:08:54.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:54 smithi019 ceph-mon[83366]: pgmap v124: 65 pgs: 8 peering, 16 active+undersized+degraded, 14 active+undersized, 27 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 63/354 objects degraded (17.797%) 2024-03-16T09:08:54.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:54 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 63/354 objects degraded (17.797%), 16 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:54 smithi007 ceph-mon[113546]: pgmap v124: 65 pgs: 8 peering, 16 active+undersized+degraded, 14 active+undersized, 27 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 63/354 objects degraded (17.797%) 2024-03-16T09:08:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:54 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 63/354 objects degraded (17.797%), 16 pgs degraded (PG_DEGRADED) 2024-03-16T09:08:56.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:56 smithi019 ceph-mon[83366]: pgmap v125: 65 pgs: 8 peering, 1 active+undersized+degraded, 3 active+undersized, 53 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 4/354 objects degraded (1.130%) 2024-03-16T09:08:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:56 smithi007 ceph-mon[113546]: pgmap v125: 65 pgs: 8 peering, 1 active+undersized+degraded, 3 active+undersized, 53 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 4/354 objects degraded (1.130%) 2024-03-16T09:08:57.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:57 smithi019 ceph-mon[83366]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/354 objects degraded (1.130%), 1 pg degraded) 2024-03-16T09:08:57.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:57 smithi019 ceph-mon[83366]: Cluster is now healthy 2024-03-16T09:08:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:57 smithi007 ceph-mon[113546]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/354 objects degraded (1.130%), 1 pg degraded) 2024-03-16T09:08:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:57 smithi007 ceph-mon[113546]: Cluster is now healthy 2024-03-16T09:08:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:08:58 smithi019 ceph-mon[83366]: pgmap v126: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:08:58.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:08:58 smithi007 ceph-mon[113546]: pgmap v126: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:00 smithi019 ceph-mon[83366]: pgmap v127: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:00.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:00 smithi007 ceph-mon[113546]: pgmap v127: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:01 smithi019 ceph-mon[83366]: pgmap v128: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:09:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:01 smithi007 ceph-mon[113546]: pgmap v128: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:01.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:09:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:03 smithi019 ceph-mon[83366]: pgmap v129: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:03.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:03 smithi007 ceph-mon[113546]: pgmap v129: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.145+0000 7f4c93c27700 1 -- 172.21.15.7:0/1431742834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 msgr2=0x7f4c8c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.145+0000 7f4c93c27700 1 --2- 172.21.15.7:0/1431742834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 0x7f4c8c1031d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4c88004940 tx=0x7f4c88009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.145+0000 7f4c93c27700 1 -- 172.21.15.7:0/1431742834 shutdown_connections 2024-03-16T09:09:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.145+0000 7f4c93c27700 1 --2- 172.21.15.7:0/1431742834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c1037a0 0x7f4c8c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.145+0000 7f4c93c27700 1 --2- 172.21.15.7:0/1431742834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 0x7f4c8c1031d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.145+0000 7f4c93c27700 1 -- 172.21.15.7:0/1431742834 >> 172.21.15.7:0/1431742834 conn(0x7f4c8c0f89b0 msgr2=0x7f4c8c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.145+0000 7f4c93c27700 1 -- 172.21.15.7:0/1431742834 shutdown_connections 2024-03-16T09:09:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.145+0000 7f4c93c27700 1 -- 172.21.15.7:0/1431742834 wait complete. 2024-03-16T09:09:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c93c27700 1 Processor -- start 2024-03-16T09:09:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c93c27700 1 -- start start 2024-03-16T09:09:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c93c27700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 0x7f4c8c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c93c27700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c1037a0 0x7f4c8c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c93c27700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0721c0 con 0x7f4c8c102e00 2024-03-16T09:09:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c93c27700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c070d40 con 0x7f4c8c1037a0 2024-03-16T09:09:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c919c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 0x7f4c8c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c919c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 0x7f4c8c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43282/0 (socket says 172.21.15.7:43282) 2024-03-16T09:09:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c919c3700 1 -- 172.21.15.7:0/2401281884 learned_addr learned my addr 172.21.15.7:0/2401281884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c911c2700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c1037a0 0x7f4c8c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c919c3700 1 -- 172.21.15.7:0/2401281884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c1037a0 msgr2=0x7f4c8c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c919c3700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c1037a0 0x7f4c8c070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.146+0000 7f4c919c3700 1 -- 172.21.15.7:0/2401281884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c88005040 con 0x7f4c8c102e00 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.147+0000 7f4c919c3700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 0x7f4c8c0702c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800b3b0 tx=0x7f4c88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.147+0000 7f4c827fc700 1 -- 172.21.15.7:0/2401281884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8800e040 con 0x7f4c8c102e00 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.147+0000 7f4c827fc700 1 -- 172.21.15.7:0/2401281884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c8800bbc0 con 0x7f4c8c102e00 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.147+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c070fc0 con 0x7f4c8c102e00 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.147+0000 7f4c827fc700 1 -- 172.21.15.7:0/2401281884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8801c3f0 con 0x7f4c8c102e00 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.147+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c11a130 con 0x7f4c8c102e00 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.148+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c100110 con 0x7f4c8c102e00 2024-03-16T09:09:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.148+0000 7f4c827fc700 1 -- 172.21.15.7:0/2401281884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4c8800bd30 con 0x7f4c8c102e00 2024-03-16T09:09:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.149+0000 7f4c827fc700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4c7806dee0 0x7f4c78070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.149+0000 7f4c827fc700 1 -- 172.21.15.7:0/2401281884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4c8808e940 con 0x7f4c8c102e00 2024-03-16T09:09:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.149+0000 7f4c911c2700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4c7806dee0 0x7f4c78070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.149+0000 7f4c911c2700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4c7806dee0 0x7f4c78070390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0719a0 tx=0x7f4c7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.154+0000 7f4c827fc700 1 -- 172.21.15.7:0/2401281884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c88057e50 con 0x7f4c8c102e00 2024-03-16T09:09:04.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.380+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c8c0717a0 con 0x7f4c7806dee0 2024-03-16T09:09:04.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.381+0000 7f4c827fc700 1 -- 172.21.15.7:0/2401281884 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4c8c0717a0 con 0x7f4c7806dee0 2024-03-16T09:09:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.383+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4c7806dee0 msgr2=0x7f4c78070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.383+0000 7f4c93c27700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4c7806dee0 0x7f4c78070390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0719a0 tx=0x7f4c7c00b040 comp rx=0 tx=0).stop 2024-03-16T09:09:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.383+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 msgr2=0x7f4c8c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.383+0000 7f4c93c27700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 0x7f4c8c0702c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800b3b0 tx=0x7f4c88009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.384+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 shutdown_connections 2024-03-16T09:09:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.384+0000 7f4c93c27700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4c7806dee0 0x7f4c78070390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.384+0000 7f4c93c27700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8c102e00 0x7f4c8c0702c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.384+0000 7f4c93c27700 1 --2- 172.21.15.7:0/2401281884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c1037a0 0x7f4c8c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.384+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 >> 172.21.15.7:0/2401281884 conn(0x7f4c8c0f89b0 msgr2=0x7f4c8c1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:04.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.384+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 shutdown_connections 2024-03-16T09:09:04.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.384+0000 7f4c93c27700 1 -- 172.21.15.7:0/2401281884 wait complete. 2024-03-16T09:09:04.396 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:09:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.487+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/781489448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd0102fd0 msgr2=0x7f9dd01033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.487+0000 7f9dd5fa1700 1 --2- 172.21.15.7:0/781489448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd0102fd0 0x7f9dd01033a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0004940 tx=0x7f9dc0009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.487+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/781489448 shutdown_connections 2024-03-16T09:09:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.487+0000 7f9dd5fa1700 1 --2- 172.21.15.7:0/781489448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd00fb540 0x7f9dd00fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.487+0000 7f9dd5fa1700 1 --2- 172.21.15.7:0/781489448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd0102fd0 0x7f9dd01033a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.487+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/781489448 >> 172.21.15.7:0/781489448 conn(0x7f9dd0071530 msgr2=0x7f9dd0071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.487+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/781489448 shutdown_connections 2024-03-16T09:09:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.487+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/781489448 wait complete. 2024-03-16T09:09:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dd5fa1700 1 Processor -- start 2024-03-16T09:09:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dd5fa1700 1 -- start start 2024-03-16T09:09:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dd5fa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd00fb540 0x7f9dd00fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dd5fa1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd0102fd0 0x7f9dd00fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dd5fa1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd00fe670 con 0x7f9dd00fb540 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dd5fa1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd00fd1a0 con 0x7f9dd0102fd0 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd00fb540 0x7f9dd00fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dcf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd00fb540 0x7f9dd00fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43304/0 (socket says 172.21.15.7:43304) 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.488+0000 7f9dcf7fe700 1 -- 172.21.15.7:0/1319471374 learned_addr learned my addr 172.21.15.7:0/1319471374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9dceffd700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd0102fd0 0x7f9dd00fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9dcf7fe700 1 -- 172.21.15.7:0/1319471374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd0102fd0 msgr2=0x7f9dd00fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9dcf7fe700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd0102fd0 0x7f9dd00fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9dcf7fe700 1 -- 172.21.15.7:0/1319471374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc0005040 con 0x7f9dd00fb540 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9dcf7fe700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd00fb540 0x7f9dd00fc720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b3b0 tx=0x7f9dc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9db7fff700 1 -- 172.21.15.7:0/1319471374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc000e040 con 0x7f9dd00fb540 2024-03-16T09:09:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9db7fff700 1 -- 172.21.15.7:0/1319471374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dc000bbc0 con 0x7f9dd00fb540 2024-03-16T09:09:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd00fd420 con 0x7f9dd00fb540 2024-03-16T09:09:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9db7fff700 1 -- 172.21.15.7:0/1319471374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc001c3f0 con 0x7f9dd00fb540 2024-03-16T09:09:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.489+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd011a070 con 0x7f9dd00fb540 2024-03-16T09:09:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.490+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd0100270 con 0x7f9dd00fb540 2024-03-16T09:09:04.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.491+0000 7f9db7fff700 1 -- 172.21.15.7:0/1319471374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f9dc000bd30 con 0x7f9dd00fb540 2024-03-16T09:09:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.491+0000 7f9db7fff700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9db806dee0 0x7f9db8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.491+0000 7f9db7fff700 1 -- 172.21.15.7:0/1319471374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9dc008e940 con 0x7f9dd00fb540 2024-03-16T09:09:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.491+0000 7f9dceffd700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9db806dee0 0x7f9db8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.492+0000 7f9dceffd700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9db806dee0 0x7f9db8070390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00fde50 tx=0x7f9dc4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:04.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.496+0000 7f9db7fff700 1 -- 172.21.15.7:0/1319471374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9dc0057e50 con 0x7f9dd00fb540 2024-03-16T09:09:04.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.721+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dd00fe900 con 0x7f9db806dee0 2024-03-16T09:09:04.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.723+0000 7f9db7fff700 1 -- 172.21.15.7:0/1319471374 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9dd00fe900 con 0x7f9db806dee0 2024-03-16T09:09:04.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9db806dee0 msgr2=0x7f9db8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9db806dee0 0x7f9db8070390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00fde50 tx=0x7f9dc4009ee0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd00fb540 msgr2=0x7f9dd00fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd00fb540 0x7f9dd00fc720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b3b0 tx=0x7f9dc0009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:04.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 shutdown_connections 2024-03-16T09:09:04.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9db806dee0 0x7f9db8070390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd00fb540 0x7f9dd00fc720 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 --2- 172.21.15.7:0/1319471374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd0102fd0 0x7f9dd00fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 >> 172.21.15.7:0/1319471374 conn(0x7f9dd0071530 msgr2=0x7f9dd00fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:04.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 shutdown_connections 2024-03-16T09:09:04.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.725+0000 7f9dd5fa1700 1 -- 172.21.15.7:0/1319471374 wait complete. 2024-03-16T09:09:04.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:04 smithi007 ceph-mon[113546]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 -- 172.21.15.7:0/3830007434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 msgr2=0x7fd99c0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 --2- 172.21.15.7:0/3830007434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 0x7fd99c0fe1e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd994009ce0 tx=0x7fd994004d10 comp rx=0 tx=0).stop 2024-03-16T09:09:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 -- 172.21.15.7:0/3830007434 shutdown_connections 2024-03-16T09:09:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 --2- 172.21.15.7:0/3830007434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd99c0fe7b0 0x7fd99c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 --2- 172.21.15.7:0/3830007434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 0x7fd99c0fe1e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 -- 172.21.15.7:0/3830007434 >> 172.21.15.7:0/3830007434 conn(0x7fd99c0f7870 msgr2=0x7fd99c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 -- 172.21.15.7:0/3830007434 shutdown_connections 2024-03-16T09:09:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 -- 172.21.15.7:0/3830007434 wait complete. 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 Processor -- start 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.828+0000 7fd9a3217700 1 -- start start 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a3217700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 0x7fd99c0fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a3217700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd99c0fe7b0 0x7fd99c0fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a3217700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0fdc20 con 0x7fd99c0fde10 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a3217700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0fc750 con 0x7fd99c0fe7b0 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a0fb3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 0x7fd99c0fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a0fb3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 0x7fd99c0fbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43312/0 (socket says 172.21.15.7:43312) 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a0fb3700 1 -- 172.21.15.7:0/2334867286 learned_addr learned my addr 172.21.15.7:0/2334867286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd99bfff700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd99c0fe7b0 0x7fd99c0fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a0fb3700 1 -- 172.21.15.7:0/2334867286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd99c0fe7b0 msgr2=0x7fd99c0fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a0fb3700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd99c0fe7b0 0x7fd99c0fc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a0fb3700 1 -- 172.21.15.7:0/2334867286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd994009940 con 0x7fd99c0fde10 2024-03-16T09:09:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.829+0000 7fd9a0fb3700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 0x7fd99c0fbcd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd994007c00 tx=0x7fd994004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.830+0000 7fd9997fa700 1 -- 172.21.15.7:0/2334867286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd99400e070 con 0x7fd99c0fde10 2024-03-16T09:09:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.830+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c0fc9d0 con 0x7fd99c0fde10 2024-03-16T09:09:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.830+0000 7fd9997fa700 1 -- 172.21.15.7:0/2334867286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9940128e0 con 0x7fd99c0fde10 2024-03-16T09:09:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.830+0000 7fd9997fa700 1 -- 172.21.15.7:0/2334867286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd99401b890 con 0x7fd99c0fde10 2024-03-16T09:09:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.830+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c11a0a0 con 0x7fd99c0fde10 2024-03-16T09:09:04.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.831+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99c066af0 con 0x7fd99c0fde10 2024-03-16T09:09:04.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.831+0000 7fd9997fa700 1 -- 172.21.15.7:0/2334867286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fd994019040 con 0x7fd99c0fde10 2024-03-16T09:09:04.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.832+0000 7fd9997fa700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd98406dee0 0x7fd984070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:04.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.832+0000 7fd9997fa700 1 -- 172.21.15.7:0/2334867286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd99408ebd0 con 0x7fd99c0fde10 2024-03-16T09:09:04.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.832+0000 7fd99bfff700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd98406dee0 0x7fd984070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:04.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.832+0000 7fd99bfff700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd98406dee0 0x7fd984070390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0fd400 tx=0x7fd99000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:04.836+0000 7fd9997fa700 1 -- 172.21.15.7:0/2334867286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd9940580e0 con 0x7fd99c0fde10 2024-03-16T09:09:04.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:04 smithi019 ceph-mon[83366]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:05.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.052+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd99c0f9490 con 0x7fd98406dee0 2024-03-16T09:09:05.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.062+0000 7fd9997fa700 1 -- 172.21.15.7:0/2334867286 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd99c0f9490 con 0x7fd98406dee0 2024-03-16T09:09:05.064 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:09:05.064 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (5m) 52s ago 15m 28.5M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:09:05.064 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (2m) 52s ago 16m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:09:05.064 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (2m) 18s ago 15m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:09:05.064 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (4m) 52s ago 15m 84.6M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:09:05.064 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (10m) 52s ago 10m 14.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (10m) 52s ago 10m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (10m) 18s ago 10m 120M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (10m) 18s ago 10m 14.7M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (6m) 52s ago 17m 503M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (6m) 18s ago 15m 450M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (3m) 52s ago 17m 51.1M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (3m) 18s ago 14m 40.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (5m) 52s ago 16m 14.7M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (5m) 18s ago 15m 15.2M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (2m) 52s ago 14m 92.2M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (94s) 52s ago 13m 91.9M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4aa10e800f72 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (58s) 52s ago 13m 12.2M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5449f945991b 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (22s) 18s ago 12m 12.7M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b bb64247744fe 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (12m) 18s ago 12m 246M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce 9ac04f07b060 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (12m) 18s ago 12m 225M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:09:05.065 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (5m) 52s ago 15m 63.4M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:09:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd98406dee0 msgr2=0x7fd984070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd98406dee0 0x7fd984070390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0fd400 tx=0x7fd99000b040 comp rx=0 tx=0).stop 2024-03-16T09:09:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 msgr2=0x7fd99c0fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 0x7fd99c0fbcd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd994007c00 tx=0x7fd994004d10 comp rx=0 tx=0).stop 2024-03-16T09:09:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 shutdown_connections 2024-03-16T09:09:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd98406dee0 0x7fd984070390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99c0fde10 0x7fd99c0fbcd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 --2- 172.21.15.7:0/2334867286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd99c0fe7b0 0x7fd99c0fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 >> 172.21.15.7:0/2334867286 conn(0x7fd99c0f7870 msgr2=0x7fd99c0f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 shutdown_connections 2024-03-16T09:09:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.064+0000 7fd9a3217700 1 -- 172.21.15.7:0/2334867286 wait complete. 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.165+0000 7f257b9c9700 1 -- 172.21.15.7:0/2646439242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fdde0 msgr2=0x7f25740fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.165+0000 7f257b9c9700 1 --2- 172.21.15.7:0/2646439242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fdde0 0x7f25740fe1b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2570004940 tx=0x7f2570009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 -- 172.21.15.7:0/2646439242 shutdown_connections 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 --2- 172.21.15.7:0/2646439242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25740fe780 0x7f2574106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 --2- 172.21.15.7:0/2646439242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fdde0 0x7f25740fe1b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 -- 172.21.15.7:0/2646439242 >> 172.21.15.7:0/2646439242 conn(0x7f25740f7890 msgr2=0x7f25740f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 -- 172.21.15.7:0/2646439242 shutdown_connections 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 -- 172.21.15.7:0/2646439242 wait complete. 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 Processor -- start 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 -- start start 2024-03-16T09:09:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25740fdde0 0x7f25741139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fe780 0x7f2574113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2574114530 con 0x7f25740fe780 2024-03-16T09:09:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.166+0000 7f257b9c9700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257410db50 con 0x7f25740fdde0 2024-03-16T09:09:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2578f64700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fe780 0x7f2574113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2578f64700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fe780 0x7f2574113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43334/0 (socket says 172.21.15.7:43334) 2024-03-16T09:09:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2578f64700 1 -- 172.21.15.7:0/3360675094 learned_addr learned my addr 172.21.15.7:0/3360675094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2579765700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25740fdde0 0x7f25741139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2578f64700 1 -- 172.21.15.7:0/3360675094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25740fdde0 msgr2=0x7f25741139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2578f64700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25740fdde0 0x7f25741139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2578f64700 1 -- 172.21.15.7:0/3360675094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2570005040 con 0x7f25740fe780 2024-03-16T09:09:05.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2578f64700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fe780 0x7f2574113ee0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2564002a50 tx=0x7f25640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:05.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2569ffb700 1 -- 172.21.15.7:0/3360675094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2564007cc0 con 0x7f25740fe780 2024-03-16T09:09:05.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2569ffb700 1 -- 172.21.15.7:0/3360675094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f256400f040 con 0x7f25740fe780 2024-03-16T09:09:05.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257410de30 con 0x7f25740fe780 2024-03-16T09:09:05.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.167+0000 7f2569ffb700 1 -- 172.21.15.7:0/3360675094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256400da30 con 0x7f25740fe780 2024-03-16T09:09:05.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.168+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257410e370 con 0x7f25740fe780 2024-03-16T09:09:05.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.169+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2574066bd0 con 0x7f25740fe780 2024-03-16T09:09:05.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.170+0000 7f2569ffb700 1 -- 172.21.15.7:0/3360675094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f256400db90 con 0x7f25740fe780 2024-03-16T09:09:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.170+0000 7f2569ffb700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f256006dee0 0x7f2560070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.170+0000 7f2569ffb700 1 -- 172.21.15.7:0/3360675094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f256408d420 con 0x7f25740fe780 2024-03-16T09:09:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.170+0000 7f2579765700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f256006dee0 0x7f2560070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.171+0000 7f2579765700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f256006dee0 0x7f2560070390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2570004f90 tx=0x7f2570009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:05.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.175+0000 7f2569ffb700 1 -- 172.21.15.7:0/3360675094 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2564056910 con 0x7f25740fe780 2024-03-16T09:09:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.469+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2574109e70 con 0x7f25740fe780 2024-03-16T09:09:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.470+0000 7f2569ffb700 1 -- 172.21.15.7:0/3360675094 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f2564059fd0 con 0x7f25740fe780 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 2, 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:09:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:09:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:09:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:09:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:09:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 6, 2024-03-16T09:09:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 8 2024-03-16T09:09:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:09:05.474 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f256006dee0 msgr2=0x7f2560070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f256006dee0 0x7f2560070390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2570004f90 tx=0x7f2570009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fe780 msgr2=0x7f2574113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fe780 0x7f2574113ee0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2564002a50 tx=0x7f25640046b0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 shutdown_connections 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25740fdde0 0x7f25741139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f256006dee0 0x7f2560070390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 --2- 172.21.15.7:0/3360675094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25740fe780 0x7f2574113ee0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 >> 172.21.15.7:0/3360675094 conn(0x7f25740f7890 msgr2=0x7f25740f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 shutdown_connections 2024-03-16T09:09:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.472+0000 7f257b9c9700 1 -- 172.21.15.7:0/3360675094 wait complete. 2024-03-16T09:09:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.575+0000 7fc4da407700 1 -- 172.21.15.7:0/4050584811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4101db0 msgr2=0x7fc4d4102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.575+0000 7fc4da407700 1 --2- 172.21.15.7:0/4050584811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4101db0 0x7fc4d4102180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4004940 tx=0x7fc4c4009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.575+0000 7fc4da407700 1 -- 172.21.15.7:0/4050584811 shutdown_connections 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.575+0000 7fc4da407700 1 --2- 172.21.15.7:0/4050584811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4d40fc4c0 0x7fc4d40fc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.575+0000 7fc4da407700 1 --2- 172.21.15.7:0/4050584811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4101db0 0x7fc4d4102180 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.575+0000 7fc4da407700 1 -- 172.21.15.7:0/4050584811 >> 172.21.15.7:0/4050584811 conn(0x7fc4d4072310 msgr2=0x7fc4d4072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.575+0000 7fc4da407700 1 -- 172.21.15.7:0/4050584811 shutdown_connections 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.575+0000 7fc4da407700 1 -- 172.21.15.7:0/4050584811 wait complete. 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4da407700 1 Processor -- start 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4da407700 1 -- start start 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4da407700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d40fc4c0 0x7fc4d4103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4da407700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4d4101db0 0x7fc4d41037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4da407700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d41051d0 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4da407700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4103d00 con 0x7fc4d4101db0 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d40fc4c0 0x7fc4d4103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d40fc4c0 0x7fc4d4103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43354/0 (socket says 172.21.15.7:43354) 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4d3fff700 1 -- 172.21.15.7:0/2280331035 learned_addr learned my addr 172.21.15.7:0/2280331035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.576+0000 7fc4d37fe700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4d4101db0 0x7fc4d41037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4d3fff700 1 -- 172.21.15.7:0/2280331035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4d4101db0 msgr2=0x7fc4d41037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4d3fff700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4d4101db0 0x7fc4d41037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4d3fff700 1 -- 172.21.15.7:0/2280331035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c4005040 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4d3fff700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d40fc4c0 0x7fc4d4103280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400b3b0 tx=0x7fc4c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:05.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4d0ff9700 1 -- 172.21.15.7:0/2280331035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4c400e040 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4d0ff9700 1 -- 172.21.15.7:0/2280331035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4c4013420 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d4103f80 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4d0ff9700 1 -- 172.21.15.7:0/2280331035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4c401c630 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.577+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d411a0a0 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.578+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d40fd4d0 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.579+0000 7fc4d0ff9700 1 -- 172.21.15.7:0/2280331035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc4c4013590 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.579+0000 7fc4d0ff9700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc4bc06dee0 0x7fc4bc070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.579+0000 7fc4d0ff9700 1 -- 172.21.15.7:0/2280331035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc4c408e7e0 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.579+0000 7fc4d37fe700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc4bc06dee0 0x7fc4bc070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.580+0000 7fc4d37fe700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc4bc06dee0 0x7fc4bc070390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4d41049b0 tx=0x7fc4c800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.584+0000 7fc4d0ff9700 1 -- 172.21.15.7:0/2280331035 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4c4057e10 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:05 smithi007 ceph-mon[113546]: pgmap v130: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:05 smithi007 ceph-mon[113546]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:05 smithi007 ceph-mon[113546]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:05 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3360675094' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.837+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc4d411a2e0 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.837+0000 7fc4d0ff9700 1 -- 172.21.15.7:0/2280331035 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1895 (secure 0 0 0) 0x7fc4c405b4d0 con 0x7fc4d40fc4c0 2024-03-16T09:09:05.839 INFO:teuthology.orchestra.run.smithi007.stdout:e20 2024-03-16T09:09:05.839 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:09:05.840 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:09:05.841 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:09:05.842 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:09:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc4bc06dee0 msgr2=0x7fc4bc070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc4bc06dee0 0x7fc4bc070390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4d41049b0 tx=0x7fc4c800b040 comp rx=0 tx=0).stop 2024-03-16T09:09:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d40fc4c0 msgr2=0x7fc4d4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d40fc4c0 0x7fc4d4103280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400b3b0 tx=0x7fc4c4009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 shutdown_connections 2024-03-16T09:09:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc4bc06dee0 0x7fc4bc070390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d40fc4c0 0x7fc4d4103280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 --2- 172.21.15.7:0/2280331035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4d4101db0 0x7fc4d41037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 >> 172.21.15.7:0/2280331035 conn(0x7fc4d4072310 msgr2=0x7fc4d41062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 shutdown_connections 2024-03-16T09:09:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.839+0000 7fc4da407700 1 -- 172.21.15.7:0/2280331035 wait complete. 2024-03-16T09:09:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:09:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.942+0000 7fa325f16700 1 -- 172.21.15.7:0/3512272349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 msgr2=0x7fa320102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.942+0000 7fa325f16700 1 --2- 172.21.15.7:0/3512272349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 0x7fa320102100 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa310004940 tx=0x7fa310009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.942+0000 7fa325f16700 1 -- 172.21.15.7:0/3512272349 shutdown_connections 2024-03-16T09:09:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.942+0000 7fa325f16700 1 --2- 172.21.15.7:0/3512272349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3201026d0 0x7fa3200fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.942+0000 7fa325f16700 1 --2- 172.21.15.7:0/3512272349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 0x7fa320102100 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.942+0000 7fa325f16700 1 -- 172.21.15.7:0/3512272349 >> 172.21.15.7:0/3512272349 conn(0x7fa3200f7860 msgr2=0x7fa3200f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.943+0000 7fa325f16700 1 -- 172.21.15.7:0/3512272349 shutdown_connections 2024-03-16T09:09:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.943+0000 7fa325f16700 1 -- 172.21.15.7:0/3512272349 wait complete. 2024-03-16T09:09:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.943+0000 7fa325f16700 1 Processor -- start 2024-03-16T09:09:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.943+0000 7fa325f16700 1 -- start start 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.943+0000 7fa325f16700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 0x7fa32010eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.943+0000 7fa325f16700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3201026d0 0x7fa32010f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.943+0000 7fa325f16700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32010f620 con 0x7fa320101d30 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.943+0000 7fa325f16700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32010f760 con 0x7fa3201026d0 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 0x7fa32010eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 0x7fa32010eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43374/0 (socket says 172.21.15.7:43374) 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31f7fe700 1 -- 172.21.15.7:0/1221427188 learned_addr learned my addr 172.21.15.7:0/1221427188 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31effd700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3201026d0 0x7fa32010f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31f7fe700 1 -- 172.21.15.7:0/1221427188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3201026d0 msgr2=0x7fa32010f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31f7fe700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3201026d0 0x7fa32010f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31f7fe700 1 -- 172.21.15.7:0/1221427188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa310005040 con 0x7fa320101d30 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31effd700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3201026d0 0x7fa32010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa31f7fe700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 0x7fa32010eba0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa31000b3b0 tx=0x7fa310009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa307fff700 1 -- 172.21.15.7:0/1221427188 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31000e040 con 0x7fa320101d30 2024-03-16T09:09:05.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32010f9e0 con 0x7fa320101d30 2024-03-16T09:09:05.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa320117fb0 con 0x7fa320101d30 2024-03-16T09:09:05.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa307fff700 1 -- 172.21.15.7:0/1221427188 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa31000bbc0 con 0x7fa320101d30 2024-03-16T09:09:05.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.944+0000 7fa307fff700 1 -- 172.21.15.7:0/1221427188 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31001c3f0 con 0x7fa320101d30 2024-03-16T09:09:05.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.945+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c0052b0 con 0x7fa320101d30 2024-03-16T09:09:05.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.946+0000 7fa307fff700 1 -- 172.21.15.7:0/1221427188 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa31000bd30 con 0x7fa320101d30 2024-03-16T09:09:05.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.946+0000 7fa307fff700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa30806dee0 0x7fa308070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:05.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.946+0000 7fa307fff700 1 -- 172.21.15.7:0/1221427188 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa31008e820 con 0x7fa320101d30 2024-03-16T09:09:05.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.946+0000 7fa31effd700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa30806dee0 0x7fa308070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:05.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.947+0000 7fa31effd700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa30806dee0 0x7fa308070390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa314006fd0 tx=0x7fa314008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:05.951+0000 7fa307fff700 1 -- 172.21.15.7:0/1221427188 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa310057d30 con 0x7fa320101d30 2024-03-16T09:09:05.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:05 smithi019 ceph-mon[83366]: pgmap v130: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:05 smithi019 ceph-mon[83366]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:05 smithi019 ceph-mon[83366]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:05.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:05 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3360675094' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.176+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa30c000bc0 con 0x7fa30806dee0 2024-03-16T09:09:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.177+0000 7fa307fff700 1 -- 172.21.15.7:0/1221427188 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa30c000bc0 con 0x7fa30806dee0 2024-03-16T09:09:06.179 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:09:06.179 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:09:06.179 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:09:06.179 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:09:06.179 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:09:06.180 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-16T09:09:06.180 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:09:06.180 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:09:06.180 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:09:06.180 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "10/21 daemons upgraded", 2024-03-16T09:09:06.180 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T09:09:06.180 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:09:06.180 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa30806dee0 msgr2=0x7fa308070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa30806dee0 0x7fa308070390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa314006fd0 tx=0x7fa314008040 comp rx=0 tx=0).stop 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 msgr2=0x7fa32010eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 0x7fa32010eba0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa31000b3b0 tx=0x7fa310009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 shutdown_connections 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa30806dee0 0x7fa308070390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa320101d30 0x7fa32010eba0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 --2- 172.21.15.7:0/1221427188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3201026d0 0x7fa32010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.179+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 >> 172.21.15.7:0/1221427188 conn(0x7fa3200f7860 msgr2=0x7fa3200f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.180+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 shutdown_connections 2024-03-16T09:09:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.180+0000 7fa325f16700 1 -- 172.21.15.7:0/1221427188 wait complete. 2024-03-16T09:09:06.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.282+0000 7f88d1026700 1 -- 172.21.15.7:0/2040084238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 msgr2=0x7f88cc0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:06.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.282+0000 7f88d1026700 1 --2- 172.21.15.7:0/2040084238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 0x7f88cc0fce10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88c0004940 tx=0x7f88c0009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:06.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.282+0000 7f88d1026700 1 -- 172.21.15.7:0/2040084238 shutdown_connections 2024-03-16T09:09:06.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.282+0000 7f88d1026700 1 --2- 172.21.15.7:0/2040084238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 0x7f88cc0fce10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.282+0000 7f88d1026700 1 --2- 172.21.15.7:0/2040084238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc102e00 0x7f88cc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.282+0000 7f88d1026700 1 -- 172.21.15.7:0/2040084238 >> 172.21.15.7:0/2040084238 conn(0x7f88cc0f89b0 msgr2=0x7f88cc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:06.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.282+0000 7f88d1026700 1 -- 172.21.15.7:0/2040084238 shutdown_connections 2024-03-16T09:09:06.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88d1026700 1 -- 172.21.15.7:0/2040084238 wait complete. 2024-03-16T09:09:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88d1026700 1 Processor -- start 2024-03-16T09:09:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88d1026700 1 -- start start 2024-03-16T09:09:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88d1026700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc102e00 0x7f88cc113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88d1026700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 0x7f88cc113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88d1026700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc114510 con 0x7f88cc1037a0 2024-03-16T09:09:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88d1026700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc10db10 con 0x7f88cc102e00 2024-03-16T09:09:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88ca59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 0x7f88cc113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:06.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.283+0000 7f88ca59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 0x7f88cc113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43400/0 (socket says 172.21.15.7:43400) 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88ca59c700 1 -- 172.21.15.7:0/865319145 learned_addr learned my addr 172.21.15.7:0/865319145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88cad9d700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc102e00 0x7f88cc113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88ca59c700 1 -- 172.21.15.7:0/865319145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc102e00 msgr2=0x7f88cc113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88ca59c700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc102e00 0x7f88cc113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88ca59c700 1 -- 172.21.15.7:0/865319145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0005040 con 0x7f88cc1037a0 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88ca59c700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 0x7f88cc113ec0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009180 tx=0x7f88c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88b37fe700 1 -- 172.21.15.7:0/865319145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88c000e040 con 0x7f88cc1037a0 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc10dd90 con 0x7f88cc1037a0 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88b37fe700 1 -- 172.21.15.7:0/865319145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88c0013420 con 0x7f88cc1037a0 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88b37fe700 1 -- 172.21.15.7:0/865319145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88c001c610 con 0x7f88cc1037a0 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.284+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc10e350 con 0x7f88cc1037a0 2024-03-16T09:09:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.285+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc1052d0 con 0x7f88cc1037a0 2024-03-16T09:09:06.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.286+0000 7f88b37fe700 1 -- 172.21.15.7:0/865319145 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f88c0013a00 con 0x7f88cc1037a0 2024-03-16T09:09:06.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.286+0000 7f88b37fe700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f88b406dee0 0x7f88b4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:06.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.286+0000 7f88b37fe700 1 -- 172.21.15.7:0/865319145 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f88c008e820 con 0x7f88cc1037a0 2024-03-16T09:09:06.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.286+0000 7f88cad9d700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f88b406dee0 0x7f88b4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:06.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.287+0000 7f88cad9d700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f88b406dee0 0x7f88b4070390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88bc004f60 tx=0x7f88bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:06.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.291+0000 7f88b37fe700 1 -- 172.21.15.7:0/865319145 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f88c005b990 con 0x7f88cc1037a0 2024-03-16T09:09:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.589+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f88cc0fe550 con 0x7f88cc1037a0 2024-03-16T09:09:06.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.589+0000 7f88b37fe700 1 -- 172.21.15.7:0/865319145 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f88c005b520 con 0x7f88cc1037a0 2024-03-16T09:09:06.592 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_OK 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f88b406dee0 msgr2=0x7f88b4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f88b406dee0 0x7f88b4070390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88bc004f60 tx=0x7f88bc00b040 comp rx=0 tx=0).stop 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 msgr2=0x7f88cc113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 0x7f88cc113ec0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009180 tx=0x7f88c0009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 shutdown_connections 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc102e00 0x7f88cc113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f88b406dee0 0x7f88b4070390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 --2- 172.21.15.7:0/865319145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc1037a0 0x7f88cc113ec0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 >> 172.21.15.7:0/865319145 conn(0x7f88cc0f89b0 msgr2=0x7f88cc102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 shutdown_connections 2024-03-16T09:09:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:06.591+0000 7f88d1026700 1 -- 172.21.15.7:0/865319145 wait complete. 2024-03-16T09:09:06.836 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:06.836 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:06 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2280331035' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:09:06.836 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:06 smithi019 ceph-mon[83366]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:06.836 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T09:09:06.836 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:06 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T09:09:06.836 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:06 smithi019 ceph-mon[83366]: Upgrade: osd.4 is safe to restart 2024-03-16T09:09:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:06 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2280331035' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:09:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:06 smithi007 ceph-mon[113546]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T09:09:06.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:06 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T09:09:06.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:06 smithi007 ceph-mon[113546]: Upgrade: osd.4 is safe to restart 2024-03-16T09:09:07.867 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:07 smithi019 ceph-mon[83366]: pgmap v131: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:07.867 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:07 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/865319145' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:09:07.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:07 smithi007 ceph-mon[113546]: pgmap v131: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:07.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:07 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/865319145' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:09:09.067 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:08 smithi019 ceph-mon[83366]: Upgrade: Updating osd.4 2024-03-16T09:09:09.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:08 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:09.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:08 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T09:09:09.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:08 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:09.068 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:08 smithi019 ceph-mon[83366]: Deploying daemon osd.4 on smithi019 2024-03-16T09:09:09.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:08 smithi007 ceph-mon[113546]: Upgrade: Updating osd.4 2024-03-16T09:09:09.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:08 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:09.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:08 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T09:09:09.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:08 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:09.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:08 smithi007 ceph-mon[113546]: Deploying daemon osd.4 on smithi019 2024-03-16T09:09:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:09 smithi019 ceph-mon[83366]: pgmap v132: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:10.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:09 smithi007 ceph-mon[113546]: pgmap v132: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:11.328 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:11 smithi019 systemd[1]: Stopping Ceph osd.4 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:09:11.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:11 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[48824]: 2024-03-16T09:09:11.337+0000 7f75df213700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:09:11.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:11 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[48824]: 2024-03-16T09:09:11.337+0000 7f75df213700 -1 osd.4 69 *** Got signal Terminated *** 2024-03-16T09:09:11.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:11 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[48824]: 2024-03-16T09:09:11.337+0000 7f75df213700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:09:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:11 smithi007 ceph-mon[113546]: pgmap v133: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:11.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:11 smithi007 ceph-mon[113546]: osd.4 marked itself down and dead 2024-03-16T09:09:12.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:11 smithi019 ceph-mon[83366]: pgmap v133: 65 pgs: 65 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:12.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:11 smithi019 ceph-mon[83366]: osd.4 marked itself down and dead 2024-03-16T09:09:12.615 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:12 smithi019 podman[97472]: 2024-03-16 09:09:12.32872131 +0000 UTC m=+1.113465920 container died 9ac04f07b0604e78aa766fa58f7ceffb2bfb0be549e854239283835841582451 (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=quincy-1df2724, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-16T09:09:12.890 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:12 smithi019 ceph-mon[83366]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:09:12.890 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:12 smithi019 ceph-mon[83366]: osdmap e70: 6 total, 5 up, 6 in 2024-03-16T09:09:12.890 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:12 smithi019 podman[97472]: 2024-03-16 09:09:12.69579056 +0000 UTC m=+1.480535170 container cleanup 9ac04f07b0604e78aa766fa58f7ceffb2bfb0be549e854239283835841582451 (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=quincy-1df2724, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-16T09:09:12.891 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:12 smithi019 bash[97472]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4 2024-03-16T09:09:13.212 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:12 smithi019 podman[97490]: 2024-03-16 09:09:12.888735676 +0000 UTC m=+0.560167565 container remove 9ac04f07b0604e78aa766fa58f7ceffb2bfb0be549e854239283835841582451 (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=quincy-1df2724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-16T09:09:13.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:12 smithi007 ceph-mon[113546]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:09:13.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:12 smithi007 ceph-mon[113546]: osdmap e70: 6 total, 5 up, 6 in 2024-03-16T09:09:13.552 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:13 smithi019 podman[97584]: 2024-03-16 09:09:13.193248771 +0000 UTC m=+0.018353983 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:09:13.552 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:13 smithi019 podman[97584]: 2024-03-16 09:09:13.322930266 +0000 UTC m=+0.148035475 container create a3dec085bb42942b7b96b5c5573c9250418fe4c9c81b26e7f12d17e8c594d4b8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:09:13.552 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:13 smithi019 podman[97584]: 2024-03-16 09:09:13.550776802 +0000 UTC m=+0.375882026 container init a3dec085bb42942b7b96b5c5573c9250418fe4c9c81b26e7f12d17e8c594d4b8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-16T09:09:13.824 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:13 smithi019 podman[97584]: 2024-03-16 09:09:13.554385542 +0000 UTC m=+0.379490752 container start a3dec085bb42942b7b96b5c5573c9250418fe4c9c81b26e7f12d17e8c594d4b8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-16T09:09:13.824 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:13 smithi019 podman[97584]: 2024-03-16 09:09:13.642138163 +0000 UTC m=+0.467243375 container attach a3dec085bb42942b7b96b5c5573c9250418fe4c9c81b26e7f12d17e8c594d4b8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-03-16T09:09:14.147 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:13 smithi019 ceph-mon[83366]: pgmap v135: 65 pgs: 6 stale+active+clean, 59 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:14.147 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:13 smithi019 ceph-mon[83366]: osdmap e71: 6 total, 5 up, 6 in 2024-03-16T09:09:14.147 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:13 smithi019 podman[97584]: 2024-03-16 09:09:13.851936477 +0000 UTC m=+0.677041687 container died a3dec085bb42942b7b96b5c5573c9250418fe4c9c81b26e7f12d17e8c594d4b8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-16T09:09:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:13 smithi007 ceph-mon[113546]: pgmap v135: 65 pgs: 6 stale+active+clean, 59 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:13 smithi007 ceph-mon[113546]: osdmap e71: 6 total, 5 up, 6 in 2024-03-16T09:09:14.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:14 smithi019 podman[97610]: 2024-03-16 09:09:14.437668469 +0000 UTC m=+0.577203695 container remove a3dec085bb42942b7b96b5c5573c9250418fe4c9c81b26e7f12d17e8c594d4b8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T09:09:14.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:14 smithi019 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-16T09:09:14.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:14 smithi019 systemd[1]: Stopped Ceph osd.4 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:09:14.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:14 smithi019 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.4.service: Consumed 59.623s CPU time. 2024-03-16T09:09:14.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:14 smithi019 systemd[1]: Starting Ceph osd.4 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:09:15.143 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:14 smithi019 ceph-mon[83366]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-16T09:09:15.143 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:14 smithi019 ceph-mon[83366]: Health check failed: Degraded data redundancy: 40/354 objects degraded (11.299%), 10 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:15.143 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:14 smithi019 podman[97711]: 2024-03-16 09:09:14.785856751 +0000 UTC m=+0.017802517 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:09:15.143 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:14 smithi019 podman[97711]: 2024-03-16 09:09:14.88892817 +0000 UTC m=+0.120873928 container create bf16ca46d28f7cf0bf9cbb69b90132d7e7018a528395a320105f2d733927598b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-16T09:09:15.143 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 podman[97711]: 2024-03-16 09:09:15.141455531 +0000 UTC m=+0.373401291 container init bf16ca46d28f7cf0bf9cbb69b90132d7e7018a528395a320105f2d733927598b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:09:15.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:14 smithi007 ceph-mon[113546]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-16T09:09:15.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:14 smithi007 ceph-mon[113546]: Health check failed: Degraded data redundancy: 40/354 objects degraded (11.299%), 10 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:15.462 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 podman[97711]: 2024-03-16 09:09:15.144989741 +0000 UTC m=+0.376935509 container start bf16ca46d28f7cf0bf9cbb69b90132d7e7018a528395a320105f2d733927598b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-16T09:09:15.463 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 podman[97711]: 2024-03-16 09:09:15.233082353 +0000 UTC m=+0.465028113 container attach bf16ca46d28f7cf0bf9cbb69b90132d7e7018a528395a320105f2d733927598b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-16T09:09:15.877 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate[97725]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T09:09:15.877 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate[97725]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate[97725]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 bash[97711]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 bash[97711]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 bash[97711]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 bash[97711]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 bash[97711]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 bash[97711]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 bash[97711]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate[97725]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate[97725]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate[97725]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate[97725]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-16T09:09:15.878 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:15 smithi019 podman[97711]: 2024-03-16 09:09:15.62014557 +0000 UTC m=+0.852091320 container died bf16ca46d28f7cf0bf9cbb69b90132d7e7018a528395a320105f2d733927598b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-16T09:09:16.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-mon[83366]: pgmap v137: 65 pgs: 10 active+undersized+degraded, 2 peering, 3 stale+active+clean, 9 active+undersized, 41 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 40/354 objects degraded (11.299%) 2024-03-16T09:09:16.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:16.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:09:16.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:15 smithi007 ceph-mon[113546]: pgmap v137: 65 pgs: 10 active+undersized+degraded, 2 peering, 3 stale+active+clean, 9 active+undersized, 41 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 40/354 objects degraded (11.299%) 2024-03-16T09:09:16.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:16.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:09:16.528 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:16 smithi019 podman[97860]: 2024-03-16 09:09:16.220386971 +0000 UTC m=+0.595380404 container remove bf16ca46d28f7cf0bf9cbb69b90132d7e7018a528395a320105f2d733927598b (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-16T09:09:16.528 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:16 smithi019 podman[97923]: 2024-03-16 09:09:16.438019532 +0000 UTC m=+0.112410446 container create e1df7d4b9eaaae0db8ec1665a91843125983dc607eafdb25342213753c6794a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True) 2024-03-16T09:09:16.528 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:16 smithi019 podman[97923]: 2024-03-16 09:09:16.344787356 +0000 UTC m=+0.019178289 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:09:16.962 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:16 smithi019 podman[97923]: 2024-03-16 09:09:16.632088106 +0000 UTC m=+0.306479030 container init e1df7d4b9eaaae0db8ec1665a91843125983dc607eafdb25342213753c6794a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:09:16.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:16 smithi019 podman[97923]: 2024-03-16 09:09:16.635079432 +0000 UTC m=+0.309470349 container start e1df7d4b9eaaae0db8ec1665a91843125983dc607eafdb25342213753c6794a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-16T09:09:16.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:16 smithi019 ceph-osd[97942]: -- 172.21.15.19:0/2185219748 <== mon.1 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe1ef7b680 con 0x55fe1ef45000 2024-03-16T09:09:16.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:16 smithi019 bash[97923]: e1df7d4b9eaaae0db8ec1665a91843125983dc607eafdb25342213753c6794a8 2024-03-16T09:09:16.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:16 smithi019 systemd[1]: Started Ceph osd.4 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:09:17.463 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:17 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:09:17.175+0000 7f76f339b7c0 -1 Falling back to public interface 2024-03-16T09:09:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:17 smithi019 ceph-mon[83366]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 2 peering, 16 active+undersized, 31 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:17 smithi007 ceph-mon[113546]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 2 peering, 16 active+undersized, 31 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:19.100 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:18 smithi019 ceph-mon[83366]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T09:09:19.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:18 smithi007 ceph-mon[113546]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T09:09:20.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:19 smithi019 ceph-mon[83366]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:19 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:19 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:19 smithi007 ceph-mon[113546]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:19 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:19 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:21.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:21 smithi019 ceph-mon[83366]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:21.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:21 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:21.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:21 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:21.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:21 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:09:21.753+0000 7f76f339b7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-16T09:09:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:21 smithi007 ceph-mon[113546]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:21 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:21 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:22.544 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:22 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:09:22.285+0000 7f76f339b7c0 -1 osd.4 69 log_to_monitors true 2024-03-16T09:09:23.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:22 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 59/354 objects degraded (16.667%), 16 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:23.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:22 smithi019 ceph-mon[83366]: from='osd.4 [v2:172.21.15.19:6808/377988924,v1:172.21.15.19:6809/377988924]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:23.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:22 smithi019 ceph-mon[83366]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:23.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:22 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 59/354 objects degraded (16.667%), 16 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:23.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:22 smithi007 ceph-mon[113546]: from='osd.4 [v2:172.21.15.19:6808/377988924,v1:172.21.15.19:6809/377988924]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:23.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:22 smithi007 ceph-mon[113546]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:24.076 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:09:23 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:09:23.752+0000 7f76e07eb700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:09:24.076 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:23 smithi019 ceph-mon[83366]: pgmap v141: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:24.077 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:23 smithi019 ceph-mon[83366]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T09:09:24.077 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:23 smithi019 ceph-mon[83366]: osdmap e72: 6 total, 5 up, 6 in 2024-03-16T09:09:24.077 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:23 smithi019 ceph-mon[83366]: from='osd.4 [v2:172.21.15.19:6808/377988924,v1:172.21.15.19:6809/377988924]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:09:24.077 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:23 smithi019 ceph-mon[83366]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:09:24.077 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:23 smithi019 ceph-mon[83366]: from='osd.4 ' entity='osd.4' 2024-03-16T09:09:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:23 smithi007 ceph-mon[113546]: pgmap v141: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:23 smithi007 ceph-mon[113546]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T09:09:24.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:23 smithi007 ceph-mon[113546]: osdmap e72: 6 total, 5 up, 6 in 2024-03-16T09:09:24.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:23 smithi007 ceph-mon[113546]: from='osd.4 [v2:172.21.15.19:6808/377988924,v1:172.21.15.19:6809/377988924]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:09:24.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:23 smithi007 ceph-mon[113546]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:09:24.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:23 smithi007 ceph-mon[113546]: from='osd.4 ' entity='osd.4' 2024-03-16T09:09:25.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:25.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:25.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:25.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:25.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:25.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: pgmap v143: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: osd.4 [v2:172.21.15.19:6808/377988924,v1:172.21.15.19:6809/377988924] boot 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: osdmap e73: 6 total, 6 up, 6 in 2024-03-16T09:09:25.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:25.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:25.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: pgmap v143: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 59/354 objects degraded (16.667%) 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: osd.4 [v2:172.21.15.19:6808/377988924,v1:172.21.15.19:6809/377988924] boot 2024-03-16T09:09:25.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: osdmap e73: 6 total, 6 up, 6 in 2024-03-16T09:09:25.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:27.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:26 smithi019 ceph-mon[83366]: osdmap e74: 6 total, 6 up, 6 in 2024-03-16T09:09:27.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:26 smithi007 ceph-mon[113546]: osdmap e74: 6 total, 6 up, 6 in 2024-03-16T09:09:28.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:27 smithi019 ceph-mon[83366]: pgmap v146: 65 pgs: 6 peering, 12 active+undersized+degraded, 12 active+undersized, 35 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44/354 objects degraded (12.429%) 2024-03-16T09:09:28.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:27 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 44/354 objects degraded (12.429%), 12 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:28.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:27 smithi007 ceph-mon[113546]: pgmap v146: 65 pgs: 6 peering, 12 active+undersized+degraded, 12 active+undersized, 35 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44/354 objects degraded (12.429%) 2024-03-16T09:09:28.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:27 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 44/354 objects degraded (12.429%), 12 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:30.050 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:29 smithi007 ceph-mon[113546]: pgmap v147: 65 pgs: 6 peering, 5 active+undersized+degraded, 5 active+undersized, 49 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25/354 objects degraded (7.062%) 2024-03-16T09:09:30.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:29 smithi019 ceph-mon[83366]: pgmap v147: 65 pgs: 6 peering, 5 active+undersized+degraded, 5 active+undersized, 49 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25/354 objects degraded (7.062%) 2024-03-16T09:09:31.705 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:31 smithi007 ceph-mon[113546]: pgmap v148: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:09:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:31 smithi007 ceph-mon[113546]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/354 objects degraded (7.062%), 5 pgs degraded) 2024-03-16T09:09:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:31 smithi007 ceph-mon[113546]: Cluster is now healthy 2024-03-16T09:09:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:31 smithi019 ceph-mon[83366]: pgmap v148: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:31 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:31 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:09:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:31 smithi019 ceph-mon[83366]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/354 objects degraded (7.062%), 5 pgs degraded) 2024-03-16T09:09:31.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:31 smithi019 ceph-mon[83366]: Cluster is now healthy 2024-03-16T09:09:33.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:33 smithi019 ceph-mon[83366]: pgmap v149: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:33.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:33 smithi007 ceph-mon[113546]: pgmap v149: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:35.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:35 smithi019 ceph-mon[83366]: pgmap v150: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:35.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:35 smithi007 ceph-mon[113546]: pgmap v150: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 -- 172.21.15.7:0/3535005277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60101d70 msgr2=0x7f3c60102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 --2- 172.21.15.7:0/3535005277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60101d70 0x7f3c60102140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c004940 tx=0x7f3c5c009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 -- 172.21.15.7:0/3535005277 shutdown_connections 2024-03-16T09:09:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 --2- 172.21.15.7:0/3535005277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c60102710 0x7f3c600fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 --2- 172.21.15.7:0/3535005277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60101d70 0x7f3c60102140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 -- 172.21.15.7:0/3535005277 >> 172.21.15.7:0/3535005277 conn(0x7f3c60072bc0 msgr2=0x7f3c60072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 -- 172.21.15.7:0/3535005277 shutdown_connections 2024-03-16T09:09:36.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 -- 172.21.15.7:0/3535005277 wait complete. 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 Processor -- start 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 -- start start 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c60101d70 0x7f3c6010c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60102710 0x7f3c6010cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6010d330 con 0x7f3c60102710 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.709+0000 7f3c6853d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6010d4a0 con 0x7f3c60101d70 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c65ad8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60102710 0x7f3c6010cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c65ad8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60102710 0x7f3c6010cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44494/0 (socket says 172.21.15.7:44494) 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c65ad8700 1 -- 172.21.15.7:0/1861823175 learned_addr learned my addr 172.21.15.7:0/1861823175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c662d9700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c60101d70 0x7f3c6010c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c65ad8700 1 -- 172.21.15.7:0/1861823175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c60101d70 msgr2=0x7f3c6010c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c65ad8700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c60101d70 0x7f3c6010c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c65ad8700 1 -- 172.21.15.7:0/1861823175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c005040 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c65ad8700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60102710 0x7f3c6010cdf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c54002a50 tx=0x7f3c540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c52ffd700 1 -- 172.21.15.7:0/1861823175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c54007cc0 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c60119f20 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c52ffd700 1 -- 172.21.15.7:0/1861823175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c5400f040 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c52ffd700 1 -- 172.21.15.7:0/1861823175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c54009630 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.710+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6011a2f0 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.711+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c600ff080 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.712+0000 7f3c52ffd700 1 -- 172.21.15.7:0/1861823175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3c54009790 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.712+0000 7f3c52ffd700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f3c4c06dee0 0x7f3c4c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.712+0000 7f3c52ffd700 1 -- 172.21.15.7:0/1861823175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f3c5408d310 con 0x7f3c60102710 2024-03-16T09:09:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.713+0000 7f3c662d9700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f3c4c06dee0 0x7f3c4c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.713+0000 7f3c662d9700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f3c4c06dee0 0x7f3c4c070390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c004f90 tx=0x7f3c5c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.717+0000 7f3c52ffd700 1 -- 172.21.15.7:0/1861823175 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c5405a400 con 0x7f3c60102710 2024-03-16T09:09:36.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.946+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c60108e70 con 0x7f3c4c06dee0 2024-03-16T09:09:36.949 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:36 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.947+0000 7f3c52ffd700 1 -- 172.21.15.7:0/1861823175 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f3c60108e70 con 0x7f3c4c06dee0 2024-03-16T09:09:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f3c4c06dee0 msgr2=0x7f3c4c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f3c4c06dee0 0x7f3c4c070390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c004f90 tx=0x7f3c5c009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60102710 msgr2=0x7f3c6010cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60102710 0x7f3c6010cdf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3c54002a50 tx=0x7f3c540046b0 comp rx=0 tx=0).stop 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 shutdown_connections 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c60101d70 0x7f3c6010c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f3c4c06dee0 0x7f3c4c070390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 --2- 172.21.15.7:0/1861823175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c60102710 0x7f3c6010cdf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 >> 172.21.15.7:0/1861823175 conn(0x7f3c60072bc0 msgr2=0x7f3c600faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 shutdown_connections 2024-03-16T09:09:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:36.949+0000 7f3c6853d700 1 -- 172.21.15.7:0/1861823175 wait complete. 2024-03-16T09:09:36.962 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:09:36.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:36 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:37.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.053+0000 7f3898719700 1 -- 172.21.15.7:0/2322365112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 msgr2=0x7f38901031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.053+0000 7f3898719700 1 --2- 172.21.15.7:0/2322365112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 0x7f38901031d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f388c004940 tx=0x7f388c009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:37.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.053+0000 7f3898719700 1 -- 172.21.15.7:0/2322365112 shutdown_connections 2024-03-16T09:09:37.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.053+0000 7f3898719700 1 --2- 172.21.15.7:0/2322365112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38901037a0 0x7f38900fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.053+0000 7f3898719700 1 --2- 172.21.15.7:0/2322365112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 0x7f38901031d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.053+0000 7f3898719700 1 -- 172.21.15.7:0/2322365112 >> 172.21.15.7:0/2322365112 conn(0x7f38900f89b0 msgr2=0x7f38900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.053+0000 7f3898719700 1 -- 172.21.15.7:0/2322365112 shutdown_connections 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.053+0000 7f3898719700 1 -- 172.21.15.7:0/2322365112 wait complete. 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f3898719700 1 Processor -- start 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f3898719700 1 -- start start 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f3898719700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 0x7f3890113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f3898719700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38901037a0 0x7f3890113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f3898719700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3890114520 con 0x7f3890102e00 2024-03-16T09:09:37.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f3898719700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389010db70 con 0x7f38901037a0 2024-03-16T09:09:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f38964b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 0x7f3890113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f38964b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 0x7f3890113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44510/0 (socket says 172.21.15.7:44510) 2024-03-16T09:09:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f38964b5700 1 -- 172.21.15.7:0/2472207240 learned_addr learned my addr 172.21.15.7:0/2472207240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:37.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.054+0000 7f3895cb4700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38901037a0 0x7f3890113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f38964b5700 1 -- 172.21.15.7:0/2472207240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38901037a0 msgr2=0x7f3890113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f38964b5700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38901037a0 0x7f3890113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f38964b5700 1 -- 172.21.15.7:0/2472207240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c005040 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f38964b5700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 0x7f3890113990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f388c009150 tx=0x7f388c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f3882ffd700 1 -- 172.21.15.7:0/2472207240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388c00e040 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f3882ffd700 1 -- 172.21.15.7:0/2472207240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f388c013420 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389010ddf0 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f3882ffd700 1 -- 172.21.15.7:0/2472207240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388c01c610 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.055+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389010e3b0 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.056+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3890100110 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.057+0000 7f3882ffd700 1 -- 172.21.15.7:0/2472207240 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f388c00baf0 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.057+0000 7f3882ffd700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f387c06dee0 0x7f387c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.057+0000 7f3882ffd700 1 -- 172.21.15.7:0/2472207240 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f388c08e7c0 con 0x7f3890102e00 2024-03-16T09:09:37.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.057+0000 7f3895cb4700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f387c06dee0 0x7f387c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.057+0000 7f3895cb4700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f387c06dee0 0x7f387c070390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f389010ece0 tx=0x7f38840043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:37.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.062+0000 7f3882ffd700 1 -- 172.21.15.7:0/2472207240 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f388c05b920 con 0x7f3890102e00 2024-03-16T09:09:37.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.288+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f389010f050 con 0x7f387c06dee0 2024-03-16T09:09:37.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.289+0000 7f3882ffd700 1 -- 172.21.15.7:0/2472207240 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f389010f050 con 0x7f387c06dee0 2024-03-16T09:09:37.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f387c06dee0 msgr2=0x7f387c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f387c06dee0 0x7f387c070390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f389010ece0 tx=0x7f38840043c0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 msgr2=0x7f3890113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 0x7f3890113990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f388c009150 tx=0x7f388c009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 shutdown_connections 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f387c06dee0 0x7f387c070390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3890102e00 0x7f3890113990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 --2- 172.21.15.7:0/2472207240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38901037a0 0x7f3890113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 >> 172.21.15.7:0/2472207240 conn(0x7f38900f89b0 msgr2=0x7f38901073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 shutdown_connections 2024-03-16T09:09:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.291+0000 7f3898719700 1 -- 172.21.15.7:0/2472207240 wait complete. 2024-03-16T09:09:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.393+0000 7f6855e6d700 1 -- 172.21.15.7:0/487771299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 msgr2=0x7f68500fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.393+0000 7f6855e6d700 1 --2- 172.21.15.7:0/487771299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 0x7f68500fbd30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6838004940 tx=0x7f6838009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.394+0000 7f6855e6d700 1 -- 172.21.15.7:0/487771299 shutdown_connections 2024-03-16T09:09:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.394+0000 7f6855e6d700 1 --2- 172.21.15.7:0/487771299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 0x7f68500fbd30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.394+0000 7f6855e6d700 1 --2- 172.21.15.7:0/487771299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6850101d20 0x7f68501020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.394+0000 7f6855e6d700 1 -- 172.21.15.7:0/487771299 >> 172.21.15.7:0/487771299 conn(0x7f68500f7870 msgr2=0x7f68500f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.394+0000 7f6855e6d700 1 -- 172.21.15.7:0/487771299 shutdown_connections 2024-03-16T09:09:37.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.394+0000 7f6855e6d700 1 -- 172.21.15.7:0/487771299 wait complete. 2024-03-16T09:09:37.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.394+0000 7f6855e6d700 1 Processor -- start 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.394+0000 7f6855e6d700 1 -- start start 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6855e6d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6850101d20 0x7f68500fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6855e6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 0x7f68500fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6855e6d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500ff8f0 con 0x7f68501026c0 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6855e6d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500fe420 con 0x7f6850101d20 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f684effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 0x7f68500fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f684effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 0x7f68500fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44522/0 (socket says 172.21.15.7:44522) 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f684effd700 1 -- 172.21.15.7:0/3147495100 learned_addr learned my addr 172.21.15.7:0/3147495100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f684f7fe700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6850101d20 0x7f68500fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f684effd700 1 -- 172.21.15.7:0/3147495100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6850101d20 msgr2=0x7f68500fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f684effd700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6850101d20 0x7f68500fd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f684effd700 1 -- 172.21.15.7:0/3147495100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838005040 con 0x7f68501026c0 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f684effd700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 0x7f68500fdee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6838009180 tx=0x7f6838009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6847fff700 1 -- 172.21.15.7:0/3147495100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f683800e040 con 0x7f68501026c0 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6847fff700 1 -- 172.21.15.7:0/3147495100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f683800bbc0 con 0x7f68501026c0 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500fe6a0 con 0x7f68501026c0 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6847fff700 1 -- 172.21.15.7:0/3147495100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f683801c3f0 con 0x7f68501026c0 2024-03-16T09:09:37.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.395+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685011a140 con 0x7f68501026c0 2024-03-16T09:09:37.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.396+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68500fd470 con 0x7f68501026c0 2024-03-16T09:09:37.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.397+0000 7f6847fff700 1 -- 172.21.15.7:0/3147495100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f683800bd30 con 0x7f68501026c0 2024-03-16T09:09:37.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.397+0000 7f6847fff700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f683c06dee0 0x7f683c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.397+0000 7f6847fff700 1 -- 172.21.15.7:0/3147495100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f683808ea10 con 0x7f68501026c0 2024-03-16T09:09:37.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.397+0000 7f684f7fe700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f683c06dee0 0x7f683c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.398+0000 7f684f7fe700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f683c06dee0 0x7f683c070390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6840004550 tx=0x7f6840009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:37.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.402+0000 7f6847fff700 1 -- 172.21.15.7:0/3147495100 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6838057fa0 con 0x7f68501026c0 2024-03-16T09:09:37.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.616+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f68500f96e0 con 0x7f683c06dee0 2024-03-16T09:09:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.626+0000 7f6847fff700 1 -- 172.21.15.7:0/3147495100 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f68500f96e0 con 0x7f683c06dee0 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (5m) 85s ago 16m 28.5M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (3m) 85s ago 16m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (3m) 17s ago 15m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (5m) 85s ago 16m 84.6M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (11m) 85s ago 11m 14.4M - 17.2.7-648-g1df27247 0bdcfccf14ce 981c80bfdb83 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (10m) 85s ago 10m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (11m) 17s ago 11m 120M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (10m) 17s ago 10m 14.8M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (7m) 85s ago 17m 503M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:09:37.629 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (6m) 17s ago 15m 450M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (4m) 85s ago 18m 51.1M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (3m) 17s ago 15m 38.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (6m) 85s ago 16m 14.7M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (6m) 17s ago 15m 15.4M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (2m) 85s ago 14m 92.2M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (2m) 85s ago 14m 91.9M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4aa10e800f72 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (90s) 85s ago 13m 12.2M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5449f945991b 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (54s) 17s ago 13m 104M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b bb64247744fe 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (21s) 17s ago 13m 12.6M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b e1df7d4b9eaa 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (12m) 17s ago 12m 225M 2583M 17.2.7-648-g1df27247 0bdcfccf14ce ee00fca555d2 2024-03-16T09:09:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (6m) 85s ago 16m 63.4M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:09:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f683c06dee0 msgr2=0x7f683c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f683c06dee0 0x7f683c070390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6840004550 tx=0x7f6840009040 comp rx=0 tx=0).stop 2024-03-16T09:09:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 msgr2=0x7f68500fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 0x7f68500fdee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6838009180 tx=0x7f6838009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 shutdown_connections 2024-03-16T09:09:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6850101d20 0x7f68500fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f683c06dee0 0x7f683c070390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 --2- 172.21.15.7:0/3147495100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68501026c0 0x7f68500fdee0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.628+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 >> 172.21.15.7:0/3147495100 conn(0x7f68500f7870 msgr2=0x7f68500f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.629+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 shutdown_connections 2024-03-16T09:09:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.629+0000 7f6855e6d700 1 -- 172.21.15.7:0/3147495100 wait complete. 2024-03-16T09:09:37.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 -- 172.21.15.7:0/711534510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda0102e00 msgr2=0x7fbda01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 --2- 172.21.15.7:0/711534510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda0102e00 0x7fbda01031d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009ce0 tx=0x7fbd90004d10 comp rx=0 tx=0).stop 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 -- 172.21.15.7:0/711534510 shutdown_connections 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 --2- 172.21.15.7:0/711534510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda01037a0 0x7fbda00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 --2- 172.21.15.7:0/711534510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda0102e00 0x7fbda01031d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 -- 172.21.15.7:0/711534510 >> 172.21.15.7:0/711534510 conn(0x7fbda00f89b0 msgr2=0x7fbda00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 -- 172.21.15.7:0/711534510 shutdown_connections 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 -- 172.21.15.7:0/711534510 wait complete. 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 Processor -- start 2024-03-16T09:09:37.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.730+0000 7fbda6251700 1 -- start start 2024-03-16T09:09:37.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbda6251700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda0102e00 0x7fbda00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbda6251700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda01037a0 0x7fbda0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbda6251700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda0070d90 con 0x7fbda01037a0 2024-03-16T09:09:37.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbda6251700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda0070ed0 con 0x7fbda0102e00 2024-03-16T09:09:37.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd9effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda01037a0 0x7fbda0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd9effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda01037a0 0x7fbda0070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44550/0 (socket says 172.21.15.7:44550) 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd9effd700 1 -- 172.21.15.7:0/317404868 learned_addr learned my addr 172.21.15.7:0/317404868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd9f7fe700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda0102e00 0x7fbda00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd9effd700 1 -- 172.21.15.7:0/317404868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda0102e00 msgr2=0x7fbda00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd9effd700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda0102e00 0x7fbda00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd9effd700 1 -- 172.21.15.7:0/317404868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90009940 con 0x7fbda01037a0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd9effd700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda01037a0 0x7fbda0070800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd88002a50 tx=0x7fbd880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.731+0000 7fbd97fff700 1 -- 172.21.15.7:0/317404868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd88007cc0 con 0x7fbda01037a0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.732+0000 7fbd97fff700 1 -- 172.21.15.7:0/317404868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd8800f040 con 0x7fbda01037a0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.732+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda0119ff0 con 0x7fbda01037a0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.732+0000 7fbd97fff700 1 -- 172.21.15.7:0/317404868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd88009630 con 0x7fbda01037a0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.732+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda011a3f0 con 0x7fbda01037a0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.733+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda0100110 con 0x7fbda01037a0 2024-03-16T09:09:37.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.733+0000 7fbd97fff700 1 -- 172.21.15.7:0/317404868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fbd88009790 con 0x7fbda01037a0 2024-03-16T09:09:37.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.733+0000 7fbd97fff700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbd8c06dee0 0x7fbd8c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:37.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.733+0000 7fbd97fff700 1 -- 172.21.15.7:0/317404868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbd8808d510 con 0x7fbda01037a0 2024-03-16T09:09:37.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.733+0000 7fbd9f7fe700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbd8c06dee0 0x7fbd8c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:37.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.734+0000 7fbd9f7fe700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbd8c06dee0 0x7fbd8c070390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000b740 tx=0x7fbd90004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:37.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:37.738+0000 7fbd97fff700 1 -- 172.21.15.7:0/317404868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd88056a20 con 0x7fbda01037a0 2024-03-16T09:09:37.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:37 smithi019 ceph-mon[83366]: pgmap v151: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:37 smithi019 ceph-mon[83366]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:37.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:37 smithi019 ceph-mon[83366]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:37 smithi007 ceph-mon[113546]: pgmap v151: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:37 smithi007 ceph-mon[113546]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:37 smithi007 ceph-mon[113546]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:38.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.034+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbda0071060 con 0x7fbda01037a0 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.034+0000 7fbd97fff700 1 -- 172.21.15.7:0/317404868 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fbda0071060 con 0x7fbda01037a0 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 1, 2024-03-16T09:09:38.037 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 5 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 4 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 5, 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 9 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:09:38.038 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:09:38.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.036+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbd8c06dee0 msgr2=0x7fbd8c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.036+0000 7fbda6251700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbd8c06dee0 0x7fbd8c070390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000b740 tx=0x7fbd90004d10 comp rx=0 tx=0).stop 2024-03-16T09:09:38.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.036+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda01037a0 msgr2=0x7fbda0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.036+0000 7fbda6251700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda01037a0 0x7fbda0070800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd88002a50 tx=0x7fbd880046b0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.036+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 shutdown_connections 2024-03-16T09:09:38.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.037+0000 7fbda6251700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbda0102e00 0x7fbda00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.037+0000 7fbda6251700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fbd8c06dee0 0x7fbd8c070390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.037+0000 7fbda6251700 1 --2- 172.21.15.7:0/317404868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda01037a0 0x7fbda0070800 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.037+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 >> 172.21.15.7:0/317404868 conn(0x7fbda00f89b0 msgr2=0x7fbda01073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:38.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.037+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 shutdown_connections 2024-03-16T09:09:38.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.037+0000 7fbda6251700 1 -- 172.21.15.7:0/317404868 wait complete. 2024-03-16T09:09:38.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 -- 172.21.15.7:0/2803621332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 msgr2=0x7fe368102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 --2- 172.21.15.7:0/2803621332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 0x7fe368102140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe350004940 tx=0x7fe350009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 -- 172.21.15.7:0/2803621332 shutdown_connections 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 --2- 172.21.15.7:0/2803621332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe368102710 0x7fe3680fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 --2- 172.21.15.7:0/2803621332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 0x7fe368102140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 -- 172.21.15.7:0/2803621332 >> 172.21.15.7:0/2803621332 conn(0x7fe368072bc0 msgr2=0x7fe368072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 -- 172.21.15.7:0/2803621332 shutdown_connections 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 -- 172.21.15.7:0/2803621332 wait complete. 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.139+0000 7fe36e755700 1 Processor -- start 2024-03-16T09:09:38.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe36e755700 1 -- start start 2024-03-16T09:09:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe36e755700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 0x7fe36810ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe36e755700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe368102710 0x7fe36810f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe36e755700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36810f700 con 0x7fe368101d70 2024-03-16T09:09:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe36e755700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36810f840 con 0x7fe368102710 2024-03-16T09:09:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe367fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 0x7fe36810ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe367fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 0x7fe36810ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44558/0 (socket says 172.21.15.7:44558) 2024-03-16T09:09:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe367fff700 1 -- 172.21.15.7:0/3413405861 learned_addr learned my addr 172.21.15.7:0/3413405861 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe367fff700 1 -- 172.21.15.7:0/3413405861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe368102710 msgr2=0x7fe36810f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe367fff700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe368102710 0x7fe36810f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe367fff700 1 -- 172.21.15.7:0/3413405861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350005040 con 0x7fe368101d70 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe367fff700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 0x7fe36810ec80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe350009b30 tx=0x7fe350009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe364ff9700 1 -- 172.21.15.7:0/3413405861 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35000e070 con 0x7fe368101d70 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe368117b30 con 0x7fe368101d70 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.140+0000 7fe364ff9700 1 -- 172.21.15.7:0/3413405861 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe350013cd0 con 0x7fe368101d70 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.141+0000 7fe364ff9700 1 -- 172.21.15.7:0/3413405861 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35000b720 con 0x7fe368101d70 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.141+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe368117fe0 con 0x7fe368101d70 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.141+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3680ff080 con 0x7fe368101d70 2024-03-16T09:09:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.142+0000 7fe364ff9700 1 -- 172.21.15.7:0/3413405861 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fe35001cab0 con 0x7fe368101d70 2024-03-16T09:09:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.142+0000 7fe364ff9700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe35406dee0 0x7fe354070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.143+0000 7fe364ff9700 1 -- 172.21.15.7:0/3413405861 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe35008f060 con 0x7fe368101d70 2024-03-16T09:09:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.143+0000 7fe3677fe700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe35406dee0 0x7fe354070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.143+0000 7fe3677fe700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe35406dee0 0x7fe354070390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe368066020 tx=0x7fe35800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:38.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.147+0000 7fe364ff9700 1 -- 172.21.15.7:0/3413405861 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe3500585f0 con 0x7fe368101d70 2024-03-16T09:09:38.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.399+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe368108e90 con 0x7fe368101d70 2024-03-16T09:09:38.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.400+0000 7fe364ff9700 1 -- 172.21.15.7:0/3413405861 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1895 (secure 0 0 0) 0x7fe350004490 con 0x7fe368101d70 2024-03-16T09:09:38.402 INFO:teuthology.orchestra.run.smithi007.stdout:e20 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:05:26.042398+0000 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:09:38.403 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 46 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=14482,1=24281} 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:09:38.404 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 14482 members: 14482,24281 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{0:14482} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.7:6828/3212461733,v1:172.21.15.7:6829/3212461733] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{1:24281} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.19:6824/4120977378,v1:172.21.15.19:6825/4120977378] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:09:38.405 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2680162193,v1:172.21.15.7:6827/2680162193] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:09:38.406 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:09:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.402+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe35406dee0 msgr2=0x7fe354070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.402+0000 7fe36e755700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe35406dee0 0x7fe354070390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe368066020 tx=0x7fe35800b040 comp rx=0 tx=0).stop 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.402+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 msgr2=0x7fe36810ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.402+0000 7fe36e755700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 0x7fe36810ec80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe350009b30 tx=0x7fe350009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.403+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 shutdown_connections 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.403+0000 7fe36e755700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe35406dee0 0x7fe354070390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.403+0000 7fe36e755700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe368101d70 0x7fe36810ec80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.403+0000 7fe36e755700 1 --2- 172.21.15.7:0/3413405861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe368102710 0x7fe36810f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.403+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 >> 172.21.15.7:0/3413405861 conn(0x7fe368072bc0 msgr2=0x7fe3680faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.403+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 shutdown_connections 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.403+0000 7fe36e755700 1 -- 172.21.15.7:0/3413405861 wait complete. 2024-03-16T09:09:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:09:38.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.506+0000 7f68fe7ad700 1 -- 172.21.15.7:0/2461542193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fbc80 msgr2=0x7f68f80fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.506+0000 7f68fe7ad700 1 --2- 172.21.15.7:0/2461542193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fbc80 0x7f68f80fc050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f68e8004940 tx=0x7f68e8009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.506+0000 7f68fe7ad700 1 -- 172.21.15.7:0/2461542193 shutdown_connections 2024-03-16T09:09:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.506+0000 7f68fe7ad700 1 --2- 172.21.15.7:0/2461542193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f80fc590 0x7f68f81092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.506+0000 7f68fe7ad700 1 --2- 172.21.15.7:0/2461542193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fbc80 0x7f68f80fc050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.506+0000 7f68fe7ad700 1 -- 172.21.15.7:0/2461542193 >> 172.21.15.7:0/2461542193 conn(0x7f68f8072bc0 msgr2=0x7f68f8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.506+0000 7f68fe7ad700 1 -- 172.21.15.7:0/2461542193 shutdown_connections 2024-03-16T09:09:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.506+0000 7f68fe7ad700 1 -- 172.21.15.7:0/2461542193 wait complete. 2024-03-16T09:09:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68fe7ad700 1 Processor -- start 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68fe7ad700 1 -- start start 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68fe7ad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f80fbc80 0x7f68f8114c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68fe7ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fc590 0x7f68f81151a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68fe7ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f81157a0 con 0x7f68f80fc590 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68fe7ad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f810fd70 con 0x7f68f80fbc80 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fc590 0x7f68f81151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fc590 0x7f68f81151a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44580/0 (socket says 172.21.15.7:44580) 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68f77fe700 1 -- 172.21.15.7:0/606407382 learned_addr learned my addr 172.21.15.7:0/606407382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68f7fff700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f80fbc80 0x7f68f8114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:38.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68f77fe700 1 -- 172.21.15.7:0/606407382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f80fbc80 msgr2=0x7f68f8114c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68f77fe700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f80fbc80 0x7f68f8114c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68f77fe700 1 -- 172.21.15.7:0/606407382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8005040 con 0x7f68f80fc590 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.507+0000 7f68f7fff700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f80fbc80 0x7f68f8114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.508+0000 7f68f77fe700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fc590 0x7f68f81151a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f68e0002a50 tx=0x7f68e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.508+0000 7f68f4ff9700 1 -- 172.21.15.7:0/606407382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e0007cc0 con 0x7f68f80fc590 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.508+0000 7f68f4ff9700 1 -- 172.21.15.7:0/606407382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68e000f040 con 0x7f68f80fc590 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.508+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f8110050 con 0x7f68f80fc590 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.508+0000 7f68f4ff9700 1 -- 172.21.15.7:0/606407382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e0009640 con 0x7f68f80fc590 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.508+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f8110610 con 0x7f68f80fc590 2024-03-16T09:09:38.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.509+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f8065e30 con 0x7f68f80fc590 2024-03-16T09:09:38.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.509+0000 7f68f4ff9700 1 -- 172.21.15.7:0/606407382 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f68e00097a0 con 0x7f68f80fc590 2024-03-16T09:09:38.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.510+0000 7f68f4ff9700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f68e406dee0 0x7f68e4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.510+0000 7f68f4ff9700 1 -- 172.21.15.7:0/606407382 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f68e008d4f0 con 0x7f68f80fc590 2024-03-16T09:09:38.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.510+0000 7f68f7fff700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f68e406dee0 0x7f68e4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:38.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.510+0000 7f68f7fff700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f68e406dee0 0x7f68e4070390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f68e8004540 tx=0x7f68e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:38.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.514+0000 7f68f4ff9700 1 -- 172.21.15.7:0/606407382 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f68e00569a0 con 0x7f68f80fc590 2024-03-16T09:09:38.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.740+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68f810c160 con 0x7f68e406dee0 2024-03-16T09:09:38.743 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:38 smithi007 ceph-mon[113546]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:38.743 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:38 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/317404868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:38.743 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:38 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3413405861' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.741+0000 7f68f4ff9700 1 -- 172.21.15.7:0/606407382 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f68f810c160 con 0x7f68e406dee0 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:09:38.744 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "11/21 daemons upgraded", 2024-03-16T09:09:38.745 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T09:09:38.745 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:09:38.745 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:09:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.743+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f68e406dee0 msgr2=0x7f68e4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.743+0000 7f68fe7ad700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f68e406dee0 0x7f68e4070390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f68e8004540 tx=0x7f68e8009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fc590 msgr2=0x7f68f81151a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fc590 0x7f68f81151a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f68e0002a50 tx=0x7f68e00046b0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 shutdown_connections 2024-03-16T09:09:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68f80fbc80 0x7f68f8114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f68e406dee0 0x7f68e4070390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 --2- 172.21.15.7:0/606407382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f80fc590 0x7f68f81151a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 >> 172.21.15.7:0/606407382 conn(0x7f68f8072bc0 msgr2=0x7f68f8105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 shutdown_connections 2024-03-16T09:09:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.744+0000 7f68fe7ad700 1 -- 172.21.15.7:0/606407382 wait complete. 2024-03-16T09:09:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.846+0000 7fa2ed87a700 1 -- 172.21.15.7:0/1287102583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 msgr2=0x7fa2e80fe220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.846+0000 7fa2ed87a700 1 --2- 172.21.15.7:0/1287102583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 0x7fa2e80fe220 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0004940 tx=0x7fa2e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.846+0000 7fa2ed87a700 1 -- 172.21.15.7:0/1287102583 shutdown_connections 2024-03-16T09:09:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.846+0000 7fa2ed87a700 1 --2- 172.21.15.7:0/1287102583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e80fe7f0 0x7fa2e8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.846+0000 7fa2ed87a700 1 --2- 172.21.15.7:0/1287102583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 0x7fa2e80fe220 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.846+0000 7fa2ed87a700 1 -- 172.21.15.7:0/1287102583 >> 172.21.15.7:0/1287102583 conn(0x7fa2e80f7850 msgr2=0x7fa2e80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.846+0000 7fa2ed87a700 1 -- 172.21.15.7:0/1287102583 shutdown_connections 2024-03-16T09:09:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.846+0000 7fa2ed87a700 1 -- 172.21.15.7:0/1287102583 wait complete. 2024-03-16T09:09:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2ed87a700 1 Processor -- start 2024-03-16T09:09:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2ed87a700 1 -- start start 2024-03-16T09:09:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2ed87a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 0x7fa2e80fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2ed87a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e80fe7f0 0x7fa2e80fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2ed87a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80fdc60 con 0x7fa2e80fde50 2024-03-16T09:09:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2ed87a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e80fc790 con 0x7fa2e80fe7f0 2024-03-16T09:09:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2e6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 0x7fa2e80fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2e6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 0x7fa2e80fbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44588/0 (socket says 172.21.15.7:44588) 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2e6ffd700 1 -- 172.21.15.7:0/842119753 learned_addr learned my addr 172.21.15.7:0/842119753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2e67fc700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e80fe7f0 0x7fa2e80fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2e6ffd700 1 -- 172.21.15.7:0/842119753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e80fe7f0 msgr2=0x7fa2e80fc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2e6ffd700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e80fe7f0 0x7fa2e80fc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.847+0000 7fa2e6ffd700 1 -- 172.21.15.7:0/842119753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0005040 con 0x7fa2e80fde50 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.848+0000 7fa2e6ffd700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 0x7fa2e80fbd10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009360 tx=0x7fa2e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.848+0000 7fa2df7fe700 1 -- 172.21.15.7:0/842119753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2e000e040 con 0x7fa2e80fde50 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.848+0000 7fa2df7fe700 1 -- 172.21.15.7:0/842119753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2e0013420 con 0x7fa2e80fde50 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.848+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e80fca10 con 0x7fa2e80fde50 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.848+0000 7fa2df7fe700 1 -- 172.21.15.7:0/842119753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2e001c630 con 0x7fa2e80fde50 2024-03-16T09:09:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.848+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e811a080 con 0x7fa2e80fde50 2024-03-16T09:09:38.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.849+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e8066a90 con 0x7fa2e80fde50 2024-03-16T09:09:38.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.849+0000 7fa2df7fe700 1 -- 172.21.15.7:0/842119753 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa2e0013a90 con 0x7fa2e80fde50 2024-03-16T09:09:38.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.850+0000 7fa2df7fe700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa2d406dee0 0x7fa2d4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:38.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.850+0000 7fa2df7fe700 1 -- 172.21.15.7:0/842119753 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa2e008e710 con 0x7fa2e80fde50 2024-03-16T09:09:38.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.850+0000 7fa2e67fc700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa2d406dee0 0x7fa2d4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:38.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.850+0000 7fa2e67fc700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa2d406dee0 0x7fa2d4070390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80fd440 tx=0x7fa2d00043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:38.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:38.855+0000 7fa2df7fe700 1 -- 172.21.15.7:0/842119753 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2e0057d40 con 0x7fa2e80fde50 2024-03-16T09:09:38.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:38 smithi019 ceph-mon[83366]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:38.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:38 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/317404868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:38.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:38 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3413405861' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:09:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.146+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa2e804c710 con 0x7fa2e80fde50 2024-03-16T09:09:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.146+0000 7fa2df7fe700 1 -- 172.21.15.7:0/842119753 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa2e005b400 con 0x7fa2e80fde50 2024-03-16T09:09:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_OK 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa2d406dee0 msgr2=0x7fa2d4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa2d406dee0 0x7fa2d4070390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80fd440 tx=0x7fa2d00043c0 comp rx=0 tx=0).stop 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 msgr2=0x7fa2e80fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 0x7fa2e80fbd10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009360 tx=0x7fa2e0009e30 comp rx=0 tx=0).stop 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 shutdown_connections 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa2d406dee0 0x7fa2d4070390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e80fde50 0x7fa2e80fbd10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 --2- 172.21.15.7:0/842119753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e80fe7f0 0x7fa2e80fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 >> 172.21.15.7:0/842119753 conn(0x7fa2e80f7850 msgr2=0x7fa2e80f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 shutdown_connections 2024-03-16T09:09:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:09:39.148+0000 7fa2ed87a700 1 -- 172.21.15.7:0/842119753 wait complete. 2024-03-16T09:09:39.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:39 smithi019 ceph-mon[83366]: pgmap v152: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:39.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:39 smithi019 ceph-mon[83366]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:39.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:39 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/842119753' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:09:39.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:39 smithi007 ceph-mon[113546]: pgmap v152: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:39.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:39 smithi007 ceph-mon[113546]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:39.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:39 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/842119753' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:09:40.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:40 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T09:09:40.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:40 smithi019 ceph-mon[83366]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T09:09:40.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:40 smithi019 ceph-mon[83366]: Upgrade: osd.5 is safe to restart 2024-03-16T09:09:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:40 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T09:09:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:40 smithi007 ceph-mon[113546]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T09:09:40.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:40 smithi007 ceph-mon[113546]: Upgrade: osd.5 is safe to restart 2024-03-16T09:09:42.582 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:42 smithi019 ceph-mon[83366]: pgmap v153: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:42.582 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:42 smithi019 ceph-mon[83366]: Upgrade: Updating osd.5 2024-03-16T09:09:42.583 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:42.583 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T09:09:42.583 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:42 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:42.583 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:42 smithi019 ceph-mon[83366]: Deploying daemon osd.5 on smithi019 2024-03-16T09:09:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:42 smithi007 ceph-mon[113546]: pgmap v153: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:42 smithi007 ceph-mon[113546]: Upgrade: Updating osd.5 2024-03-16T09:09:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T09:09:42.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:42 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:42.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:42 smithi007 ceph-mon[113546]: Deploying daemon osd.5 on smithi019 2024-03-16T09:09:44.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:44 smithi019 ceph-mon[83366]: pgmap v154: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:44 smithi007 ceph-mon[113546]: pgmap v154: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:44.853 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:44 smithi019 systemd[1]: Stopping Ceph osd.5 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:09:45.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:44 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[55734]: 2024-03-16T09:09:44.850+0000 7f36bebd2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:09:45.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:44 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[55734]: 2024-03-16T09:09:44.850+0000 7f36bebd2700 -1 osd.5 74 *** Got signal Terminated *** 2024-03-16T09:09:45.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:44 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[55734]: 2024-03-16T09:09:44.850+0000 7f36bebd2700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:09:45.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:45 smithi019 ceph-mon[83366]: osd.5 marked itself down and dead 2024-03-16T09:09:45.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:45 smithi007 ceph-mon[113546]: osd.5 marked itself down and dead 2024-03-16T09:09:45.995 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:45 smithi019 podman[102326]: 2024-03-16 09:09:45.715840395 +0000 UTC m=+0.974774777 container died ee00fca555d2b358f2fb7c309bb6b831d683a2299c142474aeb39f7009077d97 (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, RELEASE=quincy-1df2724, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-16T09:09:46.254 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:46 smithi019 podman[102326]: 2024-03-16 09:09:46.084469615 +0000 UTC m=+1.343403994 container cleanup ee00fca555d2b358f2fb7c309bb6b831d683a2299c142474aeb39f7009077d97 (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, RELEASE=quincy-1df2724, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-16T09:09:46.254 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:46 smithi019 bash[102326]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5 2024-03-16T09:09:46.704 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:46 smithi019 podman[102353]: 2024-03-16 09:09:46.252467733 +0000 UTC m=+0.536290867 container remove ee00fca555d2b358f2fb7c309bb6b831d683a2299c142474aeb39f7009077d97 (image=quay.ceph.io/ceph-ci/ceph@sha256:2b3afb7dd69d68e95126f78145ae3f8d9c669e228046f6ff990db3c13aa086fc, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-1df2724, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:09:46.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:46 smithi019 ceph-mon[83366]: pgmap v155: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:46.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:46 smithi019 ceph-mon[83366]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:09:46.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:46 smithi019 ceph-mon[83366]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-16T09:09:46.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:46 smithi019 ceph-mon[83366]: osdmap e75: 6 total, 5 up, 6 in 2024-03-16T09:09:46.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:46.705 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:09:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:46 smithi007 ceph-mon[113546]: pgmap v155: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:46 smithi007 ceph-mon[113546]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T09:09:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:46 smithi007 ceph-mon[113546]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-16T09:09:46.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:46 smithi007 ceph-mon[113546]: osdmap e75: 6 total, 5 up, 6 in 2024-03-16T09:09:46.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:46.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:09:46.962 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:46 smithi019 podman[102440]: 2024-03-16 09:09:46.703315859 +0000 UTC m=+0.109777295 container create bcfc1da2591b0955131faf3baa07dc72809210ac46846d5e4f8534ff5301482e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-16T09:09:46.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:46 smithi019 podman[102440]: 2024-03-16 09:09:46.611624079 +0000 UTC m=+0.018085516 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:09:46.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:46 smithi019 podman[102440]: 2024-03-16 09:09:46.939442678 +0000 UTC m=+0.345904121 container init bcfc1da2591b0955131faf3baa07dc72809210ac46846d5e4f8534ff5301482e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:09:46.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:46 smithi019 podman[102440]: 2024-03-16 09:09:46.943089981 +0000 UTC m=+0.349551415 container start bcfc1da2591b0955131faf3baa07dc72809210ac46846d5e4f8534ff5301482e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-16T09:09:47.360 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:47 smithi019 podman[102440]: 2024-03-16 09:09:47.0477824 +0000 UTC m=+0.454243841 container attach bcfc1da2591b0955131faf3baa07dc72809210ac46846d5e4f8534ff5301482e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-16T09:09:47.360 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:47 smithi019 podman[102440]: 2024-03-16 09:09:47.242353347 +0000 UTC m=+0.648814778 container died bcfc1da2591b0955131faf3baa07dc72809210ac46846d5e4f8534ff5301482e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-16T09:09:47.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:47 smithi019 ceph-mon[83366]: osdmap e76: 6 total, 5 up, 6 in 2024-03-16T09:09:47.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:47 smithi007 ceph-mon[113546]: osdmap e76: 6 total, 5 up, 6 in 2024-03-16T09:09:48.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:47 smithi019 podman[102466]: 2024-03-16 09:09:47.793263113 +0000 UTC m=+0.544633259 container remove bcfc1da2591b0955131faf3baa07dc72809210ac46846d5e4f8534ff5301482e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T09:09:48.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:47 smithi019 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-16T09:09:48.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:47 smithi019 systemd[1]: Stopped Ceph osd.5 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:09:48.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:47 smithi019 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.5.service: Consumed 59.488s CPU time. 2024-03-16T09:09:48.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 systemd[1]: Starting Ceph osd.5 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:09:48.473 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:48 smithi019 ceph-mon[83366]: pgmap v158: 65 pgs: 13 stale+active+clean, 52 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:48.474 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 podman[102570]: 2024-03-16 09:09:48.133034386 +0000 UTC m=+0.017522123 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:09:48.474 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 podman[102570]: 2024-03-16 09:09:48.236063612 +0000 UTC m=+0.120551342 container create e1cb43723d2546b15a451fb200df2a4b14e0e608459f9a23e46933373a1ac3c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T09:09:48.474 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 podman[102570]: 2024-03-16 09:09:48.47205338 +0000 UTC m=+0.356541123 container init e1cb43723d2546b15a451fb200df2a4b14e0e608459f9a23e46933373a1ac3c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:09:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:48 smithi007 ceph-mon[113546]: pgmap v158: 65 pgs: 13 stale+active+clean, 52 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:09:48.925 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 podman[102570]: 2024-03-16 09:09:48.476319351 +0000 UTC m=+0.360807091 container start e1cb43723d2546b15a451fb200df2a4b14e0e608459f9a23e46933373a1ac3c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:09:48.925 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 podman[102570]: 2024-03-16 09:09:48.688652605 +0000 UTC m=+0.573140350 container attach e1cb43723d2546b15a451fb200df2a4b14e0e608459f9a23e46933373a1ac3c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-03-16T09:09:49.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate[102585]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T09:09:49.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate[102585]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-16T09:09:49.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate[102585]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate[102585]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate[102585]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate[102585]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate[102585]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 bash[102570]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 bash[102570]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 bash[102570]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 bash[102570]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 bash[102570]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 bash[102570]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 bash[102570]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-16T09:09:49.214 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:48 smithi019 podman[102570]: 2024-03-16 09:09:48.948185424 +0000 UTC m=+0.832673167 container died e1cb43723d2546b15a451fb200df2a4b14e0e608459f9a23e46933373a1ac3c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-16T09:09:49.677 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:49 smithi019 ceph-mon[83366]: pgmap v159: 65 pgs: 8 active+undersized+degraded, 5 stale+active+clean, 7 active+undersized, 45 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23/354 objects degraded (6.497%) 2024-03-16T09:09:49.677 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:49 smithi019 ceph-mon[83366]: Health check failed: Degraded data redundancy: 23/354 objects degraded (6.497%), 8 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:49.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:49 smithi007 ceph-mon[113546]: pgmap v159: 65 pgs: 8 active+undersized+degraded, 5 stale+active+clean, 7 active+undersized, 45 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23/354 objects degraded (6.497%) 2024-03-16T09:09:49.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:49 smithi007 ceph-mon[113546]: Health check failed: Degraded data redundancy: 23/354 objects degraded (6.497%), 8 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:49.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:49 smithi019 podman[102710]: 2024-03-16 09:09:49.675992219 +0000 UTC m=+0.723085864 container remove e1cb43723d2546b15a451fb200df2a4b14e0e608459f9a23e46933373a1ac3c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-activate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-16T09:09:49.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:49 smithi019 podman[102774]: 2024-03-16 09:09:49.795965485 +0000 UTC m=+0.017973499 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:09:49.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:49 smithi019 podman[102774]: 2024-03-16 09:09:49.893505731 +0000 UTC m=+0.115513743 container create 5480e37181b114b6db7891620e63267f68498a94422d5e90591421e865e83639 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T09:09:50.222 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:50 smithi019 podman[102774]: 2024-03-16 09:09:50.112474895 +0000 UTC m=+0.334482906 container init 5480e37181b114b6db7891620e63267f68498a94422d5e90591421e865e83639 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-16T09:09:50.223 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:50 smithi019 podman[102774]: 2024-03-16 09:09:50.115685738 +0000 UTC m=+0.337693750 container start 5480e37181b114b6db7891620e63267f68498a94422d5e90591421e865e83639 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:09:50.223 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:50 smithi019 ceph-osd[102794]: -- 172.21.15.19:0/3093648172 <== mon.1 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620c820b680 con 0x5620c81d5000 2024-03-16T09:09:50.223 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:50 smithi019 bash[102774]: 5480e37181b114b6db7891620e63267f68498a94422d5e90591421e865e83639 2024-03-16T09:09:50.659 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:50 smithi019 systemd[1]: Started Ceph osd.5 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:09:50.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:50 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:09:50.657+0000 7f3d278777c0 -1 Falling back to public interface 2024-03-16T09:09:51.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:51 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:51.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:51 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:51.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:51 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:51.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:51 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:51.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:51 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:51.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:51 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:52.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:52 smithi019 ceph-mon[83366]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-16T09:09:52.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:52 smithi007 ceph-mon[113546]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-16T09:09:54.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:54 smithi019 ceph-mon[83366]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-16T09:09:54.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:54 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:54.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:54 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:54 smithi007 ceph-mon[113546]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-16T09:09:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:54 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:54 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:55.209 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:54 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:09:54.875+0000 7f3d278777c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-16T09:09:55.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:55 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:09:55.289+0000 7f3d278777c0 -1 osd.5 74 log_to_monitors true 2024-03-16T09:09:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:55 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:55 smithi019 ceph-mon[83366]: from='osd.5 [v2:172.21.15.19:6816/3993266086,v1:172.21.15.19:6817/3993266086]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:09:55.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:55 smithi019 ceph-mon[83366]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:09:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:55 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:55 smithi007 ceph-mon[113546]: from='osd.5 [v2:172.21.15.19:6816/3993266086,v1:172.21.15.19:6817/3993266086]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:09:55.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:55 smithi007 ceph-mon[113546]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:09:56.415 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:09:56 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:09:56.386+0000 7f3d14cc7700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:09:56.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:56 smithi019 ceph-mon[83366]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-16T09:09:56.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:56 smithi019 ceph-mon[83366]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T09:09:56.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:56 smithi019 ceph-mon[83366]: osdmap e77: 6 total, 5 up, 6 in 2024-03-16T09:09:56.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:56 smithi019 ceph-mon[83366]: from='osd.5 [v2:172.21.15.19:6816/3993266086,v1:172.21.15.19:6817/3993266086]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:09:56.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:56 smithi019 ceph-mon[83366]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:09:56.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:56 smithi019 ceph-mon[83366]: from='osd.5 ' entity='osd.5' 2024-03-16T09:09:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:56 smithi007 ceph-mon[113546]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-16T09:09:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:56 smithi007 ceph-mon[113546]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T09:09:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:56 smithi007 ceph-mon[113546]: osdmap e77: 6 total, 5 up, 6 in 2024-03-16T09:09:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:56 smithi007 ceph-mon[113546]: from='osd.5 [v2:172.21.15.19:6816/3993266086,v1:172.21.15.19:6817/3993266086]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:09:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:56 smithi007 ceph-mon[113546]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-03-16T09:09:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:56 smithi007 ceph-mon[113546]: from='osd.5 ' entity='osd.5' 2024-03-16T09:09:57.656 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:57 smithi019 ceph-mon[83366]: Health check update: Degraded data redundancy: 55/354 objects degraded (15.537%), 14 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:57.656 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:57 smithi019 ceph-mon[83366]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:09:57.656 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:57 smithi019 ceph-mon[83366]: osd.5 [v2:172.21.15.19:6816/3993266086,v1:172.21.15.19:6817/3993266086] boot 2024-03-16T09:09:57.656 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:57 smithi019 ceph-mon[83366]: osdmap e78: 6 total, 6 up, 6 in 2024-03-16T09:09:57.656 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:57 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:09:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:57 smithi007 ceph-mon[113546]: Health check update: Degraded data redundancy: 55/354 objects degraded (15.537%), 14 pgs degraded (PG_DEGRADED) 2024-03-16T09:09:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:57 smithi007 ceph-mon[113546]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T09:09:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:57 smithi007 ceph-mon[113546]: osd.5 [v2:172.21.15.19:6816/3993266086,v1:172.21.15.19:6817/3993266086] boot 2024-03-16T09:09:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:57 smithi007 ceph-mon[113546]: osdmap e78: 6 total, 6 up, 6 in 2024-03-16T09:09:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:57 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:09:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: pgmap v164: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-16T09:09:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all osd 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-16T09:09:58.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: Upgrade: Setting require_osd_release to 19 squid 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: osdmap e79: 6 total, 6 up, 6 in 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-16T09:09:58.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T09:09:58.716 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T09:09:58.716 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T09:09:58.716 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T09:09:58.716 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: osdmap e80: 6 total, 6 up, 6 in 2024-03-16T09:09:58.716 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:58 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-16T09:09:58.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:58.393+0000 7f0527833700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: pgmap v164: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all osd 2024-03-16T09:09:58.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-16T09:09:58.750 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: Upgrade: Setting require_osd_release to 19 squid 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: osdmap e79: 6 total, 6 up, 6 in 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: osdmap e80: 6 total, 6 up, 6 in 2024-03-16T09:09:58.751 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:58 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-16T09:09:59.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:09:59 smithi019 ceph-mon[83366]: pgmap v168: 65 pgs: 7 peering, 10 active+undersized+degraded, 11 active+undersized, 37 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 45/354 objects degraded (12.712%) 2024-03-16T09:09:59.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:09:59 smithi007 ceph-mon[113546]: pgmap v168: 65 pgs: 7 peering, 10 active+undersized+degraded, 11 active+undersized, 37 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 45/354 objects degraded (12.712%) 2024-03-16T09:10:00.428 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : [WRN] PG_DEGRADED: Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.2 is active+undersized+degraded, acting [1,0] 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.3 is active+undersized+degraded, acting [2,1] 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.8 is active+undersized+degraded, acting [3,0] 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.b is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.14 is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.16 is active+undersized+degraded, acting [3,2] 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.17 is active+undersized+degraded, acting [2,1] 2024-03-16T09:10:00.429 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.18 is active+undersized+degraded, acting [4,3] 2024-03-16T09:10:00.430 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.1a is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.430 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:09:59.999+0000 7f052a038700 -1 log_channel(cluster) log [ERR] : pg 2.1d is active+undersized+degraded, acting [3,0] 2024-03-16T09:10:00.430 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6828/3988729523,v1:172.21.15.7:6829/3988729523] up:boot 2024-03-16T09:10:00.430 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.19:6824/5723851,v1:172.21.15.19:6825/5723851] up:boot 2024-03-16T09:10:00.430 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:00.430 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:10:00.430 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:10:00.430 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded 2024-03-16T09:10:00.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-16T09:10:00.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: fs cephfs is degraded 2024-03-16T09:10:00.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-16T09:10:00.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: fs cephfs is offline because no MDS is active for it. 2024-03-16T09:10:00.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: [WRN] PG_DEGRADED: Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded 2024-03-16T09:10:00.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-03-16T09:10:00.678 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.b is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: Upgrade: Updating mds.cephfs.smithi007.bsxctp 2024-03-16T09:10:00.679 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:00.680 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.bsxctp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:10:00.680 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:00.680 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:00 smithi007 ceph-mon[113546]: Deploying daemon mds.cephfs.smithi007.bsxctp on smithi007 2024-03-16T09:10:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6828/3988729523,v1:172.21.15.7:6829/3988729523] up:boot 2024-03-16T09:10:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.19:6824/5723851,v1:172.21.15.19:6825/5723851] up:boot 2024-03-16T09:10:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:10:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: fs cephfs is degraded 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: fs cephfs is offline because no MDS is active for it. 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: [WRN] PG_DEGRADED: Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.b is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-03-16T09:10:00.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-03-16T09:10:00.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-03-16T09:10:00.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-03-16T09:10:00.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: Upgrade: Updating mds.cephfs.smithi007.bsxctp 2024-03-16T09:10:00.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:00.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.bsxctp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:10:00.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:00.715 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:00 smithi019 ceph-mon[83366]: Deploying daemon mds.cephfs.smithi007.bsxctp on smithi007 2024-03-16T09:10:01.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:01 smithi007 ceph-mon[113546]: pgmap v169: 65 pgs: 7 peering, 58 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:01.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:01.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:10:01.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:01 smithi007 ceph-mon[113546]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded) 2024-03-16T09:10:01.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:01 smithi019 ceph-mon[83366]: pgmap v169: 65 pgs: 7 peering, 58 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:10:01.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:01 smithi019 ceph-mon[83366]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded) 2024-03-16T09:10:03.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:03 smithi019 ceph-mon[83366]: pgmap v170: 65 pgs: 7 peering, 58 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:03.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:03 smithi007 ceph-mon[113546]: pgmap v170: 65 pgs: 7 peering, 58 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:04.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:04 smithi007 ceph-mon[113546]: osdmap e81: 6 total, 6 up, 6 in 2024-03-16T09:10:04.812 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:04 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-16T09:10:04.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:04 smithi019 ceph-mon[83366]: osdmap e81: 6 total, 6 up, 6 in 2024-03-16T09:10:04.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:04 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-16T09:10:05.918 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:05 smithi007 ceph-mon[113546]: pgmap v172: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:05.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:05 smithi019 ceph-mon[83366]: pgmap v172: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:07 smithi007 ceph-mon[113546]: pgmap v173: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:07 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:07 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:07 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:boot 2024-03-16T09:10:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:07 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:07 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:10:07.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:07 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:07 smithi019 ceph-mon[83366]: pgmap v173: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:07 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:07 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:07 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:boot 2024-03-16T09:10:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:07 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:07 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:10:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:07 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:09.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.254+0000 7f2cf1710700 1 -- 172.21.15.7:0/4287618065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec102e00 msgr2=0x7f2cec1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.254+0000 7f2cf1710700 1 --2- 172.21.15.7:0/4287618065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec102e00 0x7f2cec1031d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009ce0 tx=0x7f2ce4004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.254+0000 7f2cf1710700 1 -- 172.21.15.7:0/4287618065 shutdown_connections 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.254+0000 7f2cf1710700 1 --2- 172.21.15.7:0/4287618065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec1037a0 0x7f2cec0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.254+0000 7f2cf1710700 1 --2- 172.21.15.7:0/4287618065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec102e00 0x7f2cec1031d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.254+0000 7f2cf1710700 1 -- 172.21.15.7:0/4287618065 >> 172.21.15.7:0/4287618065 conn(0x7f2cec0f89b0 msgr2=0x7f2cec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.254+0000 7f2cf1710700 1 -- 172.21.15.7:0/4287618065 shutdown_connections 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.254+0000 7f2cf1710700 1 -- 172.21.15.7:0/4287618065 wait complete. 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cf1710700 1 Processor -- start 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cf1710700 1 -- start start 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cf1710700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec102e00 0x7f2cec113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cf1710700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec1037a0 0x7f2cec114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cf1710700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec114680 con 0x7f2cec1037a0 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cf1710700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec10dcb0 con 0x7f2cec102e00 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec1037a0 0x7f2cec114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec1037a0 0x7f2cec114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53252/0 (socket says 172.21.15.7:53252) 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.255+0000 7f2cea7fc700 1 -- 172.21.15.7:0/658037751 learned_addr learned my addr 172.21.15.7:0/658037751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2ceaffd700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec102e00 0x7f2cec113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2cea7fc700 1 -- 172.21.15.7:0/658037751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec102e00 msgr2=0x7f2cec113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2cea7fc700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec102e00 0x7f2cec113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2cea7fc700 1 -- 172.21.15.7:0/658037751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4009940 con 0x7f2cec1037a0 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2cea7fc700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec1037a0 0x7f2cec114030 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c800 tx=0x7f2cd4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2ce37fe700 1 -- 172.21.15.7:0/658037751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd400a760 con 0x7f2cec1037a0 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2ce37fe700 1 -- 172.21.15.7:0/658037751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cd400ad90 con 0x7f2cec1037a0 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2ce37fe700 1 -- 172.21.15.7:0/658037751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd40113a0 con 0x7f2cec1037a0 2024-03-16T09:10:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec10df90 con 0x7f2cec1037a0 2024-03-16T09:10:09.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.256+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec10e550 con 0x7f2cec1037a0 2024-03-16T09:10:09.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.257+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec100110 con 0x7f2cec1037a0 2024-03-16T09:10:09.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.258+0000 7f2ce37fe700 1 -- 172.21.15.7:0/658037751 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2cd4012400 con 0x7f2cec1037a0 2024-03-16T09:10:09.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.258+0000 7f2ce37fe700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2cd806dee0 0x7f2cd8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.258+0000 7f2ce37fe700 1 -- 172.21.15.7:0/658037751 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2cd408d640 con 0x7f2cec1037a0 2024-03-16T09:10:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.258+0000 7f2ceaffd700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2cd806dee0 0x7f2cd8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.259+0000 7f2ceaffd700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2cd806dee0 0x7f2cd8070390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400b740 tx=0x7f2ce4004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:09.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.263+0000 7f2ce37fe700 1 -- 172.21.15.7:0/658037751 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2cd405a720 con 0x7f2cec1037a0 2024-03-16T09:10:09.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.506+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cec10f140 con 0x7f2cd806dee0 2024-03-16T09:10:09.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.508+0000 7f2ce37fe700 1 -- 172.21.15.7:0/658037751 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2cec10f140 con 0x7f2cd806dee0 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2cd806dee0 msgr2=0x7f2cd8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2cd806dee0 0x7f2cd8070390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400b740 tx=0x7f2ce4004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec1037a0 msgr2=0x7f2cec114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec1037a0 0x7f2cec114030 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c800 tx=0x7f2cd4004470 comp rx=0 tx=0).stop 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 shutdown_connections 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cec102e00 0x7f2cec113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2cd806dee0 0x7f2cd8070390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 --2- 172.21.15.7:0/658037751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cec1037a0 0x7f2cec114030 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.510+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 >> 172.21.15.7:0/658037751 conn(0x7f2cec0f89b0 msgr2=0x7f2cec1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.511+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 shutdown_connections 2024-03-16T09:10:09.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.511+0000 7f2cf1710700 1 -- 172.21.15.7:0/658037751 wait complete. 2024-03-16T09:10:09.523 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:10:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.615+0000 7ff0defa2700 1 -- 172.21.15.7:0/4201156242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d81026b0 msgr2=0x7ff0d80fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.615+0000 7ff0defa2700 1 --2- 172.21.15.7:0/4201156242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d81026b0 0x7ff0d80fbd20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4004940 tx=0x7ff0c4009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.615+0000 7ff0defa2700 1 -- 172.21.15.7:0/4201156242 shutdown_connections 2024-03-16T09:10:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.615+0000 7ff0defa2700 1 --2- 172.21.15.7:0/4201156242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d81026b0 0x7ff0d80fbd20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.615+0000 7ff0defa2700 1 --2- 172.21.15.7:0/4201156242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d8101d10 0x7ff0d81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.616+0000 7ff0defa2700 1 -- 172.21.15.7:0/4201156242 >> 172.21.15.7:0/4201156242 conn(0x7ff0d80f7860 msgr2=0x7ff0d80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:09.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.616+0000 7ff0defa2700 1 -- 172.21.15.7:0/4201156242 shutdown_connections 2024-03-16T09:10:09.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.616+0000 7ff0defa2700 1 -- 172.21.15.7:0/4201156242 wait complete. 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.616+0000 7ff0defa2700 1 Processor -- start 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.616+0000 7ff0defa2700 1 -- start start 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0defa2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d8101d10 0x7ff0d8112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0defa2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d81026b0 0x7ff0d8112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0defa2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d8113640 con 0x7ff0d8101d10 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0defa2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d810ca30 con 0x7ff0d81026b0 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0dcd3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d8101d10 0x7ff0d8112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0dcd3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d8101d10 0x7ff0d8112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53264/0 (socket says 172.21.15.7:53264) 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0dcd3e700 1 -- 172.21.15.7:0/3352799694 learned_addr learned my addr 172.21.15.7:0/3352799694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0d7fff700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d81026b0 0x7ff0d8112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0dcd3e700 1 -- 172.21.15.7:0/3352799694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d81026b0 msgr2=0x7ff0d8112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0dcd3e700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d81026b0 0x7ff0d8112ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0dcd3e700 1 -- 172.21.15.7:0/3352799694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4005040 con 0x7ff0d8101d10 2024-03-16T09:10:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.617+0000 7ff0dcd3e700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d8101d10 0x7ff0d8112ab0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00ed20 tx=0x7ff0cc004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.618+0000 7ff0d57fa700 1 -- 172.21.15.7:0/3352799694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0cc00cd40 con 0x7ff0d8101d10 2024-03-16T09:10:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.618+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d810ccb0 con 0x7ff0d8101d10 2024-03-16T09:10:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.618+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d810d270 con 0x7ff0d8101d10 2024-03-16T09:10:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.619+0000 7ff0d57fa700 1 -- 172.21.15.7:0/3352799694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0cc011940 con 0x7ff0d8101d10 2024-03-16T09:10:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.619+0000 7ff0d57fa700 1 -- 172.21.15.7:0/3352799694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0cc010970 con 0x7ff0d8101d10 2024-03-16T09:10:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.619+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d80fd460 con 0x7ff0d8101d10 2024-03-16T09:10:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.620+0000 7ff0d57fa700 1 -- 172.21.15.7:0/3352799694 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7ff0cc011ab0 con 0x7ff0d8101d10 2024-03-16T09:10:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.621+0000 7ff0d57fa700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff0c806dee0 0x7ff0c8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.621+0000 7ff0d57fa700 1 -- 172.21.15.7:0/3352799694 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff0cc0905e0 con 0x7ff0d8101d10 2024-03-16T09:10:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.621+0000 7ff0d7fff700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff0c806dee0 0x7ff0c8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.621+0000 7ff0d7fff700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff0c806dee0 0x7ff0c8070390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0d810dbf0 tx=0x7ff0c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.629+0000 7ff0d57fa700 1 -- 172.21.15.7:0/3352799694 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff0cc05d6c0 con 0x7ff0d8101d10 2024-03-16T09:10:09.767 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:09 smithi007 ceph-mon[113546]: pgmap v174: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.863+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0d8108df0 con 0x7ff0c806dee0 2024-03-16T09:10:09.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.865+0000 7ff0d57fa700 1 -- 172.21.15.7:0/3352799694 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff0d8108df0 con 0x7ff0c806dee0 2024-03-16T09:10:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff0c806dee0 msgr2=0x7ff0c8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff0c806dee0 0x7ff0c8070390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0d810dbf0 tx=0x7ff0c4009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d8101d10 msgr2=0x7ff0d8112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d8101d10 0x7ff0d8112ab0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00ed20 tx=0x7ff0cc004570 comp rx=0 tx=0).stop 2024-03-16T09:10:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 shutdown_connections 2024-03-16T09:10:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ff0c806dee0 0x7ff0c8070390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0d8101d10 0x7ff0d8112ab0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 --2- 172.21.15.7:0/3352799694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0d81026b0 0x7ff0d8112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 >> 172.21.15.7:0/3352799694 conn(0x7ff0d80f7860 msgr2=0x7ff0d80f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 shutdown_connections 2024-03-16T09:10:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.867+0000 7ff0defa2700 1 -- 172.21.15.7:0/3352799694 wait complete. 2024-03-16T09:10:09.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:09 smithi019 ceph-mon[83366]: pgmap v174: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 -- 172.21.15.7:0/4256331437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea8101d10 msgr2=0x7fcea81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 --2- 172.21.15.7:0/4256331437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea8101d10 0x7fcea81020e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fce98009ce0 tx=0x7fce98004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 -- 172.21.15.7:0/4256331437 shutdown_connections 2024-03-16T09:10:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 --2- 172.21.15.7:0/4256331437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea81026b0 0x7fcea80fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 --2- 172.21.15.7:0/4256331437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea8101d10 0x7fcea81020e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 -- 172.21.15.7:0/4256331437 >> 172.21.15.7:0/4256331437 conn(0x7fcea80f7880 msgr2=0x7fcea80f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:09.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 -- 172.21.15.7:0/4256331437 shutdown_connections 2024-03-16T09:10:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 -- 172.21.15.7:0/4256331437 wait complete. 2024-03-16T09:10:09.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 Processor -- start 2024-03-16T09:10:09.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.969+0000 7fceaf635700 1 -- start start 2024-03-16T09:10:09.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceaf635700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea8101d10 0x7fcea8113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceaf635700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea81026b0 0x7fcea8114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceaf635700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8114670 con 0x7fcea81026b0 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceaf635700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea810dc50 con 0x7fcea8101d10 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceacbd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea81026b0 0x7fcea8114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceacbd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea81026b0 0x7fcea8114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53280/0 (socket says 172.21.15.7:53280) 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceacbd0700 1 -- 172.21.15.7:0/3342381792 learned_addr learned my addr 172.21.15.7:0/3342381792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fcead3d1700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea8101d10 0x7fcea8113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceacbd0700 1 -- 172.21.15.7:0/3342381792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea8101d10 msgr2=0x7fcea8113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceacbd0700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea8101d10 0x7fcea8113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceacbd0700 1 -- 172.21.15.7:0/3342381792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce98009940 con 0x7fcea81026b0 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fcead3d1700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea8101d10 0x7fcea8113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fceacbd0700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea81026b0 0x7fcea8114020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce9c002a50 tx=0x7fce9c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fcea5ffb700 1 -- 172.21.15.7:0/3342381792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9c007cc0 con 0x7fcea81026b0 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.970+0000 7fcea5ffb700 1 -- 172.21.15.7:0/3342381792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce9c00f040 con 0x7fcea81026b0 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.971+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea810df30 con 0x7fcea81026b0 2024-03-16T09:10:09.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.971+0000 7fcea5ffb700 1 -- 172.21.15.7:0/3342381792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9c00daf0 con 0x7fcea81026b0 2024-03-16T09:10:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.971+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea810e4f0 con 0x7fcea81026b0 2024-03-16T09:10:09.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.972+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea80fd460 con 0x7fcea81026b0 2024-03-16T09:10:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.972+0000 7fcea5ffb700 1 -- 172.21.15.7:0/3342381792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fce9c009810 con 0x7fcea81026b0 2024-03-16T09:10:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.973+0000 7fcea5ffb700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fce9006dee0 0x7fce90070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.973+0000 7fcea5ffb700 1 -- 172.21.15.7:0/3342381792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fce9c08d520 con 0x7fcea81026b0 2024-03-16T09:10:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.973+0000 7fcead3d1700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fce9006dee0 0x7fce90070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:09.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.973+0000 7fcead3d1700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fce9006dee0 0x7fce90070390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce9800b740 tx=0x7fce98004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:09.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:09.977+0000 7fcea5ffb700 1 -- 172.21.15.7:0/3342381792 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce9c05a4c0 con 0x7fcea81026b0 2024-03-16T09:10:10.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.203+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcea8108e80 con 0x7fce9006dee0 2024-03-16T09:10:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.213+0000 7fcea5ffb700 1 -- 172.21.15.7:0/3342381792 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcea8108e80 con 0x7fce9006dee0 2024-03-16T09:10:10.215 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (6m) 117s ago 17m 28.5M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (3m) 117s ago 17m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (3m) 16s ago 16m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (6m) 117s ago 16m 84.6M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 starting - - - - 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (11m) 117s ago 11m 128M - 17.2.7-648-g1df27247 0bdcfccf14ce 36eddbe38924 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (11m) 16s ago 11m 120M - 17.2.7-648-g1df27247 0bdcfccf14ce 06288e090eee 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (11m) 16s ago 11m 15.0M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (7m) 117s ago 18m 503M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (7m) 16s ago 16m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (4m) 117s ago 18m 51.1M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (4m) 16s ago 16m 40.1M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (6m) 117s ago 17m 14.7M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:10:10.216 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (6m) 16s ago 16m 15.2M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:10:10.217 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (3m) 117s ago 15m 92.2M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:10:10.217 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (2m) 117s ago 14m 91.9M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4aa10e800f72 2024-03-16T09:10:10.217 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (2m) 117s ago 14m 12.2M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5449f945991b 2024-03-16T09:10:10.217 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (87s) 16s ago 14m 105M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b bb64247744fe 2024-03-16T09:10:10.217 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (53s) 16s ago 13m 102M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b e1df7d4b9eaa 2024-03-16T09:10:10.217 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (20s) 16s ago 13m 15.0M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5480e37181b1 2024-03-16T09:10:10.217 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (6m) 117s ago 16m 63.4M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:10:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fce9006dee0 msgr2=0x7fce90070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fce9006dee0 0x7fce90070390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fce9800b740 tx=0x7fce98004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea81026b0 msgr2=0x7fcea8114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea81026b0 0x7fcea8114020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce9c002a50 tx=0x7fce9c004600 comp rx=0 tx=0).stop 2024-03-16T09:10:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 shutdown_connections 2024-03-16T09:10:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcea8101d10 0x7fcea8113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fce9006dee0 0x7fce90070390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 --2- 172.21.15.7:0/3342381792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcea81026b0 0x7fcea8114020 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 >> 172.21.15.7:0/3342381792 conn(0x7fcea80f7880 msgr2=0x7fcea80f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 shutdown_connections 2024-03-16T09:10:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.216+0000 7fceaf635700 1 -- 172.21.15.7:0/3342381792 wait complete. 2024-03-16T09:10:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.317+0000 7f7c5b234700 1 -- 172.21.15.7:0/874049178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fe820 msgr2=0x7f7c54106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.317+0000 7f7c5b234700 1 --2- 172.21.15.7:0/874049178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fe820 0x7f7c54106e60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7c40004940 tx=0x7f7c40009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.317+0000 7f7c5b234700 1 -- 172.21.15.7:0/874049178 shutdown_connections 2024-03-16T09:10:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.317+0000 7f7c5b234700 1 --2- 172.21.15.7:0/874049178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fe820 0x7f7c54106e60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.317+0000 7f7c5b234700 1 --2- 172.21.15.7:0/874049178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c540fde80 0x7f7c540fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.317+0000 7f7c5b234700 1 -- 172.21.15.7:0/874049178 >> 172.21.15.7:0/874049178 conn(0x7f7c540f7870 msgr2=0x7f7c540f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.318+0000 7f7c5b234700 1 -- 172.21.15.7:0/874049178 shutdown_connections 2024-03-16T09:10:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.318+0000 7f7c5b234700 1 -- 172.21.15.7:0/874049178 wait complete. 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.318+0000 7f7c5b234700 1 Processor -- start 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.318+0000 7f7c5b234700 1 -- start start 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.318+0000 7f7c5b234700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fde80 0x7f7c54113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.318+0000 7f7c5b234700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c540fe820 0x7f7c54113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.318+0000 7f7c5b234700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c541144c0 con 0x7f7c540fde80 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.318+0000 7f7c5b234700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5410dac0 con 0x7f7c540fe820 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c58fd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fde80 0x7f7c54113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c58fd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fde80 0x7f7c54113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53290/0 (socket says 172.21.15.7:53290) 2024-03-16T09:10:10.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c58fd0700 1 -- 172.21.15.7:0/139581600 learned_addr learned my addr 172.21.15.7:0/139581600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c53fff700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c540fe820 0x7f7c54113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c58fd0700 1 -- 172.21.15.7:0/139581600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c540fe820 msgr2=0x7f7c54113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c58fd0700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c540fe820 0x7f7c54113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c58fd0700 1 -- 172.21.15.7:0/139581600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c40005040 con 0x7f7c540fde80 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c58fd0700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fde80 0x7f7c54113930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c48002a50 tx=0x7f7c480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c517fa700 1 -- 172.21.15.7:0/139581600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c48007cc0 con 0x7f7c540fde80 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c517fa700 1 -- 172.21.15.7:0/139581600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c4800f040 con 0x7f7c540fde80 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c517fa700 1 -- 172.21.15.7:0/139581600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c4800db70 con 0x7f7c540fde80 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5410dda0 con 0x7f7c540fde80 2024-03-16T09:10:10.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.319+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5410e2e0 con 0x7f7c540fde80 2024-03-16T09:10:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.320+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c54066af0 con 0x7f7c540fde80 2024-03-16T09:10:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.321+0000 7f7c517fa700 1 -- 172.21.15.7:0/139581600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f7c4800dd10 con 0x7f7c540fde80 2024-03-16T09:10:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.321+0000 7f7c517fa700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7c4406dee0 0x7f7c44070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.321+0000 7f7c517fa700 1 -- 172.21.15.7:0/139581600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7c4808ce00 con 0x7f7c540fde80 2024-03-16T09:10:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.321+0000 7f7c53fff700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7c4406dee0 0x7f7c44070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:10.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.322+0000 7f7c53fff700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7c4406dee0 0x7f7c44070390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c5410ec80 tx=0x7f7c40009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:10.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.326+0000 7f7c517fa700 1 -- 172.21.15.7:0/139581600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7c48059e20 con 0x7f7c540fde80 2024-03-16T09:10:10.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.630+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7c54109e60 con 0x7f7c540fde80 2024-03-16T09:10:10.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.630+0000 7f7c517fa700 1 -- 172.21.15.7:0/139581600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f7c480599b0 con 0x7f7c540fde80 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:10:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 3, 2024-03-16T09:10:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T09:10:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:10:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:10:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 3, 2024-03-16T09:10:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 11 2024-03-16T09:10:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:10:10.634 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7c4406dee0 msgr2=0x7f7c44070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7c4406dee0 0x7f7c44070390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c5410ec80 tx=0x7f7c40009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fde80 msgr2=0x7f7c54113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fde80 0x7f7c54113930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c48002a50 tx=0x7f7c480046b0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 shutdown_connections 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7c4406dee0 0x7f7c44070390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c540fde80 0x7f7c54113930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 --2- 172.21.15.7:0/139581600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c540fe820 0x7f7c54113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 >> 172.21.15.7:0/139581600 conn(0x7f7c540f7870 msgr2=0x7f7c540f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 shutdown_connections 2024-03-16T09:10:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.632+0000 7f7c5b234700 1 -- 172.21.15.7:0/139581600 wait complete. 2024-03-16T09:10:10.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 -- 172.21.15.7:0/983840249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 msgr2=0x7f73bc0fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:10.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 --2- 172.21.15.7:0/983840249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 0x7f73bc0fe250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009ce0 tx=0x7f73b4004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:10.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 -- 172.21.15.7:0/983840249 shutdown_connections 2024-03-16T09:10:10.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 --2- 172.21.15.7:0/983840249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73bc0fe820 0x7f73bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 --2- 172.21.15.7:0/983840249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 0x7f73bc0fe250 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 -- 172.21.15.7:0/983840249 >> 172.21.15.7:0/983840249 conn(0x7f73bc0f7870 msgr2=0x7f73bc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:10.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 -- 172.21.15.7:0/983840249 shutdown_connections 2024-03-16T09:10:10.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 -- 172.21.15.7:0/983840249 wait complete. 2024-03-16T09:10:10.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.735+0000 7f73c1e7f700 1 Processor -- start 2024-03-16T09:10:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73c1e7f700 1 -- start start 2024-03-16T09:10:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73c1e7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 0x7f73bc0fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73c1e7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73bc0fe820 0x7f73bc0fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73c1e7f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0fdc00 con 0x7f73bc0fde80 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73c1e7f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0fc730 con 0x7f73bc0fe820 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 0x7f73bc0fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 0x7f73bc0fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53320/0 (socket says 172.21.15.7:53320) 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73bb7fe700 1 -- 172.21.15.7:0/14789 learned_addr learned my addr 172.21.15.7:0/14789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73baffd700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73bc0fe820 0x7f73bc0fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73bb7fe700 1 -- 172.21.15.7:0/14789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73bc0fe820 msgr2=0x7f73bc0fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73bb7fe700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73bc0fe820 0x7f73bc0fc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73bb7fe700 1 -- 172.21.15.7:0/14789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b4009940 con 0x7f73bc0fde80 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.736+0000 7f73bb7fe700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 0x7f73bc0fbcb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f73b4007c00 tx=0x7f73b4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.737+0000 7f73b3fff700 1 -- 172.21.15.7:0/14789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73b400e050 con 0x7f73bc0fde80 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.737+0000 7f73b3fff700 1 -- 172.21.15.7:0/14789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73b401d910 con 0x7f73bc0fde80 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.737+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc0fca10 con 0x7f73bc0fde80 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.737+0000 7f73b3fff700 1 -- 172.21.15.7:0/14789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73b40128c0 con 0x7f73bc0fde80 2024-03-16T09:10:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.737+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc11a040 con 0x7f73bc0fde80 2024-03-16T09:10:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.738+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc066b10 con 0x7f73bc0fde80 2024-03-16T09:10:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.738+0000 7f73b3fff700 1 -- 172.21.15.7:0/14789 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f73b4019040 con 0x7f73bc0fde80 2024-03-16T09:10:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.738+0000 7f73b3fff700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f73a806dee0 0x7f73a8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.739+0000 7f73b3fff700 1 -- 172.21.15.7:0/14789 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f73b408f340 con 0x7f73bc0fde80 2024-03-16T09:10:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.739+0000 7f73baffd700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f73a806dee0 0x7f73a8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.739+0000 7f73baffd700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f73a806dee0 0x7f73a8070390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0fd3e0 tx=0x7f73a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:10.743+0000 7f73b3fff700 1 -- 172.21.15.7:0/14789 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f73b40587b0 con 0x7f73bc0fde80 2024-03-16T09:10:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:10 smithi019 ceph-mon[83366]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:10 smithi019 ceph-mon[83366]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:10.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:10 smithi019 ceph-mon[83366]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:10 smithi007 ceph-mon[113546]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:10 smithi007 ceph-mon[113546]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:10.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:10 smithi007 ceph-mon[113546]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.026+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f73bc11a230 con 0x7f73bc0fde80 2024-03-16T09:10:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.026+0000 7f73b3fff700 1 -- 172.21.15.7:0/14789 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f73b405be70 con 0x7f73bc0fde80 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:e24 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:09:58.394215+0000 2024-03-16T09:10:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 80 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-03-16T09:10:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:10:11.031 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:10:11.032 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:10:11.032 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{-1:34372} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/3988729523,v1:172.21.15.7:6829/3988729523] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:10:11.032 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:10:11.032 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6824/5723851,v1:172.21.15.19:6825/5723851] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:10:11.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f73a806dee0 msgr2=0x7f73a8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f73a806dee0 0x7f73a8070390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0fd3e0 tx=0x7f73a400b040 comp rx=0 tx=0).stop 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 msgr2=0x7f73bc0fbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 0x7f73bc0fbcb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f73b4007c00 tx=0x7f73b4004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 shutdown_connections 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f73a806dee0 0x7f73a8070390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73bc0fde80 0x7f73bc0fbcb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 --2- 172.21.15.7:0/14789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73bc0fe820 0x7f73bc0fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 >> 172.21.15.7:0/14789 conn(0x7f73bc0f7870 msgr2=0x7f73bc0f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 shutdown_connections 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.029+0000 7f73c1e7f700 1 -- 172.21.15.7:0/14789 wait complete. 2024-03-16T09:10:11.033 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-16T09:10:11.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.134+0000 7f77ef245700 1 -- 172.21.15.7:0/930469411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 msgr2=0x7f77e8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.134+0000 7f77ef245700 1 --2- 172.21.15.7:0/930469411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 0x7f77e8102140 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f77dc004940 tx=0x7f77dc009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:11.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.134+0000 7f77ef245700 1 -- 172.21.15.7:0/930469411 shutdown_connections 2024-03-16T09:10:11.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.134+0000 7f77ef245700 1 --2- 172.21.15.7:0/930469411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e8102710 0x7f77e80fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.134+0000 7f77ef245700 1 --2- 172.21.15.7:0/930469411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 0x7f77e8102140 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.134+0000 7f77ef245700 1 -- 172.21.15.7:0/930469411 >> 172.21.15.7:0/930469411 conn(0x7f77e8072bc0 msgr2=0x7f77e8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:11.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.134+0000 7f77ef245700 1 -- 172.21.15.7:0/930469411 shutdown_connections 2024-03-16T09:10:11.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.134+0000 7f77ef245700 1 -- 172.21.15.7:0/930469411 wait complete. 2024-03-16T09:10:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ef245700 1 Processor -- start 2024-03-16T09:10:11.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ef245700 1 -- start start 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ef245700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 0x7f77e8113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ef245700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e8102710 0x7f77e81140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ef245700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e81146f0 con 0x7f77e8101d70 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ef245700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e810dae0 con 0x7f77e8102710 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ecfe1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 0x7f77e8113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ecfe1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 0x7f77e8113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53338/0 (socket says 172.21.15.7:53338) 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ecfe1700 1 -- 172.21.15.7:0/2675173482 learned_addr learned my addr 172.21.15.7:0/2675173482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77e7fff700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e8102710 0x7f77e81140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ecfe1700 1 -- 172.21.15.7:0/2675173482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e8102710 msgr2=0x7f77e81140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ecfe1700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e8102710 0x7f77e81140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.135+0000 7f77ecfe1700 1 -- 172.21.15.7:0/2675173482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc005040 con 0x7f77e8101d70 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.136+0000 7f77ecfe1700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 0x7f77e8113b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b3b0 tx=0x7f77dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:11.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.136+0000 7f77e57fa700 1 -- 172.21.15.7:0/2675173482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77dc00e040 con 0x7f77e8101d70 2024-03-16T09:10:11.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.136+0000 7f77e57fa700 1 -- 172.21.15.7:0/2675173482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77dc00bbc0 con 0x7f77e8101d70 2024-03-16T09:10:11.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.136+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e810dd60 con 0x7f77e8101d70 2024-03-16T09:10:11.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.136+0000 7f77e57fa700 1 -- 172.21.15.7:0/2675173482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77dc01c630 con 0x7f77e8101d70 2024-03-16T09:10:11.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.136+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e810e320 con 0x7f77e8101d70 2024-03-16T09:10:11.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.137+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e80ff080 con 0x7f77e8101d70 2024-03-16T09:10:11.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.137+0000 7f77e57fa700 1 -- 172.21.15.7:0/2675173482 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f77dc00bd30 con 0x7f77e8101d70 2024-03-16T09:10:11.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.138+0000 7f77e57fa700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f77d806dee0 0x7f77d8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:11.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.138+0000 7f77e57fa700 1 -- 172.21.15.7:0/2675173482 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f77dc08ea30 con 0x7f77e8101d70 2024-03-16T09:10:11.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.138+0000 7f77e7fff700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f77d806dee0 0x7f77d8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:11.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.138+0000 7f77e7fff700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f77d806dee0 0x7f77d8070390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009fd0 tx=0x7f77d4009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.143+0000 7f77e57fa700 1 -- 172.21.15.7:0/2675173482 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f77dc090e60 con 0x7f77e8101d70 2024-03-16T09:10:11.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.370+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77e810f220 con 0x7f77d806dee0 2024-03-16T09:10:11.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.372+0000 7f77e57fa700 1 -- 172.21.15.7:0/2675173482 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f77e810f220 con 0x7f77d806dee0 2024-03-16T09:10:11.374 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:10:11.374 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:10:11.374 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:10:11.374 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:10:11.374 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "12/21 daemons upgraded", 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:10:11.375 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:10:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f77d806dee0 msgr2=0x7f77d8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f77d806dee0 0x7f77d8070390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009fd0 tx=0x7f77d4009f20 comp rx=0 tx=0).stop 2024-03-16T09:10:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 msgr2=0x7f77e8113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 0x7f77e8113b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b3b0 tx=0x7f77dc009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 shutdown_connections 2024-03-16T09:10:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f77d806dee0 0x7f77d8070390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77e8101d70 0x7f77e8113b60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 --2- 172.21.15.7:0/2675173482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77e8102710 0x7f77e81140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 >> 172.21.15.7:0/2675173482 conn(0x7f77e8072bc0 msgr2=0x7f77e80fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 shutdown_connections 2024-03-16T09:10:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.374+0000 7f77ef245700 1 -- 172.21.15.7:0/2675173482 wait complete. 2024-03-16T09:10:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 -- 172.21.15.7:0/3662437992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc102710 msgr2=0x7fd1dc0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 --2- 172.21.15.7:0/3662437992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc102710 0x7fd1dc0fbd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0004940 tx=0x7fd1d0009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 -- 172.21.15.7:0/3662437992 shutdown_connections 2024-03-16T09:10:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 --2- 172.21.15.7:0/3662437992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc102710 0x7fd1dc0fbd80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 --2- 172.21.15.7:0/3662437992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1dc101d70 0x7fd1dc102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 -- 172.21.15.7:0/3662437992 >> 172.21.15.7:0/3662437992 conn(0x7fd1dc072bc0 msgr2=0x7fd1dc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 -- 172.21.15.7:0/3662437992 shutdown_connections 2024-03-16T09:10:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 -- 172.21.15.7:0/3662437992 wait complete. 2024-03-16T09:10:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 Processor -- start 2024-03-16T09:10:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.477+0000 7fd1e128c700 1 -- start start 2024-03-16T09:10:11.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1e128c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc101d70 0x7fd1dc113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1e128c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1dc102710 0x7fd1dc1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1e128c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc114710 con 0x7fd1dc101d70 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1e128c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc10db20 con 0x7fd1dc102710 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1daffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc101d70 0x7fd1dc113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1daffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc101d70 0x7fd1dc113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53358/0 (socket says 172.21.15.7:53358) 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1daffd700 1 -- 172.21.15.7:0/4122412587 learned_addr learned my addr 172.21.15.7:0/4122412587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1da7fc700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1dc102710 0x7fd1dc1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1daffd700 1 -- 172.21.15.7:0/4122412587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1dc102710 msgr2=0x7fd1dc1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1daffd700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1dc102710 0x7fd1dc1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1daffd700 1 -- 172.21.15.7:0/4122412587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0005040 con 0x7fd1dc101d70 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1da7fc700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1dc102710 0x7fd1dc1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:10:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1daffd700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc101d70 0x7fd1dc113b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc002a50 tx=0x7fd1cc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.478+0000 7fd1c37fe700 1 -- 172.21.15.7:0/4122412587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1cc007cc0 con 0x7fd1dc101d70 2024-03-16T09:10:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.479+0000 7fd1c37fe700 1 -- 172.21.15.7:0/4122412587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1cc00f040 con 0x7fd1dc101d70 2024-03-16T09:10:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.479+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc10de00 con 0x7fd1dc101d70 2024-03-16T09:10:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.479+0000 7fd1c37fe700 1 -- 172.21.15.7:0/4122412587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1cc009640 con 0x7fd1dc101d70 2024-03-16T09:10:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.479+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc10e3c0 con 0x7fd1dc101d70 2024-03-16T09:10:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.480+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc0fd4c0 con 0x7fd1dc101d70 2024-03-16T09:10:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.480+0000 7fd1c37fe700 1 -- 172.21.15.7:0/4122412587 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fd1cc0097c0 con 0x7fd1dc101d70 2024-03-16T09:10:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.480+0000 7fd1c37fe700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd1c406dee0 0x7fd1c4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.480+0000 7fd1c37fe700 1 -- 172.21.15.7:0/4122412587 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd1cc08d520 con 0x7fd1dc101d70 2024-03-16T09:10:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.480+0000 7fd1da7fc700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd1c406dee0 0x7fd1c4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.481+0000 7fd1da7fc700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd1c406dee0 0x7fd1c4070390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009fd0 tx=0x7fd1d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.485+0000 7fd1c37fe700 1 -- 172.21.15.7:0/4122412587 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1cc056990 con 0x7fd1dc101d70 2024-03-16T09:10:11.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:11 smithi007 ceph-mon[113546]: pgmap v175: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:11.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:11 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:11.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:11 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/139581600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:11.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:11 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/14789' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:10:11.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:11 smithi007 ceph-mon[113546]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.782+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd1dc108ee0 con 0x7fd1dc101d70 2024-03-16T09:10:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.783+0000 7fd1c37fe700 1 -- 172.21.15.7:0/4122412587 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd1cc05a050 con 0x7fd1dc101d70 2024-03-16T09:10:11.785 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-16T09:10:11.785 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-16T09:10:11.786 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-03-16T09:10:11.786 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-16T09:10:11.786 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-16T09:10:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd1c406dee0 msgr2=0x7fd1c4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd1c406dee0 0x7fd1c4070390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009fd0 tx=0x7fd1d0009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc101d70 msgr2=0x7fd1dc113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc101d70 0x7fd1dc113b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc002a50 tx=0x7fd1cc0046b0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 shutdown_connections 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd1c406dee0 0x7fd1c4070390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc101d70 0x7fd1dc113b80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 --2- 172.21.15.7:0/4122412587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1dc102710 0x7fd1dc1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 >> 172.21.15.7:0/4122412587 conn(0x7fd1dc072bc0 msgr2=0x7fd1dc0659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 shutdown_connections 2024-03-16T09:10:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:11.785+0000 7fd1e128c700 1 -- 172.21.15.7:0/4122412587 wait complete. 2024-03-16T09:10:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:11 smithi019 ceph-mon[83366]: pgmap v175: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:11 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:11 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/139581600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:11 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/14789' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:10:11.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:11 smithi019 ceph-mon[83366]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:12.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:12 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/4122412587' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:10:12.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:12 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:12.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:12 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:12 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/4122412587' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:10:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:12 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:12.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:12 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:13 smithi007 ceph-mon[113546]: pgmap v176: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:13 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:13 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:14.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:13 smithi019 ceph-mon[83366]: pgmap v176: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:14.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:13 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:14.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:13 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:16.204 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:15 smithi007 ceph-mon[113546]: pgmap v177: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:16.204 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:16.205 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:10:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:15 smithi019 ceph-mon[83366]: pgmap v177: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:16.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:10:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: pgmap v178: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:17.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:17.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:17.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:17.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:17.965 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:17 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: pgmap v178: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:17.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:17.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:17.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:17.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:17.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:17.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:17.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:17.999 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:17 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:19.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:19 smithi007 ceph-mon[113546]: pgmap v179: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:19.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:19 smithi007 ceph-mon[113546]: Upgrade: Updating mds.cephfs.smithi007.znurst 2024-03-16T09:10:19.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:19 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:19.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:19 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.znurst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:10:19.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:19 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:19.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:19 smithi007 ceph-mon[113546]: Deploying daemon mds.cephfs.smithi007.znurst on smithi007 2024-03-16T09:10:19.766 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:19 smithi019 ceph-mon[83366]: pgmap v179: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:19.766 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:19 smithi019 ceph-mon[83366]: Upgrade: Updating mds.cephfs.smithi007.znurst 2024-03-16T09:10:19.766 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:19 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:19.766 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:19 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi007.znurst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:10:19.766 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:19 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:19.766 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:19 smithi019 ceph-mon[83366]: Deploying daemon mds.cephfs.smithi007.znurst on smithi007 2024-03-16T09:10:21.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:21 smithi007 ceph-mon[113546]: pgmap v180: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:21.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:21 smithi019 ceph-mon[83366]: pgmap v180: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:22.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:22 smithi007 ceph-mon[113546]: osdmap e82: 6 total, 6 up, 6 in 2024-03-16T09:10:22.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:22 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-16T09:10:22.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:22 smithi019 ceph-mon[83366]: osdmap e82: 6 total, 6 up, 6 in 2024-03-16T09:10:22.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:22 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-16T09:10:23.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:23 smithi007 ceph-mon[113546]: pgmap v182: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:23.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:23 smithi019 ceph-mon[83366]: pgmap v182: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:25.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:25.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:25.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:25.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:25 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:boot 2024-03-16T09:10:25.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:25 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:25.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:25 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:10:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:25 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:boot 2024-03-16T09:10:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:25 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:25.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:25 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:10:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:26 smithi007 ceph-mon[113546]: pgmap v183: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:26.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:26 smithi019 ceph-mon[83366]: pgmap v183: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:28.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:28 smithi019 ceph-mon[83366]: pgmap v184: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:28.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:28 smithi007 ceph-mon[113546]: pgmap v184: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:29.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:29 smithi007 ceph-mon[113546]: pgmap v185: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:29.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:29 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:29.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:29 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:29 smithi019 ceph-mon[83366]: pgmap v185: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:29 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:29.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:29 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:30.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:30 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:30.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:30 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:30.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:30 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:10:30.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:30.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:30.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:10:31.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:31 smithi007 ceph-mon[113546]: pgmap v186: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:31.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:31 smithi019 ceph-mon[83366]: pgmap v186: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:34.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: pgmap v187: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:34.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:34.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:34.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:34.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:34.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:34.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:34.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:34.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:34.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:34.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:34 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:34.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: pgmap v187: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:34.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:34.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:34.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:34.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:34.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:34.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:34.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:34.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:34.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:34.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:34 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:36.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:35 smithi019 ceph-mon[83366]: pgmap v188: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:36.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:35 smithi019 ceph-mon[83366]: Upgrade: Updating mds.cephfs.smithi019.bncsil 2024-03-16T09:10:36.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:35 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:36.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:35 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.bncsil", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:10:36.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:35 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:36.096 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:35 smithi019 ceph-mon[83366]: Deploying daemon mds.cephfs.smithi019.bncsil on smithi019 2024-03-16T09:10:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:35 smithi007 ceph-mon[113546]: pgmap v188: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:35 smithi007 ceph-mon[113546]: Upgrade: Updating mds.cephfs.smithi019.bncsil 2024-03-16T09:10:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:35 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:35 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.bncsil", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:10:36.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:35 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:36.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:35 smithi007 ceph-mon[113546]: Deploying daemon mds.cephfs.smithi019.bncsil on smithi019 2024-03-16T09:10:38.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:37 smithi019 ceph-mon[83366]: pgmap v189: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:38.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:37 smithi007 ceph-mon[113546]: pgmap v189: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:39.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:38 smithi019 ceph-mon[83366]: osdmap e83: 6 total, 6 up, 6 in 2024-03-16T09:10:39.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:38 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-16T09:10:39.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:38 smithi007 ceph-mon[113546]: osdmap e83: 6 total, 6 up, 6 in 2024-03-16T09:10:39.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:38 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-16T09:10:40.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:39 smithi019 ceph-mon[83366]: pgmap v191: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:40.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:39 smithi007 ceph-mon[113546]: pgmap v191: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:41.450 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:41 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:41.450 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:41 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:41.450 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:41 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:41.450 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:41 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.19:6824/2889841468,v1:172.21.15.19:6825/2889841468] up:boot 2024-03-16T09:10:41.450 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:41 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:41.451 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:41 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:10:41.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:41.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:41.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:41.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:41 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.19:6824/2889841468,v1:172.21.15.19:6825/2889841468] up:boot 2024-03-16T09:10:41.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:41 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:41.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:41 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 -- 172.21.15.7:0/1025290484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 msgr2=0x7f0b941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 --2- 172.21.15.7:0/1025290484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 0x7f0b941031d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009ce0 tx=0x7f0b90004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 -- 172.21.15.7:0/1025290484 shutdown_connections 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 --2- 172.21.15.7:0/1025290484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b941037a0 0x7f0b940fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 --2- 172.21.15.7:0/1025290484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 0x7f0b941031d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 -- 172.21.15.7:0/1025290484 >> 172.21.15.7:0/1025290484 conn(0x7f0b940f89b0 msgr2=0x7f0b940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 -- 172.21.15.7:0/1025290484 shutdown_connections 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 -- 172.21.15.7:0/1025290484 wait complete. 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 Processor -- start 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 -- start start 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.905+0000 7f0b9b40b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 0x7f0b94115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b9b40b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b941037a0 0x7f0b941160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b9b40b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b94116720 con 0x7f0b94102e00 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b9b40b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9410fd00 con 0x7f0b941037a0 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b991a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 0x7f0b94115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b991a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 0x7f0b94115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38930/0 (socket says 172.21.15.7:38930) 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b991a7700 1 -- 172.21.15.7:0/4244061683 learned_addr learned my addr 172.21.15.7:0/4244061683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b989a6700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b941037a0 0x7f0b941160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b991a7700 1 -- 172.21.15.7:0/4244061683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b941037a0 msgr2=0x7f0b941160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b991a7700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b941037a0 0x7f0b941160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b991a7700 1 -- 172.21.15.7:0/4244061683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90009940 con 0x7f0b94102e00 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b991a7700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 0x7f0b94115b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000ba50 tx=0x7f0b90004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b89ffb700 1 -- 172.21.15.7:0/4244061683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b9000e040 con 0x7f0b94102e00 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9410ff80 con 0x7f0b94102e00 2024-03-16T09:10:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b89ffb700 1 -- 172.21.15.7:0/4244061683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b9001cb20 con 0x7f0b94102e00 2024-03-16T09:10:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.906+0000 7f0b89ffb700 1 -- 172.21.15.7:0/4244061683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b90012990 con 0x7f0b94102e00 2024-03-16T09:10:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.907+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b94110540 con 0x7f0b94102e00 2024-03-16T09:10:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.908+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b94107300 con 0x7f0b94102e00 2024-03-16T09:10:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.908+0000 7f0b89ffb700 1 -- 172.21.15.7:0/4244061683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0b9001cc90 con 0x7f0b94102e00 2024-03-16T09:10:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.909+0000 7f0b89ffb700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0b8006dee0 0x7f0b80070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:41.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.909+0000 7f0b89ffb700 1 -- 172.21.15.7:0/4244061683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0b9008fad0 con 0x7f0b94102e00 2024-03-16T09:10:41.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.909+0000 7f0b989a6700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0b8006dee0 0x7f0b80070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:41.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.909+0000 7f0b989a6700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0b8006dee0 0x7f0b80070390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b94111430 tx=0x7f0b8400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:41.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:41.913+0000 7f0b89ffb700 1 -- 172.21.15.7:0/4244061683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b9005ca70 con 0x7f0b94102e00 2024-03-16T09:10:42.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.140+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b94109f00 con 0x7f0b8006dee0 2024-03-16T09:10:42.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.141+0000 7f0b89ffb700 1 -- 172.21.15.7:0/4244061683 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0b94109f00 con 0x7f0b8006dee0 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.143+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0b8006dee0 msgr2=0x7f0b80070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.143+0000 7f0b9b40b700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0b8006dee0 0x7f0b80070390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b94111430 tx=0x7f0b8400b040 comp rx=0 tx=0).stop 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.143+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 msgr2=0x7f0b94115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.143+0000 7f0b9b40b700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 0x7f0b94115b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000ba50 tx=0x7f0b90004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.144+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 shutdown_connections 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.144+0000 7f0b9b40b700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0b8006dee0 0x7f0b80070390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.144+0000 7f0b9b40b700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b94102e00 0x7f0b94115b90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.144+0000 7f0b9b40b700 1 --2- 172.21.15.7:0/4244061683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b941037a0 0x7f0b941160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.144+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 >> 172.21.15.7:0/4244061683 conn(0x7f0b940f89b0 msgr2=0x7f0b940fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:42.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.144+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 shutdown_connections 2024-03-16T09:10:42.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.144+0000 7f0b9b40b700 1 -- 172.21.15.7:0/4244061683 wait complete. 2024-03-16T09:10:42.156 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:10:42.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.247+0000 7f2be3433700 1 -- 172.21.15.7:0/713105031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc0fb3b0 msgr2=0x7f2bdc0fb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.247+0000 7f2be3433700 1 --2- 172.21.15.7:0/713105031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc0fb3b0 0x7f2bdc0fb820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8004940 tx=0x7f2bd8009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.247+0000 7f2be3433700 1 -- 172.21.15.7:0/713105031 shutdown_connections 2024-03-16T09:10:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.247+0000 7f2be3433700 1 --2- 172.21.15.7:0/713105031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc0fb3b0 0x7f2bdc0fb820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.247+0000 7f2be3433700 1 --2- 172.21.15.7:0/713105031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc102f70 0x7f2bdc103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.247+0000 7f2be3433700 1 -- 172.21.15.7:0/713105031 >> 172.21.15.7:0/713105031 conn(0x7f2bdc072310 msgr2=0x7f2bdc072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.247+0000 7f2be3433700 1 -- 172.21.15.7:0/713105031 shutdown_connections 2024-03-16T09:10:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.247+0000 7f2be3433700 1 -- 172.21.15.7:0/713105031 wait complete. 2024-03-16T09:10:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be3433700 1 Processor -- start 2024-03-16T09:10:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be3433700 1 -- start start 2024-03-16T09:10:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be3433700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc0fb3b0 0x7f2bdc0fc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be3433700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc102f70 0x7f2bdc0fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be3433700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc0fe4e0 con 0x7f2bdc102f70 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be3433700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc0fd010 con 0x7f2bdc0fb3b0 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be09ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc102f70 0x7f2bdc0fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be09ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc102f70 0x7f2bdc0fcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38948/0 (socket says 172.21.15.7:38948) 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be11cf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc0fb3b0 0x7f2bdc0fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be09ce700 1 -- 172.21.15.7:0/373709886 learned_addr learned my addr 172.21.15.7:0/373709886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be09ce700 1 -- 172.21.15.7:0/373709886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc0fb3b0 msgr2=0x7f2bdc0fc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be09ce700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc0fb3b0 0x7f2bdc0fc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be09ce700 1 -- 172.21.15.7:0/373709886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8005040 con 0x7f2bdc102f70 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.248+0000 7f2be11cf700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc0fb3b0 0x7f2bdc0fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.249+0000 7f2be09ce700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc102f70 0x7f2bdc0fcad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009b30 tx=0x7f2bd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.249+0000 7f2bcdffb700 1 -- 172.21.15.7:0/373709886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd800e070 con 0x7f2bdc102f70 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.249+0000 7f2bcdffb700 1 -- 172.21.15.7:0/373709886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bd8013c00 con 0x7f2bdc102f70 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.249+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc0fd290 con 0x7f2bdc102f70 2024-03-16T09:10:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.249+0000 7f2bcdffb700 1 -- 172.21.15.7:0/373709886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd800b650 con 0x7f2bdc102f70 2024-03-16T09:10:42.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.249+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc11a030 con 0x7f2bdc102f70 2024-03-16T09:10:42.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.250+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bdc105350 con 0x7f2bdc102f70 2024-03-16T09:10:42.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.251+0000 7f2bcdffb700 1 -- 172.21.15.7:0/373709886 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2bd801ca40 con 0x7f2bdc102f70 2024-03-16T09:10:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.251+0000 7f2bcdffb700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bc806dee0 0x7f2bc8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.251+0000 7f2bcdffb700 1 -- 172.21.15.7:0/373709886 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2bd808f080 con 0x7f2bdc102f70 2024-03-16T09:10:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.251+0000 7f2be11cf700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bc806dee0 0x7f2bc8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.251+0000 7f2be11cf700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bc806dee0 0x7f2bc8070390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009b80 tx=0x7f2bd000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:42.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.256+0000 7f2bcdffb700 1 -- 172.21.15.7:0/373709886 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2bd8058570 con 0x7f2bdc102f70 2024-03-16T09:10:42.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:42 smithi019 ceph-mon[83366]: pgmap v192: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.482+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bdc0fdae0 con 0x7f2bc806dee0 2024-03-16T09:10:42.484 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:42 smithi007 ceph-mon[113546]: pgmap v192: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:42.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.483+0000 7f2bcdffb700 1 -- 172.21.15.7:0/373709886 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2bdc0fdae0 con 0x7f2bc806dee0 2024-03-16T09:10:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bc806dee0 msgr2=0x7f2bc8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bc806dee0 0x7f2bc8070390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009b80 tx=0x7f2bd000b040 comp rx=0 tx=0).stop 2024-03-16T09:10:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc102f70 msgr2=0x7f2bdc0fcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc102f70 0x7f2bdc0fcad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009b30 tx=0x7f2bd8009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 shutdown_connections 2024-03-16T09:10:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bdc0fb3b0 0x7f2bdc0fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bc806dee0 0x7f2bc8070390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 --2- 172.21.15.7:0/373709886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bdc102f70 0x7f2bdc0fcad0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 >> 172.21.15.7:0/373709886 conn(0x7f2bdc072310 msgr2=0x7f2bdc0f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 shutdown_connections 2024-03-16T09:10:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.486+0000 7f2be3433700 1 -- 172.21.15.7:0/373709886 wait complete. 2024-03-16T09:10:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.598+0000 7fa67fcf6700 1 -- 172.21.15.7:0/4096223812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678101d70 msgr2=0x7fa678102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.598+0000 7fa67fcf6700 1 --2- 172.21.15.7:0/4096223812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678101d70 0x7fa678102140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa674004940 tx=0x7fa674009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.598+0000 7fa67fcf6700 1 -- 172.21.15.7:0/4096223812 shutdown_connections 2024-03-16T09:10:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.598+0000 7fa67fcf6700 1 --2- 172.21.15.7:0/4096223812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa678102710 0x7fa6780fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.598+0000 7fa67fcf6700 1 --2- 172.21.15.7:0/4096223812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678101d70 0x7fa678102140 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.598+0000 7fa67fcf6700 1 -- 172.21.15.7:0/4096223812 >> 172.21.15.7:0/4096223812 conn(0x7fa678072bc0 msgr2=0x7fa678072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67fcf6700 1 -- 172.21.15.7:0/4096223812 shutdown_connections 2024-03-16T09:10:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67fcf6700 1 -- 172.21.15.7:0/4096223812 wait complete. 2024-03-16T09:10:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67fcf6700 1 Processor -- start 2024-03-16T09:10:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67fcf6700 1 -- start start 2024-03-16T09:10:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67fcf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa678101d70 0x7fa678113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67fcf6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678102710 0x7fa678114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67fcf6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6781146d0 con 0x7fa678102710 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67fcf6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67810dac0 con 0x7fa678101d70 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67d291700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678102710 0x7fa678114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67d291700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678102710 0x7fa678114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56880/0 (socket says 172.21.15.7:56880) 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.599+0000 7fa67d291700 1 -- 172.21.15.7:0/1324802040 learned_addr learned my addr 172.21.15.7:0/1324802040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa67da92700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa678101d70 0x7fa678113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa67d291700 1 -- 172.21.15.7:0/1324802040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa678101d70 msgr2=0x7fa678113b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa67d291700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa678101d70 0x7fa678113b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa67d291700 1 -- 172.21.15.7:0/1324802040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa674005040 con 0x7fa678102710 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa67d291700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678102710 0x7fa678114080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa668002a80 tx=0x7fa668004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa66e7fc700 1 -- 172.21.15.7:0/1324802040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa668004e80 con 0x7fa678102710 2024-03-16T09:10:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67810dda0 con 0x7fa678102710 2024-03-16T09:10:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa66e7fc700 1 -- 172.21.15.7:0/1324802040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa66800f040 con 0x7fa678102710 2024-03-16T09:10:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa66e7fc700 1 -- 172.21.15.7:0/1324802040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa66800daa0 con 0x7fa678102710 2024-03-16T09:10:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.600+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67810e360 con 0x7fa678102710 2024-03-16T09:10:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.601+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6780fd4c0 con 0x7fa678102710 2024-03-16T09:10:42.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.602+0000 7fa66e7fc700 1 -- 172.21.15.7:0/1324802040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa668007cc0 con 0x7fa678102710 2024-03-16T09:10:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.602+0000 7fa66e7fc700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa66406dee0 0x7fa664070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.602+0000 7fa66e7fc700 1 -- 172.21.15.7:0/1324802040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa66808ce40 con 0x7fa678102710 2024-03-16T09:10:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.602+0000 7fa67da92700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa66406dee0 0x7fa664070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.603+0000 7fa67da92700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa66406dee0 0x7fa664070390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa674004f90 tx=0x7fa674009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.607+0000 7fa66e7fc700 1 -- 172.21.15.7:0/1324802040 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa668059d60 con 0x7fa678102710 2024-03-16T09:10:42.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.819+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6780666a0 con 0x7fa66406dee0 2024-03-16T09:10:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.829+0000 7fa66e7fc700 1 -- 172.21.15.7:0/1324802040 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa6780666a0 con 0x7fa66406dee0 2024-03-16T09:10:42.832 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:10:42.832 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (6m) 14s ago 17m 27.1M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:10:42.832 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (4m) 14s ago 17m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:10:42.832 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (4m) 49s ago 16m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:10:42.832 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (6m) 14s ago 17m 84.6M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:10:42.832 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (36s) 14s ago 12m 12.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b fa6d147fc765 2024-03-16T09:10:42.832 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (19s) 14s ago 12m 12.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b ba11fdc24360 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 starting - - - - 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (12m) 49s ago 12m 15.0M - 17.2.7-648-g1df27247 0bdcfccf14ce de69d5d1f915 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (8m) 14s ago 18m 508M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (7m) 49s ago 16m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (5m) 14s ago 19m 56.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (4m) 49s ago 16m 40.1M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (7m) 14s ago 17m 15.2M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (7m) 49s ago 16m 15.2M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (3m) 14s ago 15m 96.7M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (3m) 14s ago 15m 95.4M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4aa10e800f72 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (2m) 14s ago 14m 87.3M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5449f945991b 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (119s) 49s ago 14m 105M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b bb64247744fe 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (86s) 49s ago 14m 102M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b e1df7d4b9eaa 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (53s) 49s ago 13m 15.0M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5480e37181b1 2024-03-16T09:10:42.833 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (7m) 14s ago 17m 66.7M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:10:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa66406dee0 msgr2=0x7fa664070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa66406dee0 0x7fa664070390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa674004f90 tx=0x7fa674009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678102710 msgr2=0x7fa678114080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678102710 0x7fa678114080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa668002a80 tx=0x7fa668004630 comp rx=0 tx=0).stop 2024-03-16T09:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 shutdown_connections 2024-03-16T09:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa678101d70 0x7fa678113b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa66406dee0 0x7fa664070390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 --2- 172.21.15.7:0/1324802040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa678102710 0x7fa678114080 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 >> 172.21.15.7:0/1324802040 conn(0x7fa678072bc0 msgr2=0x7fa678105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 shutdown_connections 2024-03-16T09:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.832+0000 7fa67fcf6700 1 -- 172.21.15.7:0/1324802040 wait complete. 2024-03-16T09:10:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.933+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/3970170271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc4070bf0 msgr2=0x7f2bc4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.933+0000 7f2bc9e3f700 1 --2- 172.21.15.7:0/3970170271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc4070bf0 0x7f2bc4109190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4004940 tx=0x7f2bb4009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.933+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/3970170271 shutdown_connections 2024-03-16T09:10:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.933+0000 7f2bc9e3f700 1 --2- 172.21.15.7:0/3970170271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc4070bf0 0x7f2bc4109190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.933+0000 7f2bc9e3f700 1 --2- 172.21.15.7:0/3970170271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc40702e0 0x7f2bc40706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.933+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/3970170271 >> 172.21.15.7:0/3970170271 conn(0x7f2bc40f89b0 msgr2=0x7f2bc40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.934+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/3970170271 shutdown_connections 2024-03-16T09:10:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.934+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/3970170271 wait complete. 2024-03-16T09:10:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.934+0000 7f2bc9e3f700 1 Processor -- start 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.934+0000 7f2bc9e3f700 1 -- start start 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc9e3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc40702e0 0x7f2bc4115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc9e3f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc4070bf0 0x7f2bc4116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc9e3f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4116760 con 0x7f2bc40702e0 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc9e3f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc410fd40 con 0x7f2bc4070bf0 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc40702e0 0x7f2bc4115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc40702e0 0x7f2bc4115bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56906/0 (socket says 172.21.15.7:56906) 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc37fe700 1 -- 172.21.15.7:0/582094218 learned_addr learned my addr 172.21.15.7:0/582094218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc2ffd700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc4070bf0 0x7f2bc4116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc37fe700 1 -- 172.21.15.7:0/582094218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc4070bf0 msgr2=0x7f2bc4116110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc37fe700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc4070bf0 0x7f2bc4116110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc37fe700 1 -- 172.21.15.7:0/582094218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb4005040 con 0x7f2bc40702e0 2024-03-16T09:10:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc2ffd700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc4070bf0 0x7f2bc4116110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:10:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc37fe700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc40702e0 0x7f2bc4115bd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c8c0 tx=0x7f2bb80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2babfff700 1 -- 172.21.15.7:0/582094218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb800a980 con 0x7f2bc40702e0 2024-03-16T09:10:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc4110020 con 0x7f2bc40702e0 2024-03-16T09:10:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2babfff700 1 -- 172.21.15.7:0/582094218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bb800ce60 con 0x7f2bc40702e0 2024-03-16T09:10:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2babfff700 1 -- 172.21.15.7:0/582094218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb80115e0 con 0x7f2bc40702e0 2024-03-16T09:10:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.935+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc4110560 con 0x7f2bc40702e0 2024-03-16T09:10:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.936+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc4071a20 con 0x7f2bc40702e0 2024-03-16T09:10:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.937+0000 7f2babfff700 1 -- 172.21.15.7:0/582094218 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2bb8012400 con 0x7f2bc40702e0 2024-03-16T09:10:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.937+0000 7f2babfff700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bac06dee0 0x7f2bac070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.937+0000 7f2babfff700 1 -- 172.21.15.7:0/582094218 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2bb808d900 con 0x7f2bc40702e0 2024-03-16T09:10:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.937+0000 7f2bc2ffd700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bac06dee0 0x7f2bac070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.938+0000 7f2bc2ffd700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bac06dee0 0x7f2bac070390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400b3b0 tx=0x7f2bb4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:42.942+0000 7f2babfff700 1 -- 172.21.15.7:0/582094218 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2bb805a8a0 con 0x7f2bc40702e0 2024-03-16T09:10:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.239+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2bc4111220 con 0x7f2bc40702e0 2024-03-16T09:10:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.240+0000 7f2babfff700 1 -- 172.21.15.7:0/582094218 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f2bb80126b0 con 0x7f2bc40702e0 2024-03-16T09:10:43.242 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:10:43.242 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 1, 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 3 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:10:43.243 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:10:43.244 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 17.2.7-648-g1df27247 (1df27247a676373423b9b5769ede23dff6dccf8f) quincy (stable)": 1, 2024-03-16T09:10:43.244 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 13 2024-03-16T09:10:43.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:10:43.244 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bac06dee0 msgr2=0x7f2bac070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bac06dee0 0x7f2bac070390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400b3b0 tx=0x7f2bb4009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc40702e0 msgr2=0x7f2bc4115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc40702e0 0x7f2bc4115bd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c8c0 tx=0x7f2bb80044c0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 shutdown_connections 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f2bac06dee0 0x7f2bac070390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc40702e0 0x7f2bc4115bd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 --2- 172.21.15.7:0/582094218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bc4070bf0 0x7f2bc4116110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 >> 172.21.15.7:0/582094218 conn(0x7f2bc40f89b0 msgr2=0x7f2bc40f9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 shutdown_connections 2024-03-16T09:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.242+0000 7f2bc9e3f700 1 -- 172.21.15.7:0/582094218 wait complete. 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.345+0000 7f8070b20700 1 -- 172.21.15.7:0/4024967032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c102e00 msgr2=0x7f806c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.345+0000 7f8070b20700 1 --2- 172.21.15.7:0/4024967032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c102e00 0x7f806c1031d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f805c004940 tx=0x7f805c009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 -- 172.21.15.7:0/4024967032 shutdown_connections 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 --2- 172.21.15.7:0/4024967032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806c1037a0 0x7f806c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 --2- 172.21.15.7:0/4024967032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c102e00 0x7f806c1031d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 -- 172.21.15.7:0/4024967032 >> 172.21.15.7:0/4024967032 conn(0x7f806c0f89b0 msgr2=0x7f806c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 -- 172.21.15.7:0/4024967032 shutdown_connections 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 -- 172.21.15.7:0/4024967032 wait complete. 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 Processor -- start 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 -- start start 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806c102e00 0x7f806c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c1037a0 0x7f806c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:43.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c114520 con 0x7f806c1037a0 2024-03-16T09:10:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8070b20700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c10db70 con 0x7f806c102e00 2024-03-16T09:10:43.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.346+0000 7f8069d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c1037a0 0x7f806c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f8069d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c1037a0 0x7f806c113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56922/0 (socket says 172.21.15.7:56922) 2024-03-16T09:10:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f806a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806c102e00 0x7f806c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f8069d9b700 1 -- 172.21.15.7:0/2134392667 learned_addr learned my addr 172.21.15.7:0/2134392667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f8069d9b700 1 -- 172.21.15.7:0/2134392667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806c102e00 msgr2=0x7f806c113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f8069d9b700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806c102e00 0x7f806c113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f8069d9b700 1 -- 172.21.15.7:0/2134392667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c005040 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f806a59c700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806c102e00 0x7f806c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f8069d9b700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c1037a0 0x7f806c113ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f806000c890 tx=0x7f80600044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f805affd700 1 -- 172.21.15.7:0/2134392667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f806000a700 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f805affd700 1 -- 172.21.15.7:0/2134392667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f806000ad30 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806c10de50 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f805affd700 1 -- 172.21.15.7:0/2134392667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80600113a0 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.347+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806c10e410 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.348+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806c100110 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.349+0000 7f805affd700 1 -- 172.21.15.7:0/2134392667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f8060012400 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.349+0000 7f805affd700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f805406dee0 0x7f8054070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.349+0000 7f805affd700 1 -- 172.21.15.7:0/2134392667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f806008d830 con 0x7f806c1037a0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.349+0000 7f806a59c700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f805406dee0 0x7f8054070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.350+0000 7f806a59c700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f805406dee0 0x7f8054070390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f805c004f90 tx=0x7f805c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.354+0000 7f805affd700 1 -- 172.21.15.7:0/2134392667 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f806005a810 con 0x7f806c1037a0 2024-03-16T09:10:43.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:43 smithi007 ceph-mon[113546]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:43.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:43 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/582094218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.610+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f806c10efe0 con 0x7f806c1037a0 2024-03-16T09:10:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.611+0000 7f805affd700 1 -- 172.21.15.7:0/2134392667 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f806005a3a0 con 0x7f806c1037a0 2024-03-16T09:10:43.613 INFO:teuthology.orchestra.run.smithi007.stdout:e28 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-16T09:10:43.614 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:09:58.394215+0000 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 80 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:10:43.615 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:up {} 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:failed 0,1 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:10:43.616 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 0 members: 2024-03-16T09:10:43.617 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:10:43.617 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:10:43.617 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:10:43.617 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:10:43.617 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:24289} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.19:6826/207631863,v1:172.21.15.19:6827/207631863] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T09:10:43.617 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:10:43.617 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{-1:34412} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:10:43.617 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{-1:44303} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6824/2889841468,v1:172.21.15.19:6825/2889841468] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f805406dee0 msgr2=0x7f8054070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f805406dee0 0x7f8054070390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f805c004f90 tx=0x7f805c009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c1037a0 msgr2=0x7f806c113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c1037a0 0x7f806c113ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f806000c890 tx=0x7f80600044c0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 shutdown_connections 2024-03-16T09:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806c102e00 0x7f806c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f805406dee0 0x7f8054070390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 --2- 172.21.15.7:0/2134392667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806c1037a0 0x7f806c113ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 >> 172.21.15.7:0/2134392667 conn(0x7f806c0f89b0 msgr2=0x7f806c1073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 shutdown_connections 2024-03-16T09:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.614+0000 7f8070b20700 1 -- 172.21.15.7:0/2134392667 wait complete. 2024-03-16T09:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-03-16T09:10:43.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:43 smithi019 ceph-mon[83366]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:43.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:43 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/582094218' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.721+0000 7ffb6bc52700 1 -- 172.21.15.7:0/214666232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fde90 msgr2=0x7ffb640fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.721+0000 7ffb6bc52700 1 --2- 172.21.15.7:0/214666232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fde90 0x7ffb640fe260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009ce0 tx=0x7ffb54004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.721+0000 7ffb6bc52700 1 -- 172.21.15.7:0/214666232 shutdown_connections 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.721+0000 7ffb6bc52700 1 --2- 172.21.15.7:0/214666232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb640fe830 0x7ffb64106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.721+0000 7ffb6bc52700 1 --2- 172.21.15.7:0/214666232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fde90 0x7ffb640fe260 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.721+0000 7ffb6bc52700 1 -- 172.21.15.7:0/214666232 >> 172.21.15.7:0/214666232 conn(0x7ffb64072bc0 msgr2=0x7ffb64072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.721+0000 7ffb6bc52700 1 -- 172.21.15.7:0/214666232 shutdown_connections 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.721+0000 7ffb6bc52700 1 -- 172.21.15.7:0/214666232 wait complete. 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb6bc52700 1 Processor -- start 2024-03-16T09:10:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb6bc52700 1 -- start start 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb6bc52700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb640fde90 0x7ffb640fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb6bc52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fe830 0x7ffb640fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb6bc52700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb640fc820 con 0x7ffb640fe830 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb6bc52700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb640fc960 con 0x7ffb640fde90 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb691ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fe830 0x7ffb640fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb691ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fe830 0x7ffb640fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56940/0 (socket says 172.21.15.7:56940) 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb691ed700 1 -- 172.21.15.7:0/3520044463 learned_addr learned my addr 172.21.15.7:0/3520044463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb699ee700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb640fde90 0x7ffb640fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb691ed700 1 -- 172.21.15.7:0/3520044463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb640fde90 msgr2=0x7ffb640fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb691ed700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb640fde90 0x7ffb640fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.722+0000 7ffb691ed700 1 -- 172.21.15.7:0/3520044463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb54009940 con 0x7ffb640fe830 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.723+0000 7ffb691ed700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fe830 0x7ffb640fc290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800eb40 tx=0x7ffb5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.723+0000 7ffb627fc700 1 -- 172.21.15.7:0/3520044463 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb58007cc0 con 0x7ffb640fe830 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.723+0000 7ffb627fc700 1 -- 172.21.15.7:0/3520044463 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb5800f040 con 0x7ffb640fe830 2024-03-16T09:10:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.723+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb640fcbb0 con 0x7ffb640fe830 2024-03-16T09:10:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.723+0000 7ffb627fc700 1 -- 172.21.15.7:0/3520044463 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb5800ca10 con 0x7ffb640fe830 2024-03-16T09:10:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.723+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6411a1e0 con 0x7ffb640fe830 2024-03-16T09:10:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.724+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb64065e30 con 0x7ffb640fe830 2024-03-16T09:10:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.724+0000 7ffb627fc700 1 -- 172.21.15.7:0/3520044463 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7ffb5800cb70 con 0x7ffb640fe830 2024-03-16T09:10:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.725+0000 7ffb627fc700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ffb4c06dee0 0x7ffb4c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.725+0000 7ffb699ee700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ffb4c06dee0 0x7ffb4c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.725+0000 7ffb627fc700 1 -- 172.21.15.7:0/3520044463 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffb5808d270 con 0x7ffb640fe830 2024-03-16T09:10:43.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.725+0000 7ffb699ee700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ffb4c06dee0 0x7ffb4c070390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb54015040 tx=0x7ffb54004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.730+0000 7ffb627fc700 1 -- 172.21.15.7:0/3520044463 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffb58056810 con 0x7ffb640fe830 2024-03-16T09:10:43.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.956+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb6404bed0 con 0x7ffb4c06dee0 2024-03-16T09:10:43.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.962+0000 7ffb627fc700 1 -- 172.21.15.7:0/3520044463 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffb6404bed0 con 0x7ffb4c06dee0 2024-03-16T09:10:43.966 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:10:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T09:10:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": true, 2024-03-16T09:10:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T09:10:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [ 2024-03-16T09:10:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-16T09:10:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "osd", 2024-03-16T09:10:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr", 2024-03-16T09:10:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "mon" 2024-03-16T09:10:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-16T09:10:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": "15/21 daemons upgraded", 2024-03-16T09:10:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "Currently upgrading mds daemons", 2024-03-16T09:10:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:10:43.967 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:10:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.964+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ffb4c06dee0 msgr2=0x7ffb4c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.964+0000 7ffb6bc52700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ffb4c06dee0 0x7ffb4c070390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb54015040 tx=0x7ffb54004d10 comp rx=0 tx=0).stop 2024-03-16T09:10:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.964+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fe830 msgr2=0x7ffb640fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.964+0000 7ffb6bc52700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fe830 0x7ffb640fc290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800eb40 tx=0x7ffb5800b040 comp rx=0 tx=0).stop 2024-03-16T09:10:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.965+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 shutdown_connections 2024-03-16T09:10:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.965+0000 7ffb6bc52700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb640fde90 0x7ffb640fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.965+0000 7ffb6bc52700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7ffb4c06dee0 0x7ffb4c070390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.965+0000 7ffb6bc52700 1 --2- 172.21.15.7:0/3520044463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb640fe830 0x7ffb640fc290 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:43.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.965+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 >> 172.21.15.7:0/3520044463 conn(0x7ffb64072bc0 msgr2=0x7ffb641037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:43.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.965+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 shutdown_connections 2024-03-16T09:10:43.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:43.965+0000 7ffb6bc52700 1 -- 172.21.15.7:0/3520044463 wait complete. 2024-03-16T09:10:44.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.069+0000 7fa197c66700 1 -- 172.21.15.7:0/4278129240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 msgr2=0x7fa1900fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:44.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.069+0000 7fa197c66700 1 --2- 172.21.15.7:0/4278129240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 0x7fa1900fce10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa180004940 tx=0x7fa180009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:44.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.069+0000 7fa197c66700 1 -- 172.21.15.7:0/4278129240 shutdown_connections 2024-03-16T09:10:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.069+0000 7fa197c66700 1 --2- 172.21.15.7:0/4278129240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 0x7fa1900fce10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.069+0000 7fa197c66700 1 --2- 172.21.15.7:0/4278129240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa190102e00 0x7fa1901031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.069+0000 7fa197c66700 1 -- 172.21.15.7:0/4278129240 >> 172.21.15.7:0/4278129240 conn(0x7fa1900f89b0 msgr2=0x7fa1900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.069+0000 7fa197c66700 1 -- 172.21.15.7:0/4278129240 shutdown_connections 2024-03-16T09:10:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.069+0000 7fa197c66700 1 -- 172.21.15.7:0/4278129240 wait complete. 2024-03-16T09:10:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa197c66700 1 Processor -- start 2024-03-16T09:10:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa197c66700 1 -- start start 2024-03-16T09:10:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa197c66700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa190102e00 0x7fa190115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa197c66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 0x7fa190116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa197c66700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa190116680 con 0x7fa1901037a0 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa197c66700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19010fc60 con 0x7fa190102e00 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195201700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 0x7fa190116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195201700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 0x7fa190116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56968/0 (socket says 172.21.15.7:56968) 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195201700 1 -- 172.21.15.7:0/4214967685 learned_addr learned my addr 172.21.15.7:0/4214967685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195a02700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa190102e00 0x7fa190115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195201700 1 -- 172.21.15.7:0/4214967685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa190102e00 msgr2=0x7fa190115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195201700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa190102e00 0x7fa190115af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195201700 1 -- 172.21.15.7:0/4214967685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180005040 con 0x7fa1901037a0 2024-03-16T09:10:44.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195a02700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa190102e00 0x7fa190115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:10:44.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.070+0000 7fa195201700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 0x7fa190116030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa180000c00 tx=0x7fa180009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:44.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.071+0000 7fa1867fc700 1 -- 172.21.15.7:0/4214967685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa180021070 con 0x7fa1901037a0 2024-03-16T09:10:44.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.071+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19010fee0 con 0x7fa1901037a0 2024-03-16T09:10:44.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.071+0000 7fa1867fc700 1 -- 172.21.15.7:0/4214967685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa180013c00 con 0x7fa1901037a0 2024-03-16T09:10:44.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.071+0000 7fa1867fc700 1 -- 172.21.15.7:0/4214967685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa18000e070 con 0x7fa1901037a0 2024-03-16T09:10:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.071+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1901103c0 con 0x7fa1901037a0 2024-03-16T09:10:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.072+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa190101780 con 0x7fa1901037a0 2024-03-16T09:10:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.072+0000 7fa1867fc700 1 -- 172.21.15.7:0/4214967685 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa18001ca60 con 0x7fa1901037a0 2024-03-16T09:10:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.073+0000 7fa1867fc700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa17c06dee0 0x7fa17c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.073+0000 7fa1867fc700 1 -- 172.21.15.7:0/4214967685 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa18008eb90 con 0x7fa1901037a0 2024-03-16T09:10:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.073+0000 7fa195a02700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa17c06dee0 0x7fa17c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.073+0000 7fa195a02700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa17c06dee0 0x7fa17c070390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa18c004fd0 tx=0x7fa18c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.077+0000 7fa1867fc700 1 -- 172.21.15.7:0/4214967685 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa18005bbb0 con 0x7fa1901037a0 2024-03-16T09:10:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.373+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa190111140 con 0x7fa1901037a0 2024-03-16T09:10:44.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.374+0000 7fa1867fc700 1 -- 172.21.15.7:0/4214967685 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa180004490 con 0x7fa1901037a0 2024-03-16T09:10:44.376 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-16T09:10:44.376 INFO:teuthology.orchestra.run.smithi007.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-16T09:10:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is degraded 2024-03-16T09:10:44.376 INFO:teuthology.orchestra.run.smithi007.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-16T09:10:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-16T09:10:44.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa17c06dee0 msgr2=0x7fa17c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:44.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa17c06dee0 0x7fa17c070390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa18c004fd0 tx=0x7fa18c00b040 comp rx=0 tx=0).stop 2024-03-16T09:10:44.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 msgr2=0x7fa190116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:44.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 0x7fa190116030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa180000c00 tx=0x7fa180009e30 comp rx=0 tx=0).stop 2024-03-16T09:10:44.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 shutdown_connections 2024-03-16T09:10:44.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa190102e00 0x7fa190115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:44.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa17c06dee0 0x7fa17c070390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:44.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 --2- 172.21.15.7:0/4214967685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1901037a0 0x7fa190116030 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:44.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 >> 172.21.15.7:0/4214967685 conn(0x7fa1900f89b0 msgr2=0x7fa1900fa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:44.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 shutdown_connections 2024-03-16T09:10:44.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:10:44.376+0000 7fa197c66700 1 -- 172.21.15.7:0/4214967685 wait complete. 2024-03-16T09:10:44.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:44 smithi019 ceph-mon[83366]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:44 smithi019 ceph-mon[83366]: pgmap v193: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:44 smithi019 ceph-mon[83366]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:44 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:44 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2134392667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:10:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:44 smithi007 ceph-mon[113546]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:44 smithi007 ceph-mon[113546]: pgmap v193: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:44 smithi007 ceph-mon[113546]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:44 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:44 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2134392667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:10:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:45 smithi019 ceph-mon[83366]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:45 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/4214967685' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:10:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:45 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:45.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:45 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:45.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:45 smithi007 ceph-mon[113546]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:45.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:45 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/4214967685' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:10:45.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:45 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:45.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:45 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:46.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:46 smithi019 ceph-mon[83366]: pgmap v194: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:46.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:10:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:46 smithi007 ceph-mon[113546]: pgmap v194: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:10:48.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: pgmap v195: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:48.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:48.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:48.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:48.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:48.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:48.647 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:48.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:48.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:48.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:48.648 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:48 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: pgmap v195: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:48.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:48 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:10:50.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:50 smithi019 ceph-mon[83366]: pgmap v196: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:50.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:50 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:50.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:50 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.gmhzgo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:10:50.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:50 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:50.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:50 smithi007 ceph-mon[113546]: pgmap v196: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:50.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:50 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:50.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:50 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi019.gmhzgo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:10:50.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:50 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:51.674 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:51 smithi019 ceph-mon[83366]: Upgrade: Updating mds.cephfs.smithi019.gmhzgo 2024-03-16T09:10:51.674 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:51 smithi019 ceph-mon[83366]: Deploying daemon mds.cephfs.smithi019.gmhzgo on smithi019 2024-03-16T09:10:51.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:51 smithi007 ceph-mon[113546]: Upgrade: Updating mds.cephfs.smithi019.gmhzgo 2024-03-16T09:10:51.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:51 smithi007 ceph-mon[113546]: Deploying daemon mds.cephfs.smithi019.gmhzgo on smithi019 2024-03-16T09:10:52.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:52 smithi019 ceph-mon[83366]: pgmap v197: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:52.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:52 smithi007 ceph-mon[113546]: pgmap v197: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:53.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:53 smithi019 ceph-mon[83366]: osdmap e84: 6 total, 6 up, 6 in 2024-03-16T09:10:53.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:53 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-16T09:10:53.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:53 smithi007 ceph-mon[113546]: osdmap e84: 6 total, 6 up, 6 in 2024-03-16T09:10:53.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:53 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-16T09:10:54.523 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:54 smithi019 ceph-mon[83366]: pgmap v199: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:54 smithi007 ceph-mon[113546]: pgmap v199: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:56.682 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:56 smithi019 ceph-mon[83366]: pgmap v200: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:56.683 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:56 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:56.683 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:56 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:56.683 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:56 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:56 smithi007 ceph-mon[113546]: pgmap v200: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:56 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:56 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:56 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:57.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:57 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.19:6826/3039505467,v1:172.21.15.19:6827/3039505467] up:boot 2024-03-16T09:10:57.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:57 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:57.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:57 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:10:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:57 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.19:6826/3039505467,v1:172.21.15.19:6827/3039505467] up:boot 2024-03-16T09:10:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:57 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:10:57.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:57 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:10:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:58 smithi019 ceph-mon[83366]: pgmap v201: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:58.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:10:58 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:58.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:58 smithi007 ceph-mon[113546]: pgmap v201: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:10:58.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:10:58.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:10:58 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:00.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:00 smithi019 ceph-mon[83366]: pgmap v202: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:00.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:00.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:00.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:00 smithi007 ceph-mon[113546]: pgmap v202: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:00.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:00.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:01.462 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:11:01.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:11:02.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:02 smithi019 ceph-mon[83366]: pgmap v203: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:02.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:02 smithi007 ceph-mon[113546]: pgmap v203: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: pgmap v204: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all mds 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.bsxctp"}]': finished 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:11:04.214 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.znurst"}]': finished 2024-03-16T09:11:04.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:11:04.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.bncsil"}]': finished 2024-03-16T09:11:04.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:11:04.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.gmhzgo"}]': finished 2024-03-16T09:11:04.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: Upgrade: Setting filesystem cephfs Joinable 2024-03-16T09:11:04.215 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:03 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-16T09:11:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: pgmap v204: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all mds 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:04.249 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.bsxctp"}]: dispatch 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.bsxctp"}]': finished 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.znurst"}]: dispatch 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi007.znurst"}]': finished 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.bncsil"}]: dispatch 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.bncsil"}]': finished 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.gmhzgo"}]: dispatch 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi019.gmhzgo"}]': finished 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: Upgrade: Setting filesystem cephfs Joinable 2024-03-16T09:11:04.250 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:03 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Standby daemon mds.cephfs.smithi007.bsxctp assigned to filesystem cephfs as rank 0 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Standby daemon mds.cephfs.smithi007.znurst assigned to filesystem cephfs as rank 1 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: fsmap cephfs:2/2 {0=cephfs.smithi007.bsxctp=up:replay,1=cephfs.smithi007.znurst=up:replay} 2 up:standby 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all rgw 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all rbd-mirror 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.464 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all ceph-exporter 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all iscsi 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all nfs 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Upgrade: Setting container_image for all nvmeof 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.465 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Upgrade: Finalizing container_image settings 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-16T09:11:05.466 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.467 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: Upgrade: Complete! 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:05.468 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Standby daemon mds.cephfs.smithi007.bsxctp assigned to filesystem cephfs as rank 0 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Standby daemon mds.cephfs.smithi007.znurst assigned to filesystem cephfs as rank 1 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: fsmap cephfs:2/2 {0=cephfs.smithi007.bsxctp=up:replay,1=cephfs.smithi007.znurst=up:replay} 2 up:standby 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all rgw 2024-03-16T09:11:05.499 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all rbd-mirror 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all ceph-exporter 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all iscsi 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all nfs 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.500 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Upgrade: Setting container_image for all nvmeof 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Upgrade: Finalizing container_image settings 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-16T09:11:05.501 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-16T09:11:05.502 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-16T09:11:05.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-16T09:11:05.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-16T09:11:05.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-16T09:11:05.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-16T09:11:05.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-16T09:11:05.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.503 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: Upgrade: Complete! 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:05.504 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:06 smithi019 ceph-mon[83366]: pgmap v205: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.7 MiB/s rd, 1 op/s 2024-03-16T09:11:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:06 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:06.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:06 smithi007 ceph-mon[113546]: pgmap v205: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.7 MiB/s rd, 1 op/s 2024-03-16T09:11:06.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:06 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:08.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:08 smithi019 ceph-mon[83366]: pgmap v206: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 7.3 MiB/s rd, 2 op/s 2024-03-16T09:11:08.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:08 smithi007 ceph-mon[113546]: pgmap v206: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 7.3 MiB/s rd, 2 op/s 2024-03-16T09:11:10.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:10 smithi019 ceph-mon[83366]: pgmap v207: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-03-16T09:11:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:10 smithi007 ceph-mon[113546]: pgmap v207: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-03-16T09:11:12.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:12 smithi019 ceph-mon[83366]: pgmap v208: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-03-16T09:11:12.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:12 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:resolve 2024-03-16T09:11:12.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:12 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:resolve 2024-03-16T09:11:12.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:12 smithi019 ceph-mon[83366]: fsmap cephfs:2/2 {0=cephfs.smithi007.bsxctp=up:resolve,1=cephfs.smithi007.znurst=up:resolve} 2 up:standby 2024-03-16T09:11:12.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:12 smithi007 ceph-mon[113546]: pgmap v208: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-03-16T09:11:12.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:12 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:resolve 2024-03-16T09:11:12.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:12 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:resolve 2024-03-16T09:11:12.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:12 smithi007 ceph-mon[113546]: fsmap cephfs:2/2 {0=cephfs.smithi007.bsxctp=up:resolve,1=cephfs.smithi007.znurst=up:resolve} 2 up:standby 2024-03-16T09:11:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:13 smithi019 ceph-mon[83366]: reconnect by client.24329 v1:172.21.15.7:0/34442062 after 0.000999986 2024-03-16T09:11:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:13 smithi019 ceph-mon[83366]: reconnect by client.24329 v1:172.21.15.7:0/34442062 after 0.000999986 2024-03-16T09:11:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:13 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:reconnect 2024-03-16T09:11:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:13 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:reconnect 2024-03-16T09:11:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:13 smithi019 ceph-mon[83366]: fsmap cephfs:2/2 {0=cephfs.smithi007.bsxctp=up:reconnect,1=cephfs.smithi007.znurst=up:reconnect} 2 up:standby 2024-03-16T09:11:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:13 smithi019 ceph-mon[83366]: reconnect by client.14540 v1:172.21.15.19:0/480350363 after 0.00499993 2024-03-16T09:11:13.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:13 smithi019 ceph-mon[83366]: reconnect by client.14540 v1:172.21.15.19:0/480350363 after 0.00499993 2024-03-16T09:11:13.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:13 smithi007 ceph-mon[113546]: reconnect by client.24329 v1:172.21.15.7:0/34442062 after 0.000999986 2024-03-16T09:11:13.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:13 smithi007 ceph-mon[113546]: reconnect by client.24329 v1:172.21.15.7:0/34442062 after 0.000999986 2024-03-16T09:11:13.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:13 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:reconnect 2024-03-16T09:11:13.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:13 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:reconnect 2024-03-16T09:11:13.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:13 smithi007 ceph-mon[113546]: fsmap cephfs:2/2 {0=cephfs.smithi007.bsxctp=up:reconnect,1=cephfs.smithi007.znurst=up:reconnect} 2 up:standby 2024-03-16T09:11:13.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:13 smithi007 ceph-mon[113546]: reconnect by client.14540 v1:172.21.15.19:0/480350363 after 0.00499993 2024-03-16T09:11:13.749 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:13 smithi007 ceph-mon[113546]: reconnect by client.14540 v1:172.21.15.19:0/480350363 after 0.00499993 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.501+0000 7fc842857700 1 -- 172.21.15.7:0/157232400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 msgr2=0x7fc83c0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.501+0000 7fc842857700 1 --2- 172.21.15.7:0/157232400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 0x7fc83c0fe1e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc82c004940 tx=0x7fc82c009e30 comp rx=0 tx=0).stop 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.501+0000 7fc842857700 1 -- 172.21.15.7:0/157232400 shutdown_connections 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.501+0000 7fc842857700 1 --2- 172.21.15.7:0/157232400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c0fe7b0 0x7fc83c106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.501+0000 7fc842857700 1 --2- 172.21.15.7:0/157232400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 0x7fc83c0fe1e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.501+0000 7fc842857700 1 -- 172.21.15.7:0/157232400 >> 172.21.15.7:0/157232400 conn(0x7fc83c0f7880 msgr2=0x7fc83c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.501+0000 7fc842857700 1 -- 172.21.15.7:0/157232400 shutdown_connections 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.501+0000 7fc842857700 1 -- 172.21.15.7:0/157232400 wait complete. 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc842857700 1 Processor -- start 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc842857700 1 -- start start 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc842857700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 0x7fc83c113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc842857700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c0fe7b0 0x7fc83c114380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc842857700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c114a60 con 0x7fc83c0fde10 2024-03-16T09:11:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc842857700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c10dde0 con 0x7fc83c0fe7b0 2024-03-16T09:11:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 0x7fc83c113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 0x7fc83c113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51280/0 (socket says 172.21.15.7:51280) 2024-03-16T09:11:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc83bfff700 1 -- 172.21.15.7:0/1862808826 learned_addr learned my addr 172.21.15.7:0/1862808826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:11:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.502+0000 7fc83b7fe700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c0fe7b0 0x7fc83c114380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc83bfff700 1 -- 172.21.15.7:0/1862808826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c0fe7b0 msgr2=0x7fc83c114380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc83bfff700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c0fe7b0 0x7fc83c114380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc83bfff700 1 -- 172.21.15.7:0/1862808826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c005040 con 0x7fc83c0fde10 2024-03-16T09:11:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc83bfff700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 0x7fc83c113e40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009180 tx=0x7fc82c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc838ff9700 1 -- 172.21.15.7:0/1862808826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc82c00e040 con 0x7fc83c0fde10 2024-03-16T09:11:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc838ff9700 1 -- 172.21.15.7:0/1862808826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc82c013420 con 0x7fc83c0fde10 2024-03-16T09:11:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83c10e060 con 0x7fc83c0fde10 2024-03-16T09:11:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc838ff9700 1 -- 172.21.15.7:0/1862808826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc82c01c630 con 0x7fc83c0fde10 2024-03-16T09:11:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.503+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83c10e5a0 con 0x7fc83c0fde10 2024-03-16T09:11:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.504+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83c1016b0 con 0x7fc83c0fde10 2024-03-16T09:11:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.505+0000 7fc838ff9700 1 -- 172.21.15.7:0/1862808826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc82c013590 con 0x7fc83c0fde10 2024-03-16T09:11:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.505+0000 7fc838ff9700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc82406def0 0x7fc8240703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.505+0000 7fc838ff9700 1 -- 172.21.15.7:0/1862808826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc82c08eb20 con 0x7fc83c0fde10 2024-03-16T09:11:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.505+0000 7fc83b7fe700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc82406def0 0x7fc8240703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.506+0000 7fc83b7fe700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc82406def0 0x7fc8240703a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc830006fd0 tx=0x7fc830008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:14.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.510+0000 7fc838ff9700 1 -- 172.21.15.7:0/1862808826 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc82c058010 con 0x7fc83c0fde10 2024-03-16T09:11:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:14 smithi019 ceph-mon[83366]: pgmap v209: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-03-16T09:11:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:14 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:rejoin 2024-03-16T09:11:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:14 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:rejoin 2024-03-16T09:11:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:14 smithi019 ceph-mon[83366]: fsmap cephfs:2/2 {0=cephfs.smithi007.bsxctp=up:rejoin,1=cephfs.smithi007.znurst=up:rejoin} 2 up:standby 2024-03-16T09:11:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:14 smithi019 ceph-mon[83366]: daemon mds.cephfs.smithi007.bsxctp is now active in filesystem cephfs as rank 0 2024-03-16T09:11:14.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:14 smithi019 ceph-mon[83366]: daemon mds.cephfs.smithi007.znurst is now active in filesystem cephfs as rank 1 2024-03-16T09:11:14.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.736+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc83c109f90 con 0x7fc82406def0 2024-03-16T09:11:14.739 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:14 smithi007 ceph-mon[113546]: pgmap v209: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-03-16T09:11:14.739 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:14 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:rejoin 2024-03-16T09:11:14.739 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:14 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:rejoin 2024-03-16T09:11:14.739 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:14 smithi007 ceph-mon[113546]: fsmap cephfs:2/2 {0=cephfs.smithi007.bsxctp=up:rejoin,1=cephfs.smithi007.znurst=up:rejoin} 2 up:standby 2024-03-16T09:11:14.739 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:14 smithi007 ceph-mon[113546]: daemon mds.cephfs.smithi007.bsxctp is now active in filesystem cephfs as rank 0 2024-03-16T09:11:14.739 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:14 smithi007 ceph-mon[113546]: daemon mds.cephfs.smithi007.znurst is now active in filesystem cephfs as rank 1 2024-03-16T09:11:14.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.737+0000 7fc838ff9700 1 -- 172.21.15.7:0/1862808826 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fc83c109f90 con 0x7fc82406def0 2024-03-16T09:11:14.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc82406def0 msgr2=0x7fc8240703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:14.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc82406def0 0x7fc8240703a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc830006fd0 tx=0x7fc830008040 comp rx=0 tx=0).stop 2024-03-16T09:11:14.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 msgr2=0x7fc83c113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:14.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 0x7fc83c113e40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009180 tx=0x7fc82c009e30 comp rx=0 tx=0).stop 2024-03-16T09:11:14.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 shutdown_connections 2024-03-16T09:11:14.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc82406def0 0x7fc8240703a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:14.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0fde10 0x7fc83c113e40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:14.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 --2- 172.21.15.7:0/1862808826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c0fe7b0 0x7fc83c114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:14.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 >> 172.21.15.7:0/1862808826 conn(0x7fc83c0f7880 msgr2=0x7fc83c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:14.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 shutdown_connections 2024-03-16T09:11:14.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:14.740+0000 7fc842857700 1 -- 172.21.15.7:0/1862808826 wait complete. 2024-03-16T09:11:15.638 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:15 smithi019 ceph-mon[83366]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T09:11:15.638 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:15 smithi019 ceph-mon[83366]: Cluster is now healthy 2024-03-16T09:11:15.638 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:15 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:active 2024-03-16T09:11:15.638 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:15 smithi019 ceph-mon[83366]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:active 2024-03-16T09:11:15.638 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:15 smithi019 ceph-mon[83366]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi007.znurst=up:active} 2 up:standby 2024-03-16T09:11:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:15 smithi007 ceph-mon[113546]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T09:11:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:15 smithi007 ceph-mon[113546]: Cluster is now healthy 2024-03-16T09:11:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:15 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] up:active 2024-03-16T09:11:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:15 smithi007 ceph-mon[113546]: mds.? [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] up:active 2024-03-16T09:11:15.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:15 smithi007 ceph-mon[113546]: fsmap cephfs:2 {0=cephfs.smithi007.bsxctp=up:active,1=cephfs.smithi007.znurst=up:active} 2 up:standby 2024-03-16T09:11:16.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:16 smithi019 ceph-mon[83366]: pgmap v210: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 170 B/s wr, 10 op/s 2024-03-16T09:11:16.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:16 smithi019 ceph-mon[83366]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:16.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:16 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:16.735 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:16 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:11:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:16 smithi007 ceph-mon[113546]: pgmap v210: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 170 B/s wr, 10 op/s 2024-03-16T09:11:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:16 smithi007 ceph-mon[113546]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:16 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:11:16.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:16 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:11:18.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:18 smithi007 ceph-mon[113546]: pgmap v211: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 341 B/s wr, 10 op/s 2024-03-16T09:11:18.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:18 smithi019 ceph-mon[83366]: pgmap v211: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 341 B/s wr, 10 op/s 2024-03-16T09:11:20.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch ps' 2024-03-16T09:11:20.637 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:11:20.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:20 smithi019 ceph-mon[83366]: pgmap v212: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-03-16T09:11:20.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:20 smithi007 ceph-mon[113546]: pgmap v212: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-03-16T09:11:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.260+0000 7f8c0df50700 1 -- 172.21.15.7:0/581832647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fe830 msgr2=0x7f8c08106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.260+0000 7f8c0df50700 1 --2- 172.21.15.7:0/581832647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fe830 0x7f8c08106f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc004940 tx=0x7f8bfc009e30 comp rx=0 tx=0).stop 2024-03-16T09:11:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.260+0000 7f8c0df50700 1 -- 172.21.15.7:0/581832647 shutdown_connections 2024-03-16T09:11:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.260+0000 7f8c0df50700 1 --2- 172.21.15.7:0/581832647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fe830 0x7f8c08106f00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.260+0000 7f8c0df50700 1 --2- 172.21.15.7:0/581832647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c080fde90 0x7f8c080fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.260+0000 7f8c0df50700 1 -- 172.21.15.7:0/581832647 >> 172.21.15.7:0/581832647 conn(0x7f8c08072bc0 msgr2=0x7f8c08072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.260+0000 7f8c0df50700 1 -- 172.21.15.7:0/581832647 shutdown_connections 2024-03-16T09:11:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.260+0000 7f8c0df50700 1 -- 172.21.15.7:0/581832647 wait complete. 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.261+0000 7f8c0df50700 1 Processor -- start 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.261+0000 7f8c0df50700 1 -- start start 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.261+0000 7f8c0df50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fde90 0x7f8c08113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.261+0000 7f8c0df50700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c080fe830 0x7f8c081140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.261+0000 7f8c0df50700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08114730 con 0x7f8c080fde90 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.261+0000 7f8c0df50700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0810db20 con 0x7f8c080fe830 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fde90 0x7f8c08113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fde90 0x7f8c08113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51286/0 (socket says 172.21.15.7:51286) 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c077fe700 1 -- 172.21.15.7:0/1883675690 learned_addr learned my addr 172.21.15.7:0/1883675690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:11:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c06ffd700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c080fe830 0x7f8c081140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c077fe700 1 -- 172.21.15.7:0/1883675690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c080fe830 msgr2=0x7f8c081140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c077fe700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c080fe830 0x7f8c081140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c077fe700 1 -- 172.21.15.7:0/1883675690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc005040 con 0x7f8c080fde90 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c077fe700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fde90 0x7f8c08113ba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8002a50 tx=0x7f8bf80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8beffff700 1 -- 172.21.15.7:0/1883675690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf8007cc0 con 0x7f8c080fde90 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0810de00 con 0x7f8c080fde90 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8beffff700 1 -- 172.21.15.7:0/1883675690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bf800f040 con 0x7f8c080fde90 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8beffff700 1 -- 172.21.15.7:0/1883675690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf800da30 con 0x7f8c080fde90 2024-03-16T09:11:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.262+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0810e310 con 0x7f8c080fde90 2024-03-16T09:11:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.264+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c08065e30 con 0x7f8c080fde90 2024-03-16T09:11:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.264+0000 7f8beffff700 1 -- 172.21.15.7:0/1883675690 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f8bf800db90 con 0x7f8c080fde90 2024-03-16T09:11:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.264+0000 7f8beffff700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8bf006def0 0x7f8bf00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.265+0000 7f8beffff700 1 -- 172.21.15.7:0/1883675690 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8bf808d520 con 0x7f8c080fde90 2024-03-16T09:11:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.265+0000 7f8c06ffd700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8bf006def0 0x7f8bf00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:22.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.265+0000 7f8c06ffd700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8bf006def0 0x7f8bf00703a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c0810ee00 tx=0x7f8bfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:22.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.273+0000 7f8beffff700 1 -- 172.21.15.7:0/1883675690 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8bf805a4c0 con 0x7f8c080fde90 2024-03-16T09:11:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.593+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c08109f10 con 0x7f8bf006def0 2024-03-16T09:11:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.604+0000 7f8beffff700 1 -- 172.21.15.7:0/1883675690 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8c08109f10 con 0x7f8bf006def0 2024-03-16T09:11:22.606 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:11:22.606 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (7m) 54s ago 18m 27.1M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:11:22.606 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (4m) 54s ago 18m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:11:22.606 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (4m) 24s ago 17m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:11:22.606 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (7m) 54s ago 18m 84.6M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (76s) 54s ago 12m 12.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b fa6d147fc765 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (59s) 54s ago 12m 12.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b ba11fdc24360 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (42s) 24s ago 12m 12.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 93b12a56fe52 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (27s) 24s ago 12m 10.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b de8a59e6e71b 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (9m) 54s ago 19m 508M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (8m) 24s ago 17m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (6m) 54s ago 19m 56.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (5m) 24s ago 17m 42.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (8m) 54s ago 18m 15.2M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:11:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (8m) 24s ago 17m 15.7M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:11:22.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (4m) 54s ago 16m 96.7M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:11:22.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (3m) 54s ago 16m 95.4M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4aa10e800f72 2024-03-16T09:11:22.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (3m) 54s ago 15m 87.3M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5449f945991b 2024-03-16T09:11:22.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (2m) 24s ago 15m 106M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b bb64247744fe 2024-03-16T09:11:22.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (2m) 24s ago 14m 103M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b e1df7d4b9eaa 2024-03-16T09:11:22.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (92s) 24s ago 14m 95.8M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5480e37181b1 2024-03-16T09:11:22.608 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (7m) 54s ago 17m 66.7M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8bf006def0 msgr2=0x7f8bf00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8bf006def0 0x7f8bf00703a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c0810ee00 tx=0x7f8bfc009e30 comp rx=0 tx=0).stop 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fde90 msgr2=0x7f8c08113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fde90 0x7f8c08113ba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8002a50 tx=0x7f8bf80046b0 comp rx=0 tx=0).stop 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 shutdown_connections 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f8bf006def0 0x7f8bf00703a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c080fde90 0x7f8c08113ba0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 --2- 172.21.15.7:0/1883675690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c080fe830 0x7f8c081140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 >> 172.21.15.7:0/1883675690 conn(0x7f8c08072bc0 msgr2=0x7f8c081037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 shutdown_connections 2024-03-16T09:11:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:22.606+0000 7f8c0df50700 1 -- 172.21.15.7:0/1883675690 wait complete. 2024-03-16T09:11:22.621 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:22 smithi007 ceph-mon[113546]: pgmap v213: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-03-16T09:11:22.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:22 smithi019 ceph-mon[83366]: pgmap v213: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-03-16T09:11:23.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade status' 2024-03-16T09:11:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:11:24.664 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:24 smithi007 ceph-mon[113546]: pgmap v214: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-03-16T09:11:24.664 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:24 smithi007 ceph-mon[113546]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:24.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:24 smithi019 ceph-mon[83366]: pgmap v214: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-03-16T09:11:24.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:24 smithi019 ceph-mon[83366]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.185+0000 7f841c891700 1 -- 172.21.15.7:0/3698400959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 msgr2=0x7f84180fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.185+0000 7f841c891700 1 --2- 172.21.15.7:0/3698400959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 0x7f84180fe260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8408004940 tx=0x7f8408009e30 comp rx=0 tx=0).stop 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.185+0000 7f841c891700 1 -- 172.21.15.7:0/3698400959 shutdown_connections 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.185+0000 7f841c891700 1 --2- 172.21.15.7:0/3698400959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84180fe830 0x7f8418106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.185+0000 7f841c891700 1 --2- 172.21.15.7:0/3698400959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 0x7f84180fe260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.185+0000 7f841c891700 1 -- 172.21.15.7:0/3698400959 >> 172.21.15.7:0/3698400959 conn(0x7f8418072bc0 msgr2=0x7f8418072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.185+0000 7f841c891700 1 -- 172.21.15.7:0/3698400959 shutdown_connections 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.185+0000 7f841c891700 1 -- 172.21.15.7:0/3698400959 wait complete. 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841c891700 1 Processor -- start 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841c891700 1 -- start start 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841c891700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 0x7f8418113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841c891700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84180fe830 0x7f84181140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841c891700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418114720 con 0x7f84180fde90 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841c891700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841810db30 con 0x7f84180fe830 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 0x7f8418113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 0x7f8418113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37226/0 (socket says 172.21.15.7:37226) 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841659c700 1 -- 172.21.15.7:0/449239802 learned_addr learned my addr 172.21.15.7:0/449239802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f8415d9b700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84180fe830 0x7f84181140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841659c700 1 -- 172.21.15.7:0/449239802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84180fe830 msgr2=0x7f84181140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841659c700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84180fe830 0x7f84181140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.186+0000 7f841659c700 1 -- 172.21.15.7:0/449239802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408005040 con 0x7f84180fde90 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.187+0000 7f841659c700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 0x7f8418113b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84080093b0 tx=0x7f8408009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.187+0000 7f8406ffd700 1 -- 172.21.15.7:0/449239802 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f840800e040 con 0x7f84180fde90 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.187+0000 7f8406ffd700 1 -- 172.21.15.7:0/449239802 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8408004290 con 0x7f84180fde90 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.187+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841810ddb0 con 0x7f84180fde90 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.187+0000 7f8406ffd700 1 -- 172.21.15.7:0/449239802 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f840801c3f0 con 0x7f84180fde90 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.187+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841810e2f0 con 0x7f84180fde90 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.188+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8418101980 con 0x7f84180fde90 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.188+0000 7f8406ffd700 1 -- 172.21.15.7:0/449239802 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f8408013890 con 0x7f84180fde90 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.189+0000 7f8406ffd700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f840006def0 0x7f84000703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.189+0000 7f8406ffd700 1 -- 172.21.15.7:0/449239802 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f840808ea30 con 0x7f84180fde90 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.189+0000 7f8415d9b700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f840006def0 0x7f84000703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.189+0000 7f8415d9b700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f840006def0 0x7f84000703a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f841810f430 tx=0x7f840c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.194+0000 7f8406ffd700 1 -- 172.21.15.7:0/449239802 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8408090e60 con 0x7f84180fde90 2024-03-16T09:11:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.452+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8418109f00 con 0x7f840006def0 2024-03-16T09:11:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.453+0000 7f8406ffd700 1 -- 172.21.15.7:0/449239802 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f8418109f00 con 0x7f840006def0 2024-03-16T09:11:25.455 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:11:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "target_image": null, 2024-03-16T09:11:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "in_progress": false, 2024-03-16T09:11:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "which": "", 2024-03-16T09:11:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "services_complete": [], 2024-03-16T09:11:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "progress": null, 2024-03-16T09:11:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-03-16T09:11:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "is_paused": false 2024-03-16T09:11:25.456 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:11:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f840006def0 msgr2=0x7f84000703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f840006def0 0x7f84000703a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f841810f430 tx=0x7f840c00b040 comp rx=0 tx=0).stop 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 msgr2=0x7f8418113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 0x7f8418113b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84080093b0 tx=0x7f8408009e30 comp rx=0 tx=0).stop 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 shutdown_connections 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f840006def0 0x7f84000703a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84180fde90 0x7f8418113b90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 --2- 172.21.15.7:0/449239802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84180fe830 0x7f84181140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 >> 172.21.15.7:0/449239802 conn(0x7f8418072bc0 msgr2=0x7f84180ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 shutdown_connections 2024-03-16T09:11:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:25.456+0000 7f841c891700 1 -- 172.21.15.7:0/449239802 wait complete. 2024-03-16T09:11:26.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph health detail' 2024-03-16T09:11:26.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:26 smithi007 ceph-mon[113546]: pgmap v215: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-03-16T09:11:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:11:26.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:26 smithi019 ceph-mon[83366]: pgmap v215: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-03-16T09:11:27.667 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:27 smithi007 ceph-mon[113546]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:27.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:27 smithi019 ceph-mon[83366]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.236+0000 7fe1c398a700 1 -- 172.21.15.7:0/865669446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fde50 msgr2=0x7fe1bc0fe220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.236+0000 7fe1c398a700 1 --2- 172.21.15.7:0/865669446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fde50 0x7fe1bc0fe220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009ce0 tx=0x7fe1ac004d10 comp rx=0 tx=0).stop 2024-03-16T09:11:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.236+0000 7fe1c398a700 1 -- 172.21.15.7:0/865669446 shutdown_connections 2024-03-16T09:11:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.236+0000 7fe1c398a700 1 --2- 172.21.15.7:0/865669446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc0fe7f0 0x7fe1bc106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.236+0000 7fe1c398a700 1 --2- 172.21.15.7:0/865669446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fde50 0x7fe1bc0fe220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.236+0000 7fe1c398a700 1 -- 172.21.15.7:0/865669446 >> 172.21.15.7:0/865669446 conn(0x7fe1bc0f7870 msgr2=0x7fe1bc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.237+0000 7fe1c398a700 1 -- 172.21.15.7:0/865669446 shutdown_connections 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.237+0000 7fe1c398a700 1 -- 172.21.15.7:0/865669446 wait complete. 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.237+0000 7fe1c398a700 1 Processor -- start 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.237+0000 7fe1c398a700 1 -- start start 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.237+0000 7fe1c398a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc0fde50 0x7fe1bc0fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.237+0000 7fe1c398a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fe7f0 0x7fe1bc0fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.237+0000 7fe1c398a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0fdc60 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.237+0000 7fe1c398a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0fc790 con 0x7fe1bc0fde50 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1c0f25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fe7f0 0x7fe1bc0fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1c0f25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fe7f0 0x7fe1bc0fc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37242/0 (socket says 172.21.15.7:37242) 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1c0f25700 1 -- 172.21.15.7:0/2369045338 learned_addr learned my addr 172.21.15.7:0/2369045338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1c1726700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc0fde50 0x7fe1bc0fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1c0f25700 1 -- 172.21.15.7:0/2369045338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc0fde50 msgr2=0x7fe1bc0fbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1c0f25700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc0fde50 0x7fe1bc0fbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1c0f25700 1 -- 172.21.15.7:0/2369045338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac009940 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1c0f25700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fe7f0 0x7fe1bc0fc250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000eb40 tx=0x7fe1b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1b9ffb700 1 -- 172.21.15.7:0/2369045338 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b0007cc0 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.238+0000 7fe1b9ffb700 1 -- 172.21.15.7:0/2369045338 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1b000f040 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.239+0000 7fe1b9ffb700 1 -- 172.21.15.7:0/2369045338 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1b000ca10 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.239+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc0fca10 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.239+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc11a020 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.240+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc066ab0 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.240+0000 7fe1b9ffb700 1 -- 172.21.15.7:0/2369045338 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fe1b000cc80 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.241+0000 7fe1b9ffb700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe1a406dee0 0x7fe1a4070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.241+0000 7fe1b9ffb700 1 -- 172.21.15.7:0/2369045338 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe1b008d260 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.241+0000 7fe1c1726700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe1a406dee0 0x7fe1a4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.241+0000 7fe1c1726700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe1a406dee0 0x7fe1a4070390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac015040 tx=0x7fe1ac004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.250+0000 7fe1b9ffb700 1 -- 172.21.15.7:0/2369045338 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe1b00566d0 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.702+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe1bc04c710 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.705 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:28 smithi007 ceph-mon[113546]: pgmap v216: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.3 KiB/s rd, 8.5 KiB/s wr, 8 op/s 2024-03-16T09:11:28.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.702+0000 7fe1b9ffb700 1 -- 172.21.15.7:0/2369045338 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe1b0059d90 con 0x7fe1bc0fe7f0 2024-03-16T09:11:28.705 INFO:teuthology.orchestra.run.smithi007.stdout:HEALTH_OK 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe1a406dee0 msgr2=0x7fe1a4070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe1a406dee0 0x7fe1a4070390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac015040 tx=0x7fe1ac004d10 comp rx=0 tx=0).stop 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fe7f0 msgr2=0x7fe1bc0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fe7f0 0x7fe1bc0fc250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000eb40 tx=0x7fe1b000b040 comp rx=0 tx=0).stop 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 shutdown_connections 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc0fde50 0x7fe1bc0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe1a406dee0 0x7fe1a4070390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 --2- 172.21.15.7:0/2369045338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc0fe7f0 0x7fe1bc0fc250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 >> 172.21.15.7:0/2369045338 conn(0x7fe1bc0f7870 msgr2=0x7fe1bc0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 shutdown_connections 2024-03-16T09:11:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:28.705+0000 7fe1c398a700 1 -- 172.21.15.7:0/2369045338 wait complete. 2024-03-16T09:11:28.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:28 smithi019 ceph-mon[83366]: pgmap v216: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.3 KiB/s rd, 8.5 KiB/s wr, 8 op/s 2024-03-16T09:11:29.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions' 2024-03-16T09:11:29.678 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:11:29.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:29 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2369045338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:11:29.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:29 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2369045338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T09:11:30.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:30 smithi019 ceph-mon[83366]: pgmap v217: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 8.3 KiB/s wr, 6 op/s 2024-03-16T09:11:30.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:30 smithi007 ceph-mon[113546]: pgmap v217: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 8.3 KiB/s wr, 6 op/s 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 -- 172.21.15.7:0/3866445903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 msgr2=0x7fcb580fbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 --2- 172.21.15.7:0/3866445903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 0x7fcb580fbf30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c004940 tx=0x7fcb4c009e30 comp rx=0 tx=0).stop 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 -- 172.21.15.7:0/3866445903 shutdown_connections 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 --2- 172.21.15.7:0/3866445903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb580fc470 0x7fcb581091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 --2- 172.21.15.7:0/3866445903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 0x7fcb580fbf30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 -- 172.21.15.7:0/3866445903 >> 172.21.15.7:0/3866445903 conn(0x7fcb580f78d0 msgr2=0x7fcb580f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 -- 172.21.15.7:0/3866445903 shutdown_connections 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 -- 172.21.15.7:0/3866445903 wait complete. 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.339+0000 7fcb5f244700 1 Processor -- start 2024-03-16T09:11:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb5f244700 1 -- start start 2024-03-16T09:11:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb5f244700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 0x7fcb58115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb5f244700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb580fc470 0x7fcb58116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb5f244700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581168e0 con 0x7fcb580fbb60 2024-03-16T09:11:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb5f244700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5810fcb0 con 0x7fcb580fc470 2024-03-16T09:11:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb5cfe0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 0x7fcb58115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb5cfe0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 0x7fcb58115d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37266/0 (socket says 172.21.15.7:37266) 2024-03-16T09:11:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb5cfe0700 1 -- 172.21.15.7:0/2713474232 learned_addr learned my addr 172.21.15.7:0/2713474232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.340+0000 7fcb57fff700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb580fc470 0x7fcb58116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb5cfe0700 1 -- 172.21.15.7:0/2713474232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb580fc470 msgr2=0x7fcb58116290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb5cfe0700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb580fc470 0x7fcb58116290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb5cfe0700 1 -- 172.21.15.7:0/2713474232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c005040 con 0x7fcb580fbb60 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb57fff700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb580fc470 0x7fcb58116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb5cfe0700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 0x7fcb58115d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009380 tx=0x7fcb4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb557fa700 1 -- 172.21.15.7:0/2713474232 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb4c00e040 con 0x7fcb580fbb60 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb557fa700 1 -- 172.21.15.7:0/2713474232 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb4c00bbc0 con 0x7fcb580fbb60 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5810ff30 con 0x7fcb580fbb60 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb557fa700 1 -- 172.21.15.7:0/2713474232 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb4c01c630 con 0x7fcb580fbb60 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.341+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581104f0 con 0x7fcb580fbb60 2024-03-16T09:11:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.342+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb58066770 con 0x7fcb580fbb60 2024-03-16T09:11:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.344+0000 7fcb557fa700 1 -- 172.21.15.7:0/2713474232 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fcb4c00bd30 con 0x7fcb580fbb60 2024-03-16T09:11:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.344+0000 7fcb557fa700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fcb4806dee0 0x7fcb48070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.345+0000 7fcb557fa700 1 -- 172.21.15.7:0/2713474232 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcb4c08f570 con 0x7fcb580fbb60 2024-03-16T09:11:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.345+0000 7fcb57fff700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fcb4806dee0 0x7fcb48070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.345+0000 7fcb57fff700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fcb4806dee0 0x7fcb48070390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009fd0 tx=0x7fcb44009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.348+0000 7fcb557fa700 1 -- 172.21.15.7:0/2713474232 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb4c05c650 con 0x7fcb580fbb60 2024-03-16T09:11:31.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:31 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:11:31.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:31 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:11:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.791+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb5804bed0 con 0x7fcb580fbb60 2024-03-16T09:11:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.792+0000 7fcb557fa700 1 -- 172.21.15.7:0/2713474232 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcb4c05c1e0 con 0x7fcb580fbb60 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T09:11:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:11:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:11:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T09:11:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:11:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:11:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-16T09:11:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:11:31.796 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fcb4806dee0 msgr2=0x7fcb48070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fcb4806dee0 0x7fcb48070390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009fd0 tx=0x7fcb44009f20 comp rx=0 tx=0).stop 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 msgr2=0x7fcb58115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 0x7fcb58115d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009380 tx=0x7fcb4c009e30 comp rx=0 tx=0).stop 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 shutdown_connections 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fcb4806dee0 0x7fcb48070390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb580fbb60 0x7fcb58115d50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 --2- 172.21.15.7:0/2713474232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb580fc470 0x7fcb58116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.794+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 >> 172.21.15.7:0/2713474232 conn(0x7fcb580f78d0 msgr2=0x7fcb580f7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.795+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 shutdown_connections 2024-03-16T09:11:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:11:31.795+0000 7fcb5f244700 1 -- 172.21.15.7:0/2713474232 wait complete. 2024-03-16T09:11:32.478 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:32 smithi007 ceph-mon[113546]: pgmap v218: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s wr, 1 op/s 2024-03-16T09:11:32.478 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:32 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2713474232' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:32.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-16T09:11:32.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:32 smithi019 ceph-mon[83366]: pgmap v218: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s wr, 1 op/s 2024-03-16T09:11:32.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:32 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2713474232' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:11:32.824 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:11:34.447 INFO:teuthology.orchestra.run.smithi007.stdout:wait for servicemap items w/ changing names to refresh 2024-03-16T09:11:34.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:34 smithi007 ceph-mon[113546]: pgmap v219: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:34.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:34 smithi019 ceph-mon[83366]: pgmap v219: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:35.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'sleep 60' 2024-03-16T09:11:35.465 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:11:36.498 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:36 smithi007 ceph-mon[113546]: pgmap v220: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:36.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:36 smithi019 ceph-mon[83366]: pgmap v220: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:38.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:38 smithi019 ceph-mon[83366]: pgmap v221: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:38.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:38 smithi007 ceph-mon[113546]: pgmap v221: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:40.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:40 smithi019 ceph-mon[83366]: pgmap v222: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:40.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:40 smithi007 ceph-mon[113546]: pgmap v222: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:42.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:42 smithi019 ceph-mon[83366]: pgmap v223: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:42.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:42 smithi007 ceph-mon[113546]: pgmap v223: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:44.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:44 smithi019 ceph-mon[83366]: pgmap v224: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:44.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:44 smithi007 ceph-mon[113546]: pgmap v224: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:46.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:46 smithi019 ceph-mon[83366]: pgmap v225: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:46.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:46 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:11:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:46 smithi007 ceph-mon[113546]: pgmap v225: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:46.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:46 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:11:48.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:48 smithi019 ceph-mon[83366]: pgmap v226: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:48.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:48 smithi007 ceph-mon[113546]: pgmap v226: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:50.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:50 smithi019 ceph-mon[83366]: pgmap v227: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:50.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:50 smithi007 ceph-mon[113546]: pgmap v227: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:52.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:52 smithi019 ceph-mon[83366]: pgmap v228: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:52.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:52 smithi007 ceph-mon[113546]: pgmap v228: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:54.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:54 smithi019 ceph-mon[83366]: pgmap v229: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:54.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:54 smithi007 ceph-mon[113546]: pgmap v229: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:56.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:56 smithi019 ceph-mon[83366]: pgmap v230: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:56.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:56 smithi007 ceph-mon[113546]: pgmap v230: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:58.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:11:58 smithi019 ceph-mon[83366]: pgmap v231: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:11:58.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:11:58 smithi007 ceph-mon[113546]: pgmap v231: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:00.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:00 smithi019 ceph-mon[83366]: pgmap v232: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:00.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:00 smithi007 ceph-mon[113546]: pgmap v232: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:01.707 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:01 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:12:01.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:01 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:12:02.714 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:02 smithi019 ceph-mon[83366]: pgmap v233: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:02.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:02 smithi007 ceph-mon[113546]: pgmap v233: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:04.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:04 smithi019 ceph-mon[83366]: pgmap v234: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:04.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:04 smithi007 ceph-mon[113546]: pgmap v234: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:05.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:12:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:05.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' 2024-03-16T09:12:06.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:06 smithi019 ceph-mon[83366]: pgmap v235: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:06.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:06 smithi007 ceph-mon[113546]: pgmap v235: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:08.712 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:08 smithi019 ceph-mon[83366]: pgmap v236: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:08.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:08 smithi007 ceph-mon[113546]: pgmap v236: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:10.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:10 smithi019 ceph-mon[83366]: pgmap v237: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:10.748 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:10 smithi007 ceph-mon[113546]: pgmap v237: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:11.706 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:11 smithi007 ceph-mon[113546]: pgmap v238: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:11.713 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:11 smithi019 ceph-mon[83366]: pgmap v238: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:14.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:13 smithi019 ceph-mon[83366]: pgmap v239: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:14.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:13 smithi007 ceph-mon[113546]: pgmap v239: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:16.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:15 smithi019 ceph-mon[83366]: pgmap v240: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:16.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:15 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:12:16.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:15 smithi007 ceph-mon[113546]: pgmap v240: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:16.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:15 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:12:18.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:17 smithi019 ceph-mon[83366]: pgmap v241: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:18.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:17 smithi007 ceph-mon[113546]: pgmap v241: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:20.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:19 smithi019 ceph-mon[83366]: pgmap v242: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:20.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:19 smithi007 ceph-mon[113546]: pgmap v242: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:21.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:21 smithi007 ceph-mon[113546]: pgmap v243: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:22.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:21 smithi019 ceph-mon[83366]: pgmap v243: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:24.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:23 smithi019 ceph-mon[83366]: pgmap v244: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:24.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:23 smithi007 ceph-mon[113546]: pgmap v244: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:26.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:25 smithi019 ceph-mon[83366]: pgmap v245: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:26.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:25 smithi007 ceph-mon[113546]: pgmap v245: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:28.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:27 smithi019 ceph-mon[83366]: pgmap v246: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:28.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:27 smithi007 ceph-mon[113546]: pgmap v246: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:30.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:29 smithi019 ceph-mon[83366]: pgmap v247: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:30.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:29 smithi007 ceph-mon[113546]: pgmap v247: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:31.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:30 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:12:31.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:30 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:12:31.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:31 smithi007 ceph-mon[113546]: pgmap v248: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:32.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:31 smithi019 ceph-mon[83366]: pgmap v248: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:34.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:33 smithi019 ceph-mon[83366]: pgmap v249: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:34.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:33 smithi007 ceph-mon[113546]: pgmap v249: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:36.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:35 smithi019 ceph-mon[83366]: pgmap v250: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:36.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:35 smithi007 ceph-mon[113546]: pgmap v250: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:37.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch ps' 2024-03-16T09:12:37.987 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:12:37.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:37 smithi007 ceph-mon[113546]: pgmap v251: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:38.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:37 smithi019 ceph-mon[83366]: pgmap v251: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.824+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/2652524434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 msgr2=0x7f0c04102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.824+0000 7f0c0b1c5700 1 --2- 172.21.15.7:0/2652524434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 0x7f0c04102140 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4004940 tx=0x7f0bf4009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.824+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/2652524434 shutdown_connections 2024-03-16T09:12:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.824+0000 7f0c0b1c5700 1 --2- 172.21.15.7:0/2652524434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04102710 0x7f0c040fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.824+0000 7f0c0b1c5700 1 --2- 172.21.15.7:0/2652524434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 0x7f0c04102140 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.824+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/2652524434 >> 172.21.15.7:0/2652524434 conn(0x7f0c04072bc0 msgr2=0x7f0c04072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.824+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/2652524434 shutdown_connections 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.824+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/2652524434 wait complete. 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c0b1c5700 1 Processor -- start 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c0b1c5700 1 -- start start 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c0b1c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 0x7f0c0410eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c0b1c5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04102710 0x7f0c0410f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c0b1c5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0410f670 con 0x7f0c04101d70 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c0b1c5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0410f7b0 con 0x7f0c04102710 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c08f61700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 0x7f0c0410eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c08f61700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 0x7f0c0410eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51870/0 (socket says 172.21.15.7:51870) 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c08f61700 1 -- 172.21.15.7:0/105519224 learned_addr learned my addr 172.21.15.7:0/105519224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.825+0000 7f0c03fff700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04102710 0x7f0c0410f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c08f61700 1 -- 172.21.15.7:0/105519224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04102710 msgr2=0x7f0c0410f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c08f61700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04102710 0x7f0c0410f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c08f61700 1 -- 172.21.15.7:0/105519224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4005040 con 0x7f0c04101d70 2024-03-16T09:12:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c08f61700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 0x7f0c0410eae0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400b8b0 tx=0x7f0bf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c017fa700 1 -- 172.21.15.7:0/105519224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf400e040 con 0x7f0c04101d70 2024-03-16T09:12:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c04117b50 con 0x7f0c04101d70 2024-03-16T09:12:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c017fa700 1 -- 172.21.15.7:0/105519224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bf4004290 con 0x7f0c04101d70 2024-03-16T09:12:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c017fa700 1 -- 172.21.15.7:0/105519224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf401c3f0 con 0x7f0c04101d70 2024-03-16T09:12:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.826+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c04118060 con 0x7f0c04101d70 2024-03-16T09:12:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.827+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c040ff080 con 0x7f0c04101d70 2024-03-16T09:12:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.828+0000 7f0c017fa700 1 -- 172.21.15.7:0/105519224 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0bf4013420 con 0x7f0c04101d70 2024-03-16T09:12:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.829+0000 7f0c017fa700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0bec06dee0 0x7f0bec070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.829+0000 7f0c017fa700 1 -- 172.21.15.7:0/105519224 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0bf408e9a0 con 0x7f0c04101d70 2024-03-16T09:12:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.829+0000 7f0c03fff700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0bec06dee0 0x7f0bec070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.829+0000 7f0c03fff700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0bec06dee0 0x7f0bec070390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c04066000 tx=0x7f0bf800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:39.834+0000 7f0c017fa700 1 -- 172.21.15.7:0/105519224 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0bf4057e90 con 0x7f0c04101d70 2024-03-16T09:12:40.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.055+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 --> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c04108ee0 con 0x7f0bec06dee0 2024-03-16T09:12:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.066+0000 7f0c017fa700 1 -- 172.21.15.7:0/105519224 <== mgr.34104 v2:172.21.15.7:6800/811946302 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0c04108ee0 con 0x7f0bec06dee0 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:alertmanager.smithi007 smithi007 *:9093,9094 running (8m) 2m ago 19m 27.1M - 0.25.0 c8568f914cd2 8fc1d2f90efe 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi007 smithi007 running (6m) 2m ago 19m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8c783262f6a9 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:crash.smithi019 smithi019 running (6m) 101s ago 18m 6660k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 050be9f17fd0 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:grafana.smithi007 smithi007 *:3000 running (8m) 2m ago 19m 84.6M - 9.4.12 a0ea5e3c46d9 9c7e0b7e2fb9 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.bsxctp smithi007 running (2m) 2m ago 14m 12.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b fa6d147fc765 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi007.znurst smithi007 running (2m) 2m ago 14m 12.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b ba11fdc24360 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.bncsil smithi019 running (2m) 101s ago 14m 12.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 93b12a56fe52 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:mds.cephfs.smithi019.gmhzgo smithi019 running (104s) 101s ago 13m 10.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b de8a59e6e71b 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi007.uamwtp smithi007 *:8443,9283,8765 running (10m) 2m ago 20m 508M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b4c329a2576 2024-03-16T09:12:40.069 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.smithi019.iuqxhi smithi019 *:8443,9283,8765 running (9m) 101s ago 18m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a2e3e1cac7b2 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi007 smithi007 running (7m) 2m ago 21m 56.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b bef77d9053ff 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:mon.smithi019 smithi019 running (6m) 101s ago 18m 42.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1abff6ceff3e 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi007 smithi007 *:9100 running (9m) 2m ago 19m 15.2M - 1.5.0 0da6a335fe13 93ae3cb20951 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:node-exporter.smithi019 smithi019 *:9100 running (9m) 101s ago 18m 15.7M - 1.5.0 0da6a335fe13 c962df798da4 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (5m) 2m ago 17m 96.7M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1f775f30a855 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi007 running (5m) 2m ago 17m 95.4M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4aa10e800f72 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi007 running (4m) 2m ago 16m 87.3M 1559M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5449f945991b 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi019 running (3m) 101s ago 16m 106M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b bb64247744fe 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi019 running (3m) 101s ago 16m 103M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b e1df7d4b9eaa 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi019 running (2m) 101s ago 15m 95.8M 2583M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5480e37181b1 2024-03-16T09:12:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:prometheus.smithi007 smithi007 *:9095 running (9m) 2m ago 19m 66.7M - 2.43.0 a07b618ecd1d 7072d391a6f5 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0bec06dee0 msgr2=0x7f0bec070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0bec06dee0 0x7f0bec070390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c04066000 tx=0x7f0bf800b040 comp rx=0 tx=0).stop 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 msgr2=0x7f0c0410eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 0x7f0c0410eae0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400b8b0 tx=0x7f0bf4009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 shutdown_connections 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0bec06dee0 0x7f0bec070390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04101d70 0x7f0c0410eae0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 --2- 172.21.15.7:0/105519224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c04102710 0x7f0c0410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 >> 172.21.15.7:0/105519224 conn(0x7f0c04072bc0 msgr2=0x7f0c040fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 shutdown_connections 2024-03-16T09:12:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:40.069+0000 7f0c0b1c5700 1 -- 172.21.15.7:0/105519224 wait complete. 2024-03-16T09:12:40.083 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:39 smithi007 ceph-mon[113546]: pgmap v252: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:40.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:39 smithi019 ceph-mon[83366]: pgmap v252: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:40.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions' 2024-03-16T09:12:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:12:41.183 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:40 smithi007 ceph-mon[113546]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:41.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:40 smithi019 ceph-mon[83366]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:42.169 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:41 smithi007 ceph-mon[113546]: pgmap v253: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:42.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:41 smithi019 ceph-mon[83366]: pgmap v253: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:42.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.792+0000 7f553e710700 1 -- 172.21.15.7:0/2344915022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 msgr2=0x7f55380fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:42.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.792+0000 7f553e710700 1 --2- 172.21.15.7:0/2344915022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 0x7f55380fce20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f552c004940 tx=0x7f552c009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:42.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.792+0000 7f553e710700 1 -- 172.21.15.7:0/2344915022 shutdown_connections 2024-03-16T09:12:42.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.792+0000 7f553e710700 1 --2- 172.21.15.7:0/2344915022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 0x7f55380fce20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:42.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.792+0000 7f553e710700 1 --2- 172.21.15.7:0/2344915022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538102e10 0x7f55381031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:42.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.792+0000 7f553e710700 1 -- 172.21.15.7:0/2344915022 >> 172.21.15.7:0/2344915022 conn(0x7f55380f89a0 msgr2=0x7f55380fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:42.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.792+0000 7f553e710700 1 -- 172.21.15.7:0/2344915022 shutdown_connections 2024-03-16T09:12:42.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.792+0000 7f553e710700 1 -- 172.21.15.7:0/2344915022 wait complete. 2024-03-16T09:12:42.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f553e710700 1 Processor -- start 2024-03-16T09:12:42.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f553e710700 1 -- start start 2024-03-16T09:12:42.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f553e710700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538102e10 0x7f5538113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:42.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f553e710700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 0x7f55381140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f553e710700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55381147b0 con 0x7f55381037b0 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f553e710700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553810db10 con 0x7f5538102e10 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f55377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 0x7f55381140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f55377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 0x7f55381140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33282/0 (socket says 172.21.15.7:33282) 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f55377fe700 1 -- 172.21.15.7:0/348386721 learned_addr learned my addr 172.21.15.7:0/348386721 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.793+0000 7f5537fff700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538102e10 0x7f5538113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f55377fe700 1 -- 172.21.15.7:0/348386721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538102e10 msgr2=0x7f5538113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f55377fe700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538102e10 0x7f5538113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f55377fe700 1 -- 172.21.15.7:0/348386721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c005040 con 0x7f55381037b0 2024-03-16T09:12:42.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f55377fe700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 0x7f55381140d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f552c009100 tx=0x7f552c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:42.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f5534ff9700 1 -- 172.21.15.7:0/348386721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f552c00e040 con 0x7f55381037b0 2024-03-16T09:12:42.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553810dd90 con 0x7f55381037b0 2024-03-16T09:12:42.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f5534ff9700 1 -- 172.21.15.7:0/348386721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f552c004290 con 0x7f55381037b0 2024-03-16T09:12:42.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f5534ff9700 1 -- 172.21.15.7:0/348386721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f552c01c3f0 con 0x7f55381037b0 2024-03-16T09:12:42.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.794+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553810e350 con 0x7f55381037b0 2024-03-16T09:12:42.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.795+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55380fe560 con 0x7f55381037b0 2024-03-16T09:12:42.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.796+0000 7f5534ff9700 1 -- 172.21.15.7:0/348386721 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f552c013690 con 0x7f55381037b0 2024-03-16T09:12:42.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.796+0000 7f5534ff9700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f552006def0 0x7f55200703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:42.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.796+0000 7f5534ff9700 1 -- 172.21.15.7:0/348386721 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f552c08e900 con 0x7f55381037b0 2024-03-16T09:12:42.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.796+0000 7f5537fff700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f552006def0 0x7f55200703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:42.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.797+0000 7f5537fff700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f552006def0 0x7f55200703a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5528009bf0 tx=0x7f55280091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:42.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:42.801+0000 7f5534ff9700 1 -- 172.21.15.7:0/348386721 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f552c05b920 con 0x7f55381037b0 2024-03-16T09:12:43.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.103+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f553805f5f0 con 0x7f55381037b0 2024-03-16T09:12:43.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.103+0000 7f5534ff9700 1 -- 172.21.15.7:0/348386721 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f552c05b4b0 con 0x7f55381037b0 2024-03-16T09:12:43.106 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-16T09:12:43.106 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": { 2024-03-16T09:12:43.106 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:12:43.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": { 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": { 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": { 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-16T09:12:43.107 INFO:teuthology.orchestra.run.smithi007.stdout: "overall": { 2024-03-16T09:12:43.108 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-16T09:12:43.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-16T09:12:43.108 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f552006def0 msgr2=0x7f55200703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f552006def0 0x7f55200703a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5528009bf0 tx=0x7f55280091a0 comp rx=0 tx=0).stop 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 msgr2=0x7f55381140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 0x7f55381140d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f552c009100 tx=0x7f552c009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 shutdown_connections 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538102e10 0x7f5538113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f552006def0 0x7f55200703a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 --2- 172.21.15.7:0/348386721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55381037b0 0x7f55381140d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:43.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 >> 172.21.15.7:0/348386721 conn(0x7f55380f89a0 msgr2=0x7f55380f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:43.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 shutdown_connections 2024-03-16T09:12:43.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:43.106+0000 7f553e710700 1 -- 172.21.15.7:0/348386721 wait complete. 2024-03-16T09:12:43.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-16T09:12:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:43 smithi007 ceph-mon[113546]: pgmap v254: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:43.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:43 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/348386721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:12:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:12:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:43 smithi019 ceph-mon[83366]: pgmap v254: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:44.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:43 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/348386721' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:12:45.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.744+0000 7f2062f72700 1 -- 172.21.15.7:0/1373244549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c101d70 msgr2=0x7f205c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:45.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.744+0000 7f2062f72700 1 --2- 172.21.15.7:0/1373244549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c101d70 0x7f205c102140 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2050004940 tx=0x7f2050009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:45.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.744+0000 7f2062f72700 1 -- 172.21.15.7:0/1373244549 shutdown_connections 2024-03-16T09:12:45.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.744+0000 7f2062f72700 1 --2- 172.21.15.7:0/1373244549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f205c102710 0x7f205c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:45.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.744+0000 7f2062f72700 1 --2- 172.21.15.7:0/1373244549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c101d70 0x7f205c102140 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:45.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.744+0000 7f2062f72700 1 -- 172.21.15.7:0/1373244549 >> 172.21.15.7:0/1373244549 conn(0x7f205c072bc0 msgr2=0x7f205c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:45.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.744+0000 7f2062f72700 1 -- 172.21.15.7:0/1373244549 shutdown_connections 2024-03-16T09:12:45.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.744+0000 7f2062f72700 1 -- 172.21.15.7:0/1373244549 wait complete. 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f2062f72700 1 Processor -- start 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f2062f72700 1 -- start start 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f2062f72700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f205c101d70 0x7f205c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f2062f72700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102710 0x7f205c103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f2062f72700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c1036a0 con 0x7f205c102710 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f2062f72700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c1037e0 con 0x7f205c101d70 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f205bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102710 0x7f205c103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f205bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102710 0x7f205c103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33308/0 (socket says 172.21.15.7:33308) 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f205bfff700 1 -- 172.21.15.7:0/834313405 learned_addr learned my addr 172.21.15.7:0/834313405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.745+0000 7f2060d0e700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f205c101d70 0x7f205c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f205bfff700 1 -- 172.21.15.7:0/834313405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f205c101d70 msgr2=0x7f205c102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f205bfff700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f205c101d70 0x7f205c102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f205bfff700 1 -- 172.21.15.7:0/834313405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050005040 con 0x7f205c102710 2024-03-16T09:12:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f205bfff700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102710 0x7f205c103110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2048002a50 tx=0x7f2048004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f20597fa700 1 -- 172.21.15.7:0/834313405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2048007cc0 con 0x7f205c102710 2024-03-16T09:12:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f20597fa700 1 -- 172.21.15.7:0/834313405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f204800f040 con 0x7f205c102710 2024-03-16T09:12:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c119e80 con 0x7f205c102710 2024-03-16T09:12:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f20597fa700 1 -- 172.21.15.7:0/834313405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f204800d9c0 con 0x7f205c102710 2024-03-16T09:12:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.746+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c11a300 con 0x7f205c102710 2024-03-16T09:12:45.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.747+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20400052b0 con 0x7f205c102710 2024-03-16T09:12:45.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.748+0000 7f20597fa700 1 -- 172.21.15.7:0/834313405 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2048009730 con 0x7f205c102710 2024-03-16T09:12:45.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.748+0000 7f20597fa700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f204c06dee0 0x7f204c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:45.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.749+0000 7f20597fa700 1 -- 172.21.15.7:0/834313405 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f204808d460 con 0x7f205c102710 2024-03-16T09:12:45.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.749+0000 7f2060d0e700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f204c06dee0 0x7f204c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:45.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.749+0000 7f2060d0e700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f204c06dee0 0x7f204c070390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2050004f90 tx=0x7f2050009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:45.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:45.753+0000 7f20597fa700 1 -- 172.21.15.7:0/834313405 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2048056850 con 0x7f205c102710 2024-03-16T09:12:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:45 smithi007 ceph-mon[113546]: pgmap v255: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:45.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:45 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:12:46.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.147+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2040005d70 con 0x7f205c102710 2024-03-16T09:12:46.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.148+0000 7f20597fa700 1 -- 172.21.15.7:0/834313405 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2048059f10 con 0x7f205c102710 2024-03-16T09:12:46.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f204c06dee0 msgr2=0x7f204c070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f204c06dee0 0x7f204c070390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2050004f90 tx=0x7f2050009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102710 msgr2=0x7f205c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102710 0x7f205c103110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2048002a50 tx=0x7f2048004600 comp rx=0 tx=0).stop 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 shutdown_connections 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f205c101d70 0x7f205c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f204c06dee0 0x7f204c070390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 --2- 172.21.15.7:0/834313405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f205c102710 0x7f205c103110 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 >> 172.21.15.7:0/834313405 conn(0x7f205c072bc0 msgr2=0x7f205c065960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:46.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 shutdown_connections 2024-03-16T09:12:46.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:46.150+0000 7f2062f72700 1 -- 172.21.15.7:0/834313405 wait complete. 2024-03-16T09:12:46.163 INFO:teuthology.orchestra.run.smithi007.stdout:true 2024-03-16T09:12:46.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:45 smithi019 ceph-mon[83366]: pgmap v255: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:46.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:45 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:12:46.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-16T09:12:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:12:47.205 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:46 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/834313405' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:12:47.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:46 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/834313405' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:12:48.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:47 smithi019 ceph-mon[83366]: pgmap v256: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:48.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:47 smithi007 ceph-mon[113546]: pgmap v256: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:48.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.847+0000 7fa1a58af700 1 -- 172.21.15.7:0/1186121044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fe820 msgr2=0x7fa1a0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:48.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.847+0000 7fa1a58af700 1 --2- 172.21.15.7:0/1186121044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fe820 0x7fa1a0106e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa194004940 tx=0x7fa194009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:48.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.847+0000 7fa1a58af700 1 -- 172.21.15.7:0/1186121044 shutdown_connections 2024-03-16T09:12:48.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.847+0000 7fa1a58af700 1 --2- 172.21.15.7:0/1186121044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fe820 0x7fa1a0106e60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.847+0000 7fa1a58af700 1 --2- 172.21.15.7:0/1186121044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a00fde80 0x7fa1a00fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.847+0000 7fa1a58af700 1 -- 172.21.15.7:0/1186121044 >> 172.21.15.7:0/1186121044 conn(0x7fa1a00f7870 msgr2=0x7fa1a00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.847+0000 7fa1a58af700 1 -- 172.21.15.7:0/1186121044 shutdown_connections 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.847+0000 7fa1a58af700 1 -- 172.21.15.7:0/1186121044 wait complete. 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa1a58af700 1 Processor -- start 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa1a58af700 1 -- start start 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa1a58af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fde80 0x7fa1a0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa1a58af700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a00fe820 0x7fa1a0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa1a58af700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a0114500 con 0x7fa1a00fde80 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa1a58af700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a010db20 con 0x7fa1a00fe820 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa19effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fde80 0x7fa1a0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa19effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fde80 0x7fa1a0113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33334/0 (socket says 172.21.15.7:33334) 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa19effd700 1 -- 172.21.15.7:0/3942570761 learned_addr learned my addr 172.21.15.7:0/3942570761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.848+0000 7fa19e7fc700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a00fe820 0x7fa1a0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa19effd700 1 -- 172.21.15.7:0/3942570761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a00fe820 msgr2=0x7fa1a0113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa19effd700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a00fe820 0x7fa1a0113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:48.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa19effd700 1 -- 172.21.15.7:0/3942570761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194005040 con 0x7fa1a00fde80 2024-03-16T09:12:48.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa19effd700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fde80 0x7fa1a0113970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa190002a50 tx=0x7fa1900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:48.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa1877fe700 1 -- 172.21.15.7:0/3942570761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa190007cc0 con 0x7fa1a00fde80 2024-03-16T09:12:48.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa1877fe700 1 -- 172.21.15.7:0/3942570761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa19000f040 con 0x7fa1a00fde80 2024-03-16T09:12:48.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa1877fe700 1 -- 172.21.15.7:0/3942570761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19000da30 con 0x7fa1a00fde80 2024-03-16T09:12:48.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a010de00 con 0x7fa1a00fde80 2024-03-16T09:12:48.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.849+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a010e340 con 0x7fa1a00fde80 2024-03-16T09:12:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.850+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a0066b10 con 0x7fa1a00fde80 2024-03-16T09:12:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.851+0000 7fa1877fe700 1 -- 172.21.15.7:0/3942570761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa19000dbb0 con 0x7fa1a00fde80 2024-03-16T09:12:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.851+0000 7fa1877fe700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa18806dee0 0x7fa188070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.851+0000 7fa1877fe700 1 -- 172.21.15.7:0/3942570761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa19008d440 con 0x7fa1a00fde80 2024-03-16T09:12:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.851+0000 7fa19e7fc700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa18806dee0 0x7fa188070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.852+0000 7fa19e7fc700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa18806dee0 0x7fa188070390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1a010f210 tx=0x7fa194009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:48.856+0000 7fa1877fe700 1 -- 172.21.15.7:0/3942570761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa190056930 con 0x7fa1a00fde80 2024-03-16T09:12:49.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.166+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa1a010efc0 con 0x7fa1a00fde80 2024-03-16T09:12:49.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.167+0000 7fa1877fe700 1 -- 172.21.15.7:0/3942570761 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa190059ff0 con 0x7fa1a00fde80 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa18806dee0 msgr2=0x7fa188070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa18806dee0 0x7fa188070390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1a010f210 tx=0x7fa194009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fde80 msgr2=0x7fa1a0113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fde80 0x7fa1a0113970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa190002a50 tx=0x7fa1900046b0 comp rx=0 tx=0).stop 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 shutdown_connections 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa18806dee0 0x7fa188070390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1a00fde80 0x7fa1a0113970 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 --2- 172.21.15.7:0/3942570761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a00fe820 0x7fa1a0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 >> 172.21.15.7:0/3942570761 conn(0x7fa1a00f7870 msgr2=0x7fa1a00f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 shutdown_connections 2024-03-16T09:12:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:49.169+0000 7fa1a58af700 1 -- 172.21.15.7:0/3942570761 wait complete. 2024-03-16T09:12:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)" 2024-03-16T09:12:49.855 DEBUG:teuthology.parallel:result is None 2024-03-16T09:12:49.855 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T09:12:49.866 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T09:12:49.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T09:12:50.143 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:49 smithi007 ceph-mon[113546]: pgmap v257: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:50.143 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:49 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3942570761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:12:50.155 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/mon.smithi007/config 2024-03-16T09:12:50.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:49 smithi019 ceph-mon[83366]: pgmap v257: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:50.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:49 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3942570761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T09:12:51.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 -- 172.21.15.7:0/2122685360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 msgr2=0x7f6538102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:51.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 --2- 172.21.15.7:0/2122685360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 0x7f6538102140 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f652c004940 tx=0x7f652c009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:51.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 -- 172.21.15.7:0/2122685360 shutdown_connections 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 --2- 172.21.15.7:0/2122685360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6538102710 0x7f65380fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 --2- 172.21.15.7:0/2122685360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 0x7f6538102140 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 -- 172.21.15.7:0/2122685360 >> 172.21.15.7:0/2122685360 conn(0x7f6538072bc0 msgr2=0x7f6538072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 -- 172.21.15.7:0/2122685360 shutdown_connections 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 -- 172.21.15.7:0/2122685360 wait complete. 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 Processor -- start 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.802+0000 7f653ed55700 1 -- start start 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653ed55700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 0x7f6538112a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653ed55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6538102710 0x7f6538112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653ed55700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65381135f0 con 0x7f6538101d70 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653ed55700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653810c9e0 con 0x7f6538102710 2024-03-16T09:12:51.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653caf1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 0x7f6538112a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:51.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653caf1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 0x7f6538112a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33342/0 (socket says 172.21.15.7:33342) 2024-03-16T09:12:51.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653caf1700 1 -- 172.21.15.7:0/106990973 learned_addr learned my addr 172.21.15.7:0/106990973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:51.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f6537fff700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6538102710 0x7f6538112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:51.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653caf1700 1 -- 172.21.15.7:0/106990973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6538102710 msgr2=0x7f6538112fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:51.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653caf1700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6538102710 0x7f6538112fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:51.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653caf1700 1 -- 172.21.15.7:0/106990973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c005040 con 0x7f6538101d70 2024-03-16T09:12:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.803+0000 7f653caf1700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 0x7f6538112a60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f652c00b3b0 tx=0x7f652c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.804+0000 7f65357fa700 1 -- 172.21.15.7:0/106990973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f652c00e040 con 0x7f6538101d70 2024-03-16T09:12:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.804+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653810cc60 con 0x7f6538101d70 2024-03-16T09:12:51.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.804+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653810d220 con 0x7f6538101d70 2024-03-16T09:12:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.804+0000 7f65357fa700 1 -- 172.21.15.7:0/106990973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f652c00bbc0 con 0x7f6538101d70 2024-03-16T09:12:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.804+0000 7f65357fa700 1 -- 172.21.15.7:0/106990973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f652c01c3f0 con 0x7f6538101d70 2024-03-16T09:12:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.805+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c0052e0 con 0x7f6538101d70 2024-03-16T09:12:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.805+0000 7f65357fa700 1 -- 172.21.15.7:0/106990973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f652c00bd30 con 0x7f6538101d70 2024-03-16T09:12:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.806+0000 7f65357fa700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f652806dce0 0x7f6528070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.806+0000 7f65357fa700 1 -- 172.21.15.7:0/106990973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f652c08e9e0 con 0x7f6538101d70 2024-03-16T09:12:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.806+0000 7f6537fff700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f652806dce0 0x7f6528070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:51.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.806+0000 7f6537fff700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f652806dce0 0x7f6528070190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6524007fd0 tx=0x7f6524009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:51.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:51.813+0000 7f65357fa700 1 -- 172.21.15.7:0/106990973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f652c05ba00 con 0x7f6538101d70 2024-03-16T09:12:52.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.073+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f651c005da0 con 0x7f6538101d70 2024-03-16T09:12:52.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.074+0000 7f65357fa700 1 -- 172.21.15.7:0/106990973 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1947 (secure 0 0 0) 0x7f652c05b590 con 0x7f6538101d70 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:e36 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 36 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-16T08:58:28.666579+0000 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-16T09:11:14.345543+0000 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-16T09:12:52.077 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 60 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 80 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 2 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=34380,1=34412} 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-16T09:12:52.078 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 1 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 34380 members: 34380,34412 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.bsxctp{0:34380} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.7:6826/2837914432,v1:172.21.15.7:6827/2837914432] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi007.znurst{1:34412} state up:active seq 16 join_fscid=1 addr [v2:172.21.15.7:6828/1434212331,v1:172.21.15.7:6829/1434212331] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-16T09:12:52.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:52.080 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.bncsil{-1:44303} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6824/2889841468,v1:172.21.15.19:6825/2889841468] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:12:52.080 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.cephfs.smithi019.gmhzgo{-1:44329} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.19:6826/3039505467,v1:172.21.15.19:6827/3039505467] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f652806dce0 msgr2=0x7f6528070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f652806dce0 0x7f6528070190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6524007fd0 tx=0x7f6524009040 comp rx=0 tx=0).stop 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 msgr2=0x7f6538112a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 0x7f6538112a60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f652c00b3b0 tx=0x7f652c009e30 comp rx=0 tx=0).stop 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 shutdown_connections 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f652806dce0 0x7f6528070190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6538101d70 0x7f6538112a60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 --2- 172.21.15.7:0/106990973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6538102710 0x7f6538112fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 >> 172.21.15.7:0/106990973 conn(0x7f6538072bc0 msgr2=0x7f65380fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 shutdown_connections 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.076+0000 7f653ed55700 1 -- 172.21.15.7:0/106990973 wait complete. 2024-03-16T09:12:52.081 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-03-16T09:12:52.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:51 smithi019 ceph-mon[83366]: pgmap v258: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:52.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:51 smithi007 ceph-mon[113546]: pgmap v258: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:52.790 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-16T09:12:52.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:12:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 -- 172.21.15.7:0/2711032894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fd1d0 msgr2=0x7f69b40fd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 --2- 172.21.15.7:0/2711032894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fd1d0 0x7f69b40fd5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f69a40048d0 tx=0x7f69a4009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 -- 172.21.15.7:0/2711032894 shutdown_connections 2024-03-16T09:12:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 --2- 172.21.15.7:0/2711032894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b40fdae0 0x7f69b41061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 --2- 172.21.15.7:0/2711032894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fd1d0 0x7f69b40fd5a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 -- 172.21.15.7:0/2711032894 >> 172.21.15.7:0/2711032894 conn(0x7f69b40f6ca0 msgr2=0x7f69b40f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 -- 172.21.15.7:0/2711032894 shutdown_connections 2024-03-16T09:12:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 -- 172.21.15.7:0/2711032894 wait complete. 2024-03-16T09:12:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 Processor -- start 2024-03-16T09:12:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 -- start start 2024-03-16T09:12:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b40fd1d0 0x7f69b4112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fdae0 0x7f69b4113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4113870 con 0x7f69b40fdae0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.910+0000 7f69b9d5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b410cec0 con 0x7f69b40fd1d0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fdae0 0x7f69b4113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fdae0 0x7f69b4113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50364/0 (socket says 172.21.15.7:50364) 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b2ffd640 1 -- 172.21.15.7:0/1871739199 learned_addr learned my addr 172.21.15.7:0/1871739199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b37fe640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b40fd1d0 0x7f69b4112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b2ffd640 1 -- 172.21.15.7:0/1871739199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b40fd1d0 msgr2=0x7f69b4112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b2ffd640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b40fd1d0 0x7f69b4112d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b2ffd640 1 -- 172.21.15.7:0/1871739199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a4005040 con 0x7f69b40fdae0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b37fe640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b40fd1d0 0x7f69b4112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b2ffd640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fdae0 0x7f69b4113270 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f69a800c830 tx=0x7f69a8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f699bfff640 1 -- 172.21.15.7:0/1871739199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a800a850 con 0x7f69b40fdae0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f699bfff640 1 -- 172.21.15.7:0/1871739199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69a800ae80 con 0x7f69b40fdae0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b410d1a0 con 0x7f69b40fdae0 2024-03-16T09:12:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f699bfff640 1 -- 172.21.15.7:0/1871739199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a8012ea0 con 0x7f69b40fdae0 2024-03-16T09:12:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.911+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b410d6c0 con 0x7f69b40fdae0 2024-03-16T09:12:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.913+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b4065dd0 con 0x7f69b40fdae0 2024-03-16T09:12:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.913+0000 7f699bfff640 1 -- 172.21.15.7:0/1871739199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f69a8012400 con 0x7f69b40fdae0 2024-03-16T09:12:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.913+0000 7f699bfff640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f699c06de50 0x7f699c070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.913+0000 7f69b37fe640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f699c06de50 0x7f699c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.913+0000 7f699bfff640 1 -- 172.21.15.7:0/1871739199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f69a808da00 con 0x7f69b40fdae0 2024-03-16T09:12:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.914+0000 7f69b37fe640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f699c06de50 0x7f699c070300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f69a4004540 tx=0x7f69a4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:52.918+0000 7f699bfff640 1 -- 172.21.15.7:0/1871739199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f69a8056ef0 con 0x7f69b40fdae0 2024-03-16T09:12:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.144+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69b4109320 con 0x7f69b40fdae0 2024-03-16T09:12:53.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.145+0000 7f699bfff640 1 -- 172.21.15.7:0/1871739199 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16516 (secure 0 0 0) 0x7f69a80126b0 con 0x7f69b40fdae0 2024-03-16T09:12:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:53.148 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","created":"2024-03-16T08:51:36.700010+0000","modified":"2024-03-16T09:10:51.672446+0000","last_up_change":"2024-03-16T09:09:57.383379+0000","last_in_change":"2024-03-16T08:56:48.805178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:55:53.635803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-16T08:58:27.077630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-16T08:58:27.912287+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f96044-56c4-4825-8138-629e5352a8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6803","nonce":2558509313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6805","nonce":2558509313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6809","nonce":2558509313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6807","nonce":2558509313}]},"public_addr":"172.21.15.7:6803/2558509313","cluster_addr":"172.21.15.7:6805/2558509313","heartbeat_back_addr":"172.21.15.7:6809/2558509313","heartbeat_front_addr":"172.21.15.7:6807/2558509313","state":["exists","up"]},{"osd":1,"uuid":"b3fcdd0c-0016-4781-803b-bbca53b4517b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6811","nonce":3984255151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6813","nonce":3984255151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6817","nonce":3984255151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6815","nonce":3984255151}]},"public_addr":"172.21.15.7:6811/3984255151","cluster_addr":"172.21.15.7:6813/3984255151","heartbeat_back_addr":"172.21.15.7:6817/3984255151","heartbeat_front_addr":"172.21.15.7:6815/3984255151","state":["exists","up"]},{"osd":2,"uuid":"4f400784-c979-4f41-bcd2-a52737e2e65e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6819","nonce":7192509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6821","nonce":7192509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6825","nonce":7192509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6823","nonce":7192509}]},"public_addr":"172.21.15.7:6819/7192509","cluster_addr":"172.21.15.7:6821/7192509","heartbeat_back_addr":"172.21.15.7:6825/7192509","heartbeat_front_addr":"172.21.15.7:6823/7192509","state":["exists","up"]},{"osd":3,"uuid":"c1fa594a-67c7-4f8f-a745-74ef06dfd43c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6801","nonce":2696958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6803","nonce":2696958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6807","nonce":2696958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6805","nonce":2696958381}]},"public_addr":"172.21.15.19:6801/2696958381","cluster_addr":"172.21.15.19:6803/2696958381","heartbeat_back_addr":"172.21.15.19:6807/2696958381","heartbeat_front_addr":"172.21.15.19:6805/2696958381","state":["exists","up"]},{"osd":4,"uuid":"55356509-5ce4-4cb1-9195-3911332ec8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6809","nonce":377988924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6811","nonce":377988924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6815","nonce":377988924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6813","nonce":377988924}]},"public_addr":"172.21.15.19:6809/377988924","cluster_addr":"172.21.15.19:6811/377988924","heartbeat_back_addr":"172.21.15.19:6815/377988924","heartbeat_front_addr":"172.21.15.19:6813/377988924","state":["exists","up"]},{"osd":5,"uuid":"576cd741-4e7d-421a-a5dd-67ceb0af2440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6817","nonce":3993266086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6819","nonce":3993266086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6823","nonce":3993266086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6821","nonce":3993266086}]},"public_addr":"172.21.15.19:6817/3993266086","cluster_addr":"172.21.15.19:6819/3993266086","heartbeat_back_addr":"172.21.15.19:6823/3993266086","heartbeat_front_addr":"172.21.15.19:6821/3993266086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:54:54.847162+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:23.867331+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:50.950660+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:14.805620+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:40.511669+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:57:05.957371+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6825/4120977378":"2024-03-17T09:09:58.393894+0000","172.21.15.19:6824/4120977378":"2024-03-17T09:09:58.393894+0000","172.21.15.7:6829/3212461733":"2024-03-17T09:09:58.393874+0000","172.21.15.7:0/3257972232":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/780594938":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6800/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3185447390":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/1599723782":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6801/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6828/3212461733":"2024-03-17T09:09:58.393874+0000","172.21.15.19:6829/3745729358":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/1002495884":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/3560098206":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6826/2352390069":"2024-03-17T08:58:44.727553+0000","172.21.15.19:0/3343811353":"2024-03-17T09:02:30.761555+0000","172.21.15.19:0/2272694341":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/1417367188":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/1939052835":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/3493394788":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6801/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3260963601":"2024-03-17T09:05:30.402092+0000","172.21.15.7:0/1698981906":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/3147176153":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3238187551":"2024-03-17T08:53:41.312979+0000","172.21.15.19:0/2491756940":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/2638653648":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/4215733143":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/3022930672":"2024-03-17T08:52:05.202127+0000","172.21.15.7:6827/2352390069":"2024-03-17T08:58:44.727553+0000","172.21.15.7:0/422599095":"2024-03-17T09:00:37.869343+0000","172.21.15.19:6828/3745729358":"2024-03-17T09:02:30.761555+0000","172.21.15.7:6800/2519997150":"2024-03-17T09:00:37.869343+0000","172.21.15.7:6801/2519997150":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/669861183":"2024-03-17T09:05:30.402092+0000","172.21.15.19:0/1724693724":"2024-03-17T09:02:30.761555+0000","172.21.15.7:6800/3493394788":"2024-03-17T09:05:30.402092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f699c06de50 msgr2=0x7f699c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f699c06de50 0x7f699c070300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f69a4004540 tx=0x7f69a4009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fdae0 msgr2=0x7f69b4113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fdae0 0x7f69b4113270 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f69a800c830 tx=0x7f69a8004430 comp rx=0 tx=0).stop 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 shutdown_connections 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f699c06de50 0x7f699c070300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b40fdae0 0x7f69b4113270 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 --2- 172.21.15.7:0/1871739199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69b40fd1d0 0x7f69b4112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.147+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 >> 172.21.15.7:0/1871739199 conn(0x7f69b40f6ca0 msgr2=0x7f69b40f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.148+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 shutdown_connections 2024-03-16T09:12:53.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.148+0000 7f69b9d5d640 1 -- 172.21.15.7:0/1871739199 wait complete. 2024-03-16T09:12:53.160 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:52 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/106990973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:12:53.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:55:53.635803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-16T08:58:27.077630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-16T08:58:27.912287+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:12:53.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:12:53.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:52 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/106990973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:12:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 -- 172.21.15.7:0/370194997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540643a0 msgr2=0x7f4554064770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 --2- 172.21.15.7:0/370194997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540643a0 0x7f4554064770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f45440048d0 tx=0x7f4544009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 -- 172.21.15.7:0/370194997 shutdown_connections 2024-03-16T09:12:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 --2- 172.21.15.7:0/370194997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4554064cb0 0x7f4554108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 --2- 172.21.15.7:0/370194997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540643a0 0x7f4554064770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 -- 172.21.15.7:0/370194997 >> 172.21.15.7:0/370194997 conn(0x7f45540710b0 msgr2=0x7f45540714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 -- 172.21.15.7:0/370194997 shutdown_connections 2024-03-16T09:12:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 -- 172.21.15.7:0/370194997 wait complete. 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 Processor -- start 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.263+0000 7f4558f9d640 1 -- start start 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4558f9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45540643a0 0x7f4554114ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4558f9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4554064cb0 0x7f4554115400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4558f9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4554115a00 con 0x7f4554064cb0 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4558f9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455410f050 con 0x7f45540643a0 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4551d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4554064cb0 0x7f4554115400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4551d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4554064cb0 0x7f4554115400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50380/0 (socket says 172.21.15.7:50380) 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4551d9b640 1 -- 172.21.15.7:0/3161018018 learned_addr learned my addr 172.21.15.7:0/3161018018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f455259c640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45540643a0 0x7f4554114ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4551d9b640 1 -- 172.21.15.7:0/3161018018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45540643a0 msgr2=0x7f4554114ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4551d9b640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45540643a0 0x7f4554114ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4551d9b640 1 -- 172.21.15.7:0/3161018018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544005040 con 0x7f4554064cb0 2024-03-16T09:12:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f455259c640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45540643a0 0x7f4554114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:12:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4551d9b640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4554064cb0 0x7f4554115400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4548002a80 tx=0x7f4548004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4542ffd640 1 -- 172.21.15.7:0/3161018018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4548007c30 con 0x7f4554064cb0 2024-03-16T09:12:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4542ffd640 1 -- 172.21.15.7:0/3161018018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f454800f040 con 0x7f4554064cb0 2024-03-16T09:12:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455410f330 con 0x7f4554064cb0 2024-03-16T09:12:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.264+0000 7f4542ffd640 1 -- 172.21.15.7:0/3161018018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45480096b0 con 0x7f4554064cb0 2024-03-16T09:12:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.265+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455410f850 con 0x7f4554064cb0 2024-03-16T09:12:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.265+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45540fb730 con 0x7f4554064cb0 2024-03-16T09:12:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.266+0000 7f4542ffd640 1 -- 172.21.15.7:0/3161018018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f454800d6b0 con 0x7f4554064cb0 2024-03-16T09:12:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.266+0000 7f4542ffd640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f453c06de50 0x7f453c070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.266+0000 7f4542ffd640 1 -- 172.21.15.7:0/3161018018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f454808d0a0 con 0x7f4554064cb0 2024-03-16T09:12:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.266+0000 7f455259c640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f453c06de50 0x7f453c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.267+0000 7f455259c640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f453c06de50 0x7f453c070300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.271+0000 7f4542ffd640 1 -- 172.21.15.7:0/3161018018 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4548056590 con 0x7f4554064cb0 2024-03-16T09:12:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.464+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4554110530 con 0x7f4554064cb0 2024-03-16T09:12:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.464+0000 7f4542ffd640 1 -- 172.21.15.7:0/3161018018 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4548054e60 con 0x7f4554064cb0 2024-03-16T09:12:53.467 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-16T09:12:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f453c06de50 msgr2=0x7f453c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f453c06de50 0x7f453c070300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4554064cb0 msgr2=0x7f4554115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4554064cb0 0x7f4554115400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4548002a80 tx=0x7f4548004600 comp rx=0 tx=0).stop 2024-03-16T09:12:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 shutdown_connections 2024-03-16T09:12:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f453c06de50 0x7f453c070300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4554064cb0 0x7f4554115400 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 --2- 172.21.15.7:0/3161018018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45540643a0 0x7f4554114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 >> 172.21.15.7:0/3161018018 conn(0x7f45540710b0 msgr2=0x7f45540f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 shutdown_connections 2024-03-16T09:12:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.467+0000 7f4558f9d640 1 -- 172.21.15.7:0/3161018018 wait complete. 2024-03-16T09:12:53.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-16T09:12:53.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.586+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/3689041082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 msgr2=0x7f6bac0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.586+0000 7f6bb1f5d640 1 --2- 172.21.15.7:0/3689041082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 0x7f6bac0fb560 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6b940048d0 tx=0x7f6b94009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.586+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/3689041082 shutdown_connections 2024-03-16T09:12:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.586+0000 7f6bb1f5d640 1 --2- 172.21.15.7:0/3689041082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 0x7f6bac0fb560 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.586+0000 7f6bb1f5d640 1 --2- 172.21.15.7:0/3689041082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bac101140 0x7f6bac101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.586+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/3689041082 >> 172.21.15.7:0/3689041082 conn(0x7f6bac0710b0 msgr2=0x7f6bac0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.586+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/3689041082 shutdown_connections 2024-03-16T09:12:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.586+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/3689041082 wait complete. 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bb1f5d640 1 Processor -- start 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bb1f5d640 1 -- start start 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bb1f5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 0x7f6bac112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bb1f5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bac101140 0x7f6bac113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bb1f5d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac113840 con 0x7f6bac0fb110 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bb1f5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac10ce90 con 0x7f6bac101140 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 0x7f6bac112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 0x7f6bac112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50394/0 (socket says 172.21.15.7:50394) 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bab7fe640 1 -- 172.21.15.7:0/2862386772 learned_addr learned my addr 172.21.15.7:0/2862386772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6ba3fff640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bac101140 0x7f6bac113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bab7fe640 1 -- 172.21.15.7:0/2862386772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bac101140 msgr2=0x7f6bac113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.587+0000 7f6bab7fe640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bac101140 0x7f6bac113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.588+0000 7f6bab7fe640 1 -- 172.21.15.7:0/2862386772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b94005040 con 0x7f6bac0fb110 2024-03-16T09:12:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.588+0000 7f6bab7fe640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 0x7f6bac112d00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c790 tx=0x7f6b9c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.588+0000 7f6ba8ff9640 1 -- 172.21.15.7:0/2862386772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b9c00a820 con 0x7f6bac0fb110 2024-03-16T09:12:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.588+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac10d170 con 0x7f6bac0fb110 2024-03-16T09:12:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.588+0000 7f6ba8ff9640 1 -- 172.21.15.7:0/2862386772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b9c00ae50 con 0x7f6bac0fb110 2024-03-16T09:12:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.588+0000 7f6ba8ff9640 1 -- 172.21.15.7:0/2862386772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b9c0115e0 con 0x7f6bac0fb110 2024-03-16T09:12:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.588+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac10d690 con 0x7f6bac0fb110 2024-03-16T09:12:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.589+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac0fc800 con 0x7f6bac0fb110 2024-03-16T09:12:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.589+0000 7f6ba8ff9640 1 -- 172.21.15.7:0/2862386772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f6b9c012400 con 0x7f6bac0fb110 2024-03-16T09:12:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.590+0000 7f6ba8ff9640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f6b9806de50 0x7f6b98070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.590+0000 7f6ba8ff9640 1 -- 172.21.15.7:0/2862386772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6b9c08d940 con 0x7f6bac0fb110 2024-03-16T09:12:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.590+0000 7f6ba3fff640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f6b9806de50 0x7f6b98070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.590+0000 7f6ba3fff640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f6b9806de50 0x7f6b98070300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6bac10e5c0 tx=0x7f6b94009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:53.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.599+0000 7f6ba8ff9640 1 -- 172.21.15.7:0/2862386772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6b9c056e30 con 0x7f6bac0fb110 2024-03-16T09:12:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.853+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f6bac10e400 con 0x7f6bac0fb110 2024-03-16T09:12:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.854+0000 7f6ba8ff9640 1 -- 172.21.15.7:0/2862386772 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6b9c05a4f0 con 0x7f6bac0fb110 2024-03-16T09:12:53.856 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-16T09:12:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f6b9806de50 msgr2=0x7f6b98070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f6b9806de50 0x7f6b98070300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6bac10e5c0 tx=0x7f6b94009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 msgr2=0x7f6bac112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 0x7f6bac112d00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c790 tx=0x7f6b9c0043c0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 shutdown_connections 2024-03-16T09:12:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f6b9806de50 0x7f6b98070300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bac101140 0x7f6bac113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 --2- 172.21.15.7:0/2862386772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bac0fb110 0x7f6bac112d00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 >> 172.21.15.7:0/2862386772 conn(0x7f6bac0710b0 msgr2=0x7f6bac0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 shutdown_connections 2024-03-16T09:12:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.856+0000 7f6bb1f5d640 1 -- 172.21.15.7:0/2862386772 wait complete. 2024-03-16T09:12:53.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-16T09:12:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.974+0000 7f28da02e640 1 -- 172.21.15.7:0/3418931884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 msgr2=0x7f28d4102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.974+0000 7f28da02e640 1 --2- 172.21.15.7:0/3418931884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 0x7f28d4102590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28c40048d0 tx=0x7f28c4009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.974+0000 7f28da02e640 1 -- 172.21.15.7:0/3418931884 shutdown_connections 2024-03-16T09:12:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.974+0000 7f28da02e640 1 --2- 172.21.15.7:0/3418931884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d4064370 0x7f28d40647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.974+0000 7f28da02e640 1 --2- 172.21.15.7:0/3418931884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 0x7f28d4102590 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.974+0000 7f28da02e640 1 -- 172.21.15.7:0/3418931884 >> 172.21.15.7:0/3418931884 conn(0x7f28d4071120 msgr2=0x7f28d4071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.974+0000 7f28da02e640 1 -- 172.21.15.7:0/3418931884 shutdown_connections 2024-03-16T09:12:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.974+0000 7f28da02e640 1 -- 172.21.15.7:0/3418931884 wait complete. 2024-03-16T09:12:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.975+0000 7f28da02e640 1 Processor -- start 2024-03-16T09:12:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.975+0000 7f28da02e640 1 -- start start 2024-03-16T09:12:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.975+0000 7f28da02e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d4064370 0x7f28d4112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.975+0000 7f28da02e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 0x7f28d4113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.975+0000 7f28da02e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4113830 con 0x7f28d41021c0 2024-03-16T09:12:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.975+0000 7f28da02e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d410ce80 con 0x7f28d4064370 2024-03-16T09:12:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 0x7f28d4113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28d2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 0x7f28d4113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50416/0 (socket says 172.21.15.7:50416) 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28d2ffd640 1 -- 172.21.15.7:0/3233844096 learned_addr learned my addr 172.21.15.7:0/3233844096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28d37fe640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d4064370 0x7f28d4112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28d2ffd640 1 -- 172.21.15.7:0/3233844096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d4064370 msgr2=0x7f28d4112cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28d2ffd640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d4064370 0x7f28d4112cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28d2ffd640 1 -- 172.21.15.7:0/3233844096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c4005040 con 0x7f28d41021c0 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28d2ffd640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 0x7f28d4113230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f28c8002a50 tx=0x7f28c8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28bbfff640 1 -- 172.21.15.7:0/3233844096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c8007c30 con 0x7f28d41021c0 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d410d160 con 0x7f28d41021c0 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28bbfff640 1 -- 172.21.15.7:0/3233844096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28c800f040 con 0x7f28d41021c0 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28bbfff640 1 -- 172.21.15.7:0/3233844096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c800db50 con 0x7f28d41021c0 2024-03-16T09:12:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.976+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d410d680 con 0x7f28d41021c0 2024-03-16T09:12:53.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.977+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d40fd8b0 con 0x7f28d41021c0 2024-03-16T09:12:53.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.978+0000 7f28bbfff640 1 -- 172.21.15.7:0/3233844096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f28c801c400 con 0x7f28d41021c0 2024-03-16T09:12:53.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.978+0000 7f28bbfff640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f28bc06de50 0x7f28bc070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:53.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.979+0000 7f28d37fe640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f28bc06de50 0x7f28bc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:53.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.979+0000 7f28bbfff640 1 -- 172.21.15.7:0/3233844096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f28c808cf90 con 0x7f28d41021c0 2024-03-16T09:12:53.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.979+0000 7f28d37fe640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f28bc06de50 0x7f28bc070300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f28c4003080 tx=0x7f28c4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:53.983+0000 7f28bbfff640 1 -- 172.21.15.7:0/3233844096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f28c80c1740 con 0x7f28d41021c0 2024-03-16T09:12:54.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.166+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f28d4109370 con 0x7f28d41021c0 2024-03-16T09:12:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.167+0000 7f28bbfff640 1 -- 172.21.15.7:0/3233844096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f28c80564b0 con 0x7f28d41021c0 2024-03-16T09:12:54.169 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f28bc06de50 msgr2=0x7f28bc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f28bc06de50 0x7f28bc070300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f28c4003080 tx=0x7f28c4009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 msgr2=0x7f28d4113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 0x7f28d4113230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f28c8002a50 tx=0x7f28c8004600 comp rx=0 tx=0).stop 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 shutdown_connections 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f28bc06de50 0x7f28bc070300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28d41021c0 0x7f28d4113230 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 --2- 172.21.15.7:0/3233844096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d4064370 0x7f28d4112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.169+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 >> 172.21.15.7:0/3233844096 conn(0x7f28d4071120 msgr2=0x7f28d40fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.170+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 shutdown_connections 2024-03-16T09:12:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.170+0000 7f28da02e640 1 -- 172.21.15.7:0/3233844096 wait complete. 2024-03-16T09:12:54.185 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:53 smithi007 ceph-mon[113546]: pgmap v259: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:54.185 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:53 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1871739199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:12:54.185 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:53 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3161018018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:12:54.185 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:53 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2862386772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T09:12:54.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:12:54.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:53 smithi019 ceph-mon[83366]: pgmap v259: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:54.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:53 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1871739199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:12:54.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:53 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3161018018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:12:54.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:53 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2862386772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T09:12:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.294+0000 7fdfc706e640 1 -- 172.21.15.7:0/3297989418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 msgr2=0x7fdfc0106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.294+0000 7fdfc706e640 1 --2- 172.21.15.7:0/3297989418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 0x7fdfc0106320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80048d0 tx=0x7fdfa8009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.294+0000 7fdfc706e640 1 -- 172.21.15.7:0/3297989418 shutdown_connections 2024-03-16T09:12:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.294+0000 7fdfc706e640 1 --2- 172.21.15.7:0/3297989418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 0x7fdfc0106320 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.294+0000 7fdfc706e640 1 --2- 172.21.15.7:0/3297989418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc00fd2b0 0x7fdfc00fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.294+0000 7fdfc706e640 1 -- 172.21.15.7:0/3297989418 >> 172.21.15.7:0/3297989418 conn(0x7fdfc00f6d70 msgr2=0x7fdfc00f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.295+0000 7fdfc706e640 1 -- 172.21.15.7:0/3297989418 shutdown_connections 2024-03-16T09:12:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.295+0000 7fdfc706e640 1 -- 172.21.15.7:0/3297989418 wait complete. 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.295+0000 7fdfc706e640 1 Processor -- start 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.295+0000 7fdfc706e640 1 -- start start 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfc706e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc00fd2b0 0x7fdfc0112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfc706e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 0x7fdfc0113340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfc706e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0113940 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfc706e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc010cf90 con 0x7fdfc00fd2b0 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 0x7fdfc0113340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 0x7fdfc0113340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50436/0 (socket says 172.21.15.7:50436) 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbffff640 1 -- 172.21.15.7:0/3113035862 learned_addr learned my addr 172.21.15.7:0/3113035862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfc4e0a640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc00fd2b0 0x7fdfc0112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbffff640 1 -- 172.21.15.7:0/3113035862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc00fd2b0 msgr2=0x7fdfc0112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbffff640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc00fd2b0 0x7fdfc0112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbffff640 1 -- 172.21.15.7:0/3113035862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa8005040 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbffff640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 0x7fdfc0113340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8004930 tx=0x7fdfa8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbd7fa640 1 -- 172.21.15.7:0/3113035862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa80094c0 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbd7fa640 1 -- 172.21.15.7:0/3113035862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfa8032e50 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc010d210 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.296+0000 7fdfbd7fa640 1 -- 172.21.15.7:0/3113035862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa80433b0 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.297+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc010d6d0 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.298+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc0065db0 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.298+0000 7fdfbd7fa640 1 -- 172.21.15.7:0/3113035862 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdfa802f070 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.298+0000 7fdfbd7fa640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdfac06de50 0x7fdfac070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.298+0000 7fdfbd7fa640 1 -- 172.21.15.7:0/3113035862 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdfa80af590 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.298+0000 7fdfc4e0a640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdfac06de50 0x7fdfac070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.299+0000 7fdfc4e0a640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdfac06de50 0x7fdfac070300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8004fd0 tx=0x7fdfb8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:54.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.303+0000 7fdfbd7fa640 1 -- 172.21.15.7:0/3113035862 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdfa8078ab0 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.565+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfc0109420 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.565+0000 7fdfbd7fa640 1 -- 172.21.15.7:0/3113035862 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5224 (secure 0 0 0) 0x7fdfa807c170 con 0x7fdfc00fdbc0 2024-03-16T09:12:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:54.568 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44329,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/3039505467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3039505467},{"type":"v1","addr":"172.21.15.19:6827","nonce":3039505467}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:11:14.345543+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":0,"incarnation":32,"state":"up:active","state_seq":21,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi007.znurst","rank":1,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34380,"qdb_cluster":[34380,34412]},"id":1}]} 2024-03-16T09:12:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-03-16T09:12:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdfac06de50 msgr2=0x7fdfac070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdfac06de50 0x7fdfac070300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8004fd0 tx=0x7fdfb8009040 comp rx=0 tx=0).stop 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 msgr2=0x7fdfc0113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 0x7fdfc0113340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8004930 tx=0x7fdfa8009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 shutdown_connections 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdfac06de50 0x7fdfac070300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfc00fdbc0 0x7fdfc0113340 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 --2- 172.21.15.7:0/3113035862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfc00fd2b0 0x7fdfc0112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 >> 172.21.15.7:0/3113035862 conn(0x7fdfc00f6d70 msgr2=0x7fdfc0102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 shutdown_connections 2024-03-16T09:12:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.568+0000 7fdfc706e640 1 -- 172.21.15.7:0/3113035862 wait complete. 2024-03-16T09:12:54.583 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 18} 2024-03-16T09:12:54.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-03-16T09:12:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.684+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2957531891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab98102c70 msgr2=0x7fab980fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.684+0000 7fab9eb9d640 1 --2- 172.21.15.7:0/2957531891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab98102c70 0x7fab980fc1f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab840048d0 tx=0x7fab84009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.684+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2957531891 shutdown_connections 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.684+0000 7fab9eb9d640 1 --2- 172.21.15.7:0/2957531891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab98102c70 0x7fab980fc1f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.684+0000 7fab9eb9d640 1 --2- 172.21.15.7:0/2957531891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab981022d0 0x7fab981026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.684+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2957531891 >> 172.21.15.7:0/2957531891 conn(0x7fab980f7d90 msgr2=0x7fab980fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.684+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2957531891 shutdown_connections 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.684+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2957531891 wait complete. 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9eb9d640 1 Processor -- start 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9eb9d640 1 -- start start 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9eb9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab981022d0 0x7fab9806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9eb9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab98102c70 0x7fab9806fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9eb9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab980716b0 con 0x7fab981022d0 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9eb9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab98070250 con 0x7fab98102c70 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9c939640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab981022d0 0x7fab9806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9c939640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab981022d0 0x7fab9806f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50460/0 (socket says 172.21.15.7:50460) 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9c939640 1 -- 172.21.15.7:0/2291496183 learned_addr learned my addr 172.21.15.7:0/2291496183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab97fff640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab98102c70 0x7fab9806fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.685+0000 7fab9c939640 1 -- 172.21.15.7:0/2291496183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab98102c70 msgr2=0x7fab9806fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab9c939640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab98102c70 0x7fab9806fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab9c939640 1 -- 172.21.15.7:0/2291496183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab84005040 con 0x7fab981022d0 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab97fff640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab98102c70 0x7fab9806fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab9c939640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab981022d0 0x7fab9806f7d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00eb70 tx=0x7fab8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab957fa640 1 -- 172.21.15.7:0/2291496183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab8c007c30 con 0x7fab981022d0 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab980704d0 con 0x7fab981022d0 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab957fa640 1 -- 172.21.15.7:0/2291496183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab8c00f040 con 0x7fab981022d0 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab957fa640 1 -- 172.21.15.7:0/2291496183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab8c0095b0 con 0x7fab981022d0 2024-03-16T09:12:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.686+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab981194e0 con 0x7fab981022d0 2024-03-16T09:12:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.687+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9804bcc0 con 0x7fab981022d0 2024-03-16T09:12:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.688+0000 7fab957fa640 1 -- 172.21.15.7:0/2291496183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fab8c009730 con 0x7fab981022d0 2024-03-16T09:12:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.688+0000 7fab957fa640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fab8806de60 0x7fab88070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.689+0000 7fab97fff640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fab8806de60 0x7fab88070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.689+0000 7fab957fa640 1 -- 172.21.15.7:0/2291496183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fab8c08d510 con 0x7fab981022d0 2024-03-16T09:12:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.689+0000 7fab97fff640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fab8806de60 0x7fab88070310 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab840048a0 tx=0x7fab84009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:54.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.693+0000 7fab957fa640 1 -- 172.21.15.7:0/2291496183 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab8c056980 con 0x7fab981022d0 2024-03-16T09:12:54.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.973+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fab980fdaa0 con 0x7fab981022d0 2024-03-16T09:12:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.974+0000 7fab957fa640 1 -- 172.21.15.7:0/2291496183 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v36) v1 ==== 107+0+4864 (secure 0 0 0) 0x7fab8c05a040 con 0x7fab981022d0 2024-03-16T09:12:54.976 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:54.976 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2680162193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2680162193},{"type":"v1","addr":"172.21.15.7:6827","nonce":2680162193}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:00:35.303896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14482,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14482":{"gid":14482,"name":"cephfs.smithi007.znurst","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.7:6829/3212461733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3212461733},{"type":"v1","addr":"172.21.15.7:6829","nonce":3212461733}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi019.bncsil","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6825/4120977378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":4120977378},{"type":"v1","addr":"172.21.15.19:6825","nonce":4120977378}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fab8806de60 msgr2=0x7fab88070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fab8806de60 0x7fab88070310 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab840048a0 tx=0x7fab84009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab981022d0 msgr2=0x7fab9806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab981022d0 0x7fab9806f7d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00eb70 tx=0x7fab8c00b040 comp rx=0 tx=0).stop 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 shutdown_connections 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fab8806de60 0x7fab88070310 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab98102c70 0x7fab9806fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 --2- 172.21.15.7:0/2291496183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab981022d0 0x7fab9806f7d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 >> 172.21.15.7:0/2291496183 conn(0x7fab980f7d90 msgr2=0x7fab980f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 shutdown_connections 2024-03-16T09:12:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:54.976+0000 7fab9eb9d640 1 -- 172.21.15.7:0/2291496183 wait complete. 2024-03-16T09:12:54.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-16T09:12:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 -- 172.21.15.7:0/3527053825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 msgr2=0x7f1ad81014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 --2- 172.21.15.7:0/3527053825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 0x7f1ad81014a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009c50 tx=0x7f1ad0004d10 comp rx=0 tx=0).stop 2024-03-16T09:12:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 -- 172.21.15.7:0/3527053825 shutdown_connections 2024-03-16T09:12:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 --2- 172.21.15.7:0/3527053825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad8101a70 0x7f1ad80fb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 --2- 172.21.15.7:0/3527053825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 0x7f1ad81014a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 -- 172.21.15.7:0/3527053825 >> 172.21.15.7:0/3527053825 conn(0x7f1ad80f6ca0 msgr2=0x7f1ad80f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 -- 172.21.15.7:0/3527053825 shutdown_connections 2024-03-16T09:12:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 -- 172.21.15.7:0/3527053825 wait complete. 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 Processor -- start 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.116+0000 7f1adf15d640 1 -- start start 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adf15d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 0x7f1ad810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adf15d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad8101a70 0x7f1ad810e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adf15d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8065fc0 con 0x7f1ad81010d0 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adf15d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad810e8f0 con 0x7f1ad8101a70 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adcef9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 0x7f1ad810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adcef9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 0x7f1ad810de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50478/0 (socket says 172.21.15.7:50478) 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adcef9640 1 -- 172.21.15.7:0/3230394178 learned_addr learned my addr 172.21.15.7:0/3230394178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1ad7fff640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad8101a70 0x7f1ad810e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adcef9640 1 -- 172.21.15.7:0/3230394178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad8101a70 msgr2=0x7f1ad810e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adcef9640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad8101a70 0x7f1ad810e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adcef9640 1 -- 172.21.15.7:0/3230394178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00098b0 con 0x7f1ad81010d0 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.117+0000 7f1adcef9640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 0x7f1ad810de70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0004a30 tx=0x7f1ad0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:55.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.118+0000 7f1ad57fa640 1 -- 172.21.15.7:0/3230394178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ad003a070 con 0x7f1ad81010d0 2024-03-16T09:12:55.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.118+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad810eb70 con 0x7f1ad81010d0 2024-03-16T09:12:55.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.118+0000 7f1ad57fa640 1 -- 172.21.15.7:0/3230394178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ad0035b20 con 0x7f1ad81010d0 2024-03-16T09:12:55.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.118+0000 7f1ad57fa640 1 -- 172.21.15.7:0/3230394178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ad003ea70 con 0x7f1ad81010d0 2024-03-16T09:12:55.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.118+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad8117130 con 0x7f1ad81010d0 2024-03-16T09:12:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.119+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad80fc840 con 0x7f1ad81010d0 2024-03-16T09:12:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.120+0000 7f1ad57fa640 1 -- 172.21.15.7:0/3230394178 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f1ad0032070 con 0x7f1ad81010d0 2024-03-16T09:12:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.120+0000 7f1ad57fa640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1ac006de50 0x7f1ac0070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.120+0000 7f1ad57fa640 1 -- 172.21.15.7:0/3230394178 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1ad00af970 con 0x7f1ad81010d0 2024-03-16T09:12:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.120+0000 7f1ad7fff640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1ac006de50 0x7f1ac0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.120+0000 7f1ad7fff640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1ac006de50 0x7f1ac0070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009af0 tx=0x7f1acc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.127+0000 7f1ad57fa640 1 -- 172.21.15.7:0/3230394178 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1ad0078e90 con 0x7f1ad81010d0 2024-03-16T09:12:55.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:54 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3233844096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T09:12:55.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:54 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3113035862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:12:55.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:54 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3233844096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T09:12:55.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:54 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3113035862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:12:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.363+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f1ad804bcc0 con 0x7f1ad81010d0 2024-03-16T09:12:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.363+0000 7f1ad57fa640 1 -- 172.21.15.7:0/3230394178 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v36) v1 ==== 107+0+4865 (secure 0 0 0) 0x7f1ad007c550 con 0x7f1ad81010d0 2024-03-16T09:12:55.366 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:55.366 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2680162193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2680162193},{"type":"v1","addr":"172.21.15.7:6827","nonce":2680162193}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:01:31.167053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14482,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14482":{"gid":14482,"name":"cephfs.smithi007.znurst","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.7:6829/3212461733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3212461733},{"type":"v1","addr":"172.21.15.7:6829","nonce":3212461733}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi019.bncsil","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6825/4120977378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":4120977378},{"type":"v1","addr":"172.21.15.19:6825","nonce":4120977378}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1ac006de50 msgr2=0x7f1ac0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1ac006de50 0x7f1ac0070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009af0 tx=0x7f1acc00b040 comp rx=0 tx=0).stop 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 msgr2=0x7f1ad810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 0x7f1ad810de70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0004a30 tx=0x7f1ad0004d10 comp rx=0 tx=0).stop 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 shutdown_connections 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1ac006de50 0x7f1ac0070300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ad8101a70 0x7f1ad810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 --2- 172.21.15.7:0/3230394178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ad81010d0 0x7f1ad810de70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 >> 172.21.15.7:0/3230394178 conn(0x7f1ad80f6ca0 msgr2=0x7f1ad80f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 shutdown_connections 2024-03-16T09:12:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.366+0000 7f1adf15d640 1 -- 172.21.15.7:0/3230394178 wait complete. 2024-03-16T09:12:55.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-16T09:12:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.483+0000 7fdb6195d640 1 -- 172.21.15.7:0/3027930617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c0643a0 msgr2=0x7fdb5c0647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.483+0000 7fdb6195d640 1 --2- 172.21.15.7:0/3027930617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c0643a0 0x7fdb5c0647d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdb500048d0 tx=0x7fdb50009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.483+0000 7fdb6195d640 1 -- 172.21.15.7:0/3027930617 shutdown_connections 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.483+0000 7fdb6195d640 1 --2- 172.21.15.7:0/3027930617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c0643a0 0x7fdb5c0647d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.483+0000 7fdb6195d640 1 --2- 172.21.15.7:0/3027930617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb5c102240 0x7fdb5c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.483+0000 7fdb6195d640 1 -- 172.21.15.7:0/3027930617 >> 172.21.15.7:0/3027930617 conn(0x7fdb5c0710b0 msgr2=0x7fdb5c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.483+0000 7fdb6195d640 1 -- 172.21.15.7:0/3027930617 shutdown_connections 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.483+0000 7fdb6195d640 1 -- 172.21.15.7:0/3027930617 wait complete. 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb6195d640 1 Processor -- start 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb6195d640 1 -- start start 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb6195d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb5c0643a0 0x7fdb5c112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb6195d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c102240 0x7fdb5c113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb6195d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c113810 con 0x7fdb5c102240 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb6195d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c10ce60 con 0x7fdb5c0643a0 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb5a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c102240 0x7fdb5c113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb5a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c102240 0x7fdb5c113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50488/0 (socket says 172.21.15.7:50488) 2024-03-16T09:12:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb5a7fc640 1 -- 172.21.15.7:0/2784241154 learned_addr learned my addr 172.21.15.7:0/2784241154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb5affd640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb5c0643a0 0x7fdb5c112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb5a7fc640 1 -- 172.21.15.7:0/2784241154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb5c0643a0 msgr2=0x7fdb5c112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb5a7fc640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb5c0643a0 0x7fdb5c112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.484+0000 7fdb5a7fc640 1 -- 172.21.15.7:0/2784241154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb50005040 con 0x7fdb5c102240 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.485+0000 7fdb5affd640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb5c0643a0 0x7fdb5c112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.485+0000 7fdb5a7fc640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c102240 0x7fdb5c113210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb50004930 tx=0x7fdb50009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.485+0000 7fdb437fe640 1 -- 172.21.15.7:0/2784241154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb500094c0 con 0x7fdb5c102240 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.485+0000 7fdb437fe640 1 -- 172.21.15.7:0/2784241154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb50032e50 con 0x7fdb5c102240 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.485+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c10d0e0 con 0x7fdb5c102240 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.485+0000 7fdb437fe640 1 -- 172.21.15.7:0/2784241154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb500433b0 con 0x7fdb5c102240 2024-03-16T09:12:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.485+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c10d5a0 con 0x7fdb5c102240 2024-03-16T09:12:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.486+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c0fd940 con 0x7fdb5c102240 2024-03-16T09:12:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.486+0000 7fdb437fe640 1 -- 172.21.15.7:0/2784241154 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdb5002f070 con 0x7fdb5c102240 2024-03-16T09:12:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.487+0000 7fdb437fe640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb4406de50 0x7fdb44070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.487+0000 7fdb437fe640 1 -- 172.21.15.7:0/2784241154 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fdb500af7a0 con 0x7fdb5c102240 2024-03-16T09:12:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.487+0000 7fdb5affd640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb4406de50 0x7fdb44070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.487+0000 7fdb5affd640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb4406de50 0x7fdb44070300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0041d0 tx=0x7fdb4c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.492+0000 7fdb437fe640 1 -- 172.21.15.7:0/2784241154 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb50078c10 con 0x7fdb5c102240 2024-03-16T09:12:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.714+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fdb5c109320 con 0x7fdb5c102240 2024-03-16T09:12:55.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.714+0000 7fdb437fe640 1 -- 172.21.15.7:0/2784241154 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v36) v1 ==== 107+0+4864 (secure 0 0 0) 0x7fdb5007c2d0 con 0x7fdb5c102240 2024-03-16T09:12:55.717 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2680162193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2680162193},{"type":"v1","addr":"172.21.15.7:6827","nonce":2680162193}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:04:55.865707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14482,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14482":{"gid":14482,"name":"cephfs.smithi007.znurst","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.7:6829/3212461733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3212461733},{"type":"v1","addr":"172.21.15.7:6829","nonce":3212461733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi019.bncsil","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6825/4120977378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":4120977378},{"type":"v1","addr":"172.21.15.19:6825","nonce":4120977378}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:55.717 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-03-16T09:12:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb4406de50 msgr2=0x7fdb44070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb4406de50 0x7fdb44070300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0041d0 tx=0x7fdb4c009040 comp rx=0 tx=0).stop 2024-03-16T09:12:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c102240 msgr2=0x7fdb5c113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c102240 0x7fdb5c113210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb50004930 tx=0x7fdb50009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 shutdown_connections 2024-03-16T09:12:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fdb4406de50 0x7fdb44070300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb5c102240 0x7fdb5c113210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 --2- 172.21.15.7:0/2784241154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb5c0643a0 0x7fdb5c112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 >> 172.21.15.7:0/2784241154 conn(0x7fdb5c0710b0 msgr2=0x7fdb5c0f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 shutdown_connections 2024-03-16T09:12:55.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.716+0000 7fdb6195d640 1 -- 172.21.15.7:0/2784241154 wait complete. 2024-03-16T09:12:55.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-16T09:12:55.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.833+0000 7f708531d640 1 -- 172.21.15.7:0/4230820007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 msgr2=0x7f7080101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.833+0000 7f708531d640 1 --2- 172.21.15.7:0/4230820007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 0x7f7080101500 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70700048d0 tx=0x7f7070009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.833+0000 7f708531d640 1 -- 172.21.15.7:0/4230820007 shutdown_connections 2024-03-16T09:12:55.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.833+0000 7f708531d640 1 --2- 172.21.15.7:0/4230820007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080101a40 0x7f70800fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.833+0000 7f708531d640 1 --2- 172.21.15.7:0/4230820007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 0x7f7080101500 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.833+0000 7f708531d640 1 -- 172.21.15.7:0/4230820007 >> 172.21.15.7:0/4230820007 conn(0x7f7080070880 msgr2=0x7f7080070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:55.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.833+0000 7f708531d640 1 -- 172.21.15.7:0/4230820007 shutdown_connections 2024-03-16T09:12:55.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.833+0000 7f708531d640 1 -- 172.21.15.7:0/4230820007 wait complete. 2024-03-16T09:12:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.834+0000 7f708531d640 1 Processor -- start 2024-03-16T09:12:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.834+0000 7f708531d640 1 -- start start 2024-03-16T09:12:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.834+0000 7f708531d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 0x7f7080111c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.834+0000 7f708531d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080101a40 0x7f70801121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.834+0000 7f708531d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70801127b0 con 0x7f7080101130 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.834+0000 7f708531d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708010be00 con 0x7f7080101a40 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.834+0000 7f707effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 0x7f7080111c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f707effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 0x7f7080111c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50504/0 (socket says 172.21.15.7:50504) 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f707effd640 1 -- 172.21.15.7:0/4125146492 learned_addr learned my addr 172.21.15.7:0/4125146492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f707e7fc640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080101a40 0x7f70801121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f707effd640 1 -- 172.21.15.7:0/4125146492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080101a40 msgr2=0x7f70801121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f707effd640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080101a40 0x7f70801121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f707effd640 1 -- 172.21.15.7:0/4125146492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070005040 con 0x7f7080101130 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f707effd640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 0x7f7080111c70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7070030640 tx=0x7f7070009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f70677fe640 1 -- 172.21.15.7:0/4125146492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7070030820 con 0x7f7080101130 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708010c080 con 0x7f7080101130 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f70677fe640 1 -- 172.21.15.7:0/4125146492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70700092e0 con 0x7f7080101130 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f70677fe640 1 -- 172.21.15.7:0/4125146492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7070043400 con 0x7f7080101130 2024-03-16T09:12:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.835+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708010c5a0 con 0x7f7080101130 2024-03-16T09:12:55.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.836+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70800fc9c0 con 0x7f7080101130 2024-03-16T09:12:55.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.837+0000 7f70677fe640 1 -- 172.21.15.7:0/4125146492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f707003b420 con 0x7f7080101130 2024-03-16T09:12:55.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.837+0000 7f70677fe640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f706806de50 0x7f7068070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.837+0000 7f70677fe640 1 -- 172.21.15.7:0/4125146492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f70700b0420 con 0x7f7080101130 2024-03-16T09:12:55.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.837+0000 7f707e7fc640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f706806de50 0x7f7068070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.838+0000 7f707e7fc640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f706806de50 0x7f7068070300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f708010d530 tx=0x7f707400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:55.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:55.842+0000 7f70677fe640 1 -- 172.21.15.7:0/4125146492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f70700799d0 con 0x7f7080101130 2024-03-16T09:12:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:55 smithi007 ceph-mon[113546]: pgmap v260: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:55 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2291496183' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-16T09:12:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:55 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3230394178' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-16T09:12:55.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:55 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2784241154' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-16T09:12:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.089+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f708004bcc0 con 0x7f7080101130 2024-03-16T09:12:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.090+0000 7f70677fe640 1 -- 172.21.15.7:0/4125146492 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v36) v1 ==== 107+0+4878 (secure 0 0 0) 0x7f707007d090 con 0x7f7080101130 2024-03-16T09:12:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:56.092 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2680162193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2680162193},{"type":"v1","addr":"172.21.15.7:6827","nonce":2680162193}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:05:26.042398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14482,"mds_1":24281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14482":{"gid":14482,"name":"cephfs.smithi007.znurst","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.7:6829/3212461733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3212461733},{"type":"v1","addr":"172.21.15.7:6829","nonce":3212461733}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24281":{"gid":24281,"name":"cephfs.smithi019.bncsil","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6825/4120977378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":4120977378},{"type":"v1","addr":"172.21.15.19:6825","nonce":4120977378}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14482,"qdb_cluster":[14482,24281]},"id":1}]} 2024-03-16T09:12:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-03-16T09:12:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f706806de50 msgr2=0x7f7068070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f706806de50 0x7f7068070300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f708010d530 tx=0x7f707400b040 comp rx=0 tx=0).stop 2024-03-16T09:12:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 msgr2=0x7f7080111c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 0x7f7080111c70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7070030640 tx=0x7f7070009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 shutdown_connections 2024-03-16T09:12:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f706806de50 0x7f7068070300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7080101a40 0x7f70801121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 --2- 172.21.15.7:0/4125146492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7080101130 0x7f7080111c70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 >> 172.21.15.7:0/4125146492 conn(0x7f7080070880 msgr2=0x7f70800f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 shutdown_connections 2024-03-16T09:12:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.092+0000 7f708531d640 1 -- 172.21.15.7:0/4125146492 wait complete. 2024-03-16T09:12:56.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-16T09:12:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.204+0000 7fd32079d640 1 -- 172.21.15.7:0/1288099595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 msgr2=0x7fd3180fb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.204+0000 7fd32079d640 1 --2- 172.21.15.7:0/1288099595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 0x7fd3180fb310 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3080048d0 tx=0x7fd308009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.204+0000 7fd32079d640 1 -- 172.21.15.7:0/1288099595 shutdown_connections 2024-03-16T09:12:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.204+0000 7fd32079d640 1 --2- 172.21.15.7:0/1288099595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 0x7fd3180fb310 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.204+0000 7fd32079d640 1 --2- 172.21.15.7:0/1288099595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3181012e0 0x7fd3181016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.204+0000 7fd32079d640 1 -- 172.21.15.7:0/1288099595 >> 172.21.15.7:0/1288099595 conn(0x7fd3180f6e70 msgr2=0x7fd3180f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.204+0000 7fd32079d640 1 -- 172.21.15.7:0/1288099595 shutdown_connections 2024-03-16T09:12:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.204+0000 7fd32079d640 1 -- 172.21.15.7:0/1288099595 wait complete. 2024-03-16T09:12:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd32079d640 1 Processor -- start 2024-03-16T09:12:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd32079d640 1 -- start start 2024-03-16T09:12:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd32079d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3181012e0 0x7fd318112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd32079d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 0x7fd318113430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd32079d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318113a30 con 0x7fd318101c80 2024-03-16T09:12:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd32079d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318113ba0 con 0x7fd3181012e0 2024-03-16T09:12:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd31dd38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 0x7fd318113430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd31dd38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 0x7fd318113430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50530/0 (socket says 172.21.15.7:50530) 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd31dd38640 1 -- 172.21.15.7:0/2646668041 learned_addr learned my addr 172.21.15.7:0/2646668041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd31e539640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3181012e0 0x7fd318112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd31dd38640 1 -- 172.21.15.7:0/2646668041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3181012e0 msgr2=0x7fd318112ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd31dd38640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3181012e0 0x7fd318112ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.205+0000 7fd31dd38640 1 -- 172.21.15.7:0/2646668041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308005040 con 0x7fd318101c80 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.206+0000 7fd31dd38640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 0x7fd318113430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd308004930 tx=0x7fd308009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.206+0000 7fd30effd640 1 -- 172.21.15.7:0/2646668041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3080094c0 con 0x7fd318101c80 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.206+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31810d1d0 con 0x7fd318101c80 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.206+0000 7fd30effd640 1 -- 172.21.15.7:0/2646668041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd308032e50 con 0x7fd318101c80 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.206+0000 7fd30effd640 1 -- 172.21.15.7:0/2646668041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3080433b0 con 0x7fd318101c80 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.206+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31810d690 con 0x7fd318101c80 2024-03-16T09:12:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.207+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3180fca50 con 0x7fd318101c80 2024-03-16T09:12:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.208+0000 7fd30effd640 1 -- 172.21.15.7:0/2646668041 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fd30802f070 con 0x7fd318101c80 2024-03-16T09:12:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.208+0000 7fd30effd640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd30406de50 0x7fd304070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.208+0000 7fd31e539640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd30406de50 0x7fd304070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.208+0000 7fd30effd640 1 -- 172.21.15.7:0/2646668041 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd3080af590 con 0x7fd318101c80 2024-03-16T09:12:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.209+0000 7fd31e539640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd30406de50 0x7fd304070300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd314004eb0 tx=0x7fd31400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:56.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:55 smithi019 ceph-mon[83366]: pgmap v260: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:55 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2291496183' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-16T09:12:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:55 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3230394178' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-16T09:12:56.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:55 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2784241154' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-16T09:12:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.214+0000 7fd30effd640 1 -- 172.21.15.7:0/2646668041 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd308078ab0 con 0x7fd318101c80 2024-03-16T09:12:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.444+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fd31804bcc0 con 0x7fd318101c80 2024-03-16T09:12:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.444+0000 7fd30effd640 1 -- 172.21.15.7:0/2646668041 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v36) v1 ==== 107+0+3287 (secure 0 0 0) 0x7fd30803fcf0 con 0x7fd318101c80 2024-03-16T09:12:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:56.447 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2680162193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2680162193},{"type":"v1","addr":"172.21.15.7:6827","nonce":2680162193}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 21 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd30406de50 msgr2=0x7fd304070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd30406de50 0x7fd304070300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd314004eb0 tx=0x7fd31400b040 comp rx=0 tx=0).stop 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 msgr2=0x7fd318113430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 0x7fd318113430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd308004930 tx=0x7fd308009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 shutdown_connections 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fd30406de50 0x7fd304070300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd318101c80 0x7fd318113430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 --2- 172.21.15.7:0/2646668041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3181012e0 0x7fd318112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.446+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 >> 172.21.15.7:0/2646668041 conn(0x7fd3180f6e70 msgr2=0x7fd3180f7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.447+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 shutdown_connections 2024-03-16T09:12:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.447+0000 7fd32079d640 1 -- 172.21.15.7:0/2646668041 wait complete. 2024-03-16T09:12:56.460 DEBUG:tasks.fs:max_mds not reduced in epoch 21 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:56.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-16T09:12:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 -- 172.21.15.7:0/1986211312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fb1b0 msgr2=0x7f03380fb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 --2- 172.21.15.7:0/1986211312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fb1b0 0x7f03380fb580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03280048d0 tx=0x7f0328009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 -- 172.21.15.7:0/1986211312 shutdown_connections 2024-03-16T09:12:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 --2- 172.21.15.7:0/1986211312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03380fbac0 0x7f0338108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 --2- 172.21.15.7:0/1986211312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fb1b0 0x7f03380fb580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 -- 172.21.15.7:0/1986211312 >> 172.21.15.7:0/1986211312 conn(0x7f03380f6ea0 msgr2=0x7f03380f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 -- 172.21.15.7:0/1986211312 shutdown_connections 2024-03-16T09:12:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 -- 172.21.15.7:0/1986211312 wait complete. 2024-03-16T09:12:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 Processor -- start 2024-03-16T09:12:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.560+0000 7f033e51d640 1 -- start start 2024-03-16T09:12:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f033e51d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03380fb1b0 0x7f03381150d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f033e51d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fbac0 0x7f0338115610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f033e51d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338115c10 con 0x7f03380fbac0 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f033e51d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033810f260 con 0x7f03380fb1b0 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f03377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fbac0 0x7f0338115610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f03377fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fbac0 0x7f0338115610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50544/0 (socket says 172.21.15.7:50544) 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f03377fe640 1 -- 172.21.15.7:0/2184205265 learned_addr learned my addr 172.21.15.7:0/2184205265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f03377fe640 1 -- 172.21.15.7:0/2184205265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03380fb1b0 msgr2=0x7f03381150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f0337fff640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03380fb1b0 0x7f03381150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f03377fe640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03380fb1b0 0x7f03381150d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f03377fe640 1 -- 172.21.15.7:0/2184205265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328005040 con 0x7f03380fbac0 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f03377fe640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fbac0 0x7f0338115610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f032c002a50 tx=0x7f032c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f0334ff9640 1 -- 172.21.15.7:0/2184205265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f032c007c30 con 0x7f03380fbac0 2024-03-16T09:12:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033810f540 con 0x7f03380fbac0 2024-03-16T09:12:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f0334ff9640 1 -- 172.21.15.7:0/2184205265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f032c00f040 con 0x7f03380fbac0 2024-03-16T09:12:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f0334ff9640 1 -- 172.21.15.7:0/2184205265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f032c00db50 con 0x7f03380fbac0 2024-03-16T09:12:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.561+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033810fa60 con 0x7f03380fbac0 2024-03-16T09:12:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.563+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0338065d70 con 0x7f03380fbac0 2024-03-16T09:12:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.563+0000 7f0334ff9640 1 -- 172.21.15.7:0/2184205265 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f032c01c400 con 0x7f03380fbac0 2024-03-16T09:12:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.564+0000 7f0334ff9640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f032006de50 0x7f0320070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.564+0000 7f0334ff9640 1 -- 172.21.15.7:0/2184205265 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f032c08d040 con 0x7f03380fbac0 2024-03-16T09:12:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.564+0000 7f0337fff640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f032006de50 0x7f0320070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.564+0000 7f0337fff640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f032006de50 0x7f0320070300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f03280043d0 tx=0x7f0328009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.572+0000 7f0334ff9640 1 -- 172.21.15.7:0/2184205265 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f032c0564b0 con 0x7f03380fbac0 2024-03-16T09:12:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.817+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f033810b760 con 0x7f03380fbac0 2024-03-16T09:12:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.818+0000 7f0334ff9640 1 -- 172.21.15.7:0/2184205265 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v36) v1 ==== 107+0+4833 (secure 0 0 0) 0x7f032c059b70 con 0x7f03380fbac0 2024-03-16T09:12:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:56.820 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2680162193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2680162193},{"type":"v1","addr":"172.21.15.7:6827","nonce":2680162193}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34372,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/3988729523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3988729523},{"type":"v1","addr":"172.21.15.7:6829","nonce":3988729523}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44275,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/5723851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":5723851},{"type":"v1","addr":"172.21.15.19:6825","nonce":5723851}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-03-16T09:12:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f032006de50 msgr2=0x7f0320070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f032006de50 0x7f0320070300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f03280043d0 tx=0x7f0328009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fbac0 msgr2=0x7f0338115610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fbac0 0x7f0338115610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f032c002a50 tx=0x7f032c004600 comp rx=0 tx=0).stop 2024-03-16T09:12:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 shutdown_connections 2024-03-16T09:12:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f032006de50 0x7f0320070300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380fbac0 0x7f0338115610 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 --2- 172.21.15.7:0/2184205265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03380fb1b0 0x7f03381150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 >> 172.21.15.7:0/2184205265 conn(0x7f03380f6ea0 msgr2=0x7f03380f72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 shutdown_connections 2024-03-16T09:12:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.820+0000 7f033e51d640 1 -- 172.21.15.7:0/2184205265 wait complete. 2024-03-16T09:12:56.833 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:56.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-16T09:12:56.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.934+0000 7f7f4ec65640 1 -- 172.21.15.7:0/3974100473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f480643a0 msgr2=0x7f7f480647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.934+0000 7f7f4ec65640 1 --2- 172.21.15.7:0/3974100473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f480643a0 0x7f7f480647d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0048d0 tx=0x7f7f3c009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.934+0000 7f7f4ec65640 1 -- 172.21.15.7:0/3974100473 shutdown_connections 2024-03-16T09:12:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.934+0000 7f7f4ec65640 1 --2- 172.21.15.7:0/3974100473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f480643a0 0x7f7f480647d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.934+0000 7f7f4ec65640 1 --2- 172.21.15.7:0/3974100473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f48102240 0x7f7f48102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.934+0000 7f7f4ec65640 1 -- 172.21.15.7:0/3974100473 >> 172.21.15.7:0/3974100473 conn(0x7f7f480710b0 msgr2=0x7f7f480714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.934+0000 7f7f4ec65640 1 -- 172.21.15.7:0/3974100473 shutdown_connections 2024-03-16T09:12:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.934+0000 7f7f4ec65640 1 -- 172.21.15.7:0/3974100473 wait complete. 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f4ec65640 1 Processor -- start 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f4ec65640 1 -- start start 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f4ec65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f480643a0 0x7f7f480fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f4ec65640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f48102240 0x7f7f480fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f4ec65640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480fd0c0 con 0x7f7f48102240 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f4ec65640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480fbc60 con 0x7f7f480643a0 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f48102240 0x7f7f480fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f47fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f48102240 0x7f7f480fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50564/0 (socket says 172.21.15.7:50564) 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.935+0000 7f7f47fff640 1 -- 172.21.15.7:0/149857821 learned_addr learned my addr 172.21.15.7:0/149857821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f4ca01640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f480643a0 0x7f7f480fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f47fff640 1 -- 172.21.15.7:0/149857821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f480643a0 msgr2=0x7f7f480fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f47fff640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f480643a0 0x7f7f480fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f47fff640 1 -- 172.21.15.7:0/149857821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c005040 con 0x7f7f48102240 2024-03-16T09:12:56.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f47fff640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f48102240 0x7f7f480fb720 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004930 tx=0x7f7f3c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:56.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f457fa640 1 -- 172.21.15.7:0/149857821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3c0094c0 con 0x7f7f48102240 2024-03-16T09:12:56.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f480fbee0 con 0x7f7f48102240 2024-03-16T09:12:56.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f457fa640 1 -- 172.21.15.7:0/149857821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f3c032e50 con 0x7f7f48102240 2024-03-16T09:12:56.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f457fa640 1 -- 172.21.15.7:0/149857821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3c0433b0 con 0x7f7f48102240 2024-03-16T09:12:56.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.936+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f48119390 con 0x7f7f48102240 2024-03-16T09:12:56.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.937+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f480fd940 con 0x7f7f48102240 2024-03-16T09:12:56.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.938+0000 7f7f457fa640 1 -- 172.21.15.7:0/149857821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f7f3c02f070 con 0x7f7f48102240 2024-03-16T09:12:56.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.938+0000 7f7f457fa640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7f3006de50 0x7f7f30070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:56.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.939+0000 7f7f457fa640 1 -- 172.21.15.7:0/149857821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7f3c0af5b0 con 0x7f7f48102240 2024-03-16T09:12:56.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.939+0000 7f7f4ca01640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7f3006de50 0x7f7f30070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:56.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.939+0000 7f7f4ca01640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7f3006de50 0x7f7f30070300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009470 tx=0x7f7f38009ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:56.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:56.943+0000 7f7f457fa640 1 -- 172.21.15.7:0/149857821 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f3c078a20 con 0x7f7f48102240 2024-03-16T09:12:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.166+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f7f48109330 con 0x7f7f48102240 2024-03-16T09:12:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.167+0000 7f7f457fa640 1 -- 172.21.15.7:0/149857821 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v36) v1 ==== 107+0+4057 (secure 0 0 0) 0x7f7f3c07c0e0 con 0x7f7f48102240 2024-03-16T09:12:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:57.169 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34372,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/3988729523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3988729523},{"type":"v1","addr":"172.21.15.7:6829","nonce":3988729523}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44275,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/5723851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":5723851},{"type":"v1","addr":"172.21.15.19:6825","nonce":5723851}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-03-16T09:12:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7f3006de50 msgr2=0x7f7f30070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7f3006de50 0x7f7f30070300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009470 tx=0x7f7f38009ed0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f48102240 msgr2=0x7f7f480fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f48102240 0x7f7f480fb720 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004930 tx=0x7f7f3c009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 shutdown_connections 2024-03-16T09:12:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f7f3006de50 0x7f7f30070300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f48102240 0x7f7f480fb720 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 --2- 172.21.15.7:0/149857821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f480643a0 0x7f7f480fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 >> 172.21.15.7:0/149857821 conn(0x7f7f480710b0 msgr2=0x7f7f480f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 shutdown_connections 2024-03-16T09:12:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.169+0000 7f7f4ec65640 1 -- 172.21.15.7:0/149857821 wait complete. 2024-03-16T09:12:57.181 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:56 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/4125146492' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-16T09:12:57.181 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:56 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2646668041' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-16T09:12:57.181 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:56 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2184205265' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-16T09:12:57.182 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:57.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-16T09:12:57.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:56 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/4125146492' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-16T09:12:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:56 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2646668041' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-16T09:12:57.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:56 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2184205265' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-16T09:12:57.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.289+0000 7f4f15265640 1 -- 172.21.15.7:0/780158642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102bf0 msgr2=0x7f4f100fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.289+0000 7f4f15265640 1 --2- 172.21.15.7:0/780158642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102bf0 0x7f4f100fc280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80048d0 tx=0x7f4ef8009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.289+0000 7f4f15265640 1 -- 172.21.15.7:0/780158642 shutdown_connections 2024-03-16T09:12:57.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.289+0000 7f4f15265640 1 --2- 172.21.15.7:0/780158642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102bf0 0x7f4f100fc280 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 --2- 172.21.15.7:0/780158642 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f10102250 0x7f4f10102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 -- 172.21.15.7:0/780158642 >> 172.21.15.7:0/780158642 conn(0x7f4f100f7e60 msgr2=0x7f4f100fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 -- 172.21.15.7:0/780158642 shutdown_connections 2024-03-16T09:12:57.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 -- 172.21.15.7:0/780158642 wait complete. 2024-03-16T09:12:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 Processor -- start 2024-03-16T09:12:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 -- start start 2024-03-16T09:12:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102250 0x7f4f10112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f10102bf0 0x7f4f101131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.290+0000 7f4f15265640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f101137f0 con 0x7f4f10102250 2024-03-16T09:12:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f15265640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1010ce40 con 0x7f4f10102bf0 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f0effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102250 0x7f4f10112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f0effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102250 0x7f4f10112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50582/0 (socket says 172.21.15.7:50582) 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f0effd640 1 -- 172.21.15.7:0/3308067429 learned_addr learned my addr 172.21.15.7:0/3308067429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f07fff640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f10102bf0 0x7f4f101131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f0effd640 1 -- 172.21.15.7:0/3308067429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f10102bf0 msgr2=0x7f4f101131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f0effd640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f10102bf0 0x7f4f101131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f0effd640 1 -- 172.21.15.7:0/3308067429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8005040 con 0x7f4f10102250 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f07fff640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f10102bf0 0x7f4f101131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f0effd640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102250 0x7f4f10112cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000c790 tx=0x7f4f000043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f077fe640 1 -- 172.21.15.7:0/3308067429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f0000a820 con 0x7f4f10102250 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f077fe640 1 -- 172.21.15.7:0/3308067429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f0000ae50 con 0x7f4f10102250 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1010d120 con 0x7f4f10102250 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f077fe640 1 -- 172.21.15.7:0/3308067429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f000115e0 con 0x7f4f10102250 2024-03-16T09:12:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.291+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1010d610 con 0x7f4f10102250 2024-03-16T09:12:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.292+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f100fd9c0 con 0x7f4f10102250 2024-03-16T09:12:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.293+0000 7f4f077fe640 1 -- 172.21.15.7:0/3308067429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4f00012400 con 0x7f4f10102250 2024-03-16T09:12:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.293+0000 7f4f077fe640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4efc06de50 0x7f4efc070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.293+0000 7f4f077fe640 1 -- 172.21.15.7:0/3308067429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4f0008d980 con 0x7f4f10102250 2024-03-16T09:12:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.293+0000 7f4f07fff640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4efc06de50 0x7f4efc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.294+0000 7f4f07fff640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4efc06de50 0x7f4efc070300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f100f9530 tx=0x7f4ef8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.298+0000 7f4f077fe640 1 -- 172.21.15.7:0/3308067429 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f00056e30 con 0x7f4f10102250 2024-03-16T09:12:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.535+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f4f10104690 con 0x7f4f10102250 2024-03-16T09:12:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.536+0000 7f4f077fe640 1 -- 172.21.15.7:0/3308067429 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v36) v1 ==== 107+0+4901 (secure 0 0 0) 0x7f4f0005a4f0 con 0x7f4f10102250 2024-03-16T09:12:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:57.538 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34372,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/3988729523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3988729523},{"type":"v1","addr":"172.21.15.7:6829","nonce":3988729523}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44275,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/5723851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":5723851},{"type":"v1","addr":"172.21.15.19:6825","nonce":5723851}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-16T09:12:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4efc06de50 msgr2=0x7f4efc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4efc06de50 0x7f4efc070300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f100f9530 tx=0x7f4ef8009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102250 msgr2=0x7f4f10112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102250 0x7f4f10112cb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000c790 tx=0x7f4f000043c0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 shutdown_connections 2024-03-16T09:12:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f4efc06de50 0x7f4efc070300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f10102bf0 0x7f4f101131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 --2- 172.21.15.7:0/3308067429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f10102250 0x7f4f10112cb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.538+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 >> 172.21.15.7:0/3308067429 conn(0x7f4f100f7e60 msgr2=0x7f4f100f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.539+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 shutdown_connections 2024-03-16T09:12:57.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.539+0000 7f4f15265640 1 -- 172.21.15.7:0/3308067429 wait complete. 2024-03-16T09:12:57.552 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:57.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-16T09:12:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.650+0000 7fc76c26e640 1 -- 172.21.15.7:0/3436719137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc764102240 msgr2=0x7fc764102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.650+0000 7fc76c26e640 1 --2- 172.21.15.7:0/3436719137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc764102240 0x7fc764102610 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc7600048d0 tx=0x7fc760009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.650+0000 7fc76c26e640 1 -- 172.21.15.7:0/3436719137 shutdown_connections 2024-03-16T09:12:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.650+0000 7fc76c26e640 1 --2- 172.21.15.7:0/3436719137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7640643a0 0x7fc7640647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.650+0000 7fc76c26e640 1 --2- 172.21.15.7:0/3436719137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc764102240 0x7fc764102610 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.650+0000 7fc76c26e640 1 -- 172.21.15.7:0/3436719137 >> 172.21.15.7:0/3436719137 conn(0x7fc7640710b0 msgr2=0x7fc7640714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.650+0000 7fc76c26e640 1 -- 172.21.15.7:0/3436719137 shutdown_connections 2024-03-16T09:12:57.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.650+0000 7fc76c26e640 1 -- 172.21.15.7:0/3436719137 wait complete. 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.651+0000 7fc76c26e640 1 Processor -- start 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.651+0000 7fc76c26e640 1 -- start start 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.651+0000 7fc76c26e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7640643a0 0x7fc764112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.651+0000 7fc76c26e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc764102240 0x7fc764113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.651+0000 7fc76c26e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc764113810 con 0x7fc7640643a0 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.651+0000 7fc76c26e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76410ce60 con 0x7fc764102240 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.651+0000 7fc76a00a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7640643a0 0x7fc764112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc76a00a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7640643a0 0x7fc764112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50604/0 (socket says 172.21.15.7:50604) 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc76a00a640 1 -- 172.21.15.7:0/3987337088 learned_addr learned my addr 172.21.15.7:0/3987337088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc769809640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc764102240 0x7fc764113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc76a00a640 1 -- 172.21.15.7:0/3987337088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc764102240 msgr2=0x7fc764113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc76a00a640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc764102240 0x7fc764113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc76a00a640 1 -- 172.21.15.7:0/3987337088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760005040 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc76a00a640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7640643a0 0x7fc764112cd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc7600047f0 tx=0x7fc760009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc75affd640 1 -- 172.21.15.7:0/3987337088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7600092b0 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc75affd640 1 -- 172.21.15.7:0/3987337088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc76003b400 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76410d0e0 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc75affd640 1 -- 172.21.15.7:0/3987337088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7600435b0 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.652+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76410d5a0 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.653+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7640fd940 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.654+0000 7fc75affd640 1 -- 172.21.15.7:0/3987337088 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc76002f070 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.654+0000 7fc75affd640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc75006de60 0x7fc750070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.654+0000 7fc75affd640 1 -- 172.21.15.7:0/3987337088 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc7600af420 con 0x7fc7640643a0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.654+0000 7fc769809640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc75006de60 0x7fc750070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.655+0000 7fc769809640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc75006de60 0x7fc750070310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc76410e590 tx=0x7fc75400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.659+0000 7fc75affd640 1 -- 172.21.15.7:0/3987337088 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc760078910 con 0x7fc7640643a0 2024-03-16T09:12:57.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.955+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fc76404bcc0 con 0x7fc7640643a0 2024-03-16T09:12:57.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.955+0000 7fc75affd640 1 -- 172.21.15.7:0/3987337088 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v36) v1 ==== 107+0+4125 (secure 0 0 0) 0x7fc76007bfd0 con 0x7fc7640643a0 2024-03-16T09:12:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:57.958 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44275,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/5723851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":5723851},{"type":"v1","addr":"172.21.15.19:6825","nonce":5723851}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:57.958 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-16T09:12:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc75006de60 msgr2=0x7fc750070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc75006de60 0x7fc750070310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc76410e590 tx=0x7fc75400b040 comp rx=0 tx=0).stop 2024-03-16T09:12:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7640643a0 msgr2=0x7fc764112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7640643a0 0x7fc764112cd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc7600047f0 tx=0x7fc760009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 shutdown_connections 2024-03-16T09:12:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc75006de60 0x7fc750070310 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc764102240 0x7fc764113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 --2- 172.21.15.7:0/3987337088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7640643a0 0x7fc764112cd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.957+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 >> 172.21.15.7:0/3987337088 conn(0x7fc7640710b0 msgr2=0x7fc7640f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.958+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 shutdown_connections 2024-03-16T09:12:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:57.958+0000 7fc76c26e640 1 -- 172.21.15.7:0/3987337088 wait complete. 2024-03-16T09:12:57.971 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:57.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-16T09:12:58.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.069+0000 7f554f55d640 1 -- 172.21.15.7:0/1937089727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fdba0 msgr2=0x7f55481061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.069+0000 7f554f55d640 1 --2- 172.21.15.7:0/1937089727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fdba0 0x7f55481061e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f55380048d0 tx=0x7f5538009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.069+0000 7f554f55d640 1 -- 172.21.15.7:0/1937089727 shutdown_connections 2024-03-16T09:12:58.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.069+0000 7f554f55d640 1 --2- 172.21.15.7:0/1937089727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fdba0 0x7f55481061e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.069+0000 7f554f55d640 1 --2- 172.21.15.7:0/1937089727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55480fd200 0x7f55480fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.069+0000 7f554f55d640 1 -- 172.21.15.7:0/1937089727 >> 172.21.15.7:0/1937089727 conn(0x7f55480f6cb0 msgr2=0x7f55480f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:58.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.069+0000 7f554f55d640 1 -- 172.21.15.7:0/1937089727 shutdown_connections 2024-03-16T09:12:58.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554f55d640 1 -- 172.21.15.7:0/1937089727 wait complete. 2024-03-16T09:12:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554f55d640 1 Processor -- start 2024-03-16T09:12:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554f55d640 1 -- start start 2024-03-16T09:12:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554f55d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fd200 0x7f5548112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554f55d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55480fdba0 0x7f5548113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554f55d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548113880 con 0x7f55480fd200 2024-03-16T09:12:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554f55d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554810ced0 con 0x7f55480fdba0 2024-03-16T09:12:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554d2f9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fd200 0x7f5548112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554d2f9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fd200 0x7f5548112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50622/0 (socket says 172.21.15.7:50622) 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.070+0000 7f554d2f9640 1 -- 172.21.15.7:0/875633429 learned_addr learned my addr 172.21.15.7:0/875633429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f554caf8640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55480fdba0 0x7f5548113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f554d2f9640 1 -- 172.21.15.7:0/875633429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55480fdba0 msgr2=0x7f5548113280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f554d2f9640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55480fdba0 0x7f5548113280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f554d2f9640 1 -- 172.21.15.7:0/875633429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538005040 con 0x7f55480fd200 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f554d2f9640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fd200 0x7f5548112d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f554400eb70 tx=0x7f554400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f553dffb640 1 -- 172.21.15.7:0/875633429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5544007c30 con 0x7f55480fd200 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f553dffb640 1 -- 172.21.15.7:0/875633429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f554400f040 con 0x7f55480fd200 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554810d150 con 0x7f55480fd200 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f553dffb640 1 -- 172.21.15.7:0/875633429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55440095b0 con 0x7f55480fd200 2024-03-16T09:12:58.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.071+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554810d640 con 0x7f55480fd200 2024-03-16T09:12:58.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.072+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5548065980 con 0x7f55480fd200 2024-03-16T09:12:58.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.073+0000 7f553dffb640 1 -- 172.21.15.7:0/875633429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f5544009730 con 0x7f55480fd200 2024-03-16T09:12:58.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.073+0000 7f553dffb640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.073+0000 7f553dffb640 1 -- 172.21.15.7:0/875633429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f554408d510 con 0x7f55480fd200 2024-03-16T09:12:58.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.073+0000 7f554caf8640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.074+0000 7f554caf8640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55480fe570 tx=0x7f5538009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:58.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.078+0000 7f553dffb640 1 -- 172.21.15.7:0/875633429 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5544056980 con 0x7f55480fd200 2024-03-16T09:12:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:57 smithi019 ceph-mon[83366]: pgmap v261: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:57 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/149857821' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-16T09:12:58.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:57 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3308067429' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-16T09:12:58.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:57 smithi007 ceph-mon[113546]: pgmap v261: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:12:58.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:57 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/149857821' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-16T09:12:58.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:57 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3308067429' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-16T09:12:58.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.300+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f5548109350 con 0x7f55480fd200 2024-03-16T09:12:58.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.301+0000 7f553dffb640 1 -- 172.21.15.7:0/875633429 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v36) v1 ==== 107+0+4969 (secure 0 0 0) 0x7f554405a040 con 0x7f55480fd200 2024-03-16T09:12:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:58.304 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34412,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44275,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/5723851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":5723851},{"type":"v1","addr":"172.21.15.19:6825","nonce":5723851}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:58.304 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 msgr2=0x7f5534070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55480fe570 tx=0x7f5538009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fd200 msgr2=0x7f5548112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fd200 0x7f5548112d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f554400eb70 tx=0x7f554400b040 comp rx=0 tx=0).stop 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 shutdown_connections 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55480fdba0 0x7f5548113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 --2- 172.21.15.7:0/875633429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55480fd200 0x7f5548112d40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 >> 172.21.15.7:0/875633429 conn(0x7f55480f6cb0 msgr2=0x7f55480f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:58.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 shutdown_connections 2024-03-16T09:12:58.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.304+0000 7f554f55d640 1 -- 172.21.15.7:0/875633429 wait complete. 2024-03-16T09:12:58.317 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:58.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-16T09:12:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.419+0000 7fba88b9d640 1 -- 172.21.15.7:0/1826020989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba84101a70 msgr2=0x7fba840fb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.419+0000 7fba88b9d640 1 --2- 172.21.15.7:0/1826020989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba84101a70 0x7fba840fb100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0048d0 tx=0x7fba6c009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.419+0000 7fba88b9d640 1 -- 172.21.15.7:0/1826020989 shutdown_connections 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.419+0000 7fba88b9d640 1 --2- 172.21.15.7:0/1826020989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba84101a70 0x7fba840fb100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.419+0000 7fba88b9d640 1 --2- 172.21.15.7:0/1826020989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba841010d0 0x7fba841014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.419+0000 7fba88b9d640 1 -- 172.21.15.7:0/1826020989 >> 172.21.15.7:0/1826020989 conn(0x7fba840f6ca0 msgr2=0x7fba840f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.419+0000 7fba88b9d640 1 -- 172.21.15.7:0/1826020989 shutdown_connections 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.419+0000 7fba88b9d640 1 -- 172.21.15.7:0/1826020989 wait complete. 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba88b9d640 1 Processor -- start 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba88b9d640 1 -- start start 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba88b9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba841010d0 0x7fba8410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba88b9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba84101a70 0x7fba8410e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba88b9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba84065fc0 con 0x7fba841010d0 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba88b9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8410e900 con 0x7fba84101a70 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba8259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba841010d0 0x7fba8410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba8259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba841010d0 0x7fba8410de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50638/0 (socket says 172.21.15.7:50638) 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba8259c640 1 -- 172.21.15.7:0/3370612210 learned_addr learned my addr 172.21.15.7:0/3370612210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba79d9b640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba84101a70 0x7fba8410e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba8259c640 1 -- 172.21.15.7:0/3370612210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba84101a70 msgr2=0x7fba8410e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba8259c640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba84101a70 0x7fba8410e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba8259c640 1 -- 172.21.15.7:0/3370612210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c005040 con 0x7fba841010d0 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.420+0000 7fba79d9b640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba84101a70 0x7fba8410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:12:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.421+0000 7fba8259c640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba841010d0 0x7fba8410de80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fba74002a50 tx=0x7fba74004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.421+0000 7fba7b7fe640 1 -- 172.21.15.7:0/3370612210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba74007c30 con 0x7fba841010d0 2024-03-16T09:12:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.421+0000 7fba7b7fe640 1 -- 172.21.15.7:0/3370612210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba7400f040 con 0x7fba841010d0 2024-03-16T09:12:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.421+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8410ebe0 con 0x7fba841010d0 2024-03-16T09:12:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.421+0000 7fba7b7fe640 1 -- 172.21.15.7:0/3370612210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7400db50 con 0x7fba841010d0 2024-03-16T09:12:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.421+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba841171a0 con 0x7fba841010d0 2024-03-16T09:12:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.422+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba840fc840 con 0x7fba841010d0 2024-03-16T09:12:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.423+0000 7fba7b7fe640 1 -- 172.21.15.7:0/3370612210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fba7401c400 con 0x7fba841010d0 2024-03-16T09:12:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.423+0000 7fba7b7fe640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba7006de50 0x7fba70070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.423+0000 7fba7b7fe640 1 -- 172.21.15.7:0/3370612210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fba7408d060 con 0x7fba841010d0 2024-03-16T09:12:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.423+0000 7fba79d9b640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba7006de50 0x7fba70070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.424+0000 7fba79d9b640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba7006de50 0x7fba70070300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fba84065810 tx=0x7fba6c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.428+0000 7fba7b7fe640 1 -- 172.21.15.7:0/3370612210 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fba740564d0 con 0x7fba841010d0 2024-03-16T09:12:58.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.650+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fba8404bcc0 con 0x7fba841010d0 2024-03-16T09:12:58.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.651+0000 7fba7b7fe640 1 -- 172.21.15.7:0/3370612210 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v36) v1 ==== 107+0+4199 (secure 0 0 0) 0x7fba74059b90 con 0x7fba841010d0 2024-03-16T09:12:58.654 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:58.654 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34412,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:58.654 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba7006de50 msgr2=0x7fba70070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba7006de50 0x7fba70070300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fba84065810 tx=0x7fba6c009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba841010d0 msgr2=0x7fba8410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba841010d0 0x7fba8410de80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fba74002a50 tx=0x7fba74004600 comp rx=0 tx=0).stop 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 shutdown_connections 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba7006de50 0x7fba70070300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba84101a70 0x7fba8410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 --2- 172.21.15.7:0/3370612210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba841010d0 0x7fba8410de80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 >> 172.21.15.7:0/3370612210 conn(0x7fba840f6ca0 msgr2=0x7fba840f7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:58.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 shutdown_connections 2024-03-16T09:12:58.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.653+0000 7fba88b9d640 1 -- 172.21.15.7:0/3370612210 wait complete. 2024-03-16T09:12:58.668 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:58.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-16T09:12:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.765+0000 7fa588a6e640 1 -- 172.21.15.7:0/2145092808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fdb90 msgr2=0x7fa584106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.765+0000 7fa588a6e640 1 --2- 172.21.15.7:0/2145092808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fdb90 0x7fa584106260 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa56c004900 tx=0x7fa56c009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.765+0000 7fa588a6e640 1 -- 172.21.15.7:0/2145092808 shutdown_connections 2024-03-16T09:12:58.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.765+0000 7fa588a6e640 1 --2- 172.21.15.7:0/2145092808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fdb90 0x7fa584106260 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.765+0000 7fa588a6e640 1 --2- 172.21.15.7:0/2145092808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5840fd1f0 0x7fa5840fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.765+0000 7fa588a6e640 1 -- 172.21.15.7:0/2145092808 >> 172.21.15.7:0/2145092808 conn(0x7fa5840f6cb0 msgr2=0x7fa5840f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.765+0000 7fa588a6e640 1 -- 172.21.15.7:0/2145092808 shutdown_connections 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.765+0000 7fa588a6e640 1 -- 172.21.15.7:0/2145092808 wait complete. 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa588a6e640 1 Processor -- start 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa588a6e640 1 -- start start 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa588a6e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fd1f0 0x7fa5840fb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa588a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5840fdb90 0x7fa5840fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa588a6e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840fcf90 con 0x7fa5840fd1f0 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa588a6e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840fbb30 con 0x7fa5840fdb90 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa58259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fd1f0 0x7fa5840fb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa58259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fd1f0 0x7fa5840fb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50654/0 (socket says 172.21.15.7:50654) 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa58259c640 1 -- 172.21.15.7:0/3851418761 learned_addr learned my addr 172.21.15.7:0/3851418761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:58.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.766+0000 7fa57bfff640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5840fdb90 0x7fa5840fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa58259c640 1 -- 172.21.15.7:0/3851418761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5840fdb90 msgr2=0x7fa5840fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa58259c640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5840fdb90 0x7fa5840fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa58259c640 1 -- 172.21.15.7:0/3851418761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c005040 con 0x7fa5840fd1f0 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa57bfff640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5840fdb90 0x7fa5840fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa58259c640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fd1f0 0x7fa5840fb0b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa574002a50 tx=0x7fa574004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa57affd640 1 -- 172.21.15.7:0/3851418761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa574007c30 con 0x7fa5840fd1f0 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa57affd640 1 -- 172.21.15.7:0/3851418761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa57400f040 con 0x7fa5840fd1f0 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5840fbe10 con 0x7fa5840fd1f0 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa57affd640 1 -- 172.21.15.7:0/3851418761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57400dbe0 con 0x7fa5840fd1f0 2024-03-16T09:12:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.767+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa584119490 con 0x7fa5840fd1f0 2024-03-16T09:12:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.768+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa584065900 con 0x7fa5840fd1f0 2024-03-16T09:12:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.769+0000 7fa57affd640 1 -- 172.21.15.7:0/3851418761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa57400dd40 con 0x7fa5840fd1f0 2024-03-16T09:12:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.769+0000 7fa57affd640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa57006de50 0x7fa570070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.769+0000 7fa57affd640 1 -- 172.21.15.7:0/3851418761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa57408d730 con 0x7fa5840fd1f0 2024-03-16T09:12:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.769+0000 7fa57bfff640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa57006de50 0x7fa570070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.770+0000 7fa57bfff640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa57006de50 0x7fa570070300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009fa0 tx=0x7fa56c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:58.777+0000 7fa57affd640 1 -- 172.21.15.7:0/3851418761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa574056ba0 con 0x7fa5840fd1f0 2024-03-16T09:12:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:58 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3987337088' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-16T09:12:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:58 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/875633429' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-16T09:12:58.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:58 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3370612210' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-16T09:12:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.043+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fa584109340 con 0x7fa5840fd1f0 2024-03-16T09:12:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.043+0000 7fa57affd640 1 -- 172.21.15.7:0/3851418761 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v36) v1 ==== 107+0+5046 (secure 0 0 0) 0x7fa57405a260 con 0x7fa5840fd1f0 2024-03-16T09:12:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:59.046 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24289,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.19:6827/207631863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":207631863},{"type":"v1","addr":"172.21.15.19:6827","nonce":207631863}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34412,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa57006de50 msgr2=0x7fa570070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa57006de50 0x7fa570070300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009fa0 tx=0x7fa56c009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fd1f0 msgr2=0x7fa5840fb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fd1f0 0x7fa5840fb0b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa574002a50 tx=0x7fa574004600 comp rx=0 tx=0).stop 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 shutdown_connections 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fa57006de50 0x7fa570070300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5840fdb90 0x7fa5840fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 --2- 172.21.15.7:0/3851418761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5840fd1f0 0x7fa5840fb0b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.045+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 >> 172.21.15.7:0/3851418761 conn(0x7fa5840f6cb0 msgr2=0x7fa5840f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:59.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.046+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 shutdown_connections 2024-03-16T09:12:59.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.046+0000 7fa588a6e640 1 -- 172.21.15.7:0/3851418761 wait complete. 2024-03-16T09:12:59.059 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:59.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-16T09:12:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1343459116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41023e0 msgr2=0x7f1cc41027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 --2- 172.21.15.7:0/1343459116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41023e0 0x7f1cc41027b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40048d0 tx=0x7f1cb4009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1343459116 shutdown_connections 2024-03-16T09:12:59.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 --2- 172.21.15.7:0/1343459116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc4102d80 0x7f1cc40fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 --2- 172.21.15.7:0/1343459116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41023e0 0x7f1cc41027b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1343459116 >> 172.21.15.7:0/1343459116 conn(0x7f1cc40f7fb0 msgr2=0x7f1cc40fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:59.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1343459116 shutdown_connections 2024-03-16T09:12:59.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1343459116 wait complete. 2024-03-16T09:12:59.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.158+0000 7f1cc9d5d640 1 Processor -- start 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc9d5d640 1 -- start start 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc9d5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc41023e0 0x7f1cc4112f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc9d5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102d80 0x7f1cc4113440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc9d5d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc4113a40 con 0x7f1cc4102d80 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc9d5d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc410d090 con 0x7f1cc41023e0 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102d80 0x7f1cc4113440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102d80 0x7f1cc4113440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50662/0 (socket says 172.21.15.7:50662) 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc2ffd640 1 -- 172.21.15.7:0/1311922600 learned_addr learned my addr 172.21.15.7:0/1311922600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc37fe640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc41023e0 0x7f1cc4112f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc2ffd640 1 -- 172.21.15.7:0/1311922600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc41023e0 msgr2=0x7f1cc4112f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc2ffd640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc41023e0 0x7f1cc4112f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc2ffd640 1 -- 172.21.15.7:0/1311922600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4005040 con 0x7f1cc4102d80 2024-03-16T09:12:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.159+0000 7f1cc2ffd640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102d80 0x7f1cc4113440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8002a50 tx=0x7f1cb8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.160+0000 7f1cabfff640 1 -- 172.21.15.7:0/1311922600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb8007c30 con 0x7f1cc4102d80 2024-03-16T09:12:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.160+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc410d370 con 0x7f1cc4102d80 2024-03-16T09:12:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.160+0000 7f1cabfff640 1 -- 172.21.15.7:0/1311922600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cb800f040 con 0x7f1cc4102d80 2024-03-16T09:12:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.160+0000 7f1cabfff640 1 -- 172.21.15.7:0/1311922600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb800db50 con 0x7f1cc4102d80 2024-03-16T09:12:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.160+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc410d890 con 0x7f1cc4102d80 2024-03-16T09:12:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.161+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc40fdb50 con 0x7f1cc4102d80 2024-03-16T09:12:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.161+0000 7f1cabfff640 1 -- 172.21.15.7:0/1311922600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f1cb801c400 con 0x7f1cc4102d80 2024-03-16T09:12:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.162+0000 7f1cabfff640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1cac06de50 0x7f1cac070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.162+0000 7f1cabfff640 1 -- 172.21.15.7:0/1311922600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1cb808d0c0 con 0x7f1cc4102d80 2024-03-16T09:12:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.162+0000 7f1cc37fe640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1cac06de50 0x7f1cac070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.163+0000 7f1cc37fe640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1cac06de50 0x7f1cac070300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4003080 tx=0x7f1cb4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.167+0000 7f1cabfff640 1 -- 172.21.15.7:0/1311922600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1cb80564b0 con 0x7f1cc4102d80 2024-03-16T09:12:59.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:58 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3987337088' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-16T09:12:59.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:58 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/875633429' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-16T09:12:59.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:58 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3370612210' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-16T09:12:59.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.438+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f1cc4109540 con 0x7f1cc4102d80 2024-03-16T09:12:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.439+0000 7f1cabfff640 1 -- 172.21.15.7:0/1311922600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v36) v1 ==== 107+0+4270 (secure 0 0 0) 0x7f1cb8059b70 con 0x7f1cc4102d80 2024-03-16T09:12:59.441 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:59.441 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34412,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1cac06de50 msgr2=0x7f1cac070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1cac06de50 0x7f1cac070300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4003080 tx=0x7f1cb4009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102d80 msgr2=0x7f1cc4113440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102d80 0x7f1cc4113440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8002a50 tx=0x7f1cb8004600 comp rx=0 tx=0).stop 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 shutdown_connections 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f1cac06de50 0x7f1cac070300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102d80 0x7f1cc4113440 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 --2- 172.21.15.7:0/1311922600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cc41023e0 0x7f1cc4112f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 >> 172.21.15.7:0/1311922600 conn(0x7f1cc40f7fb0 msgr2=0x7f1cc40f8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 shutdown_connections 2024-03-16T09:12:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.441+0000 7f1cc9d5d640 1 -- 172.21.15.7:0/1311922600 wait complete. 2024-03-16T09:12:59.455 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:59.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.557+0000 7f554af1d640 1 -- 172.21.15.7:0/3132400685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 msgr2=0x7f55440fa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.557+0000 7f554af1d640 1 --2- 172.21.15.7:0/3132400685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 0x7f55440fa8a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f55300048d0 tx=0x7f5530009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.557+0000 7f554af1d640 1 -- 172.21.15.7:0/3132400685 shutdown_connections 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.557+0000 7f554af1d640 1 --2- 172.21.15.7:0/3132400685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 0x7f55440fa8a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.557+0000 7f554af1d640 1 --2- 172.21.15.7:0/3132400685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544102250 0x7f5544102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.557+0000 7f554af1d640 1 -- 172.21.15.7:0/3132400685 >> 172.21.15.7:0/3132400685 conn(0x7f5544070880 msgr2=0x7f5544070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.557+0000 7f554af1d640 1 -- 172.21.15.7:0/3132400685 shutdown_connections 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.557+0000 7f554af1d640 1 -- 172.21.15.7:0/3132400685 wait complete. 2024-03-16T09:12:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f554af1d640 1 Processor -- start 2024-03-16T09:12:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f554af1d640 1 -- start start 2024-03-16T09:12:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f554af1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544102250 0x7f5544112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f554af1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 0x7f5544113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f554af1d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5544113800 con 0x7f5544102b60 2024-03-16T09:12:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f554af1d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5544113970 con 0x7f5544102250 2024-03-16T09:12:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f5543fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 0x7f5544113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f5543fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 0x7f5544113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50676/0 (socket says 172.21.15.7:50676) 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.558+0000 7f5543fff640 1 -- 172.21.15.7:0/1072212714 learned_addr learned my addr 172.21.15.7:0/1072212714 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f5548cb9640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544102250 0x7f5544112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f5543fff640 1 -- 172.21.15.7:0/1072212714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544102250 msgr2=0x7f5544112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f5543fff640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544102250 0x7f5544112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f5543fff640 1 -- 172.21.15.7:0/1072212714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530005040 con 0x7f5544102b60 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f5543fff640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 0x7f5544113200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f55300043d0 tx=0x7f5530009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f55417fa640 1 -- 172.21.15.7:0/1072212714 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5530030870 con 0x7f5544102b60 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f55417fa640 1 -- 172.21.15.7:0/1072212714 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55300092e0 con 0x7f5544102b60 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554410cfa0 con 0x7f5544102b60 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f55417fa640 1 -- 172.21.15.7:0/1072212714 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5530043400 con 0x7f5544102b60 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.559+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554410d4c0 con 0x7f5544102b60 2024-03-16T09:12:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.560+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55440640e0 con 0x7f5544102b60 2024-03-16T09:12:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.561+0000 7f55417fa640 1 -- 172.21.15.7:0/1072212714 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f553003b420 con 0x7f5544102b60 2024-03-16T09:12:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.561+0000 7f55417fa640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.561+0000 7f55417fa640 1 -- 172.21.15.7:0/1072212714 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f55300b04d0 con 0x7f5544102b60 2024-03-16T09:12:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.561+0000 7f5548cb9640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.562+0000 7f5548cb9640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5538004eb0 tx=0x7f553800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:59.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.570+0000 7f55417fa640 1 -- 172.21.15.7:0/1072212714 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55300799c0 con 0x7f5544102b60 2024-03-16T09:12:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.868+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f554404bcc0 con 0x7f5544102b60 2024-03-16T09:12:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.868+0000 7f55417fa640 1 -- 172.21.15.7:0/1072212714 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v36) v1 ==== 107+0+5117 (secure 0 0 0) 0x7f553007d080 con 0x7f5544102b60 2024-03-16T09:12:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:12:59.871 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34412,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44329,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/3039505467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3039505467},{"type":"v1","addr":"172.21.15.19:6827","nonce":3039505467}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:09:58.394215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:12:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 msgr2=0x7f5534070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5538004eb0 tx=0x7f553800b040 comp rx=0 tx=0).stop 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 msgr2=0x7f5544113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 0x7f5544113200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f55300043d0 tx=0x7f5530009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 shutdown_connections 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f553406de50 0x7f5534070300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5544102b60 0x7f5544113200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 --2- 172.21.15.7:0/1072212714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544102250 0x7f5544112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 >> 172.21.15.7:0/1072212714 conn(0x7f5544070880 msgr2=0x7f55440ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 shutdown_connections 2024-03-16T09:12:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.871+0000 7f554af1d640 1 -- 172.21.15.7:0/1072212714 wait complete. 2024-03-16T09:12:59.887 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-16T09:12:59.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.989+0000 7f0802e6e640 1 -- 172.21.15.7:0/1403645659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fdb70 msgr2=0x7f07fc1061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.989+0000 7f0802e6e640 1 --2- 172.21.15.7:0/1403645659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fdb70 0x7f07fc1061b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07e80048d0 tx=0x7f07e8009da0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.989+0000 7f0802e6e640 1 -- 172.21.15.7:0/1403645659 shutdown_connections 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.989+0000 7f0802e6e640 1 --2- 172.21.15.7:0/1403645659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fdb70 0x7f07fc1061b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.989+0000 7f0802e6e640 1 --2- 172.21.15.7:0/1403645659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07fc0fd1d0 0x7f07fc0fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.989+0000 7f0802e6e640 1 -- 172.21.15.7:0/1403645659 >> 172.21.15.7:0/1403645659 conn(0x7f07fc0f6ca0 msgr2=0x7f07fc0f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.989+0000 7f0802e6e640 1 -- 172.21.15.7:0/1403645659 shutdown_connections 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.989+0000 7f0802e6e640 1 -- 172.21.15.7:0/1403645659 wait complete. 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0802e6e640 1 Processor -- start 2024-03-16T09:12:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0802e6e640 1 -- start start 2024-03-16T09:12:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0802e6e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fd1d0 0x7f07fc112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0802e6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07fc0fdb70 0x7f07fc113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0802e6e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc113860 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0802e6e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc10ceb0 con 0x7f07fc0fdb70 2024-03-16T09:12:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0800c0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fd1d0 0x7f07fc112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0800c0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fd1d0 0x7f07fc112d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50700/0 (socket says 172.21.15.7:50700) 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f0800c0a640 1 -- 172.21.15.7:0/1847483454 learned_addr learned my addr 172.21.15.7:0/1847483454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.990+0000 7f07fbfff640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07fc0fdb70 0x7f07fc113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f0800c0a640 1 -- 172.21.15.7:0/1847483454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07fc0fdb70 msgr2=0x7f07fc113260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f0800c0a640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07fc0fdb70 0x7f07fc113260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f0800c0a640 1 -- 172.21.15.7:0/1847483454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8005040 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f0800c0a640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fd1d0 0x7f07fc112d20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f07f000eb70 tx=0x7f07f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f07f97fa640 1 -- 172.21.15.7:0/1847483454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07f0007c30 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc10d130 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc10d620 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f07f97fa640 1 -- 172.21.15.7:0/1847483454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07f000f040 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.991+0000 7f07f97fa640 1 -- 172.21.15.7:0/1847483454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07f00094b0 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.992+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e0005210 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.992+0000 7f07f97fa640 1 -- 172.21.15.7:0/1847483454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f07f0009670 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.993+0000 7f07f97fa640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f07ec06de50 0x7f07ec070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:59.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.993+0000 7f07f97fa640 1 -- 172.21.15.7:0/1847483454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f07f008d380 con 0x7f07fc0fd1d0 2024-03-16T09:12:59.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.993+0000 7f07fbfff640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f07ec06de50 0x7f07ec070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:59.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.993+0000 7f07fbfff640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f07ec06de50 0x7f07ec070300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0fe540 tx=0x7f07e8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:00.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:12:59.998+0000 7f07f97fa640 1 -- 172.21.15.7:0/1847483454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f07f00568a0 con 0x7f07fc0fd1d0 2024-03-16T09:13:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:59 smithi019 ceph-mon[83366]: pgmap v262: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:13:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:59 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3851418761' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-16T09:13:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:59 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1311922600' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-16T09:13:00.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:12:59 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1072212714' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-16T09:13:00.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.221+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f07e00054c0 con 0x7f07fc0fd1d0 2024-03-16T09:13:00.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.222+0000 7f07f97fa640 1 -- 172.21.15.7:0/1847483454 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v36) v1 ==== 107+0+5116 (secure 0 0 0) 0x7f07f0056030 con 0x7f07fc0fd1d0 2024-03-16T09:13:00.224 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:13:00.224 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34412,"name":"cephfs.smithi007.znurst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44329,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/3039505467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3039505467},{"type":"v1","addr":"172.21.15.19:6827","nonce":3039505467}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:11:03.019265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:13:00.224 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 31 2024-03-16T09:13:00.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f07ec06de50 msgr2=0x7f07ec070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f07ec06de50 0x7f07ec070300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0fe540 tx=0x7f07e8009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fd1d0 msgr2=0x7f07fc112d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fd1d0 0x7f07fc112d20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f07f000eb70 tx=0x7f07f000b040 comp rx=0 tx=0).stop 2024-03-16T09:13:00.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 shutdown_connections 2024-03-16T09:13:00.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f07ec06de50 0x7f07ec070300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07fc0fdb70 0x7f07fc113260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 --2- 172.21.15.7:0/1847483454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07fc0fd1d0 0x7f07fc112d20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 >> 172.21.15.7:0/1847483454 conn(0x7f07fc0f6ca0 msgr2=0x7f07fc0f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:00.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 shutdown_connections 2024-03-16T09:13:00.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.224+0000 7f0802e6e640 1 -- 172.21.15.7:0/1847483454 wait complete. 2024-03-16T09:13:00.239 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:59 smithi007 ceph-mon[113546]: pgmap v262: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:13:00.239 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:59 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3851418761' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-16T09:13:00.239 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:59 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1311922600' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-16T09:13:00.239 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:12:59 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1072212714' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-16T09:13:00.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-16T09:13:00.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.350+0000 7fba21d9d640 1 -- 172.21.15.7:0/1367650236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c102240 msgr2=0x7fba1c102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.350+0000 7fba21d9d640 1 --2- 172.21.15.7:0/1367650236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c102240 0x7fba1c102610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0048d0 tx=0x7fba0c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.350+0000 7fba21d9d640 1 -- 172.21.15.7:0/1367650236 shutdown_connections 2024-03-16T09:13:00.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 --2- 172.21.15.7:0/1367650236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c0643a0 0x7fba1c0647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 --2- 172.21.15.7:0/1367650236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c102240 0x7fba1c102610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 -- 172.21.15.7:0/1367650236 >> 172.21.15.7:0/1367650236 conn(0x7fba1c0710b0 msgr2=0x7fba1c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:00.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 -- 172.21.15.7:0/1367650236 shutdown_connections 2024-03-16T09:13:00.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 -- 172.21.15.7:0/1367650236 wait complete. 2024-03-16T09:13:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 Processor -- start 2024-03-16T09:13:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 -- start start 2024-03-16T09:13:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c0643a0 0x7fba1c112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c102240 0x7fba1c113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c113850 con 0x7fba1c0643a0 2024-03-16T09:13:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.351+0000 7fba21d9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c10cea0 con 0x7fba1c102240 2024-03-16T09:13:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba1b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c0643a0 0x7fba1c112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba1b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c0643a0 0x7fba1c112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50714/0 (socket says 172.21.15.7:50714) 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba1b7fe640 1 -- 172.21.15.7:0/1963415930 learned_addr learned my addr 172.21.15.7:0/1963415930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba1affd640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c102240 0x7fba1c113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba1b7fe640 1 -- 172.21.15.7:0/1963415930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c102240 msgr2=0x7fba1c113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba1b7fe640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c102240 0x7fba1c113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba1b7fe640 1 -- 172.21.15.7:0/1963415930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c005040 con 0x7fba1c0643a0 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba1b7fe640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c0643a0 0x7fba1c112d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fba0c004930 tx=0x7fba0c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba03fff640 1 -- 172.21.15.7:0/1963415930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0c0094c0 con 0x7fba1c0643a0 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c10d120 con 0x7fba1c0643a0 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba03fff640 1 -- 172.21.15.7:0/1963415930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba0c032e50 con 0x7fba1c0643a0 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba03fff640 1 -- 172.21.15.7:0/1963415930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0c0433b0 con 0x7fba1c0643a0 2024-03-16T09:13:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.352+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c10d5e0 con 0x7fba1c0643a0 2024-03-16T09:13:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.354+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c0fd940 con 0x7fba1c0643a0 2024-03-16T09:13:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.354+0000 7fba03fff640 1 -- 172.21.15.7:0/1963415930 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fba0c02f070 con 0x7fba1c0643a0 2024-03-16T09:13:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.354+0000 7fba03fff640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba0406de50 0x7fba04070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.355+0000 7fba03fff640 1 -- 172.21.15.7:0/1963415930 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fba0c0af490 con 0x7fba1c0643a0 2024-03-16T09:13:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.355+0000 7fba1affd640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba0406de50 0x7fba04070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.355+0000 7fba1affd640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba0406de50 0x7fba04070300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba1c10e5d0 tx=0x7fba1000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.363+0000 7fba03fff640 1 -- 172.21.15.7:0/1963415930 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fba0c0789b0 con 0x7fba1c0643a0 2024-03-16T09:13:00.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.617+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fba1c10e380 con 0x7fba1c0643a0 2024-03-16T09:13:00.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.618+0000 7fba03fff640 1 -- 172.21.15.7:0/1963415930 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v36) v1 ==== 107+0+5207 (secure 0 0 0) 0x7fba0c07c070 con 0x7fba1c0643a0 2024-03-16T09:13:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:13:00.620 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44329,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/3039505467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3039505467},{"type":"v1","addr":"172.21.15.19:6827","nonce":3039505467}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:11:04.019878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi007.znurst","rank":1,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:13:00.620 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-03-16T09:13:00.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba0406de50 msgr2=0x7fba04070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba0406de50 0x7fba04070300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba1c10e5d0 tx=0x7fba1000b040 comp rx=0 tx=0).stop 2024-03-16T09:13:00.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c0643a0 msgr2=0x7fba1c112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c0643a0 0x7fba1c112d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fba0c004930 tx=0x7fba0c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 shutdown_connections 2024-03-16T09:13:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fba0406de50 0x7fba04070300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba1c102240 0x7fba1c113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 --2- 172.21.15.7:0/1963415930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c0643a0 0x7fba1c112d10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 >> 172.21.15.7:0/1963415930 conn(0x7fba1c0710b0 msgr2=0x7fba1c0f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 shutdown_connections 2024-03-16T09:13:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.621+0000 7fba21d9d640 1 -- 172.21.15.7:0/1963415930 wait complete. 2024-03-16T09:13:00.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-16T09:13:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 -- 172.21.15.7:0/872141982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9370101a60 msgr2=0x7f93700fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 --2- 172.21.15.7:0/872141982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9370101a60 0x7f93700fb0f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f935c0048d0 tx=0x7f935c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 -- 172.21.15.7:0/872141982 shutdown_connections 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 --2- 172.21.15.7:0/872141982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9370101a60 0x7f93700fb0f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 --2- 172.21.15.7:0/872141982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93701010c0 0x7f9370101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 -- 172.21.15.7:0/872141982 >> 172.21.15.7:0/872141982 conn(0x7f93700f6cb0 msgr2=0x7f93700f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 -- 172.21.15.7:0/872141982 shutdown_connections 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 -- 172.21.15.7:0/872141982 wait complete. 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.762+0000 7f9377014640 1 Processor -- start 2024-03-16T09:13:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9377014640 1 -- start start 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9377014640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93701010c0 0x7f937010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9377014640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9370101a60 0x7f937010e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9377014640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9370066080 con 0x7f93701010c0 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9377014640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937010e940 con 0x7f9370101a60 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9374db0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93701010c0 0x7f937010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9374db0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93701010c0 0x7f937010dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50724/0 (socket says 172.21.15.7:50724) 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9374db0640 1 -- 172.21.15.7:0/1070346559 learned_addr learned my addr 172.21.15.7:0/1070346559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f936ffff640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9370101a60 0x7f937010e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9374db0640 1 -- 172.21.15.7:0/1070346559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9370101a60 msgr2=0x7f937010e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9374db0640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9370101a60 0x7f937010e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.763+0000 7f9374db0640 1 -- 172.21.15.7:0/1070346559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c005040 con 0x7f93701010c0 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.764+0000 7f9374db0640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93701010c0 0x7f937010dec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9364002a50 tx=0x7f9364004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.764+0000 7f936d7fa640 1 -- 172.21.15.7:0/1070346559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9364007c30 con 0x7f93701010c0 2024-03-16T09:13:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.764+0000 7f936d7fa640 1 -- 172.21.15.7:0/1070346559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f936400f040 con 0x7f93701010c0 2024-03-16T09:13:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.764+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937010ec20 con 0x7f93701010c0 2024-03-16T09:13:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.764+0000 7f936d7fa640 1 -- 172.21.15.7:0/1070346559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f936400dbe0 con 0x7f93701010c0 2024-03-16T09:13:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.764+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93701171e0 con 0x7f93701010c0 2024-03-16T09:13:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.765+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93700fc830 con 0x7f93701010c0 2024-03-16T09:13:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.766+0000 7f936d7fa640 1 -- 172.21.15.7:0/1070346559 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f936400dd40 con 0x7f93701010c0 2024-03-16T09:13:00.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.767+0000 7f936d7fa640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f936006de50 0x7f9360070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.767+0000 7f936d7fa640 1 -- 172.21.15.7:0/1070346559 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f936408d720 con 0x7f93701010c0 2024-03-16T09:13:00.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.767+0000 7f936ffff640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f936006de50 0x7f9360070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.767+0000 7f936ffff640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f936006de50 0x7f9360070300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f935c003080 tx=0x7f935c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:00.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:00.773+0000 7f936d7fa640 1 -- 172.21.15.7:0/1070346559 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9364056b90 con 0x7f93701010c0 2024-03-16T09:13:01.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.011+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f937004bcc0 con 0x7f93701010c0 2024-03-16T09:13:01.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.012+0000 7f936d7fa640 1 -- 172.21.15.7:0/1070346559 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v36) v1 ==== 107+0+5211 (secure 0 0 0) 0x7f936405a250 con 0x7f93701010c0 2024-03-16T09:13:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:13:01.014 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44329,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/3039505467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3039505467},{"type":"v1","addr":"172.21.15.19:6827","nonce":3039505467}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:11:11.272500+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":0,"incarnation":32,"state":"up:resolve","state_seq":18,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi007.znurst","rank":1,"incarnation":32,"state":"up:resolve","state_seq":13,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:13:01.015 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f936006de50 msgr2=0x7f9360070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f936006de50 0x7f9360070300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f935c003080 tx=0x7f935c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93701010c0 msgr2=0x7f937010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93701010c0 0x7f937010dec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9364002a50 tx=0x7f9364004600 comp rx=0 tx=0).stop 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 shutdown_connections 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f936006de50 0x7f9360070300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9370101a60 0x7f937010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 --2- 172.21.15.7:0/1070346559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93701010c0 0x7f937010dec0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 >> 172.21.15.7:0/1070346559 conn(0x7f93700f6cb0 msgr2=0x7f93700f7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.014+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 shutdown_connections 2024-03-16T09:13:01.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.015+0000 7f9377014640 1 -- 172.21.15.7:0/1070346559 wait complete. 2024-03-16T09:13:01.029 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 -- 172.21.15.7:0/1119300931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 msgr2=0x7f19200fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 --2- 172.21.15.7:0/1119300931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 0x7f19200fc410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f19100048d0 tx=0x7f1910009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 -- 172.21.15.7:0/1119300931 shutdown_connections 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 --2- 172.21.15.7:0/1119300931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 0x7f19200fc410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 --2- 172.21.15.7:0/1119300931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19201023e0 0x7f19201027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 -- 172.21.15.7:0/1119300931 >> 172.21.15.7:0/1119300931 conn(0x7f19200f7fb0 msgr2=0x7f19200fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 -- 172.21.15.7:0/1119300931 shutdown_connections 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 -- 172.21.15.7:0/1119300931 wait complete. 2024-03-16T09:13:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 Processor -- start 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.159+0000 7f192731d640 1 -- start start 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f192731d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19201023e0 0x7f192006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f192731d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 0x7f192006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f192731d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19200716b0 con 0x7f1920102d80 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f192731d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1920070250 con 0x7f19201023e0 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19248b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 0x7f192006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19248b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 0x7f192006fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50732/0 (socket says 172.21.15.7:50732) 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19248b8640 1 -- 172.21.15.7:0/2196835280 learned_addr learned my addr 172.21.15.7:0/2196835280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19250b9640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19201023e0 0x7f192006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19248b8640 1 -- 172.21.15.7:0/2196835280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19201023e0 msgr2=0x7f192006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19248b8640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19201023e0 0x7f192006f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19248b8640 1 -- 172.21.15.7:0/2196835280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910005040 con 0x7f1920102d80 2024-03-16T09:13:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19250b9640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19201023e0 0x7f192006f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f19248b8640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 0x7f192006fd10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f19100047f0 tx=0x7f1910009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f1915ffb640 1 -- 172.21.15.7:0/2196835280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19100094c0 con 0x7f1920102d80 2024-03-16T09:13:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.160+0000 7f1915ffb640 1 -- 172.21.15.7:0/2196835280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1910032e50 con 0x7f1920102d80 2024-03-16T09:13:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.161+0000 7f1915ffb640 1 -- 172.21.15.7:0/2196835280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19100435f0 con 0x7f1920102d80 2024-03-16T09:13:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.161+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19200704d0 con 0x7f1920102d80 2024-03-16T09:13:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.161+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19201195f0 con 0x7f1920102d80 2024-03-16T09:13:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.162+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19200fdb50 con 0x7f1920102d80 2024-03-16T09:13:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.162+0000 7f1915ffb640 1 -- 172.21.15.7:0/2196835280 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f191002f070 con 0x7f1920102d80 2024-03-16T09:13:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.163+0000 7f1915ffb640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f190c06de60 0x7f190c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.163+0000 7f1915ffb640 1 -- 172.21.15.7:0/2196835280 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f19100af6b0 con 0x7f1920102d80 2024-03-16T09:13:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.163+0000 7f19250b9640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f190c06de60 0x7f190c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.163+0000 7f19250b9640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f190c06de60 0x7f190c070310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f191c004c60 tx=0x7f191c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:01.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.167+0000 7f1915ffb640 1 -- 172.21.15.7:0/2196835280 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1910078b20 con 0x7f1920102d80 2024-03-16T09:13:01.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:00 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1847483454' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-16T09:13:01.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:00 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:13:01.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:00 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1963415930' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-16T09:13:01.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:00 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1847483454' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-16T09:13:01.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:00 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T09:13:01.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:00 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1963415930' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-16T09:13:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.465+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f192004bcc0 con 0x7f1920102d80 2024-03-16T09:13:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.466+0000 7f1915ffb640 1 -- 172.21.15.7:0/2196835280 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v36) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f191007c1e0 con 0x7f1920102d80 2024-03-16T09:13:01.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:13:01.468 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44329,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/3039505467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3039505467},{"type":"v1","addr":"172.21.15.19:6827","nonce":3039505467}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:11:11.368008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi007.znurst","rank":1,"incarnation":32,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:13:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-03-16T09:13:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f190c06de60 msgr2=0x7f190c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f190c06de60 0x7f190c070310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f191c004c60 tx=0x7f191c009040 comp rx=0 tx=0).stop 2024-03-16T09:13:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 msgr2=0x7f192006fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 0x7f192006fd10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f19100047f0 tx=0x7f1910009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 shutdown_connections 2024-03-16T09:13:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f190c06de60 0x7f190c070310 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1920102d80 0x7f192006fd10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 --2- 172.21.15.7:0/2196835280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19201023e0 0x7f192006f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 >> 172.21.15.7:0/2196835280 conn(0x7f19200f7fb0 msgr2=0x7f19200f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 shutdown_connections 2024-03-16T09:13:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.468+0000 7f192731d640 1 -- 172.21.15.7:0/2196835280 wait complete. 2024-03-16T09:13:01.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-16T09:13:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.592+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/2217531015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d68102b70 msgr2=0x7f5d680fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.592+0000 7f5d6dc6e640 1 --2- 172.21.15.7:0/2217531015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d68102b70 0x7f5d680fc200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0048d0 tx=0x7f5d5c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/2217531015 shutdown_connections 2024-03-16T09:13:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 --2- 172.21.15.7:0/2217531015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d68102b70 0x7f5d680fc200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 --2- 172.21.15.7:0/2217531015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d681021d0 0x7f5d681025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/2217531015 >> 172.21.15.7:0/2217531015 conn(0x7f5d680f7dc0 msgr2=0x7f5d680fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/2217531015 shutdown_connections 2024-03-16T09:13:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/2217531015 wait complete. 2024-03-16T09:13:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 Processor -- start 2024-03-16T09:13:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 -- start start 2024-03-16T09:13:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d681021d0 0x7f5d68112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:01.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68102b70 0x7f5d68113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68113830 con 0x7f5d681021d0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.593+0000 7f5d6dc6e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6810ce80 con 0x7f5d68102b70 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d681021d0 0x7f5d68112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d681021d0 0x7f5d68112cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50760/0 (socket says 172.21.15.7:50760) 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d677fe640 1 -- 172.21.15.7:0/1144441564 learned_addr learned my addr 172.21.15.7:0/1144441564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d66ffd640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68102b70 0x7f5d68113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d677fe640 1 -- 172.21.15.7:0/1144441564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68102b70 msgr2=0x7f5d68113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d677fe640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68102b70 0x7f5d68113230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d677fe640 1 -- 172.21.15.7:0/1144441564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c005040 con 0x7f5d681021d0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d677fe640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d681021d0 0x7f5d68112cf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800eb70 tx=0x7f5d5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d4ffff640 1 -- 172.21.15.7:0/1144441564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d58007c30 con 0x7f5d681021d0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6810d100 con 0x7f5d681021d0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d4ffff640 1 -- 172.21.15.7:0/1144441564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d5800f040 con 0x7f5d681021d0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d4ffff640 1 -- 172.21.15.7:0/1144441564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d580095b0 con 0x7f5d681021d0 2024-03-16T09:13:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.594+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6810d5f0 con 0x7f5d681021d0 2024-03-16T09:13:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.595+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d680fd940 con 0x7f5d681021d0 2024-03-16T09:13:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.596+0000 7f5d4ffff640 1 -- 172.21.15.7:0/1144441564 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f5d58009730 con 0x7f5d681021d0 2024-03-16T09:13:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.596+0000 7f5d4ffff640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d5006de50 0x7f5d50070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.597+0000 7f5d66ffd640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d5006de50 0x7f5d50070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.597+0000 7f5d4ffff640 1 -- 172.21.15.7:0/1144441564 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5d5808d430 con 0x7f5d681021d0 2024-03-16T09:13:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.597+0000 7f5d66ffd640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d5006de50 0x7f5d50070300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5d680fa840 tx=0x7f5d5c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:01.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.605+0000 7f5d4ffff640 1 -- 172.21.15.7:0/1144441564 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d58056950 con 0x7f5d681021d0 2024-03-16T09:13:01.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.870+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f5d6810e360 con 0x7f5d681021d0 2024-03-16T09:13:01.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.871+0000 7f5d4ffff640 1 -- 172.21.15.7:0/1144441564 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v36) v1 ==== 107+0+5209 (secure 0 0 0) 0x7f5d5805a010 con 0x7f5d681021d0 2024-03-16T09:13:01.873 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:13:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44303,"name":"cephfs.smithi019.bncsil","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6825/2889841468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2889841468},{"type":"v1","addr":"172.21.15.19:6825","nonce":2889841468}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44329,"name":"cephfs.smithi019.gmhzgo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6827/3039505467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3039505467},{"type":"v1","addr":"172.21.15.19:6827","nonce":3039505467}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T08:58:28.666579+0000","modified":"2024-03-16T09:11:12.345759+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi007.bsxctp","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.7:6827/2837914432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2837914432},{"type":"v1","addr":"172.21.15.7:6827","nonce":2837914432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi007.znurst","rank":1,"incarnation":32,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.7:6829/1434212331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1434212331},{"type":"v1","addr":"172.21.15.7:6829","nonce":1434212331}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T09:13:01.873 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d5006de50 msgr2=0x7f5d50070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d5006de50 0x7f5d50070300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5d680fa840 tx=0x7f5d5c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d681021d0 msgr2=0x7f5d68112cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d681021d0 0x7f5d68112cf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800eb70 tx=0x7f5d5800b040 comp rx=0 tx=0).stop 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 shutdown_connections 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d5006de50 0x7f5d50070300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68102b70 0x7f5d68113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 --2- 172.21.15.7:0/1144441564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d681021d0 0x7f5d68112cf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:01.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 >> 172.21.15.7:0/1144441564 conn(0x7f5d680f7dc0 msgr2=0x7f5d680f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:01.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 shutdown_connections 2024-03-16T09:13:01.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:01.873+0000 7f5d6dc6e640 1 -- 172.21.15.7:0/1144441564 wait complete. 2024-03-16T09:13:01.886 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-16T09:13:01.897 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-16T09:13:01.897 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T09:13:01.898 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T09:13:01.915 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-16T09:13:01.915 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T09:13:01.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:13:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/3706416618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d581021d0 msgr2=0x7f5d581025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 --2- 172.21.15.7:0/3706416618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d581021d0 0x7f5d581025a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009c50 tx=0x7f5d48004d10 comp rx=0 tx=0).stop 2024-03-16T09:13:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/3706416618 shutdown_connections 2024-03-16T09:13:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 --2- 172.21.15.7:0/3706416618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d58102b70 0x7f5d580fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 --2- 172.21.15.7:0/3706416618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d581021d0 0x7f5d581025a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/3706416618 >> 172.21.15.7:0/3706416618 conn(0x7f5d580f7d60 msgr2=0x7f5d580fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/3706416618 shutdown_connections 2024-03-16T09:13:02.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/3706416618 wait complete. 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.109+0000 7f5d5dc2e640 1 Processor -- start 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d5dc2e640 1 -- start start 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d5dc2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581021d0 0x7f5d58112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d5dc2e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58102b70 0x7f5d58113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d5dc2e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d581138e0 con 0x7f5d58102b70 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d5dc2e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5810cea0 con 0x7f5d581021d0 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d56ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58102b70 0x7f5d58113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d56ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58102b70 0x7f5d58113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50786/0 (socket says 172.21.15.7:50786) 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d56ffd640 1 -- 172.21.15.7:0/1215689270 learned_addr learned my addr 172.21.15.7:0/1215689270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:02.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d577fe640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581021d0 0x7f5d58112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d56ffd640 1 -- 172.21.15.7:0/1215689270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581021d0 msgr2=0x7f5d58112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d56ffd640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581021d0 0x7f5d58112d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.110+0000 7f5d56ffd640 1 -- 172.21.15.7:0/1215689270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480098b0 con 0x7f5d58102b70 2024-03-16T09:13:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.111+0000 7f5d56ffd640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58102b70 0x7f5d58113250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b790 tx=0x7f5d4c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.111+0000 7f5d3ffff640 1 -- 172.21.15.7:0/1215689270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d4c00c700 con 0x7f5d58102b70 2024-03-16T09:13:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.111+0000 7f5d3ffff640 1 -- 172.21.15.7:0/1215689270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d4c00cd30 con 0x7f5d58102b70 2024-03-16T09:13:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.111+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5810d180 con 0x7f5d58102b70 2024-03-16T09:13:02.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.112+0000 7f5d3ffff640 1 -- 172.21.15.7:0/1215689270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d4c013370 con 0x7f5d58102b70 2024-03-16T09:13:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.112+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5810d6a0 con 0x7f5d58102b70 2024-03-16T09:13:02.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.112+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d580fd940 con 0x7f5d58102b70 2024-03-16T09:13:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.113+0000 7f5d3ffff640 1 -- 172.21.15.7:0/1215689270 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f5d4c012070 con 0x7f5d58102b70 2024-03-16T09:13:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.114+0000 7f5d3ffff640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d4006df10 0x7f5d400703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.114+0000 7f5d3ffff640 1 -- 172.21.15.7:0/1215689270 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5d4c08d160 con 0x7f5d58102b70 2024-03-16T09:13:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.114+0000 7f5d577fe640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d4006df10 0x7f5d400703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.114+0000 7f5d577fe640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d4006df10 0x7f5d400703c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d48004a30 tx=0x7f5d48037040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:02.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.119+0000 7f5d3ffff640 1 -- 172.21.15.7:0/1215689270 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d4c056710 con 0x7f5d58102b70 2024-03-16T09:13:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:01 smithi019 ceph-mon[83366]: pgmap v263: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:13:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:01 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1070346559' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-16T09:13:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:01 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2196835280' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-16T09:13:02.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:01 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1144441564' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-16T09:13:02.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:01 smithi007 ceph-mon[113546]: pgmap v263: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:13:02.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:01 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1070346559' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-16T09:13:02.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:01 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2196835280' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-16T09:13:02.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:01 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1144441564' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-16T09:13:02.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.307+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d58109390 con 0x7f5d58102b70 2024-03-16T09:13:02.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.308+0000 7f5d3ffff640 1 -- 172.21.15.7:0/1215689270 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16516 (secure 0 0 0) 0x7f5d4c059dd0 con 0x7f5d58102b70 2024-03-16T09:13:02.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:13:02.311 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","created":"2024-03-16T08:51:36.700010+0000","modified":"2024-03-16T09:10:51.672446+0000","last_up_change":"2024-03-16T09:09:57.383379+0000","last_in_change":"2024-03-16T08:56:48.805178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:55:53.635803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-16T08:58:27.077630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-16T08:58:27.912287+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f96044-56c4-4825-8138-629e5352a8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6803","nonce":2558509313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6805","nonce":2558509313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6809","nonce":2558509313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6807","nonce":2558509313}]},"public_addr":"172.21.15.7:6803/2558509313","cluster_addr":"172.21.15.7:6805/2558509313","heartbeat_back_addr":"172.21.15.7:6809/2558509313","heartbeat_front_addr":"172.21.15.7:6807/2558509313","state":["exists","up"]},{"osd":1,"uuid":"b3fcdd0c-0016-4781-803b-bbca53b4517b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6811","nonce":3984255151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6813","nonce":3984255151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6817","nonce":3984255151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6815","nonce":3984255151}]},"public_addr":"172.21.15.7:6811/3984255151","cluster_addr":"172.21.15.7:6813/3984255151","heartbeat_back_addr":"172.21.15.7:6817/3984255151","heartbeat_front_addr":"172.21.15.7:6815/3984255151","state":["exists","up"]},{"osd":2,"uuid":"4f400784-c979-4f41-bcd2-a52737e2e65e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6819","nonce":7192509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6821","nonce":7192509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6825","nonce":7192509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6823","nonce":7192509}]},"public_addr":"172.21.15.7:6819/7192509","cluster_addr":"172.21.15.7:6821/7192509","heartbeat_back_addr":"172.21.15.7:6825/7192509","heartbeat_front_addr":"172.21.15.7:6823/7192509","state":["exists","up"]},{"osd":3,"uuid":"c1fa594a-67c7-4f8f-a745-74ef06dfd43c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6801","nonce":2696958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6803","nonce":2696958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6807","nonce":2696958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6805","nonce":2696958381}]},"public_addr":"172.21.15.19:6801/2696958381","cluster_addr":"172.21.15.19:6803/2696958381","heartbeat_back_addr":"172.21.15.19:6807/2696958381","heartbeat_front_addr":"172.21.15.19:6805/2696958381","state":["exists","up"]},{"osd":4,"uuid":"55356509-5ce4-4cb1-9195-3911332ec8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6809","nonce":377988924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6811","nonce":377988924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6815","nonce":377988924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6813","nonce":377988924}]},"public_addr":"172.21.15.19:6809/377988924","cluster_addr":"172.21.15.19:6811/377988924","heartbeat_back_addr":"172.21.15.19:6815/377988924","heartbeat_front_addr":"172.21.15.19:6813/377988924","state":["exists","up"]},{"osd":5,"uuid":"576cd741-4e7d-421a-a5dd-67ceb0af2440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6817","nonce":3993266086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6819","nonce":3993266086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6823","nonce":3993266086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6821","nonce":3993266086}]},"public_addr":"172.21.15.19:6817/3993266086","cluster_addr":"172.21.15.19:6819/3993266086","heartbeat_back_addr":"172.21.15.19:6823/3993266086","heartbeat_front_addr":"172.21.15.19:6821/3993266086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:54:54.847162+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:23.867331+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:50.950660+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:14.805620+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:40.511669+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:57:05.957371+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6825/4120977378":"2024-03-17T09:09:58.393894+0000","172.21.15.19:6824/4120977378":"2024-03-17T09:09:58.393894+0000","172.21.15.7:6829/3212461733":"2024-03-17T09:09:58.393874+0000","172.21.15.7:0/3257972232":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/780594938":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6800/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3185447390":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/1599723782":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6801/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6828/3212461733":"2024-03-17T09:09:58.393874+0000","172.21.15.19:6829/3745729358":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/1002495884":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/3560098206":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6826/2352390069":"2024-03-17T08:58:44.727553+0000","172.21.15.19:0/3343811353":"2024-03-17T09:02:30.761555+0000","172.21.15.19:0/2272694341":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/1417367188":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/1939052835":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/3493394788":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6801/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3260963601":"2024-03-17T09:05:30.402092+0000","172.21.15.7:0/1698981906":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/3147176153":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3238187551":"2024-03-17T08:53:41.312979+0000","172.21.15.19:0/2491756940":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/2638653648":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/4215733143":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/3022930672":"2024-03-17T08:52:05.202127+0000","172.21.15.7:6827/2352390069":"2024-03-17T08:58:44.727553+0000","172.21.15.7:0/422599095":"2024-03-17T09:00:37.869343+0000","172.21.15.19:6828/3745729358":"2024-03-17T09:02:30.761555+0000","172.21.15.7:6800/2519997150":"2024-03-17T09:00:37.869343+0000","172.21.15.7:6801/2519997150":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/669861183":"2024-03-17T09:05:30.402092+0000","172.21.15.19:0/1724693724":"2024-03-17T09:02:30.761555+0000","172.21.15.7:6800/3493394788":"2024-03-17T09:05:30.402092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:13:02.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d4006df10 msgr2=0x7f5d400703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d4006df10 0x7f5d400703c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d48004a30 tx=0x7f5d48037040 comp rx=0 tx=0).stop 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58102b70 msgr2=0x7f5d58113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58102b70 0x7f5d58113250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b790 tx=0x7f5d4c0091d0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 shutdown_connections 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f5d4006df10 0x7f5d400703c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58102b70 0x7f5d58113250 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 --2- 172.21.15.7:0/1215689270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d581021d0 0x7f5d58112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 >> 172.21.15.7:0/1215689270 conn(0x7f5d580f7d60 msgr2=0x7f5d580f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 shutdown_connections 2024-03-16T09:13:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.310+0000 7f5d5dc2e640 1 -- 172.21.15.7:0/1215689270 wait complete. 2024-03-16T09:13:02.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:55:53.635803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-16T08:58:27.077630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-16T08:58:27.912287+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:13:02.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:13:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.427+0000 7f3159cae640 1 -- 172.21.15.7:0/2865275239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 msgr2=0x7f31541026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.427+0000 7f3159cae640 1 --2- 172.21.15.7:0/2865275239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 0x7f31541026a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f31440048d0 tx=0x7f3144009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.427+0000 7f3159cae640 1 -- 172.21.15.7:0/2865275239 shutdown_connections 2024-03-16T09:13:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.427+0000 7f3159cae640 1 --2- 172.21.15.7:0/2865275239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154102c70 0x7f31540fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.427+0000 7f3159cae640 1 --2- 172.21.15.7:0/2865275239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 0x7f31541026a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.427+0000 7f3159cae640 1 -- 172.21.15.7:0/2865275239 >> 172.21.15.7:0/2865275239 conn(0x7f31540f7d90 msgr2=0x7f31540fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.427+0000 7f3159cae640 1 -- 172.21.15.7:0/2865275239 shutdown_connections 2024-03-16T09:13:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.427+0000 7f3159cae640 1 -- 172.21.15.7:0/2865275239 wait complete. 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f3159cae640 1 Processor -- start 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f3159cae640 1 -- start start 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f3159cae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 0x7f3154112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f3159cae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154102c70 0x7f3154113320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f3159cae640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3154113920 con 0x7f31541022d0 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f3159cae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315410cf70 con 0x7f3154102c70 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f31537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 0x7f3154112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f31537fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 0x7f3154112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49970/0 (socket says 172.21.15.7:49970) 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.428+0000 7f31537fe640 1 -- 172.21.15.7:0/3930726789 learned_addr learned my addr 172.21.15.7:0/3930726789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f314bfff640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154102c70 0x7f3154113320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f31537fe640 1 -- 172.21.15.7:0/3930726789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154102c70 msgr2=0x7f3154113320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f31537fe640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154102c70 0x7f3154113320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f31537fe640 1 -- 172.21.15.7:0/3930726789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3144005040 con 0x7f31541022d0 2024-03-16T09:13:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f31537fe640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 0x7f3154112de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3144004930 tx=0x7f3144009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f3150ff9640 1 -- 172.21.15.7:0/3930726789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31440094c0 con 0x7f31541022d0 2024-03-16T09:13:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f3150ff9640 1 -- 172.21.15.7:0/3930726789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3144032e50 con 0x7f31541022d0 2024-03-16T09:13:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315410d1f0 con 0x7f31541022d0 2024-03-16T09:13:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f3150ff9640 1 -- 172.21.15.7:0/3930726789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31440433b0 con 0x7f31541022d0 2024-03-16T09:13:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.429+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315410d6b0 con 0x7f31541022d0 2024-03-16T09:13:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.430+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315404bcc0 con 0x7f31541022d0 2024-03-16T09:13:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.431+0000 7f3150ff9640 1 -- 172.21.15.7:0/3930726789 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f314402f070 con 0x7f31541022d0 2024-03-16T09:13:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.431+0000 7f3150ff9640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f314006de50 0x7f3140070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.431+0000 7f314bfff640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f314006de50 0x7f3140070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.431+0000 7f3150ff9640 1 -- 172.21.15.7:0/3930726789 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f31440af500 con 0x7f31541022d0 2024-03-16T09:13:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.432+0000 7f314bfff640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f314006de50 0x7f3140070300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f315410e6a0 tx=0x7f313c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.437+0000 7f3150ff9640 1 -- 172.21.15.7:0/3930726789 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3144078a20 con 0x7f31541022d0 2024-03-16T09:13:02.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.694+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31540fd960 con 0x7f31541022d0 2024-03-16T09:13:02.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.695+0000 7f3150ff9640 1 -- 172.21.15.7:0/3930726789 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f314407c0e0 con 0x7f31541022d0 2024-03-16T09:13:02.697 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-16T09:13:02.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f314006de50 msgr2=0x7f3140070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f314006de50 0x7f3140070300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f315410e6a0 tx=0x7f313c00b040 comp rx=0 tx=0).stop 2024-03-16T09:13:02.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 msgr2=0x7f3154112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 0x7f3154112de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3144004930 tx=0x7f3144009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 shutdown_connections 2024-03-16T09:13:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f314006de50 0x7f3140070300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3154102c70 0x7f3154113320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 --2- 172.21.15.7:0/3930726789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31541022d0 0x7f3154112de0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 >> 172.21.15.7:0/3930726789 conn(0x7f31540f7d90 msgr2=0x7f31540f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 shutdown_connections 2024-03-16T09:13:02.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.697+0000 7f3159cae640 1 -- 172.21.15.7:0/3930726789 wait complete. 2024-03-16T09:13:02.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-16T09:13:02.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 -- 172.21.15.7:0/1377858992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc0643a0 msgr2=0x7f41bc064770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 --2- 172.21.15.7:0/1377858992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc0643a0 0x7f41bc064770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f41b80048d0 tx=0x7f41b8009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 -- 172.21.15.7:0/1377858992 shutdown_connections 2024-03-16T09:13:02.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 --2- 172.21.15.7:0/1377858992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc064cb0 0x7f41bc108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 --2- 172.21.15.7:0/1377858992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc0643a0 0x7f41bc064770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 -- 172.21.15.7:0/1377858992 >> 172.21.15.7:0/1377858992 conn(0x7f41bc0710b0 msgr2=0x7f41bc0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:02.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 -- 172.21.15.7:0/1377858992 shutdown_connections 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 -- 172.21.15.7:0/1377858992 wait complete. 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 Processor -- start 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 -- start start 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0643a0 0x7f41bc114ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc064cb0 0x7f41bc115400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc115a00 con 0x7f41bc064cb0 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.813+0000 7f41c435d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc10f050 con 0x7f41bc0643a0 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c18f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc064cb0 0x7f41bc115400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c18f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc064cb0 0x7f41bc115400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49978/0 (socket says 172.21.15.7:49978) 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c18f8640 1 -- 172.21.15.7:0/2744108650 learned_addr learned my addr 172.21.15.7:0/2744108650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c20f9640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0643a0 0x7f41bc114ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c18f8640 1 -- 172.21.15.7:0/2744108650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0643a0 msgr2=0x7f41bc114ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c18f8640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0643a0 0x7f41bc114ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c18f8640 1 -- 172.21.15.7:0/2744108650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b8005040 con 0x7f41bc064cb0 2024-03-16T09:13:02.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c18f8640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc064cb0 0x7f41bc115400 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f41ac002a50 tx=0x7f41ac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41b2ffd640 1 -- 172.21.15.7:0/2744108650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ac007c30 con 0x7f41bc064cb0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41b2ffd640 1 -- 172.21.15.7:0/2744108650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41ac00f040 con 0x7f41bc064cb0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc10f330 con 0x7f41bc064cb0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41b2ffd640 1 -- 172.21.15.7:0/2744108650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ac00dbe0 con 0x7f41bc064cb0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.814+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc10f850 con 0x7f41bc064cb0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.816+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc0fb730 con 0x7f41bc064cb0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.816+0000 7f41b2ffd640 1 -- 172.21.15.7:0/2744108650 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f41ac00dd40 con 0x7f41bc064cb0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.816+0000 7f41b2ffd640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f41a806de50 0x7f41a8070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.816+0000 7f41b2ffd640 1 -- 172.21.15.7:0/2744108650 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f41ac08d730 con 0x7f41bc064cb0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.816+0000 7f41c20f9640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f41a806de50 0x7f41a8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.817+0000 7f41c20f9640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f41a806de50 0x7f41a8070300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f41b8003080 tx=0x7f41b8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:02.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:02.824+0000 7f41b2ffd640 1 -- 172.21.15.7:0/2744108650 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41ac056ba0 con 0x7f41bc064cb0 2024-03-16T09:13:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:02 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1215689270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:02.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:02 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/3930726789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:13:03.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.050+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f41bc04bcc0 con 0x7f41bc064cb0 2024-03-16T09:13:03.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.050+0000 7f41b2ffd640 1 -- 172.21.15.7:0/2744108650 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f41ac05a260 con 0x7f41bc064cb0 2024-03-16T09:13:03.052 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-16T09:13:03.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f41a806de50 msgr2=0x7f41a8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f41a806de50 0x7f41a8070300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f41b8003080 tx=0x7f41b8009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc064cb0 msgr2=0x7f41bc115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc064cb0 0x7f41bc115400 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f41ac002a50 tx=0x7f41ac004600 comp rx=0 tx=0).stop 2024-03-16T09:13:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 shutdown_connections 2024-03-16T09:13:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f41a806de50 0x7f41a8070300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc064cb0 0x7f41bc115400 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 --2- 172.21.15.7:0/2744108650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0643a0 0x7f41bc114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 >> 172.21.15.7:0/2744108650 conn(0x7f41bc0710b0 msgr2=0x7f41bc0f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 shutdown_connections 2024-03-16T09:13:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.053+0000 7f41c435d640 1 -- 172.21.15.7:0/2744108650 wait complete. 2024-03-16T09:13:03.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-16T09:13:03.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.196+0000 7f292a79d640 1 -- 172.21.15.7:0/3145631993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29240fa8a0 msgr2=0x7f29240facf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.196+0000 7f292a79d640 1 --2- 172.21.15.7:0/3145631993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29240fa8a0 0x7f29240facf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29180048d0 tx=0x7f2918009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.196+0000 7f292a79d640 1 -- 172.21.15.7:0/3145631993 shutdown_connections 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.196+0000 7f292a79d640 1 --2- 172.21.15.7:0/3145631993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29240fa8a0 0x7f29240facf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.196+0000 7f292a79d640 1 --2- 172.21.15.7:0/3145631993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29241008b0 0x7f2924100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.196+0000 7f292a79d640 1 -- 172.21.15.7:0/3145631993 >> 172.21.15.7:0/3145631993 conn(0x7f2924070880 msgr2=0x7f2924070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.196+0000 7f292a79d640 1 -- 172.21.15.7:0/3145631993 shutdown_connections 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.196+0000 7f292a79d640 1 -- 172.21.15.7:0/3145631993 wait complete. 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f292a79d640 1 Processor -- start 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f292a79d640 1 -- start start 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f292a79d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29240fa8a0 0x7f2924114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f292a79d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29241008b0 0x7f2924115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f292a79d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2924115a10 con 0x7f29241008b0 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f292a79d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292410f060 con 0x7f29240fa8a0 2024-03-16T09:13:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f29237fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29241008b0 0x7f2924115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f29237fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29241008b0 0x7f2924115410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49988/0 (socket says 172.21.15.7:49988) 2024-03-16T09:13:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f29237fe640 1 -- 172.21.15.7:0/1096302513 learned_addr learned my addr 172.21.15.7:0/1096302513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f2923fff640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29240fa8a0 0x7f2924114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f29237fe640 1 -- 172.21.15.7:0/1096302513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29240fa8a0 msgr2=0x7f2924114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f29237fe640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29240fa8a0 0x7f2924114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.197+0000 7f29237fe640 1 -- 172.21.15.7:0/1096302513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918005040 con 0x7f29241008b0 2024-03-16T09:13:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.198+0000 7f29237fe640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29241008b0 0x7f2924115410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2918004930 tx=0x7f2918009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.198+0000 7f2920ff9640 1 -- 172.21.15.7:0/1096302513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29180094c0 con 0x7f29241008b0 2024-03-16T09:13:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.198+0000 7f2920ff9640 1 -- 172.21.15.7:0/1096302513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2918032e50 con 0x7f29241008b0 2024-03-16T09:13:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.198+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292410f2e0 con 0x7f29241008b0 2024-03-16T09:13:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.198+0000 7f2920ff9640 1 -- 172.21.15.7:0/1096302513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29180433b0 con 0x7f29241008b0 2024-03-16T09:13:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.198+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292410f7a0 con 0x7f29241008b0 2024-03-16T09:13:03.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.199+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29240fbf90 con 0x7f29241008b0 2024-03-16T09:13:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.199+0000 7f2920ff9640 1 -- 172.21.15.7:0/1096302513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f291802f070 con 0x7f29241008b0 2024-03-16T09:13:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.200+0000 7f2920ff9640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f290c06de50 0x7f290c070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.200+0000 7f2920ff9640 1 -- 172.21.15.7:0/1096302513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f29180af580 con 0x7f29241008b0 2024-03-16T09:13:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.200+0000 7f2923fff640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f290c06de50 0x7f290c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.200+0000 7f2923fff640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f290c06de50 0x7f290c070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2914004ed0 tx=0x7f291400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.207+0000 7f2920ff9640 1 -- 172.21.15.7:0/1096302513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2918078a20 con 0x7f29241008b0 2024-03-16T09:13:03.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:02 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1215689270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:03.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:02 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/3930726789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:13:03.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.389+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f29241079b0 con 0x7f29241008b0 2024-03-16T09:13:03.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.390+0000 7f2920ff9640 1 -- 172.21.15.7:0/1096302513 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f291807c0e0 con 0x7f29241008b0 2024-03-16T09:13:03.392 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-16T09:13:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f290c06de50 msgr2=0x7f290c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f290c06de50 0x7f290c070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2914004ed0 tx=0x7f291400b040 comp rx=0 tx=0).stop 2024-03-16T09:13:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29241008b0 msgr2=0x7f2924115410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29241008b0 0x7f2924115410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2918004930 tx=0x7f2918009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 shutdown_connections 2024-03-16T09:13:03.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f290c06de50 0x7f290c070300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29241008b0 0x7f2924115410 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 --2- 172.21.15.7:0/1096302513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29240fa8a0 0x7f2924114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 >> 172.21.15.7:0/1096302513 conn(0x7f2924070880 msgr2=0x7f29240fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:03.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 shutdown_connections 2024-03-16T09:13:03.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.392+0000 7f292a79d640 1 -- 172.21.15.7:0/1096302513 wait complete. 2024-03-16T09:13:03.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 -- 172.21.15.7:0/4044251268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 msgr2=0x7f0e6c0647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 --2- 172.21.15.7:0/4044251268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 0x7f0e6c0647d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0048d0 tx=0x7f0e5c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 -- 172.21.15.7:0/4044251268 shutdown_connections 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 --2- 172.21.15.7:0/4044251268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 0x7f0e6c0647d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 --2- 172.21.15.7:0/4044251268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c102240 0x7f0e6c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 -- 172.21.15.7:0/4044251268 >> 172.21.15.7:0/4044251268 conn(0x7f0e6c0710b0 msgr2=0x7f0e6c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 -- 172.21.15.7:0/4044251268 shutdown_connections 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 -- 172.21.15.7:0/4044251268 wait complete. 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.528+0000 7f0e73b9d640 1 Processor -- start 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e73b9d640 1 -- start start 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e73b9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 0x7f0e6c112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e73b9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c102240 0x7f0e6c113150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e73b9d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c113750 con 0x7f0e6c0643a0 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e73b9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1138c0 con 0x7f0e6c102240 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e71939640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 0x7f0e6c112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e71939640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 0x7f0e6c112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50004/0 (socket says 172.21.15.7:50004) 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e71939640 1 -- 172.21.15.7:0/991093452 learned_addr learned my addr 172.21.15.7:0/991093452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e71939640 1 -- 172.21.15.7:0/991093452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c102240 msgr2=0x7f0e6c113150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e71939640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c102240 0x7f0e6c113150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e71939640 1 -- 172.21.15.7:0/991093452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c005040 con 0x7f0e6c0643a0 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e71939640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 0x7f0e6c112c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c830 tx=0x7f0e68004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.529+0000 7f0e627fc640 1 -- 172.21.15.7:0/991093452 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e6800a850 con 0x7f0e6c0643a0 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.530+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c10cfc0 con 0x7f0e6c0643a0 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.530+0000 7f0e627fc640 1 -- 172.21.15.7:0/991093452 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e6800ae80 con 0x7f0e6c0643a0 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.530+0000 7f0e627fc640 1 -- 172.21.15.7:0/991093452 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e68012ea0 con 0x7f0e6c0643a0 2024-03-16T09:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.530+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c10d4e0 con 0x7f0e6c0643a0 2024-03-16T09:13:03.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.531+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c0fd940 con 0x7f0e6c0643a0 2024-03-16T09:13:03.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.531+0000 7f0e627fc640 1 -- 172.21.15.7:0/991093452 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0e68012400 con 0x7f0e6c0643a0 2024-03-16T09:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.532+0000 7f0e627fc640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e5806de50 0x7f0e58070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.532+0000 7f0e71138640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e5806de50 0x7f0e58070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.532+0000 7f0e627fc640 1 -- 172.21.15.7:0/991093452 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0e6808da00 con 0x7f0e6c0643a0 2024-03-16T09:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.532+0000 7f0e71138640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e5806de50 0x7f0e58070300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0043d0 tx=0x7f0e5c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:03.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.536+0000 7f0e627fc640 1 -- 172.21.15.7:0/991093452 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e68056ef0 con 0x7f0e6c0643a0 2024-03-16T09:13:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.855+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0e6c109390 con 0x7f0e6c0643a0 2024-03-16T09:13:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.856+0000 7f0e627fc640 1 -- 172.21.15.7:0/991093452 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v84) v1 ==== 81+0+2280 (secure 0 0 0) 0x7f0e680126b0 con 0x7f0e6c0643a0 2024-03-16T09:13:03.858 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:6825/4120977378 2024-03-17T09:09:58.393894+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:6824/4120977378 2024-03-17T09:09:58.393894+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6829/3212461733 2024-03-17T09:09:58.393874+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3257972232 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/780594938 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2343923853 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/490303564 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3185447390 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1599723782 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2343923853 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3435258711 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:03.859 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6828/3212461733 2024-03-17T09:09:58.393874+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:6829/3745729358 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1002495884 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3560098206 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6826/2352390069 2024-03-17T08:58:44.727553+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:0/3343811353 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:0/2272694341 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1417367188 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1939052835 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3435258711 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3493394788 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/490303564 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3260963601 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:03.860 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1698981906 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3147176153 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3238187551 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:0/2491756940 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2638653648 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4215733143 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3022930672 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6827/2352390069 2024-03-17T08:58:44.727553+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/422599095 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:6828/3745729358 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2519997150 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2519997150 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:03.861 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/669861183 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:0/1724693724 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3493394788 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:listed 39 entries 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e5806de50 msgr2=0x7f0e58070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e5806de50 0x7f0e58070300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0043d0 tx=0x7f0e5c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 msgr2=0x7f0e6c112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 0x7f0e6c112c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c830 tx=0x7f0e68004430 comp rx=0 tx=0).stop 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 shutdown_connections 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f0e5806de50 0x7f0e58070300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6c102240 0x7f0e6c113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 --2- 172.21.15.7:0/991093452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c0643a0 0x7f0e6c112c10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 >> 172.21.15.7:0/991093452 conn(0x7f0e6c0710b0 msgr2=0x7f0e6c0f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 shutdown_connections 2024-03-16T09:13:03.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:03.858+0000 7f0e73b9d640 1 -- 172.21.15.7:0/991093452 wait complete. 2024-03-16T09:13:03.871 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-16T09:13:03.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:03.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-16T09:13:03.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T09:13:03.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:03.904 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T09:13:03.969 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T09:13:03.970 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T09:13:03.991 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-16T09:13:03.991 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T09:13:04.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:13:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 -- 172.21.15.7:0/3933587140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c1021b0 msgr2=0x7fe84c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 --2- 172.21.15.7:0/3933587140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c1021b0 0x7fe84c102580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe848009c50 tx=0x7fe848004d10 comp rx=0 tx=0).stop 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 -- 172.21.15.7:0/3933587140 shutdown_connections 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 --2- 172.21.15.7:0/3933587140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe84c102b50 0x7fe84c0fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 --2- 172.21.15.7:0/3933587140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c1021b0 0x7fe84c102580 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 -- 172.21.15.7:0/3933587140 >> 172.21.15.7:0/3933587140 conn(0x7fe84c0f7da0 msgr2=0x7fe84c0fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 -- 172.21.15.7:0/3933587140 shutdown_connections 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 -- 172.21.15.7:0/3933587140 wait complete. 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 Processor -- start 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.147+0000 7fe85451d640 1 -- start start 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe85451d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe84c1021b0 0x7fe84c112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe85451d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c102b50 0x7fe84c113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe85451d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c113840 con 0x7fe84c102b50 2024-03-16T09:13:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe85451d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c10ce90 con 0x7fe84c1021b0 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe851ab8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c102b50 0x7fe84c113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe851ab8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c102b50 0x7fe84c113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50022/0 (socket says 172.21.15.7:50022) 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe851ab8640 1 -- 172.21.15.7:0/2078462333 learned_addr learned my addr 172.21.15.7:0/2078462333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe8522b9640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe84c1021b0 0x7fe84c112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe851ab8640 1 -- 172.21.15.7:0/2078462333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe84c1021b0 msgr2=0x7fe84c112d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe851ab8640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe84c1021b0 0x7fe84c112d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe851ab8640 1 -- 172.21.15.7:0/2078462333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480098b0 con 0x7fe84c102b50 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe851ab8640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c102b50 0x7fe84c113240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00eb70 tx=0x7fe83c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe842ffd640 1 -- 172.21.15.7:0/2078462333 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83c007c30 con 0x7fe84c102b50 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe842ffd640 1 -- 172.21.15.7:0/2078462333 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe83c00f040 con 0x7fe84c102b50 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c10d110 con 0x7fe84c102b50 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe842ffd640 1 -- 172.21.15.7:0/2078462333 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83c0095b0 con 0x7fe84c102b50 2024-03-16T09:13:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.148+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c10d6b0 con 0x7fe84c102b50 2024-03-16T09:13:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.150+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84c0fd920 con 0x7fe84c102b50 2024-03-16T09:13:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.150+0000 7fe842ffd640 1 -- 172.21.15.7:0/2078462333 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fe83c009730 con 0x7fe84c102b50 2024-03-16T09:13:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.151+0000 7fe842ffd640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe83806de50 0x7fe838070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.151+0000 7fe8522b9640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe83806de50 0x7fe838070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.151+0000 7fe842ffd640 1 -- 172.21.15.7:0/2078462333 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe83c08d510 con 0x7fe84c102b50 2024-03-16T09:13:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.151+0000 7fe8522b9640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe83806de50 0x7fe838070300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe848004a30 tx=0x7fe84802e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:04.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.156+0000 7fe842ffd640 1 -- 172.21.15.7:0/2078462333 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe83c056980 con 0x7fe84c102b50 2024-03-16T09:13:04.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:03 smithi019 ceph-mon[83366]: pgmap v264: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:13:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:03 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2744108650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T09:13:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:03 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1096302513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T09:13:04.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:03 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/991093452' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T09:13:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:03 smithi007 ceph-mon[113546]: pgmap v264: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T09:13:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:03 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2744108650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T09:13:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:03 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1096302513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T09:13:04.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:03 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/991093452' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T09:13:04.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.347+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe84c109310 con 0x7fe84c102b50 2024-03-16T09:13:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.348+0000 7fe842ffd640 1 -- 172.21.15.7:0/2078462333 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16516 (secure 0 0 0) 0x7fe83c05a040 con 0x7fe84c102b50 2024-03-16T09:13:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-16T09:13:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"4c46b77e-e372-11ee-95c9-87774f69a715","created":"2024-03-16T08:51:36.700010+0000","modified":"2024-03-16T09:10:51.672446+0000","last_up_change":"2024-03-16T09:09:57.383379+0000","last_in_change":"2024-03-16T08:56:48.805178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T08:55:53.635803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-16T08:58:27.077630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-16T08:58:27.912287+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f96044-56c4-4825-8138-629e5352a8cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6803","nonce":2558509313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6805","nonce":2558509313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6809","nonce":2558509313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2558509313},{"type":"v1","addr":"172.21.15.7:6807","nonce":2558509313}]},"public_addr":"172.21.15.7:6803/2558509313","cluster_addr":"172.21.15.7:6805/2558509313","heartbeat_back_addr":"172.21.15.7:6809/2558509313","heartbeat_front_addr":"172.21.15.7:6807/2558509313","state":["exists","up"]},{"osd":1,"uuid":"b3fcdd0c-0016-4781-803b-bbca53b4517b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6811","nonce":3984255151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6813","nonce":3984255151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6817","nonce":3984255151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3984255151},{"type":"v1","addr":"172.21.15.7:6815","nonce":3984255151}]},"public_addr":"172.21.15.7:6811/3984255151","cluster_addr":"172.21.15.7:6813/3984255151","heartbeat_back_addr":"172.21.15.7:6817/3984255151","heartbeat_front_addr":"172.21.15.7:6815/3984255151","state":["exists","up"]},{"osd":2,"uuid":"4f400784-c979-4f41-bcd2-a52737e2e65e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6819","nonce":7192509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6821","nonce":7192509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6825","nonce":7192509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":7192509},{"type":"v1","addr":"172.21.15.7:6823","nonce":7192509}]},"public_addr":"172.21.15.7:6819/7192509","cluster_addr":"172.21.15.7:6821/7192509","heartbeat_back_addr":"172.21.15.7:6825/7192509","heartbeat_front_addr":"172.21.15.7:6823/7192509","state":["exists","up"]},{"osd":3,"uuid":"c1fa594a-67c7-4f8f-a745-74ef06dfd43c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6801","nonce":2696958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6803","nonce":2696958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6807","nonce":2696958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2696958381},{"type":"v1","addr":"172.21.15.19:6805","nonce":2696958381}]},"public_addr":"172.21.15.19:6801/2696958381","cluster_addr":"172.21.15.19:6803/2696958381","heartbeat_back_addr":"172.21.15.19:6807/2696958381","heartbeat_front_addr":"172.21.15.19:6805/2696958381","state":["exists","up"]},{"osd":4,"uuid":"55356509-5ce4-4cb1-9195-3911332ec8fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6809","nonce":377988924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6811","nonce":377988924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6815","nonce":377988924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":377988924},{"type":"v1","addr":"172.21.15.19:6813","nonce":377988924}]},"public_addr":"172.21.15.19:6809/377988924","cluster_addr":"172.21.15.19:6811/377988924","heartbeat_back_addr":"172.21.15.19:6815/377988924","heartbeat_front_addr":"172.21.15.19:6813/377988924","state":["exists","up"]},{"osd":5,"uuid":"576cd741-4e7d-421a-a5dd-67ceb0af2440","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6817","nonce":3993266086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6819","nonce":3993266086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6823","nonce":3993266086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3993266086},{"type":"v1","addr":"172.21.15.19:6821","nonce":3993266086}]},"public_addr":"172.21.15.19:6817/3993266086","cluster_addr":"172.21.15.19:6819/3993266086","heartbeat_back_addr":"172.21.15.19:6823/3993266086","heartbeat_front_addr":"172.21.15.19:6821/3993266086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:54:54.847162+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:23.867331+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:55:50.950660+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:14.805620+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:56:40.511669+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T08:57:05.957371+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6825/4120977378":"2024-03-17T09:09:58.393894+0000","172.21.15.19:6824/4120977378":"2024-03-17T09:09:58.393894+0000","172.21.15.7:6829/3212461733":"2024-03-17T09:09:58.393874+0000","172.21.15.7:0/3257972232":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/780594938":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6800/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3185447390":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/1599723782":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/2343923853":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6801/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6828/3212461733":"2024-03-17T09:09:58.393874+0000","172.21.15.19:6829/3745729358":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/1002495884":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/3560098206":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6826/2352390069":"2024-03-17T08:58:44.727553+0000","172.21.15.19:0/3343811353":"2024-03-17T09:02:30.761555+0000","172.21.15.19:0/2272694341":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/1417367188":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/1939052835":"2024-03-17T08:53:41.312979+0000","172.21.15.7:6800/3435258711":"2024-03-17T08:52:36.181013+0000","172.21.15.7:6801/3493394788":"2024-03-17T09:05:30.402092+0000","172.21.15.7:6801/490303564":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3260963601":"2024-03-17T09:05:30.402092+0000","172.21.15.7:0/1698981906":"2024-03-17T08:53:41.312979+0000","172.21.15.7:0/3147176153":"2024-03-17T08:52:05.202127+0000","172.21.15.7:0/3238187551":"2024-03-17T08:53:41.312979+0000","172.21.15.19:0/2491756940":"2024-03-17T09:02:30.761555+0000","172.21.15.7:0/2638653648":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/4215733143":"2024-03-17T08:52:36.181013+0000","172.21.15.7:0/3022930672":"2024-03-17T08:52:05.202127+0000","172.21.15.7:6827/2352390069":"2024-03-17T08:58:44.727553+0000","172.21.15.7:0/422599095":"2024-03-17T09:00:37.869343+0000","172.21.15.19:6828/3745729358":"2024-03-17T09:02:30.761555+0000","172.21.15.7:6800/2519997150":"2024-03-17T09:00:37.869343+0000","172.21.15.7:6801/2519997150":"2024-03-17T09:00:37.869343+0000","172.21.15.7:0/669861183":"2024-03-17T09:05:30.402092+0000","172.21.15.19:0/1724693724":"2024-03-17T09:02:30.761555+0000","172.21.15.7:6800/3493394788":"2024-03-17T09:05:30.402092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:13:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe83806de50 msgr2=0x7fe838070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe83806de50 0x7fe838070300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe848004a30 tx=0x7fe84802e040 comp rx=0 tx=0).stop 2024-03-16T09:13:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c102b50 msgr2=0x7fe84c113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c102b50 0x7fe84c113240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00eb70 tx=0x7fe83c00b040 comp rx=0 tx=0).stop 2024-03-16T09:13:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 shutdown_connections 2024-03-16T09:13:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fe83806de50 0x7fe838070300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe84c102b50 0x7fe84c113240 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 --2- 172.21.15.7:0/2078462333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe84c1021b0 0x7fe84c112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 >> 172.21.15.7:0/2078462333 conn(0x7fe84c0f7da0 msgr2=0x7fe84c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 shutdown_connections 2024-03-16T09:13:04.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.350+0000 7fe85451d640 1 -- 172.21.15.7:0/2078462333 wait complete. 2024-03-16T09:13:04.363 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T08:55:53.635803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-16T08:58:27.077630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-16T08:58:27.912287+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:13:04.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.466+0000 7fc2a635d640 1 -- 172.21.15.7:0/3113530421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 msgr2=0x7fc2a00fd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.466+0000 7fc2a635d640 1 --2- 172.21.15.7:0/3113530421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 0x7fc2a00fd5d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc2900048d0 tx=0x7fc290009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.466+0000 7fc2a635d640 1 -- 172.21.15.7:0/3113530421 shutdown_connections 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.466+0000 7fc2a635d640 1 --2- 172.21.15.7:0/3113530421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a00fdba0 0x7fc2a01061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.466+0000 7fc2a635d640 1 --2- 172.21.15.7:0/3113530421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 0x7fc2a00fd5d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.466+0000 7fc2a635d640 1 -- 172.21.15.7:0/3113530421 >> 172.21.15.7:0/3113530421 conn(0x7fc2a00f6cb0 msgr2=0x7fc2a00f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc2a635d640 1 -- 172.21.15.7:0/3113530421 shutdown_connections 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc2a635d640 1 -- 172.21.15.7:0/3113530421 wait complete. 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc2a635d640 1 Processor -- start 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc2a635d640 1 -- start start 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc2a635d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 0x7fc2a0112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc2a635d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a00fdba0 0x7fc2a01131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc2a635d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a01137d0 con 0x7fc2a00fd200 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc2a635d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0113940 con 0x7fc2a00fdba0 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.467+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 0x7fc2a0112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 0x7fc2a0112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50048/0 (socket says 172.21.15.7:50048) 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29ffff640 1 -- 172.21.15.7:0/885905312 learned_addr learned my addr 172.21.15.7:0/885905312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29f7fe640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a00fdba0 0x7fc2a01131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29ffff640 1 -- 172.21.15.7:0/885905312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a00fdba0 msgr2=0x7fc2a01131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29ffff640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a00fdba0 0x7fc2a01131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29ffff640 1 -- 172.21.15.7:0/885905312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290005040 con 0x7fc2a00fd200 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29ffff640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 0x7fc2a0112c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc290004930 tx=0x7fc290009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29cff9640 1 -- 172.21.15.7:0/885905312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2900094c0 con 0x7fc2a00fd200 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a010cfe0 con 0x7fc2a00fd200 2024-03-16T09:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29cff9640 1 -- 172.21.15.7:0/885905312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc290032e50 con 0x7fc2a00fd200 2024-03-16T09:13:04.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc29cff9640 1 -- 172.21.15.7:0/885905312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2900433b0 con 0x7fc2a00fd200 2024-03-16T09:13:04.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.468+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a010d4a0 con 0x7fc2a00fd200 2024-03-16T09:13:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.469+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a0065980 con 0x7fc2a00fd200 2024-03-16T09:13:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.470+0000 7fc29cff9640 1 -- 172.21.15.7:0/885905312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc29002f070 con 0x7fc2a00fd200 2024-03-16T09:13:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.471+0000 7fc29cff9640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28806de50 0x7fc288070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.471+0000 7fc29cff9640 1 -- 172.21.15.7:0/885905312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc2900af590 con 0x7fc2a00fd200 2024-03-16T09:13:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.471+0000 7fc29f7fe640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28806de50 0x7fc288070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.471+0000 7fc29f7fe640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28806de50 0x7fc288070300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0101aa0 tx=0x7fc29400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.478+0000 7fc29cff9640 1 -- 172.21.15.7:0/885905312 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc290078ab0 con 0x7fc2a00fd200 2024-03-16T09:13:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.665+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2a01093c0 con 0x7fc2a00fd200 2024-03-16T09:13:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.666+0000 7fc29cff9640 1 -- 172.21.15.7:0/885905312 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc29007c170 con 0x7fc2a00fd200 2024-03-16T09:13:04.668 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-16T09:13:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28806de50 msgr2=0x7fc288070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28806de50 0x7fc288070300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0101aa0 tx=0x7fc29400b040 comp rx=0 tx=0).stop 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 msgr2=0x7fc2a0112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 0x7fc2a0112c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc290004930 tx=0x7fc290009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 shutdown_connections 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28806de50 0x7fc288070300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a00fdba0 0x7fc2a01131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 --2- 172.21.15.7:0/885905312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fd200 0x7fc2a0112c90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 >> 172.21.15.7:0/885905312 conn(0x7fc2a00f6cb0 msgr2=0x7fc2a00f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 shutdown_connections 2024-03-16T09:13:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.668+0000 7fc2a635d640 1 -- 172.21.15.7:0/885905312 wait complete. 2024-03-16T09:13:04.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-16T09:13:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 -- 172.21.15.7:0/3587303006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd40643a0 msgr2=0x7f9dd40647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 --2- 172.21.15.7:0/3587303006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd40643a0 0x7f9dd40647d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40048d0 tx=0x7f9dc4009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 -- 172.21.15.7:0/3587303006 shutdown_connections 2024-03-16T09:13:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 --2- 172.21.15.7:0/3587303006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd40643a0 0x7f9dd40647d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 --2- 172.21.15.7:0/3587303006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd4102240 0x7f9dd4102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 -- 172.21.15.7:0/3587303006 >> 172.21.15.7:0/3587303006 conn(0x7f9dd40710b0 msgr2=0x7f9dd40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 -- 172.21.15.7:0/3587303006 shutdown_connections 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 -- 172.21.15.7:0/3587303006 wait complete. 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 Processor -- start 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 -- start start 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.782+0000 7f9ddb66e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd40643a0 0x7f9dd40fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9ddb66e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd4102240 0x7f9dd40fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9ddb66e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40fd0c0 con 0x7f9dd4102240 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9ddb66e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40fbc60 con 0x7f9dd40643a0 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dd8c09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd4102240 0x7f9dd40fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dd8c09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd4102240 0x7f9dd40fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50068/0 (socket says 172.21.15.7:50068) 2024-03-16T09:13:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dd8c09640 1 -- 172.21.15.7:0/319181720 learned_addr learned my addr 172.21.15.7:0/319181720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dd940a640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd40643a0 0x7f9dd40fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dd8c09640 1 -- 172.21.15.7:0/319181720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd40643a0 msgr2=0x7f9dd40fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dd8c09640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd40643a0 0x7f9dd40fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dd8c09640 1 -- 172.21.15.7:0/319181720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4005040 con 0x7f9dd4102240 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dd8c09640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd4102240 0x7f9dd40fb720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4004930 tx=0x7f9dc4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dc9ffb640 1 -- 172.21.15.7:0/319181720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc40094c0 con 0x7f9dd4102240 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dc9ffb640 1 -- 172.21.15.7:0/319181720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dc4032e50 con 0x7f9dd4102240 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd40fbee0 con 0x7f9dd4102240 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.783+0000 7f9dc9ffb640 1 -- 172.21.15.7:0/319181720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc40433b0 con 0x7f9dd4102240 2024-03-16T09:13:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.784+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd41193a0 con 0x7f9dd4102240 2024-03-16T09:13:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.785+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd40fd940 con 0x7f9dd4102240 2024-03-16T09:13:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.785+0000 7f9dc9ffb640 1 -- 172.21.15.7:0/319181720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f9dc402f070 con 0x7f9dd4102240 2024-03-16T09:13:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.786+0000 7f9dc9ffb640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9dc006de50 0x7f9dc0070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.786+0000 7f9dc9ffb640 1 -- 172.21.15.7:0/319181720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9dc40af610 con 0x7f9dd4102240 2024-03-16T09:13:04.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.786+0000 7f9dd940a640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9dc006de50 0x7f9dc0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.786+0000 7f9dd940a640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9dc006de50 0x7f9dc0070300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0004550 tx=0x7f9dd0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:04.794+0000 7f9dc9ffb640 1 -- 172.21.15.7:0/319181720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9dc4078b30 con 0x7f9dd4102240 2024-03-16T09:13:04.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:04 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/2078462333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:04.998 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:04 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/885905312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:13:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.056+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f9dd404bcc0 con 0x7f9dd4102240 2024-03-16T09:13:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.057+0000 7f9dc9ffb640 1 -- 172.21.15.7:0/319181720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9dc407c1f0 con 0x7f9dd4102240 2024-03-16T09:13:05.059 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-16T09:13:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9dc006de50 msgr2=0x7f9dc0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9dc006de50 0x7f9dc0070300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0004550 tx=0x7f9dd0009040 comp rx=0 tx=0).stop 2024-03-16T09:13:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd4102240 msgr2=0x7f9dd40fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd4102240 0x7f9dd40fb720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4004930 tx=0x7f9dc4009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 shutdown_connections 2024-03-16T09:13:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f9dc006de50 0x7f9dc0070300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dd4102240 0x7f9dd40fb720 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 --2- 172.21.15.7:0/319181720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd40643a0 0x7f9dd40fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 >> 172.21.15.7:0/319181720 conn(0x7f9dd40710b0 msgr2=0x7f9dd40f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 shutdown_connections 2024-03-16T09:13:05.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.059+0000 7f9ddb66e640 1 -- 172.21.15.7:0/319181720 wait complete. 2024-03-16T09:13:05.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-16T09:13:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 -- 172.21.15.7:0/989952052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 msgr2=0x7fc2a00fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 --2- 172.21.15.7:0/989952052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 0x7fc2a00fad10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc2900048d0 tx=0x7fc290009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 -- 172.21.15.7:0/989952052 shutdown_connections 2024-03-16T09:13:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 --2- 172.21.15.7:0/989952052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 0x7fc2a00fad10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 --2- 172.21.15.7:0/989952052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102250 0x7fc2a0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 -- 172.21.15.7:0/989952052 >> 172.21.15.7:0/989952052 conn(0x7fc2a0070880 msgr2=0x7fc2a0070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 -- 172.21.15.7:0/989952052 shutdown_connections 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 -- 172.21.15.7:0/989952052 wait complete. 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 Processor -- start 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.180+0000 7fc2a76ae640 1 -- start start 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a76ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 0x7fc2a0112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a76ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102250 0x7fc2a01131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a76ae640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a01137a0 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a76ae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a010ce40 con 0x7fc2a0102250 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a544a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 0x7fc2a0112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a544a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 0x7fc2a0112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50082/0 (socket says 172.21.15.7:50082) 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a544a640 1 -- 172.21.15.7:0/973860143 learned_addr learned my addr 172.21.15.7:0/973860143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a4c49640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102250 0x7fc2a01131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a544a640 1 -- 172.21.15.7:0/973860143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102250 msgr2=0x7fc2a01131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a544a640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102250 0x7fc2a01131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a544a640 1 -- 172.21.15.7:0/973860143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290005040 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a4c49640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102250 0x7fc2a01131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc2a544a640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 0x7fc2a0112c60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c830 tx=0x7fc29c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc295ffb640 1 -- 172.21.15.7:0/973860143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29c00a850 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.181+0000 7fc295ffb640 1 -- 172.21.15.7:0/973860143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc29c00ae80 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.182+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a010d120 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.182+0000 7fc295ffb640 1 -- 172.21.15.7:0/973860143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29c012ea0 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.182+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a010d610 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.183+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a00640f0 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.183+0000 7fc295ffb640 1 -- 172.21.15.7:0/973860143 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc29c012400 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.184+0000 7fc295ffb640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28c06de50 0x7fc28c070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.184+0000 7fc295ffb640 1 -- 172.21.15.7:0/973860143 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc29c08d8f0 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.184+0000 7fc2a4c49640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28c06de50 0x7fc28c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.184+0000 7fc2a4c49640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28c06de50 0x7fc28c070300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00f8cc0 tx=0x7fc290009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.189+0000 7fc295ffb640 1 -- 172.21.15.7:0/973860143 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc29c056ea0 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.212 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:04 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/2078462333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:05.213 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:04 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/885905312' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:13:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.382+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fc2a004bcc0 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.382+0000 7fc295ffb640 1 -- 172.21.15.7:0/973860143 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc29c0126b0 con 0x7fc2a00fa8a0 2024-03-16T09:13:05.385 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28c06de50 msgr2=0x7fc28c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28c06de50 0x7fc28c070300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00f8cc0 tx=0x7fc290009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 msgr2=0x7fc2a0112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 0x7fc2a0112c60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c830 tx=0x7fc29c004430 comp rx=0 tx=0).stop 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 shutdown_connections 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7fc28c06de50 0x7fc28c070300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2a0102250 0x7fc2a01131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 --2- 172.21.15.7:0/973860143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a00fa8a0 0x7fc2a0112c60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 >> 172.21.15.7:0/973860143 conn(0x7fc2a0070880 msgr2=0x7fc2a00f9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 shutdown_connections 2024-03-16T09:13:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.384+0000 7fc2a76ae640 1 -- 172.21.15.7:0/973860143 wait complete. 2024-03-16T09:13:05.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T09:13:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.526+0000 7f012e66e640 1 -- 172.21.15.7:0/1264251580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 msgr2=0x7f01280647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.526+0000 7f012e66e640 1 --2- 172.21.15.7:0/1264251580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 0x7f01280647d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f011c0048d0 tx=0x7f011c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.526+0000 7f012e66e640 1 -- 172.21.15.7:0/1264251580 shutdown_connections 2024-03-16T09:13:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.526+0000 7f012e66e640 1 --2- 172.21.15.7:0/1264251580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 0x7f01280647d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.526+0000 7f012e66e640 1 --2- 172.21.15.7:0/1264251580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128102240 0x7f0128102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.526+0000 7f012e66e640 1 -- 172.21.15.7:0/1264251580 >> 172.21.15.7:0/1264251580 conn(0x7f01280710b0 msgr2=0x7f01280714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.526+0000 7f012e66e640 1 -- 172.21.15.7:0/1264251580 shutdown_connections 2024-03-16T09:13:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.526+0000 7f012e66e640 1 -- 172.21.15.7:0/1264251580 wait complete. 2024-03-16T09:13:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f012e66e640 1 Processor -- start 2024-03-16T09:13:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f012e66e640 1 -- start start 2024-03-16T09:13:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f012e66e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 0x7f01280fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f012e66e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128102240 0x7f01280fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f012e66e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01280fd0c0 con 0x7f01280643a0 2024-03-16T09:13:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f012e66e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01280fbc60 con 0x7f0128102240 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f0127fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 0x7f01280fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f0127fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 0x7f01280fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50094/0 (socket says 172.21.15.7:50094) 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f0127fff640 1 -- 172.21.15.7:0/1901439162 learned_addr learned my addr 172.21.15.7:0/1901439162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.527+0000 7f01277fe640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128102240 0x7f01280fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f01277fe640 1 -- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 msgr2=0x7f01280fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f01277fe640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 0x7f01280fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f01277fe640 1 -- 172.21.15.7:0/1901439162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c005040 con 0x7f0128102240 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f0127fff640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 0x7f01280fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f01277fe640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128102240 0x7f01280fb720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f011c034040 tx=0x7f011c009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f0124ff9640 1 -- 172.21.15.7:0/1901439162 <== mon.1 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f011c0092b0 con 0x7f0128102240 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01280fbee0 con 0x7f0128102240 2024-03-16T09:13:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f0124ff9640 1 -- 172.21.15.7:0/1901439162 <== mon.1 v2:172.21.15.19:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f011c03d450 con 0x7f0128102240 2024-03-16T09:13:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f0124ff9640 1 -- 172.21.15.7:0/1901439162 <== mon.1 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f011c03b640 con 0x7f0128102240 2024-03-16T09:13:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.528+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01281193a0 con 0x7f0128102240 2024-03-16T09:13:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.529+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01280fd940 con 0x7f0128102240 2024-03-16T09:13:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.530+0000 7f0124ff9640 1 -- 172.21.15.7:0/1901439162 <== mon.1 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 39) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f011c02f070 con 0x7f0128102240 2024-03-16T09:13:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.530+0000 7f0124ff9640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f011006de50 0x7f0110070300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.530+0000 7f0124ff9640 1 -- 172.21.15.7:0/1901439162 <== mon.1 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f011c0af920 con 0x7f0128102240 2024-03-16T09:13:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.530+0000 7f0127fff640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f011006de50 0x7f0110070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.531+0000 7f0127fff640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f011006de50 0x7f0110070300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0118007e60 tx=0x7f011800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:05.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.535+0000 7f0124ff9640 1 -- 172.21.15.7:0/1901439162 <== mon.1 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f011c078e50 con 0x7f0128102240 2024-03-16T09:13:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.734+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0128119530 con 0x7f0128102240 2024-03-16T09:13:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.735+0000 7f0124ff9640 1 -- 172.21.15.7:0/1901439162 <== mon.1 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v84) v1 ==== 81+0+2280 (secure 0 0 0) 0x7f011c07c510 con 0x7f0128102240 2024-03-16T09:13:05.737 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:6825/4120977378 2024-03-17T09:09:58.393894+0000 2024-03-16T09:13:05.737 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:6824/4120977378 2024-03-17T09:09:58.393894+0000 2024-03-16T09:13:05.737 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6829/3212461733 2024-03-17T09:09:58.393874+0000 2024-03-16T09:13:05.737 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3257972232 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:05.737 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/780594938 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:05.737 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/490303564 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:05.737 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3185447390 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/669861183 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:0/2272694341 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1599723782 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2343923853 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3435258711 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6828/3212461733 2024-03-17T09:09:58.393874+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:6829/3745729358 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3435258711 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3493394788 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/490303564 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3260963601 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1698981906 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3147176153 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3238187551 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2638653648 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:0/2491756940 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:0/3343811353 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6826/2352390069 2024-03-17T08:58:44.727553+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3560098206 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1002495884 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2343923853 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1939052835 2024-03-17T08:53:41.312979+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1417367188 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/4215733143 2024-03-17T08:52:36.181013+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3022930672 2024-03-17T08:52:05.202127+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6827/2352390069 2024-03-17T08:58:44.727553+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/422599095 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3493394788 2024-03-17T09:05:30.402092+0000 2024-03-16T09:13:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:0/1724693724 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:05.740 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.19:6828/3745729358 2024-03-17T09:02:30.761555+0000 2024-03-16T09:13:05.740 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2519997150 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:05.740 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2519997150 2024-03-17T09:00:37.869343+0000 2024-03-16T09:13:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:listed 39 entries 2024-03-16T09:13:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f011006de50 msgr2=0x7f0110070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f011006de50 0x7f0110070300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0118007e60 tx=0x7f011800b040 comp rx=0 tx=0).stop 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128102240 msgr2=0x7f01280fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128102240 0x7f01280fb720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f011c034040 tx=0x7f011c009da0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 shutdown_connections 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:6800/811946302,v1:172.21.15.7:6801/811946302] conn(0x7f011006de50 0x7f0110070300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0128102240 0x7f01280fb720 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 --2- 172.21.15.7:0/1901439162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01280643a0 0x7f01280fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 >> 172.21.15.7:0/1901439162 conn(0x7f01280710b0 msgr2=0x7f01280f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 shutdown_connections 2024-03-16T09:13:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-16T09:13:05.737+0000 7f012e66e640 1 -- 172.21.15.7:0/1901439162 wait complete. 2024-03-16T09:13:05.754 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-16T09:13:05.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:05.754 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-16T09:13:05.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-16T09:13:05.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:05.784 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-16T09:13:05.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:05.848 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-16T09:13:05.866 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T09:13:05.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:05.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T09:13:05.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:05.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-03-16T09:13:06.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:05 smithi007 ceph-mon[113546]: pgmap v265: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T09:13:06.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:05 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/319181720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T09:13:06.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:05 smithi007 ceph-mon[113546]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:13:06.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:05 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/973860143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T09:13:06.003 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:05 smithi007 ceph-mon[113546]: from='client.? 172.21.15.7:0/1901439162' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T09:13:06.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:06.049 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-16T09:13:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-16T09:13:06.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:06.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T09:13:06.127 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:05 smithi019 ceph-mon[83366]: pgmap v265: 65 pgs: 65 active+clean; 300 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T09:13:06.127 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:05 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/319181720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T09:13:06.127 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:05 smithi019 ceph-mon[83366]: from='mgr.34104 172.21.15.7:0/1416444117' entity='mgr.smithi007.uamwtp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:13:06.128 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:05 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/973860143' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T09:13:06.128 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:05 smithi019 ceph-mon[83366]: from='client.? 172.21.15.7:0/1901439162' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T09:13:06.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:13:06.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-03-16T09:13:06.220 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-16T09:13:06.231 INFO:tasks.cephadm:Teardown begin 2024-03-16T09:13:06.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:13:06.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:13:06.290 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-16T09:13:06.291 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T09:13:06.318 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T09:13:06.345 INFO:tasks.cephadm:Stopping all daemons... 2024-03-16T09:13:06.345 INFO:tasks.cephadm.mon.smithi007:Stopping mon.smithi007... 2024-03-16T09:13:06.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007 2024-03-16T09:13:06.692 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:06 smithi007 systemd[1]: Stopping Ceph mon.smithi007 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:13:06.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:13:06.689+0000 7f052d03e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi007 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:13:06.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:06 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007[113542]: 2024-03-16T09:13:06.689+0000 7f052d03e700 -1 mon.smithi007@0(leader) e3 *** Got Signal Terminated *** 2024-03-16T09:13:06.954 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:06 smithi007 podman[155103]: 2024-03-16 09:13:06.730947332 +0000 UTC m=+0.154840707 container died bef77d9053ffa9cc11166b4943ae8a30326313b6d51c3f0fe37ebd7607609591 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True) 2024-03-16T09:13:07.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:07 smithi007 podman[155103]: 2024-03-16 09:13:07.160185952 +0000 UTC m=+0.584079332 container cleanup bef77d9053ffa9cc11166b4943ae8a30326313b6d51c3f0fe37ebd7607609591 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=) 2024-03-16T09:13:07.248 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:07 smithi007 bash[155103]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007 2024-03-16T09:13:07.617 INFO:journalctl@ceph.mon.smithi007.smithi007.stdout:Mar 16 09:13:07 smithi007 podman[155160]: 2024-03-16 09:13:07.353677908 +0000 UTC m=+0.614908690 container remove bef77d9053ffa9cc11166b4943ae8a30326313b6d51c3f0fe37ebd7607609591 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi007, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T09:13:07.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi007.service' 2024-03-16T09:13:07.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:13:07.679 INFO:tasks.cephadm.mon.smithi007:Stopped mon.smithi007 2024-03-16T09:13:07.679 INFO:tasks.cephadm.mon.smithi019:Stopping mon.smithi019... 2024-03-16T09:13:07.679 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi019 2024-03-16T09:13:07.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:07 smithi019 systemd[1]: Stopping Ceph mon.smithi019 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:13:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:07 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019[83362]: 2024-03-16T09:13:07.978+0000 7fbb480c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi019 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:13:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:07 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019[83362]: 2024-03-16T09:13:07.978+0000 7fbb480c6700 -1 mon.smithi019@1(peon) e3 *** Got Signal Terminated *** 2024-03-16T09:13:08.463 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:08 smithi019 podman[115268]: 2024-03-16 09:13:08.194032223 +0000 UTC m=+0.318778146 container died 1abff6ceff3e6a1a9c348636ce5782a4d68198ab269f8ea7c74b42416e0b06c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-16T09:13:08.962 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:08 smithi019 podman[115268]: 2024-03-16 09:13:08.755269038 +0000 UTC m=+0.880014959 container cleanup 1abff6ceff3e6a1a9c348636ce5782a4d68198ab269f8ea7c74b42416e0b06c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T09:13:08.963 INFO:journalctl@ceph.mon.smithi019.smithi019.stdout:Mar 16 09:13:08 smithi019 bash[115268]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-mon-smithi019 2024-03-16T09:13:09.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@mon.smithi019.service' 2024-03-16T09:13:09.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:13:09.240 INFO:tasks.cephadm.mon.smithi019:Stopped mon.smithi019 2024-03-16T09:13:09.240 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-16T09:13:09.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.0 2024-03-16T09:13:09.558 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:09 smithi007 systemd[1]: Stopping Ceph osd.0 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:13:09.998 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:09 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[125287]: 2024-03-16T09:13:09.555+0000 7f270071e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:13:09.998 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:09 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[125287]: 2024-03-16T09:13:09.555+0000 7f270071e700 -1 osd.0 84 *** Got signal Terminated *** 2024-03-16T09:13:09.998 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:09 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0[125287]: 2024-03-16T09:13:09.555+0000 7f270071e700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:13:15.235 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:14 smithi007 podman[155265]: 2024-03-16 09:13:14.946453672 +0000 UTC m=+5.510765693 container died 1f775f30a8555e232b7e6e11b2cf81032f7f9229e38266a098b2ed8532715b65 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-16T09:13:15.498 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:15 smithi007 podman[155265]: 2024-03-16 09:13:15.431992378 +0000 UTC m=+5.996304405 container cleanup 1f775f30a8555e232b7e6e11b2cf81032f7f9229e38266a098b2ed8532715b65 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-16T09:13:15.498 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:15 smithi007 bash[155265]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0 2024-03-16T09:13:15.998 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:15 smithi007 podman[155284]: 2024-03-16 09:13:15.691925987 +0000 UTC m=+0.742200658 container remove 1f775f30a8555e232b7e6e11b2cf81032f7f9229e38266a098b2ed8532715b65 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:16.314 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:16 smithi007 podman[155369]: 2024-03-16 09:13:15.943008856 +0000 UTC m=+0.022452204 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:16.314 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:16 smithi007 podman[155369]: 2024-03-16 09:13:16.042817659 +0000 UTC m=+0.122260988 container create 70961dfdf8868d5e200953c1604ddbd98dc19f7269362ca15342134167e94bf8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-16T09:13:16.314 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:16 smithi007 podman[155369]: 2024-03-16 09:13:16.312595455 +0000 UTC m=+0.392038799 container init 70961dfdf8868d5e200953c1604ddbd98dc19f7269362ca15342134167e94bf8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307) 2024-03-16T09:13:16.648 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:16 smithi007 podman[155369]: 2024-03-16 09:13:16.316977054 +0000 UTC m=+0.396420384 container start 70961dfdf8868d5e200953c1604ddbd98dc19f7269362ca15342134167e94bf8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-16T09:13:16.648 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:16 smithi007 podman[155369]: 2024-03-16 09:13:16.421070932 +0000 UTC m=+0.500514288 container attach 70961dfdf8868d5e200953c1604ddbd98dc19f7269362ca15342134167e94bf8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:13:16.998 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:16 smithi007 podman[155369]: 2024-03-16 09:13:16.646750611 +0000 UTC m=+0.726193952 container died 70961dfdf8868d5e200953c1604ddbd98dc19f7269362ca15342134167e94bf8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-16T09:13:17.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.0.service' 2024-03-16T09:13:17.430 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:17 smithi007 podman[155396]: 2024-03-16 09:13:17.383885415 +0000 UTC m=+0.731631711 container remove 70961dfdf8868d5e200953c1604ddbd98dc19f7269362ca15342134167e94bf8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-16T09:13:17.430 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:17 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-16T09:13:17.430 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:17 smithi007 systemd[1]: Stopped Ceph osd.0 for 4c46b77e-e372-11ee-95c9-87774f69a715. 2024-03-16T09:13:17.430 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 16 09:13:17 smithi007 systemd[1]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.0.service: Consumed 8.118s CPU time. 2024-03-16T09:13:17.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:13:17.446 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-16T09:13:17.446 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-16T09:13:17.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.1 2024-03-16T09:13:17.730 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:17 smithi007 systemd[1]: Stopping Ceph osd.1 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:13:17.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:17 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[131183]: 2024-03-16T09:13:17.837+0000 7f9015f5c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:13:17.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:17 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[131183]: 2024-03-16T09:13:17.837+0000 7f9015f5c700 -1 osd.1 84 *** Got signal Terminated *** 2024-03-16T09:13:17.998 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:17 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1[131183]: 2024-03-16T09:13:17.837+0000 7f9015f5c700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:13:23.482 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:23 smithi007 podman[155475]: 2024-03-16 09:13:23.229884959 +0000 UTC m=+5.529301378 container died 4aa10e800f7285b7813be7848ff2e0c0c41e50e276022899d5b20fa59fd11cec (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-16T09:13:23.748 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:23 smithi007 podman[155475]: 2024-03-16 09:13:23.578835981 +0000 UTC m=+5.878252388 container cleanup 4aa10e800f7285b7813be7848ff2e0c0c41e50e276022899d5b20fa59fd11cec (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-16T09:13:23.748 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:23 smithi007 bash[155475]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1 2024-03-16T09:13:24.218 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:23 smithi007 podman[155504]: 2024-03-16 09:13:23.855149088 +0000 UTC m=+0.623374229 container remove 4aa10e800f7285b7813be7848ff2e0c0c41e50e276022899d5b20fa59fd11cec (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:13:24.477 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:24 smithi007 podman[155599]: 2024-03-16 09:13:24.120051241 +0000 UTC m=+0.022501522 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:24.477 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:24 smithi007 podman[155599]: 2024-03-16 09:13:24.247821081 +0000 UTC m=+0.150271353 container create 8114b7255f946be66d0102c7474c7696a74441c4f580122df91de1886cd36062 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-16T09:13:24.477 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:24 smithi007 podman[155599]: 2024-03-16 09:13:24.475361774 +0000 UTC m=+0.377812044 container init 8114b7255f946be66d0102c7474c7696a74441c4f580122df91de1886cd36062 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-16T09:13:24.748 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:24 smithi007 podman[155599]: 2024-03-16 09:13:24.480050441 +0000 UTC m=+0.382500720 container start 8114b7255f946be66d0102c7474c7696a74441c4f580122df91de1886cd36062 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-16T09:13:24.748 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:24 smithi007 podman[155599]: 2024-03-16 09:13:24.575527351 +0000 UTC m=+0.477977631 container attach 8114b7255f946be66d0102c7474c7696a74441c4f580122df91de1886cd36062 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-16T09:13:25.091 INFO:journalctl@ceph.osd.1.smithi007.stdout:Mar 16 09:13:24 smithi007 podman[155599]: 2024-03-16 09:13:24.787726684 +0000 UTC m=+0.690176965 container died 8114b7255f946be66d0102c7474c7696a74441c4f580122df91de1886cd36062 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-16T09:13:25.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.1.service' 2024-03-16T09:13:25.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:13:25.458 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-16T09:13:25.458 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-16T09:13:25.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.2 2024-03-16T09:13:25.748 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:25 smithi007 systemd[1]: Stopping Ceph osd.2 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:13:26.248 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:25 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[136980]: 2024-03-16T09:13:25.850+0000 7f316e6f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:13:26.248 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:25 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[136980]: 2024-03-16T09:13:25.850+0000 7f316e6f0700 -1 osd.2 84 *** Got signal Terminated *** 2024-03-16T09:13:26.248 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:25 smithi007 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2[136980]: 2024-03-16T09:13:25.850+0000 7f316e6f0700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:13:31.487 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:31 smithi007 podman[155703]: 2024-03-16 09:13:31.223782872 +0000 UTC m=+5.509686673 container died 5449f945991b5364849d62d1e363a170e718059e472d93786b334cab793c264a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-16T09:13:31.748 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:31 smithi007 podman[155703]: 2024-03-16 09:13:31.58471304 +0000 UTC m=+5.870616851 container cleanup 5449f945991b5364849d62d1e363a170e718059e472d93786b334cab793c264a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-16T09:13:31.748 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:31 smithi007 bash[155703]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2 2024-03-16T09:13:32.229 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:31 smithi007 podman[155720]: 2024-03-16 09:13:31.861552984 +0000 UTC m=+0.633869206 container remove 5449f945991b5364849d62d1e363a170e718059e472d93786b334cab793c264a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-16T09:13:32.372 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:32 smithi007 podman[155811]: 2024-03-16 09:13:32.131282037 +0000 UTC m=+0.020644295 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:32.372 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:32 smithi007 podman[155811]: 2024-03-16 09:13:32.254176723 +0000 UTC m=+0.143538990 container create f36ab27e10c38f96a2ac1fb897b8980ac4251e070848846490df34d74e18dfb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:13:32.498 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:32 smithi007 podman[155811]: 2024-03-16 09:13:32.456938993 +0000 UTC m=+0.346301271 container init f36ab27e10c38f96a2ac1fb897b8980ac4251e070848846490df34d74e18dfb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=) 2024-03-16T09:13:32.498 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:32 smithi007 podman[155811]: 2024-03-16 09:13:32.461012461 +0000 UTC m=+0.350374725 container start f36ab27e10c38f96a2ac1fb897b8980ac4251e070848846490df34d74e18dfb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:32.831 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:32 smithi007 podman[155811]: 2024-03-16 09:13:32.565362259 +0000 UTC m=+0.454724533 container attach f36ab27e10c38f96a2ac1fb897b8980ac4251e070848846490df34d74e18dfb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:13:33.128 INFO:journalctl@ceph.osd.2.smithi007.stdout:Mar 16 09:13:32 smithi007 podman[155811]: 2024-03-16 09:13:32.829883682 +0000 UTC m=+0.719245940 container died f36ab27e10c38f96a2ac1fb897b8980ac4251e070848846490df34d74e18dfb9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-16T09:13:33.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.2.service' 2024-03-16T09:13:33.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:13:33.478 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-16T09:13:33.478 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-16T09:13:33.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.3 2024-03-16T09:13:33.793 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:33 smithi019 systemd[1]: Stopping Ceph osd.3 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:13:34.058 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:33 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[93068]: 2024-03-16T09:13:33.791+0000 7fa0fc48c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:13:34.058 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:33 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[93068]: 2024-03-16T09:13:33.791+0000 7fa0fc48c700 -1 osd.3 84 *** Got signal Terminated *** 2024-03-16T09:13:34.058 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:33 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3[93068]: 2024-03-16T09:13:33.791+0000 7fa0fc48c700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:13:38.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:37 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:37.920+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:39.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:38 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:38.957+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:39.680 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:39 smithi019 podman[115390]: 2024-03-16 09:13:39.280518457 +0000 UTC m=+5.610121473 container died bb64247744fe74069e60e14781ae07a9abb1e6270c76d69c239172c12d960fbe (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:39.957 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:39 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:39.678+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:13.415464+0000 front 2024-03-16T09:13:13.415501+0000 (oldest deadline 2024-03-16T09:13:38.715425+0000) 2024-03-16T09:13:39.958 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:39 smithi019 podman[115390]: 2024-03-16 09:13:39.891759714 +0000 UTC m=+6.221362727 container cleanup bb64247744fe74069e60e14781ae07a9abb1e6270c76d69c239172c12d960fbe (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-16T09:13:39.958 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:39 smithi019 bash[115390]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3 2024-03-16T09:13:40.213 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:40 smithi019 podman[115418]: 2024-03-16 09:13:40.110358152 +0000 UTC m=+0.822625611 container remove bb64247744fe74069e60e14781ae07a9abb1e6270c76d69c239172c12d960fbe (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T09:13:40.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:39 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:39.955+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:40.873 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:40 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:40.668+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:13.415464+0000 front 2024-03-16T09:13:13.415501+0000 (oldest deadline 2024-03-16T09:13:38.715425+0000) 2024-03-16T09:13:40.874 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:40 smithi019 podman[115509]: 2024-03-16 09:13:40.459314437 +0000 UTC m=+0.017426122 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:40.874 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:40 smithi019 podman[115509]: 2024-03-16 09:13:40.561117998 +0000 UTC m=+0.119229675 container create b82236142eb93667742b7348bf510179c5ef47e075b1f1b9345900d64ae66800 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-16T09:13:40.874 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:40 smithi019 podman[115509]: 2024-03-16 09:13:40.788781439 +0000 UTC m=+0.346893117 container init b82236142eb93667742b7348bf510179c5ef47e075b1f1b9345900d64ae66800 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:13:40.874 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:40 smithi019 podman[115509]: 2024-03-16 09:13:40.791920604 +0000 UTC m=+0.350032290 container start b82236142eb93667742b7348bf510179c5ef47e075b1f1b9345900d64ae66800 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:40.874 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:40 smithi019 podman[115509]: 2024-03-16 09:13:40.871822239 +0000 UTC m=+0.429933917 container attach b82236142eb93667742b7348bf510179c5ef47e075b1f1b9345900d64ae66800 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-16T09:13:41.128 INFO:journalctl@ceph.osd.3.smithi019.stdout:Mar 16 09:13:41 smithi019 podman[115509]: 2024-03-16 09:13:41.09358266 +0000 UTC m=+0.651694329 container died b82236142eb93667742b7348bf510179c5ef47e075b1f1b9345900d64ae66800 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-16T09:13:41.128 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:40 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:40.959+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:41.713 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:41 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:41.632+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:13.415464+0000 front 2024-03-16T09:13:13.415501+0000 (oldest deadline 2024-03-16T09:13:38.715425+0000) 2024-03-16T09:13:41.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.3.service' 2024-03-16T09:13:41.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:13:41.824 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-16T09:13:41.824 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-16T09:13:41.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.4 2024-03-16T09:13:42.139 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:41 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:41.929+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:42.139 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:41 smithi019 systemd[1]: Stopping Ceph osd.4 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:13:42.139 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:42.136+0000 7f76e9a02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:13:42.139 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:42.136+0000 7f76e9a02700 -1 osd.4 84 *** Got signal Terminated *** 2024-03-16T09:13:42.139 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:42.136+0000 7f76e9a02700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:13:42.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:42.903+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:42.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:42 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:42.678+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:13.415464+0000 front 2024-03-16T09:13:13.415501+0000 (oldest deadline 2024-03-16T09:13:38.715425+0000) 2024-03-16T09:13:43.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:43 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:43.683+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:13.415464+0000 front 2024-03-16T09:13:13.415501+0000 (oldest deadline 2024-03-16T09:13:38.715425+0000) 2024-03-16T09:13:43.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:43 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:43.930+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:44.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:44 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:44.947+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:44.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:44 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:44.708+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:13.415464+0000 front 2024-03-16T09:13:13.415501+0000 (oldest deadline 2024-03-16T09:13:38.715425+0000) 2024-03-16T09:13:44.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:44 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:44.708+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:22.816112+0000 front 2024-03-16T09:13:22.816222+0000 (oldest deadline 2024-03-16T09:13:44.516177+0000) 2024-03-16T09:13:45.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:45 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:45.672+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:13.415464+0000 front 2024-03-16T09:13:13.415501+0000 (oldest deadline 2024-03-16T09:13:38.715425+0000) 2024-03-16T09:13:45.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:45 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:45.672+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:22.816112+0000 front 2024-03-16T09:13:22.816222+0000 (oldest deadline 2024-03-16T09:13:44.516177+0000) 2024-03-16T09:13:46.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:45 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:45.983+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:46.991 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:46 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:46.715+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:13.415464+0000 front 2024-03-16T09:13:13.415501+0000 (oldest deadline 2024-03-16T09:13:38.715425+0000) 2024-03-16T09:13:46.992 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:46 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4[97938]: 2024-03-16T09:13:46.715+0000 7f76e1fee700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:22.816112+0000 front 2024-03-16T09:13:22.816222+0000 (oldest deadline 2024-03-16T09:13:44.516177+0000) 2024-03-16T09:13:47.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:46 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:46.989+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:47.739 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:47 smithi019 podman[115615]: 2024-03-16 09:13:47.464113043 +0000 UTC m=+5.451031414 container died e1df7d4b9eaaae0db8ec1665a91843125983dc607eafdb25342213753c6794a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-16T09:13:48.213 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:47 smithi019 podman[115615]: 2024-03-16 09:13:47.912054709 +0000 UTC m=+5.898973139 container cleanup e1df7d4b9eaaae0db8ec1665a91843125983dc607eafdb25342213753c6794a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-16T09:13:48.213 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:47 smithi019 bash[115615]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4 2024-03-16T09:13:48.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:47 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:47.979+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:48.213 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:47 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:47.979+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:17.820399+0000 front 2024-03-16T09:13:17.820473+0000 (oldest deadline 2024-03-16T09:13:47.820930+0000) 2024-03-16T09:13:48.688 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:48 smithi019 podman[115632]: 2024-03-16 09:13:48.288788211 +0000 UTC m=+0.825497118 container remove e1df7d4b9eaaae0db8ec1665a91843125983dc607eafdb25342213753c6794a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-03-16T09:13:48.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:48 smithi019 podman[115726]: 2024-03-16 09:13:48.590235243 +0000 UTC m=+0.018246606 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:48.963 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:48 smithi019 podman[115726]: 2024-03-16 09:13:48.706406736 +0000 UTC m=+0.134418132 container create 4e0e4447c7a45ef13bdd2448bf8af1dec9f0015eb649cd822e7eea2322d36645 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-16T09:13:49.284 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:48 smithi019 podman[115726]: 2024-03-16 09:13:48.984926401 +0000 UTC m=+0.412937759 container init 4e0e4447c7a45ef13bdd2448bf8af1dec9f0015eb649cd822e7eea2322d36645 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-16T09:13:49.284 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:48 smithi019 podman[115726]: 2024-03-16 09:13:48.988243284 +0000 UTC m=+0.416254638 container start 4e0e4447c7a45ef13bdd2448bf8af1dec9f0015eb649cd822e7eea2322d36645 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:13:49.284 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:49 smithi019 podman[115726]: 2024-03-16 09:13:49.067348565 +0000 UTC m=+0.495359933 container attach 4e0e4447c7a45ef13bdd2448bf8af1dec9f0015eb649cd822e7eea2322d36645 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-16T09:13:49.285 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:49 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:49.023+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:49.285 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:49 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:49.023+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:17.820399+0000 front 2024-03-16T09:13:17.820473+0000 (oldest deadline 2024-03-16T09:13:47.820930+0000) 2024-03-16T09:13:49.571 INFO:journalctl@ceph.osd.4.smithi019.stdout:Mar 16 09:13:49 smithi019 podman[115726]: 2024-03-16 09:13:49.283182068 +0000 UTC m=+0.711193426 container died 4e0e4447c7a45ef13bdd2448bf8af1dec9f0015eb649cd822e7eea2322d36645 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-16T09:13:49.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.4.service' 2024-03-16T09:13:49.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:13:49.895 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-16T09:13:49.895 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-16T09:13:49.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.5 2024-03-16T09:13:50.183 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:49 smithi019 systemd[1]: Stopping Ceph osd.5 for 4c46b77e-e372-11ee-95c9-87774f69a715... 2024-03-16T09:13:50.183 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:50 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:50.019+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:50.184 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:50 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:50.019+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:17.820399+0000 front 2024-03-16T09:13:17.820473+0000 (oldest deadline 2024-03-16T09:13:47.820930+0000) 2024-03-16T09:13:50.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:50 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:50.181+0000 7f3d1dede700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:13:50.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:50 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:50.181+0000 7f3d1dede700 -1 osd.5 84 *** Got signal Terminated *** 2024-03-16T09:13:50.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:50 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:50.181+0000 7f3d1dede700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:13:51.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:51 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:51.003+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:51.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:51 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:51.003+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:17.820399+0000 front 2024-03-16T09:13:17.820473+0000 (oldest deadline 2024-03-16T09:13:47.820930+0000) 2024-03-16T09:13:52.462 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:52 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:52.001+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:52.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:52 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:52.001+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:17.820399+0000 front 2024-03-16T09:13:17.820473+0000 (oldest deadline 2024-03-16T09:13:47.820930+0000) 2024-03-16T09:13:53.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:53 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:53.014+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:53.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:53 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:53.014+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:17.820399+0000 front 2024-03-16T09:13:17.820473+0000 (oldest deadline 2024-03-16T09:13:47.820930+0000) 2024-03-16T09:13:54.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:54 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:54.034+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:54.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:54 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:54.034+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:17.820399+0000 front 2024-03-16T09:13:17.820473+0000 (oldest deadline 2024-03-16T09:13:47.820930+0000) 2024-03-16T09:13:55.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:55 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:55.049+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-16T09:13:12.519899+0000 front 2024-03-16T09:13:12.519921+0000 (oldest deadline 2024-03-16T09:13:37.820089+0000) 2024-03-16T09:13:55.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:55 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:55.049+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6814 osd.1 since back 2024-03-16T09:13:17.820399+0000 front 2024-03-16T09:13:17.820473+0000 (oldest deadline 2024-03-16T09:13:47.820930+0000) 2024-03-16T09:13:55.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:55 smithi019 ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5[102790]: 2024-03-16T09:13:55.049+0000 7f3d164ca700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.7:6822 osd.2 since back 2024-03-16T09:13:30.721616+0000 front 2024-03-16T09:13:30.721666+0000 (oldest deadline 2024-03-16T09:13:54.221572+0000) 2024-03-16T09:13:56.082 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:55 smithi019 podman[115832]: 2024-03-16 09:13:55.726458716 +0000 UTC m=+5.650363543 container died 5480e37181b114b6db7891620e63267f68498a94422d5e90591421e865e83639 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-16T09:13:56.083 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:56 smithi019 podman[115832]: 2024-03-16 09:13:56.081169327 +0000 UTC m=+6.005074156 container cleanup 5480e37181b114b6db7891620e63267f68498a94422d5e90591421e865e83639 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:13:56.462 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:56 smithi019 bash[115832]: ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5 2024-03-16T09:13:56.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:56 smithi019 podman[115851]: 2024-03-16 09:13:56.249425891 +0000 UTC m=+0.519160824 container remove 5480e37181b114b6db7891620e63267f68498a94422d5e90591421e865e83639 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-16T09:13:56.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:56 smithi019 podman[115942]: 2024-03-16 09:13:56.600535932 +0000 UTC m=+0.018579724 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:56.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:56 smithi019 podman[115942]: 2024-03-16 09:13:56.700355095 +0000 UTC m=+0.118398884 container create 11cd31c7f484665c7d5fd9b38f3abf197276d85f27d6a3e0ff149db58b3a3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T09:13:56.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:56 smithi019 podman[115942]: 2024-03-16 09:13:56.902675766 +0000 UTC m=+0.320719571 container init 11cd31c7f484665c7d5fd9b38f3abf197276d85f27d6a3e0ff149db58b3a3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-16T09:13:56.963 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:56 smithi019 podman[115942]: 2024-03-16 09:13:56.905913018 +0000 UTC m=+0.323956808 container start 11cd31c7f484665c7d5fd9b38f3abf197276d85f27d6a3e0ff149db58b3a3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-03-16T09:13:57.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:56 smithi019 podman[115942]: 2024-03-16 09:13:56.995543151 +0000 UTC m=+0.413586959 container attach 11cd31c7f484665c7d5fd9b38f3abf197276d85f27d6a3e0ff149db58b3a3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:13:57.463 INFO:journalctl@ceph.osd.5.smithi019.stdout:Mar 16 09:13:57 smithi019 podman[115942]: 2024-03-16 09:13:57.20174856 +0000 UTC m=+0.619792350 container died 11cd31c7f484665c7d5fd9b38f3abf197276d85f27d6a3e0ff149db58b3a3745 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-4c46b77e-e372-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-16T09:13:57.763 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4c46b77e-e372-11ee-95c9-87774f69a715@osd.5.service' 2024-03-16T09:13:57.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:13:57.808 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-16T09:13:57.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T09:14:19.851 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T09:14:33.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:14:33.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:14:33.280 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-16T09:14:33.283 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305/remote/smithi007/crash 2024-03-16T09:14:33.283 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/crash -- . 2024-03-16T09:14:33.321 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T09:14:33.321 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-03-16T09:14:33.323 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305/remote/smithi019/crash 2024-03-16T09:14:33.323 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/crash -- . 2024-03-16T09:14:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T09:14:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-03-16T09:14:33.351 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-16T09:14:33.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T09:14:33.396 INFO:teuthology.orchestra.run.smithi007.stdout:2024-03-16T09:06:52.482544+0000 mon.smithi007 (mon.0) 246 : cluster [WRN] Health check failed: Degraded data redundancy: 68/354 objects degraded (19.209%), 19 pgs degraded (PG_DEGRADED) 2024-03-16T09:14:33.397 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-16T09:14:33.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[SEC\]' /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T09:14:33.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]' /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T09:14:33.545 INFO:teuthology.orchestra.run.smithi007.stdout:2024-03-16T09:10:00.000172+0000 mon.smithi007 (mon.0) 552 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded 2024-03-16T09:14:33.546 INFO:tasks.cephadm:Compressing logs... 2024-03-16T09:14:33.546 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T09:14:33.588 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T09:14:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T09:14:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:: No such file or directory 2024-03-16T09:14:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T09:14:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-16T09:14:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mon.smithi007.log 2024-03-16T09:14:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.audit.log: /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mgr.smithi007.uamwtp.log 2024-03-16T09:14:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi007.znurst.log 2024-03-16T09:14:33.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mon.smithi007.log: /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mgr.smithi007.uamwtp.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-16T09:14:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-16T09:14:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T09:14:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi019.bncsil.log 2024-03-16T09:14:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T09:14:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-16T09:14:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi019.bncsil.log: /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mgr.smithi019.iuqxhi.log 2024-03-16T09:14:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi019.gmhzgo.log 2024-03-16T09:14:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.4.log: /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mgr.smithi019.iuqxhi.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-16T09:14:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi019.gmhzgo.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-16T09:14:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mon.smithi019.log 2024-03-16T09:14:33.620 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.3.log: 2024-03-16T09:14:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi007.znurst.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T09:14:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.0.log: 91.0% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T09:14:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log 2024-03-16T09:14:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T09:14:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-16T09:14:33.625 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T09:14:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T09:14:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mon.smithi019.log: gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log 2024-03-16T09:14:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.cephadm.log: 93.8% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi019.gmhzgo.log.gz 2024-03-16T09:14:33.626 INFO:teuthology.orchestra.run.smithi019.stderr: 85.4% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T09:14:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T09:14:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log: /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-volume.log: 87.6% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T09:14:33.627 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log: 87.6% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T09:14:33.627 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T09:14:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.2.log: 90.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T09:14:33.631 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi007.bsxctp.log 2024-03-16T09:14:33.645 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T09:14:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi007.bsxctp.log: 91.1% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T09:14:33.665 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mgr.smithi019.iuqxhi.log.gz 2024-03-16T09:14:33.914 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mon.smithi019.log.gz 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mgr.smithi007.uamwtp.log.gz 2024-03-16T09:14:34.635 INFO:teuthology.orchestra.run.smithi007.stderr: 90.6% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mon.smithi007.log.gz 2024-03-16T09:14:37.642 INFO:teuthology.orchestra.run.smithi007.stderr: 93.4% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-16T09:14:37.701 INFO:teuthology.orchestra.run.smithi019.stderr: 93.3% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-16T09:14:38.507 INFO:teuthology.orchestra.run.smithi007.stderr: 95.0% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi007.bsxctp.log.gz 2024-03-16T09:14:38.849 INFO:teuthology.orchestra.run.smithi019.stderr: 93.4% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-16T09:14:38.873 INFO:teuthology.orchestra.run.smithi007.stderr: 93.4% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-16T09:14:39.427 INFO:teuthology.orchestra.run.smithi019.stderr: 93.3% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-16T09:14:39.603 INFO:teuthology.orchestra.run.smithi007.stderr: 93.3% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-16T09:15:18.195 INFO:teuthology.orchestra.run.smithi019.stderr: 93.2% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi019.bncsil.log.gz 2024-03-16T09:15:18.196 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-16T09:15:18.196 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m44.594s 2024-03-16T09:15:18.196 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m58.190s 2024-03-16T09:15:18.196 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m1.822s 2024-03-16T09:15:28.003 INFO:teuthology.orchestra.run.smithi007.stderr: 93.0% -- replaced with /var/log/ceph/4c46b77e-e372-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi007.znurst.log.gz 2024-03-16T09:15:28.005 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-03-16T09:15:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m54.405s 2024-03-16T09:15:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:user 1m13.662s 2024-03-16T09:15:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m2.156s 2024-03-16T09:15:28.006 INFO:tasks.cephadm:Archiving logs... 2024-03-16T09:15:28.007 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305/remote/smithi007/log 2024-03-16T09:15:28.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T09:15:38.470 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305/remote/smithi019/log 2024-03-16T09:15:38.471 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T09:15:46.851 INFO:tasks.cephadm:Removing cluster... 2024-03-16T09:15:46.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 --force 2024-03-16T09:15:47.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4c46b77e-e372-11ee-95c9-87774f69a715 --force 2024-03-16T09:15:47.968 INFO:tasks.cephadm:Removing cephadm ... 2024-03-16T09:15:47.969 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T09:15:47.984 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T09:15:47.998 INFO:tasks.cephadm:Teardown complete 2024-03-16T09:15:47.999 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-16T09:15:48.010 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-16T09:15:48.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T09:15:48.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T09:15:48.120 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-16T09:15:48.131 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-16T09:15:48.131 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:15:48.133 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:15:48.145 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:15:48.147 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:15:48.150 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:15:48.150 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-03-16T09:15:48.150 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 24 +3529us[+3529us] +/- 73ms 2024-03-16T09:15:48.150 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -2098us[-2319us] +/- 49ms 2024-03-16T09:15:48.150 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -606us[ -606us] +/- 90ms 2024-03-16T09:15:48.150 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:15:48.152 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:15:48.152 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-03-16T09:15:48.152 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +3508us[+3291us] +/- 73ms 2024-03-16T09:15:48.152 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -2092us[-2309us] +/- 49ms 2024-03-16T09:15:48.152 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 -597us[ -814us] +/- 90ms 2024-03-16T09:15:48.152 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:15:48.152 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-16T09:15:48.163 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-16T09:15:48.163 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-16T09:15:48.180 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T09:15:48.279 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T09:15:48.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T09:15:48.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:15:48.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T09:15:48.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:15:48.420 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-16T09:15:48.431 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-16T09:15:48.463 INFO:teuthology.task.internal:Duration was 1905.425985 seconds 2024-03-16T09:15:48.464 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-16T09:15:48.479 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-16T09:15:48.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T09:15:48.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T09:15:48.522 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T09:15:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T09:15:48.799 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-16T09:15:48.800 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-03-16T09:15:48.800 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T09:15:48.859 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-03-16T09:15:48.860 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T09:15:48.893 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-16T09:15:48.893 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T09:15:48.902 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T09:15:48.961 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-16T09:15:48.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T09:15:48.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T09:15:49.589 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-16T09:15:49.600 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-16T09:15:49.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T09:15:49.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T09:15:49.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-16T09:15:49.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:15:49.670 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:15:49.693 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-03-16T09:15:49.721 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-03-16T09:15:49.736 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:15:49.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:15:49.763 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:15:49.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:15:49.790 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-16T09:15:49.804 INFO:teuthology.task.internal:Transferring archived files... 2024-03-16T09:15:49.805 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305/remote/smithi007 2024-03-16T09:15:49.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T09:15:49.855 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305/remote/smithi019 2024-03-16T09:15:49.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T09:15:49.895 INFO:teuthology.task.internal:Removing archive directory... 2024-03-16T09:15:49.895 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T09:15:49.897 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T09:15:49.943 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-16T09:15:49.956 INFO:teuthology.task.internal:Not uploading archives. 2024-03-16T09:15:49.956 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-16T09:15:49.972 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-16T09:15:49.972 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T09:15:49.975 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T09:15:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 09:15 /home/ubuntu/cephtest 2024-03-16T09:15:49.996 INFO:teuthology.orchestra.run.smithi019.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 09:15 /home/ubuntu/cephtest 2024-03-16T09:15:49.997 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-16T09:15:50.009 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-16T09:15:50.047 INFO:teuthology.nuke:Checking targets against current locks 2024-03-16T09:15:50.074 DEBUG:teuthology.nuke:shortname: smithi007 2024-03-16T09:15:50.075 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T09:15:50.096 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:34:22.360652', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTbtdlTgoD62hC8MA7sGYj7zhxCZE928MyRAME/9TidtgTgSQZSKV4FTznZXn2dbL4FRWoyT2Ti/kY27swpgxE='} 2024-03-16T09:15:50.103 DEBUG:teuthology.nuke:shortname: smithi019 2024-03-16T09:15:50.103 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T09:15:50.125 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602305', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:34:22.362401', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPjtRrt2ZraSIWALpw5k1RtZcMzXMjx8UNRAfILXNtwlAFBpbw7dKfex/N5L/pe0C8pvGn7YZi/qiT9ohiYJ4+k='} 2024-03-16T09:15:50.137 INFO:teuthology.orchestra.console.smithi007:Power off 2024-03-16T09:15:50.138 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T09:15:50.162 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-03-16T09:15:50.162 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:15:50.181 INFO:teuthology.orchestra.console.smithi019:Power off 2024-03-16T09:15:50.181 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T09:15:50.191 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-03-16T09:15:50.204 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-03-16T09:15:50.204 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:15:50.229 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-16T09:15:54.192 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:15:54.230 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:15:54.302 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-03-16T09:15:54.340 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-16T09:15:58.307 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:15:58.342 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:15:58.444 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-03-16T09:15:58.453 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-16T09:15:58.546 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-03-16T09:16:02.457 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:16:02.568 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-03-16T09:16:02.668 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-03-16T09:16:02.773 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1905.4259848594666 failure_reason: '"2024-03-16T09:10:00.000172+0000 mon.smithi007 (mon.0) 552 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-16T09:16:02.773 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T09:16:02.864 INFO:teuthology.run:FAIL